If you wish to see earlier Inside Risks columns, most of
those through December 2003
are at
http://www.csl.sri.com/neumann/insiderisks.html
For 2004 columns, see
http://www.csl.sri.com/neumann/insiderisks04.html
For 2005, see
http://www.csl.sri.com/neumann/insiderisks05.html
For 2006, see
http://www.csl.sri.com/neumann/insiderisks06.html.
Inside Risks 210, CACM 50, 12, December 2007
Building surveillance technologies into communications networks is risky. Two years ago, Greece discovered that legally installed surveillance software in a cellphone network had been surreptitiously activated by unknown persons: over one hundred senior members of their government were tapped for almost a year. Things were no better in Italy, where a number of employees at Telecom Italia were arrested for illegal wiretapping (with attempts at blackmail). In the U.S., recently released documents show that an FBI-designed communications interception system has security problems --- difficulty providing auditing, relying on passwords rather than token-based or biometric authentication, having no unprivileged userids --- leaving the system potentially vulnerable to insider attack.
Although we focus here on U.S. legislation, the security and privacy risks are global. For example, consider the Protect America Act (PAA), the August 2007 wiretap law updating the 1978 Foreign Intelligence Surveillance Act (FISA). FISA permitted warrantless interception of radio communications traveling outside the U.S. if the communications didn't involve targeted ``U.S. persons'' (citizens, residents, or U.S. corporations). The value of the exemption comes from the U.S. role as a hub: communications from other continents often transit the U.S. This role has been a real boon to the National Security Agency (NSA), the U.S. signals intelligence organization.
In recent years, however, cable has broadly replaced satellites. NSA pressed for broadening the radio exemption, arguing it had become inadequate. The PAA went significantly further, allowing warrantless wiretapping whenever one end of the communications is ``reasonably believed'' to be outside the U.S. Beyond eliminating the delays associated with warrants, the PAA arguably allows using the communication carriers' transactional data, including histories for real-time determination of interception targets.
Determining the origin of a communication in real time is not always easy. Locating the source of a phone call depends on the accuracy of information from the remote phone switch, but technologies such as VoIP and PBXes may alter the data. An Internet address reveals neither a computer's geographic location nor the user's identity, and techniques for inferring the rough location are not always accurate. While most calls outside the U.S. involve foreigners talking to foreigners, most communications within the U.S. are constitutionally-protected U.S. persons talking to U.S. persons. Any surveillance system built to satisfy the new law (or later amended versions) increases the chances that communications of Americans will be inadvertently collected.
When you build a system to spy on yourself, you entail an awesome risk. By building a communications surveillance system itself --- and saving its enemies the effort --- the U.S. government is creating three distinct serious security risks: danger of exploitation of the system by unauthorized users, danger of criminal misuse by trusted insiders, and danger of misuse by government agents. How can these risks be mitigated?
Minimization matters. An architecture that minimizes the collection of communications lowers the risk of exploitation by outsiders and exposure to insider attacks. Collect traffic at international cableheads rather than at tandem switches or backbone routers, which also carry purely domestic traffic. Regardless of where communications are intercepted, collected traffic could be subjected to multiple tests to determine whether its source and destinations are inside the U.S., and, if so, discarded before any further processing is done.
Architecture matters. Using real-time transactional information to intercept high volume traffic makes architectural choices critical. Robust auditing and logging systems must be part of the system design. Communication providers, who have technical expertise and decades of experience protecting the security and privacy of their customers' communications, should have an active role in both design and operation. ``Two-person control'' --- control by two authorized parties who know how the system should work --- is as applicable to organizations as to individuals.
Oversight matters. The new system is likely to operate differently from previous wiretapping regimes, and likely to be using new technologies for purposes of targeting wiretaps. There should be appropriate oversight by publicly accountable bodies. While the details of problems may remain classified, there should be a publicly known system for handling situations when ``mistakes are made.''
Surveillance built into communications networks must include minimization, robust controls, and oversight into system design. Otherwise, Trojan horses could threaten the central nervous systems of entire nations, where the threats to citizens' security and privacy could be immense.
This column is based on a paper by the named authors, ``Risking Communications Security: Potential Hazards of the ``Protect America Act'', http://crypto.com/paa.pdf.
========================================================
Inside Risks 209, CACM 50, 11, November 2007
Electronic voting has spread throughout the U.S. and the world without sufficient attention to reliability, security, or transparency. Today's e-voting systems use proprietary code, and vendors have often asserted the confidentiality of this code when independent reviews of certified systems were requested. This confidentiality conflicts with the transparency required for public elections.
In order to provide an independent assessment of the voting systems certified for use in California, Secretary of State Debra Bowen initiated a top-to-bottom review of those e-voting systems. She asked us to recruit a team of experts and gave us access to all the equipment, source code, and technical information that the Secretary of State's office had.
The results showed that the systems appeared not to be designed or implemented with security in mind. The design and implementation ignored basic security principles, and we found serious security vulnerabilities in all three vendors' systems. The security flaws were systemic and surprisingly similar across the three systems.
For example, malicious code could exploit vulnerabilities in the voting software to spread virally from machine to machine. As a result, when the voting machines return results to election central to count the votes, a virus could infect the county's election management systems. At the next election, the infected election management systems could then infect every voting machine in the county.
This virus could be introduced at several points in the process. An attacker could tamper with an e-voting machine while it is stored unattended overnight in a polling place. For some of the systems, a voter could introduce malicious code in under a minute, while voting.
Many flaws resulted from elementary mistakes such as straightforward buffer overrun vulnerabilities and flawed cryptography. One piece of voting software appends a three-letter suffix to a password and sends this ``encrypted'' result over the network. Another has encryption keys hard-coded in the source code, meaning the keys are the same for all machines using that software -- an obvious security flaw. One of the manufacturers used its own name as a hard-wired password. Our public reports had to be written carefully to convey the depth of the problem without providing a "road map" for attackers.
We drew several lessons from this exercise.
First, the national regulatory system has not worked well. Federal testing repeatedly failed to detect flaws in voting systems. Election officials relied in good faith upon these certifications when they purchased, deployed, and used these voting systems. They, and voters, deserve better.
This should provide a strong impetus to reform the oversight system so that states do not have to bear the cost of securing voting systems one state at a time. Vendors will build whatever the regulatory system allows and the marketplace demands. So far these forces have failed to weed out flawed voting systems.
Fortunately, the results of the top-to-bottom review give us an opportunity to change the regulatory process to make it effective. Federal officials are currently preparing a major revision of the federal voting standards, and we encourage the computing community to become more involved.
Secondly, applying technology to solve one problem may introduce other problems. E-voting systems were introduced to eliminate paper and problems such as hanging chads. However, without paper, voters cannot check that their vote is correctly recorded and cannot independently validate vote totals. Thus the solution to one problem introduced another: the violation of a fundamental tenet, that there must be an independent means for verifying results.
This problem can be mitigated with voter-verified paper records that election officials audit after each election. However, only 16 states currently require this. The security vulnerabilities we found highlight the importance of election auditing: without audits, there may be no way to rebut suspicion of tampering.
Electronic voting systems form a critical part of the election process. We have far to go to ensure they are a transparent and secure part of that process.
Matt Bishop (bishop@cs.ucdavis.edu) is a professor in the Department of Computer Science at the University of California at Davis. He teaches and does research in computer security and information assurance. David Wagner (daw@cs.berkeley.edu) is a professor in the computer science division at the University of California at Berkeley, a cofounder of the ACCURATE center on voting, and a member of the federal advisory committee charged with helping draft the next-generation voting standard.
========================================================
Inside Risks 208, CACM 50, 10, October 2007
Inside Risks readers are familiar with descriptions of cybersecurity threats and potentially dire consequences, particularly as more critical activities become dependent on cyberspace. They are also recognize the high ongoing burden of living with and defending against cyberattacks. At the same time, many policymakers (including those with responsibility for security) have a wide range of views about the severity of the threats. The resulting lack of urgency has impeded progress towards a more secure cyberspace.
A recent National Research Council report (Toward a Safer and More Secure Cyberspace, National Academies Press, Washington, DC, 2007, http://www.nap.edu, analyzes aspects of the current cybersecurity dilemma and articulates five key points regarding cybersecurity: First, the report argues that we should focus more attention on cybersecurity for both public safety and economic reasons. Increasing dependence on information technology (IT) steadily increases the risks that cyberattacks could impact the safe and orderly operation of systems across the board, including for critical infrastructures. From an economic perspective, the costs and risks related to computer security are high and could begin to outweigh the benefits of the increasing use of IT. This in turn could cause us to reject key opportunities to apply IT more widely.
Second, particularly in huge, open, networked systems, the quest for cybersecurity must be seen as an ongoing, multifaceted battle that will never end. New threats will emerge for many reasons -- for example, as responses to new defenses. New vulnerabilities will emerge as innovation adds applications and changes underlying systems. Our increasing dependency on information technology amplifies the incentives to compromise it. In important ways, the problems of cybersecurity are similar to security problems arising in other domains. Cybersecurity is a problem here to stay.
Third, there are important implications for the appropriate cybersecurity research agenda. Because of the breadth of the problem, there are no silver bullets that will allow government policymakers, corporate decision makers, or individual users to take cybersecurity problems off the radar screen, and so it is futile to seek narrowly focused research agendas that will decisively "solve the problem." Consequently, the research agenda must be both broad and interdisciplinary, and draw not only on computer science and software engineering but expertise in human factors, law, economics, political science, and psychology as well.
The report divides the research agenda into six categories. Research is needed on blocking and limiting the impact of compromise; enabling users to limit anyone or anything with access to a system component (computing device, sensor, actuator, network, and so on) accountable for the results of such access; promoting deployment of good cybersecurity technologies and practices; and deterring would-be attackers. A fifth category is aimed at cross-cutting problem-focused research addressing applications as well as systems. A sixth category focuses on unorthodox ``out-of-the-box'' research that may offer potentially greater gains. As just two examples, the report proposes research on how to motivate creation of more robust software, and the maintenance of information provenance from information creation to modification. The report contains about 150 pages of details on cybersecurity research opportunities.
Fourth, recognizing that cybersecurity will improve only if many agree on its objectives and increase their focus, the NRC study proposed a Cybersecurity Bill of Rights (CBoR) that is a statement of security goals or expectations, depending on whether the reader views himself an IT creator or user. It illustrates what society could reasonably expect in the way of security in its information technology. Because most of today's information technologies are not designed or implemented with the goals of the CBoR in mind, it also vividly illustrates the enormous gap between what information technologies should do and what they presently do.
The ten provisions of the CBoR are in four categories: Holistic system properties related to availability, recoverability, and control of systems; Traditional security properties related to confidentiality, authentication, provenance, and authorization; Cross-cutting properties such as safe access to information, confident invocation of important transactions particularly including those control physical devices, and knowledge of what security exists in a system; and Appropriate justice related to cybersecurity.
Finally, the report argues that current cybersecurity research funding is insufficiently directed to meeting the objectives defined by the CBoR. The report recommends that policymakers establish budgets to support a significant fraction of the good ideas proposed for long-term cybersecurity research.
[ADDED NOTE: The entire NRC report is now available without cost: http://www.cyber.st.dhs.gov/docs/Toward_a_Safer_and_More_Secure_Cyberspace-Full_report.pdf.]
========================================================
Inside Risks 207, CACM 50, 9, September 2007
The recent denial-of-service attacks in Estonia reprise a running theme in this column space: the prevalance of inherent weaknesses regarding security, survivability, and resilience of our computer-communication infrastructures. These attacks provide another warning of things to come. They also suggest further reasons why renewed energy should be expended on improving the trustworthiness of our infrastructures and on somehow reducing our critical dependencies on systems, networks, and people that are not trustworthy.
In the July column, Charles Perrow suggested reducing the surface of exploitable vulnerabilities and providing greater diversity. In the August column, David Parnas pointed out some remaining difficulties, noting that there is still much more to be done --- including insisting on precise specifications and applicable standards, and transcending simplistic approaches. He might have added the urgent need for pervasive use of the good software engineering practices that he himself has been pioneering since the early 1970s.
The Estonian attacks began at the end of April 2007, and demonstrated quite profoundly how widely distributed and loosely coordinated denial-of-service attacks could affect an entire nation. (Estonia popularly refers to itself as E-stonia, because almost everyone there has Internet access.) Initially, only Estonian governmental computer systems were attacked, which later spread via the Internet throughout the country -- to newspapers, TV stations, schools, and banks. The attacks intensified on 3 May (which coincided with protests in Moscow against the Estonian removal of a Soviet-era war monument) and again on 8-9 May (when Europe commemorated the end of World War II).
Although some Russian government server systems were involved, many other systems participated in the attacks (presumably unsuspectingly, as compromised zombie systems). The Estonian government suspected Russian complicity. A Kremlin spokesman denied any Russian governmental involvement. Analysis is apparently still ongoing.
Many opportunities exist for creating major service denials, often requiring no direct access to system resources. Much more serious potential damage can result from the implantation of Trojan horses and other forms of malware (worms, viruses, and so on) --- some of which could have effects that were largely undetectable (such as surreptitious surveillance), others of which could trigger outages and situations from which recovery would be exceedingly difficult. Corruption of software delivery pipelines could add further risks. In some cases, the consequences could be irreversible.
The Internet is of course a world-wide medium, and risks are not confined by national boundaries. However, the Estonian case apparently represents the first case in which a specific country was the target nationwide. As such, it illustrates only the tip of an iceberg with respect to the enormous potential for disruption -- whether widespread or specifically targeted. (In contrast, the 1988 Internet Worm targeted only BSD Unix systems, although performance degradation and unsubstantiated fear affected other systems as well.)
The technological expertise and nontechnological incentives required to carry out much more damaging attacks are already widely available to nation states, organizations, and even individuals. The Estonian case serves to remind us once again of the need for more coherent serious measures to reduce the risks.
What can you do to help? First, study and act upon Toward a Safer and More Secure Cyberspace, National Academies Press, 2007, a new report from the National Research Council Computer Science and Telecommunications Board that assesses the overall situation -- coming to some old conclusions that are still valid, as well as new ones. Second, make your concerns known to your elected government officials, at all levels. Third, partake in research and development that can help overcome the underlying problems. Above all, if you are developing systems or teaching how to do it, apply security principles and wisdom to network routers, secure name services, operating systems, application software, critical national and global infrastructures, and consider how such endeavors depend on one another. Today, all of these have serious associated risks. Assume that weaknesses will exist, and try to compensate for them.
The feeling that ``we have not yet had the electronic tsunami or Chernobyl, so why should we worry?'' tends to undermine attempts at constructive remediation. It is time to recognize that computer-related disasters can happen anywhere or everywhere, irrespective of geographical borders.
========================================================
Inside Risks 206, CACM 50, 8, August 2007
It is Computer Science ``folk wisdom'' that our computer systems, particularly the networks, are unnecessarily vulnerable because so many of our systems are either made by Microsoft, highly dependent on Microsoft software, or required to interact with Microsoft software. Many see this as a single point of failure, an Achilles' heel. Analogies are drawn to situations such as many people concentrated in a dangerous area, large quantities of hazardous materials stored in one place, or systems reliant on a single power source. Many propose that we can decrease our vulnerability by insisting on the use of non-Windows operating systems - thereby increasing diversity. This column questions that view.
Diversity, when combined with redundancy, is a well-established approach to increasing the reliability of safety-critical systems. For example, having two independent pumps, either of which is adequate, may decrease the probability of a complete outage; using two pumps of diverse manufacture may make it less likely that both fail at once.
Predictions that diversity will increase reliability assume:
* Redundancy: The system must be functional even if one pump fails.
* Independence: The failure of one pump must not make the failure of the other more likely, and the pumps must not depend on shared resources.
* Deep diversity: The pumps must be fundamentally different so that they are unlikely to have common design faults.
* Interoperability: The pumps must function well together. If not, the number of failures may increase. The validity of these assumptions must be carefully examined in each individual situation; they do not seem to apply to today's computer systems.
That increasing diversity does not always improve reliability is obvious if we think of automobile traffic. We would not make our roads safer by demanding that 30% of us switch to the other side of the road. On the roads there is limited redundancy and independence. All drivers are essential; a failure of one affects many others. Further, cars driven with different rules would not be interoperable. Examining the case at hand we see:
* In today's computer systems, there is remarkably little redundancy. When a Canadian tax system failed, no tax returns could be submitted. Basing the Irish system on a different OS would not have helped. Having two diverse Canadian systems would help only if they implemented exactly the same rules.
* In today's OS market, diversity is shallow. Linux, and the various UNIX versions are very much alike and Apple has joined this club. More subtly, studying Windows and the UNIX family, one will see that the developers have all ``learned from'' each other or from common sources. More generally, programmers often overlook the same situations and write diverse programs that fail on the same cases. Independence is equally questionable. Two communicating systems constitute a single system. A failure of one can cause problems for the other. Frequently, networks stop while all the elements patiently wait for one to finish. One false message can trigger a cavalcade of failures.
* Interoperability of independently developed software systems is very difficult to achieve. Communication protocols that are ``almost alike'' often fail to work together. Many experienced ITD managers wisely insist on a monoculture in their networks because they fear incompatibility and cannot deal with the finger pointing that occurs when one supplier's system does not work well with another. If one product is updated to remove a fault (or feature) on which another depends, the combined system fails and it is not clear who should fix it.
Were we to insist on a diverse mix of operating systems, their failure to work together properly could actually reduce reliability and increase vulnerability. In some cases, the whole system would be no more reliable than its weakest link. When I buy a light bulb, a tire, or a car, I benefit from competition and limited diversity because there are tight standards that allow me to replace one brand with another. We do not have comparable standards for operating systems. Each upgrade causes some trouble in application software.
This column is neither pro-monopoly nor pro-Microsoft. It is pro-realism. If we want the advantages of diversity and competition in support software, there is much hard work to do. We need precise specifications for systems that are to operate in our networks; we also need the ability to enforce those standards. Otherwise, increasing diversity might make the situation worse.
David Lorge Parnas is Director of the Software Quality Research Laboratory (http://www.sqrl.ul.ie/ at the University of Limerick in Ireland.
========================================================
Inside Risks 205, CACM 50, 7, July 2007
The U.S. is not good at preventing or handling disasters, so it is time to consider reducing the size of our vulnerable targets: the concentrations of populations in risky areas, the concentrations of hazardous materials, and the concentration of economic power in the huge corporations that sit astride our critical infrastructures.
The devastation of New Orleans was amply predicted, with a large population so obviously at risk. In Katrina, nature deconcentrated the population at great human cost. New Orleans could be one third the size if rebuilt on its higher ground and still be one of the biggest U.S. ports, and yet have fewer but safer levees. Hurricane Andrew in 1992 barely missed Miami; a direct hit would mean enormous devastation. Miami could be downsized by charging realistic, unsubsidized insurance.
Other concentrations of populations in risky areas include California's Delta, where there are settlements 20 feet below the water level behind inadequately maintained levees. A major earthquake could breach levees and flood large areas, letting in salt water from San Francisco Bay and salinating over half of the state's fresh water supply. The St.~Louis area, which is repeatedly flooded, continues to promote intensive settlements on the floodplains, and the state legislature forbids counties from raising levee heights beyond the state standards.
Not just populations are concentrated in risky areas; so are hazardous materials. In the Mississippi Flood of 1993, a large propane tank farm in St. Louis barely escaped having a massive explosion. Our landscape is littered with such potential agents of mass destruction, often in highly populated areas. Large cities are sprinkled with windowless telecommunication `hotels' that store huge quantities of diesel fuel to keep the servers cool. They should be smaller and moved away. A large oil company that failed to secure its oil storage tanks before Katrina hit made 18,000 homes uninhabitable. Did it have to be in a residential area of New Orleans? Fortunately, only a tiny part of our railroad tracks pass through cities, so most derailments occur in rural or uninhabited areas, but a Baltimore tunnel fire in 2001 was devastating. 90-ton tank cars of deadly chlorine gas pass through our cities all the time; an accident or a bomb could put a million people at risk, according to official estimates.
About 85% of our critical infrastructures are privately owned, and increasingly highly concentrated. Concentration in the electric power industry has increased since deregulation in the 1990s, and utility rates, outages, and the scope of blackouts have all increased as a result. Concentration results in more long-distance transmission and less of the local control that was more likely to insist upon proper maintenance and investment in the grid. A third of the U.S. plants store their spent fuel rods in the open, vulnerable to a primitive terrorist attack or tornado or industrial accident that could release more radiation than the plant itself in a meltdown. The U.S. has 103 of these aging, largely unprotected, often poorly run potential agents of mass destruction within 50 miles of half of our population. The plants cannot be deconcentrated or downsized, but antitrust actions could deconcentrate the industry, reducing its political power. This would allow higher transmission standards to be set and enforced, and more effective regulation of our nuclear plants.
Although these problems are more severe in the US than in Europe, because of Europe's stronger government regulations, one particular concentration is transnational: computer operating systems and applications. In the U.S., software is ubiquitous, and the industry appears highly deconcentrated. But the Departments of Homeland Security and Defense (for example) are heavily dependent on Microsoft products. In business and industry, SAP and IBM's CICS are increasingly linked with Microsoft operating systems and applications. Alarmingly, this software runs parts of our critical infrastructures--making them increasingly vulnerable to software errors and malware from hackers and possibly terrorists. If there were more variety in operating systems, these critical systems could choose the vendors that offered greater reliability and security.
Charles Perrow is the author of Normal Accidents: Living With High Risk Technologies (Princeton, 1999) and The Next Catastrophe: Reducing our Vulnerability to Natural, Industrial, and Terrorist Disasters (Princeton, 2007), from which this column is drawn.
========================================================
Inside Risks 204, CACM 50, 6, June 2007
In his February 2007 column, Peter Neumann mentioned some failures that resulted from inadequate attention to the architecture of the overall system when considering components. But many developers cannot influence or even comprehend the system architecture. So, how can they be held responsible in such a situation? Although many system failures can be detected and prevented without reference to the system architecture, professionals working on isolated components still have professional -- indeed, moral -- duties to ensure their results are as risk-free as -- -- possible.
The aphorism ``He can't see the forest for the trees,'' comes to mind. From my perspective, there are two issues: Are there ``tools'' that permit those of you working at the ``tree'' level to see the larger context? Do you use them?
Here, ``tools'' means representations and analysis methods (supported by tools in the usual connotation) that can represent more than small, individual components of a larger system -- the ``forest.'' We have a number of representations -- wiring diagrams, flow charts, structure charts, UML, more formal techniques, and so on -- but they have (at least) two major faults: First, at best they can only incompletely represent the full scope of a complete system architecture and its environment. For example, they may have no way of representing the unexpected external event, or they may represent the physical parts of a system but not the software, or incompletely describe interactions between the system under consideration and the rest of the world, or not be able to fully represent potential (damaging) behaviors, and so on. The result is consequences that are not foreseen. Second, they typically support only the most limited forms of rigorous analysis techniques.
Various proposed analysis techniques can be applied to representations of computer-based systems, but many are neither proven nor widely used. The prevalence of ``testing'' rather than ``proving'' or ``assuring via simulation'' in the world of computer systems is a clear indication of the lack of practical, intellectual tools that are so vital in other areas of engineering. Consider how a new airplane is simulated many times before ever being test-flown -- and, fortunately, how rarely a new plane design crashes on initial flight! Or, how a new chip design is rigorously checked before being burned into silicon, even though some errors still do occur.
Substantial academic research for at least forty years has been aimed at addressing this lack of intellectual tools. While R&D continues and has produced some useful results, it has not produced a ``silver bullet.'' Nonetheless, I still believe strongly that research will eventually improve our stock of intellectual tools and produce automated aids for applying them to large complex systems. The alternative is to continue to flounder in the dark.
In the meantime, however, each of us must address the second issue -- using those tools that we do have. But, how can you do that in today's world of competitive pressures and failure up the line to insist on good engineering practice, at least in the case of software?
Put starkly, you must have the guts to apply what you do know how to do, demand training on those tools that may apply, and insist as professionals that you will not tolerate less. If you are a requirements analyst, insist that security issues be made a part of any overall requirements statement. If you are a systems designer, utilize available techniques for risk analysis carefully and rigorously. If you are a programmer or a component or sub-system designer, make sure your parts fit into the larger architecture in a way that minimizes risks. If you are a manager or supervisor at any level, enable and insist on these behaviors. If you are an educator, make sure your students learn to take the larger view and to consider risks.
Ultimately, it isn't sufficient just for individual tree planters to do the right thing. If a large-scale system is to be as risk-free as possible, the planners of the forest and the planters of the trees must be able to communicate and be incentivized to do so. If you are a system designer or development manager, you have broader purview and authority; thus, it is your responsibility even more. In short, risks are everyone's responsibility! The challenges are significant, but more research, development, successful examples, and human understanding are needed.
Peter A. Freeman ( http://www.cc.gatech.edu/staff/f/freeman/) is Emeritus Dean and Professor at Georgia Tech, and immediate past Assistant Director of NSF for CISE.
========================================================
Inside Risks 203, CACM 50, 5, May 2007
The information security literature is filled with risk pathologies, heuristics that we use to help us evaluate risks. I've collected them from many different sources.
Risks of Risks:
Exaggerated risks vs. Downplayed risks
----------------------------------------------
Spectacular vs. Pedestrian
Rare vs. Common
Personified vs. Anonymous
Beyond one's control vs. More under control
Externally imposed vs. Taken willingly
Talked about vs. Not discussed
Intentional or man-made vs. Natural
Immediate vs. Long-term or diffuse
Sudden vs. Evolving slowly over time
Affecting them personally vs. Affecting others
New and unfamiliar vs. Familiar
Uncertain vs. Well understood
Directed against their children vs. Directed towards themselves
Morally offensive vs. Morally desirable
Entirely without redeeming features vs. Associated with some ancillary benefit
Not like their current situation vs. Like their current situation
----------------------------------------------
When you look over the list, the most remarkable thing is how reasonable so many of them seem. This makes sense for two reasons. One, our perceptions of risk are deeply ingrained in our brains, the result of millions of years of evolution. And two, our perceptions of risk are generally pretty good, and are what have kept us alive and reproducing during those millions of years of evolution.
This is an important point. A general intuition about risks is central to life on this planet. Imagine a rabbit sitting in a field, eating clover. Suddenly, he spies a fox. He's going to make risk evaluation: stay or flee? The rabbits that are good at making these evaluations are going to reproduce, while the others are either going to get eaten or starve. This means that, as a successful species on the planet, humans should be really good at evaluating risks.
And yet, at the same time we seem hopelessly bad at it. We exaggerate some risks while minimizing others. We misunderstand or mischaracterize risks. Even simple security we get wrong, wrong, wrong-again and again. It's a seeming paradox.
The truth is that we are very well adapted to dealing with the security environment endemic to hominids living in small family groups on the highland plains of East Africa. However, the environment of New York in 2007 is different from Kenya circa 100,000 BC. And so our perception of risk diverges from the reality of risk, and we get things wrong.
When our risk perceptions fail today, it's because of new situations that have occurred at a faster rate than evolution: situations that exist in the world of 2007, but didn't in the world of 100,000 BC. Like a squirrel whose predator-evasion techniques fail when confronted with a car, or a passenger pigeon who finds that evolution prepared him to survive the hawk but not the shotgun, our innate capabilities to deal with risk can fail when confronted with such things as modern human society, technology, and the media. And, even worse, they can be made to fail by others-politicians, marketers, and so on-who exploit our natural failures for their gain.
This topic is explored more fully in Bruce's essay, The Psychology of
Security: http://www.schneier.com/essay-155.htm
. Bruce Schneier
========================================================
Inside Risks 202, CACM 50, 4, April 2007
The software engineering and online gaming communities have been rocked
by the conviction in a Texas court of an Icelandic programmer for crimes
committed in the secret virtual world Third Life, which is entered via a
hidden trap door in Second Life(r).
Two years ago, the Fifth Third Life Bank (FTLB) outsourced the
programming of a "virtually invulnerable" security system to David
Josephssen, known online as dvdjo. Following a major theft from FTLB
last year, he was charged with malpractice, being an accessory to grand
theft before and after the fact, and practicing software engineering
without a license--which is an aggravated felony in Texas, if done in
furtherance of a crime.
The charges of malpractice and accessory before the fact were based on
the success of the theft. FTLB's CIO testified "It was supposed to be
invulnerable. He either bungled or betrayed us." The charge of
accessory after the fact was based on the absence of any evidence of
intrusion in the FTLB security log. He was supposedly the only person
who had the password to remove these records.
After the trial, the jury foreman said that the only charge the jury
really understood was practicing software engineering without a license.
"But we figured, if he did that, he's a hacker, and heck, he probably
did all those fancy virtual things the DA said."
In a related civil case, the Recording Institute of American Avatars
(RIAA) has sued demanding "the return of items stolen from the virtual
vault of Fifth Third Life Bank, namely 1,023 virtual cloaks of
invisibility and 255 virtual gold master CDs, and all copies thereof,"
plus $100,000,000 in exemplary damages. Second Hollywood insiders say
that the RIAA was about to launch a campaign in Second Life featuring
CDs as costume jewelry. The cloaks of invisibility were to assist RIAA
avatars tracking down avatars listening to CDs instead of wearing them
(part of a "Bling, not Sing" campaign). Ima Ghofer, spokesman for
Dewey, Cheatham, and Howe, which is representing the RIAA in the case,
refused to comment, "because the matter is being litigated."
Josephssen was lured from Iceland to Texas for the trial by a pretexting
avatar who met dvdjo in an encrypted Third Life chat room, surrounded by
a virtual Faraday cage. Excerpts from the log of the meeting were
placed in evidence: "xyzzy. gota j0b 4u dvdjo. RL & vir $MMz, chaNc 2
bcum 133t haxorz karamat, lotsa RL sx," "TBH n2ez d00d. I cnt expln it
hEr. we tink d NSA mA hav brkn our cript0. JIC LMIRL," and "ul hav 2 cum
2 TX 2 get d Dtails."
Legal experts say that there were several reasons to bring the actions
in Texas. It is one of the few states that requires software engineers
to be licensed. Its courts follow the doctrine of habeas grabus;
how a criminal in custody was captured is irrelevant. And the death
penalty can be based on the gravity of the crime, rather than the guilt
of the accused.
The court in the small town of Diagon is known to trial lawyers as the
"Kazoom Courtroom," for its obscurity, quick trials, and hanging juries.
Perhaps the RIAA hoped that Josephssen could be executed before his case
drew the attention of the national media. ("Once dead, dvdjo would make a
great scarecrow," said an RIAA insider.) And indeed, that could have
happened, except for the posts of a lone blogger, writing as texasdirt at http://westofpecos.blogspot.com.
His reports of the trial and sentence stirred up a storm in the blogosphere
that attracted the attention of the ACLU and Amnesty International.
In its emergency appeal, the ACLU cited the inadequacy of Josephssen's
defense. According to West of Pecos, the public defender played
Tetris throughout the trial, failed to cross-examine any witnesses, and
when asked if he wished to call defense witnesses, said "Hell, no,
judge. I'm getting paid by the case." He also waived the right to
appeal, apparently over Josephssen's objections. AI's amicus
brief said that the pretexter violated federal law by using promises of
sex to lure a teenager across state lines.
But other groups are unsympathetic. The Motion Picture Artists
Association says "hanging is the traditional remedy for piracy," and
Electronic Frontier Fighters says "vigilante justice is a vital part of
any frontier."
Asked to comment on the severity of his sentence, Judge Roy Bean
replied, "This is Texas. If you can't stand the heat, stay off the
Internet."
Jim Horning (horning@acm.org) is a co-founder of Computer Professionals
for April Foolishness (CPfAF).
========================================================
Inside Risks 201, CACM 50, 3, March 2007
Information Technology trade publications report increasing information
security losses, questionable risk management and risk assessments, and
underfunding and understaffing. Government departments receive low grades
in security. Legislators react by adopting draconian laws such as
Sarbanes-Oxley. The poor state of information security derives from a
fundamental risk-based approach to security.
Management deals with risks every day, and risk reduction justification
makes it too easy to accept security vulnerabilities in exchange for other
benefits. It must be imperative and unavoidable that management support
effective security by our insisting on demonstrable diligence to avoid
negligence, addressing ethical aspects, complying with laws to avoid
penalties, and enabling businesses to be competitive and deliver secure
services within budget. This diligence method does better than what others
are doing, by exceeding benchmarks under similar circumstances, exceeding
standards and requirements of law, and using well-known good and new more
powerful controls.
It is relatively easy to justify increased security to stop or control
ongoing significant loss incidents such as virus attacks --- because they
are certainties, rather than intangible security risks. We can justify
security against incident loss certainties by straightforward calculation of
return on investment (RoI) based on real experience. The more difficult
problem is making a successful case for adequate security against rare but
significant threats such as enemies engaged in fraud, espionage, and
sabotage. Information security departments have attempted to justify
expending security resources to address these rare problems by managing and
reducing security risks. To manage, they must control; to control, they try
to measure the benefits of information security ``scientifically'' based on
risk reduction. However, security risk reduction is generally not
measurable.
A security risk is defined to be an adversity, but measuring security risk
requires anticipating frequency and impact of rare loss events in a specific
security setting. Security risk is different than measurable business risk
that consists of voluntarily investing resources to produce a profit or meet
a goal. Security risk is not measurable, because the frequencies and
impacts of future incidents are mutually dependent variables with unknown
mutual dependency under control of unknown and often irrational enemies with
unknown skills, knowledge, resources, authority, motives, and objectives ---
operating from unknown locations at unknown future times with the possible
intent of attacking known but untreated vulnerabilities and vulnerabilities
that are known to the attackers but unknown to the defenders (a constant
problem in our technologically complex environments). In addition, when
enemies cannot exploit one vulnerability, they often attack other
vulnerabilities to accomplish their goals. Therefore, risks are related in
unknown complex ways so that reducing one risk may increase or decrease
other risks. Also, the impact may be minimal in major attacks and major in
minor attacks. For example, recall the complete failure of Barings Bank in
London resulting from a lack of simple separation of duties in just one
branch in Singapore. You never know what amount of liability, litigation,
or secondary effects may ensue after even a minor incident. "For want of a
nail the war was lost."
Many security efforts may affect one risk, and one security effort may
affect many risks --- such as occurs today with use of powerful security
software packages. Thus, risks and vulnerabilities cannot be paired in
simple ways. There are too many interrelated unknown and known variables,
with unknown values. They all change in unknown ways over time, depending
on unknown future circumstances such as system and business changes, labor
disputes, social and political changes, unknown enemies' failures and
successes, and enemy and defender frailties and irrationalities. It is
generally agreed that there is insufficient valid loss experience data to
support quantitative risk assessment applied to a specific instance, because
of victims' needs for confidentiality. Also, humans are notoriously bad at
qualitative risk assessment. Finally, there is no proof of effectiveness or
reported experience of performing security risk assessments cited in the
security literature, because they are proprietary and confidential.
Many security managers support my position on the failure of security based
on intangible risks, but they are so committed to it through job titles,
standards, advice in the literature, requirements in the law, and policies
that they are afraid to declare it inoperative. Risk-based security and
security risk management are the emperor's new clothes and must be replaced
with diligence-based security consisting of measurable and tangible
diligence, ethical practice, compliance, and enablement.
Donn B. Parker (donnlorna@aol.com), retired from SRI International,
has 40 years of experience as a researcher, management consultant,
and prolific writer/lecturer, on computer crime and information security.
========================================================
Inside Risks 200, CACM 50, 2, February 2007
Networking problems offer many potential lessons that need to be assimilated
by researchers, developers, and operators of highly distributed systems such
as computer networks and electric power distribution. Here we briefly
revisit some widespread outages and other extreme network behavior, and
explores ever increasing needs for trustworthy distributed control.
The 1980 four-hour ARPANET collapse (a combination of unchecked memory
errors and a weak garbage-collection algorithm) and the 1990 half-day AT&T
long-distance network collapse (a bug in newly installed software for
autorecovery from telephone switch failures) are examples of iteratively
propagating system failures. Similarly, numerous widespread power outages
have occurred, notably the Northeast U.S.~blackout of November 1965, the New
York State blackout in July 1977, blackout of ten western U.S. states in
October 1984, separate outages in July and August 1996 that took out much of
the Western U.S. (the second of which also affected parts of Canada and Baja
California in Mexico), and the Northeastern U.S. again in August 2003.
In each case, a single-point failure triggered a cascading effect.
Various other serious power outages include the month-long outage in Quebec
in January 1998 (due to the collapse of ice-laden transmission towers) and
the week-long area outages in Queens NY in July 2006 (due to fires and
failures in 100-year-old wiring). In addition, many less dramatic outages
have been reported, including a power failure that triggered fire alarms and
evacuation of Oregon's Portland Convention Center during the ACM OOPSLA
conference on October 26, 2006, and also shut down the surrounding area
including the light rail system, for almost an hour. Additional power
outages were compounded by failures of backup power systems, such as the
1991 four-hour shutdown of New York City's three airports and an ESS
telephone system (due to misconfiguration of the backup power, which instead
of being driven by the activated generators was running on standby batteries
until they were drained).
The most recent case of a cascading power outage occurred on November 4,
2006, and affected about 10 million people in Germany, Austria, Italy,
France, Spain and Portugal. Among other disruptions, 100 regional Deutsche
Bahn trains were disrupted. The widespread outage began with a supposedly
routine event, in which a power line over the Ems River in northern Germany
was shut down to allow a ship (the Norwegian Pearl) to pass safely.
Following the so-called "N-1 criterion" for stability that must be
applied before prospective reconfiguration can be authorized, alternative
power was procured to compensate for the line that would be shut down, and
simulations were run to demonstrate the power sufficiency of that preventive
measure. However, a second-order reevaluation was not conducted following
the reconfiguration to analyze the increased loads caused by the shutdown
itself, and the resulting overload propagated across Europe, lasting about
four hours before power could be restored.
What is perhaps most alarming about these and other unanticipated outages is
that such disruptions continue to occur --- despite remedial efforts or in
the absence of such efforts. Furthermore, the effects are not simply
cascading or unidirectionally propagating, because in many cases feedback
loops exacerbate the breadth and speed of the outages. Clearly, more
proactive efforts are needed to analyze systems for potential widespread
fault modes and ensuing risks, and to enable appropriate real-time responses
and automated remediations. (Of course, proactive defenses are also
desirable against environmental disasters such as tidal waves, global
warming, and hurricanes.)
Distributed control of distributed systems with distributed sensors and
distributed actuators is typically more vulnerable to widespread outages and
other perverse failure modes such as deadlocks and other unrecognized hidden
interdependencies (particularly among components that are untrustworthy and
perhaps even hidden), race conditions and other timing quirks, coordinated
denial-of-service attacks, and so on. Achieving reliability, fault
tolerance, system survivability, security and integrity in the face of
adversities in highly distributed systems is problematic; formal analyses
and system testing are highly desirable, but are potentially more complex
than in systems with more centralized control.
Proactive system architectures and further analytic efforts are needed to
prevent and quickly remediate such problems in power distribution and
information networks. Long-term planning, closer oversight, and
multipartite coordination are also essential, along with improvements in
university curricula and operational management.
========================================================
Inside Risks 199, CACM 50, 1, January 2007
Former FCC Chairman William Kennard recently characterized the
Internet network neutrality debate as nothing but a battle between
the "extremely wealthy" and the "merely rich," suggesting it
was a distraction from truly important telecom-related issues.
Kennard misses the point. The outcome of this controversy will
affect everybody who ever comes into contact with the Internet, and
as usual the interests of ordinary consumers are being left in the
lurch. Anti-neutrality forces (primarily the increasingly
conglomerated telecommunications provider giants) have been
manipulating this controversy to their own advantage, and to the
detriment of nearly everyone else.
Starting from Defense Department research origins, the Internet and
its ancestors have thrived by providing essentially neutral channels
of communications, with the networks themselves not imposing
unreasonable constraints on applications using their facilities, from
e-mail to video streaming, and everything in between. The Internet
is now integral to our lives, and we depend upon straightforward
access to these services from firms and organizations of all sizes.
But to the telephone companies and their ilk, neutral transmission
isn't an adequate profit center. They want a cut of everybody's
action, as exemplified by AT&T's CEO Edward Whitacre infamously
swiping at Google and other major Internet services, claiming that
they were using "his pipes" for free.
His assertion is an utter fallacy, and the anti-neutrality folks know
it. We all already pay for our Internet access. Google pays for
their connectivity -- undoubtedly not petty cash either. Every
DSL or Internet cable hookup is feeding
money into telecom company coffers. Even if we choose to use VoIP
phone services, we're still paying a phone company or cable TV firm
for the underlying Internet circuits.
Much of the anti-neutrality argument is simple greed in action. The
telecom providers have watched their traditional business models
decay around them, and true to their roots, are looking for new ways
to strangle any real competition, in league with a public relations
and lobbying spin aimed at obscuring this fundamental fact.
If that sounds too strong, let's remember that the telecom landscape is
littered with the broken promises and unfair tactics of the dominant
telephone companies in particular -- promised broadband rollouts never
delivered, "cherry-picking" of lucrative neighborhoods for advanced services
deployments, major rate hikes when regulatory scrutiny is lifted, and so on.
These guys' methods are textbook examples of predatory practices. No wonder
it's so difficult to believe these telecom firms now, or why so many
observers feel that laws mandating neutrality -- enacted today, before
neutrality slips away -- are the only practical approach to maintaining
Internet fairness.
The telecom providers' historical behaviors, increasingly restrictive
ISP Terms of Service requirements, and the generally lackadaisical or
even consumer-hostile attitudes of the FCC and most other related
regulators are a clear warning. Outrageously skewed bandwidth
pricing, demands for profit participation, blocking or throttling of
services that are viewed as competitive, and other punitive telecom
provider actions are likely to occur if it's possible to get away
with them.
Anti-neutrality proponents have incorrectly suggested that
pro-neutrality arguments are invalidated by a large and powerful
firm such as Google taking a strong stance in favor of network
neutrality. To be sure, Google does have a financial interest in the
outcome, but so do the rest of us. In the non-neutral Internet
world dreamt of by the telecom providers -- with only after the
fact, glacially slow antitrust suits as the main possible redress
for abuses -- it's doubtful that Google, Vonage, eBay, or many other
household Internet names could have afforded to really get off the
ground in the first place.
A non-neutral Net would likely be a death knell for an entire future
of competitive Internet entrepreneurs who might otherwise have
brought us a vast range of useful new services, especially start-ups
and other initially small businesses. Neutrality is an aspect of
the Internet that is so taken for granted that it seems invisible and
intrinsic, but it has been critical to the Internet's success to
date.
It's unfortunate the network neutrality controversy has
escalated to an emotional level, which sometimes obscures underlying
facts. But most Internet users simply don't realize how drastically
and negatively they could be affected if anti-neutrality arguments
hold sway.
Getting true network neutrality back after it's been lost is likely
to be effectively impossible. Except for the anti-neutrality camp
itself, we'd all be worse off with a non-neutral Internet, and
that's a risk that we simply must not accept.
Lauren Weinstein (lauren@pfir.org) is co-founder of People For Internet
Responsibility (http://www.pfir.org). He moderates the Privacy Forum
(http://www.vortex.com/privacy).
========================================================
Risks of Virtual Professionalism
Jim HorningRisks of Risk-Based Security
Donn B. ParkerWidespread Network Failures
Peter G. NeumannMa Bell's Revenge: The Battle for Network Neutrality
Lauren Weinstein