CACM Inside Risks

Following the clickable table of contents, these columns are given in REVERSE CHRONOLOGICAL ORDER. In order not to break existing hot links to old columns, but also not to allow the primary file to grow without bounds in the future, the columns beginning with January 2004 are now kept in a separate files, one per year, requiring an indirect download.

The text here is not necessarily completely identical to the actual printed versions, in which some ACM editing has taken place (for example, due to space limitations). Of some particular interest recently, discussion of computer-related voting can be found in the columns of January 2001, November 2000, June 2000, and in earlier columns November 1993, November 1992, and November 1990 appended below, which you will find in the continuation of the menu. Other columns prior to December 1997 can be added on request.

========================================================

Inside Risks 162, CACM 46, 12, December 2003

The Devil You Know

Question: What's worse than buggy software? Answer: Patches and upgrades that make things even worse. This is a dilemma critical to many applications. How should we cope with the untold millions of computers that are constantly subjected to penetrations, viruses, worms, and other nasties that exploit a steady stream of security weaknesses and flaws? Is finessing, coercing, or even forcing users to install updates a solution -- or just an invitation to further aggravation and potential disasters?

The underlying problem is obvious. Much commercial software is a mess on the inside. Get past the flashy graphics and the fancy user interfaces, and you frequently descend into a nightmarish realm of twisted spaghetti-like code that might better belong in a Salvador Dali painting. One recurring type of software security bug -- buffer overflows -- dates back to the dawn of computing, but only recently are we seeing some serious attempts to limit this vulnerability systemically.

Meanwhile, the 800-pound gorilla of PC software, Microsoft, sends forth a stream of patches intending to correct what they themselves designate as ``critical'' security flaws in their systems, applications, and even their own previous patches. Microsoft certainly isn't alone when it comes to software flaws, but as the massively dominant desktop system vendor, their software and support decisions tend to have much more influence on most consumers, businesses, and other organizations than those of other firms.

Microsoft has expressed continuing concerns about user behavior. They seem to say, in essence, ``If we could just find some way to get users to install each and every patch forever, the bugs in our software wouldn't really matter so much.'' This seems somewhat akin to a vampire, after having bitten your throat and transformed you into one of the living dead, pointing out that vampirism really wasn't so bad as long as you got plenty of blood every night and stayed out of the sun.

Many computer users pay little if any attention to the issues of security bugs. They take the unfortunate but understandable view that if something seems to be working adequately, don't try to fix it. In the security realm, this can indeed be a very dangerous attitude.

On the other hand, many expert computer users (particularly those using Microsoft products) don't ignore patches -- they're simply terrified of them. Too often, installing seemingly innocuous ``fixes'' into working systems results in instability, crashes, or even total unusability. Interactions between patches and other software, particularly already-installed third-party packages, can result in widespread disruption to both application and system software. And often there's been no going back without total system restores. For example, Microsoft patches have often been incapable of being effectively removed in case of problems. Microsoft has now announced the move to (more organized) monthly aggregated patches -- but has already had to issue additional interim patches to patch their monthly patches!

For a time, it was reported that Microsoft was considering the possibility of forcing virtually all users of their systems to accept Microsoft's Internet-delivered updates. More recently, they've been talking about changing the defaults for their ``home user'' systems to automatically accept Microsoft-provided ``critical'' Internet-delivered patches, unless specifically instructed otherwise by users. Not only is it unclear how to accurately delineate this ``home users'' category, but there may be in such a segregation an ominous attitude -- that it's somehow less serious to screw-up home users' computers than those of businesses and other more well-heeled customers. This would be an unacceptable outcome.

Widely deployed automatic updating systems for PCs could carry with them another very real and serious risk -- the possibility of hackers cracking the Internet-connected update mechanisms, either at the user systems themselves or at central servers, then using them as convenient portals for their own nefarious payloads. Weaknesses in autonomous updating environments (and we know from experience that there almost certainly will be weaknesses) could provide yet another endless series of field days for worms, viruses, and other software nightmares.

Users (and/or system administrators, as appropriate) have the need and right to fully control their own computers. No particular class of users should be subjected to defaults considered too risky for another group, nor should we need to risk having our operational systems sidelined by possibly unstable vendor patches that may do more damage than the original bugs. A plethora of patches will never be a substitute for true quality software.

Lauren Weinstein (lauren@pfir.org) is co-founder of People For Internet Responsibility http://www.pfir.org. He moderates the Privacy Forum http://www.vortex.com/privacy.

========================================================

Inside Risks 161, CACM 46, 11, November 2003

Security by Insecurity

The belief that code secrecy can make a system more secure is commonly known as security by obscurity. Certainly, vendors have the right to use Trade Secret protection for their products in order to extend ownership beyond the terms afforded under Copyright and Patent law. But some software systems must satisfy critical requirements under intensive challenges, and thus must be trustworthy. The following scenarios illustrate the limitations of the myth of security by obscurity.

The Ostrich. Metaphorically, many people think (falsely) that ostriches put their heads in the sand in the belief that they are invisible. Some designers think that by restricting access to their system code, exploitable vulnerabilities will not be exposed. The fallacy in this line of reasoning was evident in Matt Blaze's 1994 discovery of a flaw in the Escrowed Encryption Standard (Clipper) that could be used to circumvent law-enforcement monitoring [http://www.risks.org, risks-16.11 and 16.12, June 1994]. Surprisingly, Blaze's method allowed for even easier access to secure communication through the proliferation of Clipper products than was heretofore possible, without access to any keys, backdoors, or weaknesses in the encryption algorithm. (Hiding cryptographic keys is of course necessarily a form of security by obscurity.)

The Emperor Has No Clothes. A fabled trusted entourage agrees with a foolish assertion because each observer fears retribution. It takes a child with no reason to kowtow to authorities to reveal the truth. Software is not like Coca-Cola(R), where for decades a handful of key employees have been trusted with a secret recipe and production process. Many computer systems are constructed in environments where a host of developers, debuggers, integrators, evaluators, and end users (with shared or possibly conflicting interests) require access to the proprietary product. Each of these individuals or agencies (collectively and individually) may hold the ``keys to the kingdom'', not only because they have knowledge of some or all of the secret code, but because they may be aware of limitations and constrained from revealing or sharing this information. Also, organizational culture may discourage whistle-blowing, even when dire consequences are possible. This happened in both Space Shuttle disasters, where the O-ring and debris damage problems were known within the NASA community before the fateful missions.

I've Got a Secret. As Benjamin Franklin observed, ``three may keep a secret if two of them are dead.'' The ease with which digital files can be transmitted (willingly or inadvertently) compounds the software secrecy problem. Earlier this year, a number of program files relating to a proprietary voting system were discovered on a subcontractor's unsecured FTP site. Diebold (the vendor) argued that the software subsequently reviewed at Johns Hopkins [Kohno, Stubblefield, Rubin, and Wallach, Analysis of an Electronic Voting System, July 23, 2003, http://www.avirubin.com/vote.pdf] ``represents a very small percentage of the entire code needed to conduct an election'' [Diebold Election Systems exposes flaws found in recent voting system report, July 29, 2003, http://www.diebold.com/election.htm]. Of course, this does not excuse the lax security that allowed the code to be downloaded from the Internet in the first place.

The Shell Game. Here, a trickster uses slight-of-hand to keep an object from view. In the above voting system example, the Johns Hopkins analysis team found numerous security flaws in the code, one of which involved the use of a vulnerable DES encryption protocol, along with a hardcoded key in the source file [http://www.avirubin.com/vote.pdf]. Diebold defended its system in a rebuttal report, claiming that the examined software ``was an older version'' that had ``passed rigorous functional tests and reviews'' [Checks and balances in elections equipment and procedures prevent alleged fraud scenarios, July 30, 2003, http://www.diebold.com/checksandbalances.pdf]. However, many election equipment tests are performed in secret, thus making it impossible to ascertain the level of rigor applied. One such examiner, Douglas Jones, had served on Iowa's Board of Examiners and, based on his analysis of a federal test report, had asserted to Global (Diebold's predecessor) in 1997 and the House Science Committee in 2001 that inappropriate key management was being used with this firm's products [Douglas W. Jones, The Case Against the Diebold AccuVote TS, July 28, 2003, http://www.cs.uiowa.edu/~jones/voting/dieboldacm.html]. It will be difficult to know whether such problems have been adequately resolved, because Diebold plans to continue its closed-source practices.

As noted here in October 2003, open source by itself does not provide a solution to these problems. Risk assessment, examination, and testing appropriate to deployment settings are fundamental to security assurance -- which should not be hampered by vendors' refusals to disclose critical components where a need to know can be demonstrated. Furthermore, customers should not cling to the false hopes of security by obscurity.

Rebecca Mercuri (mercuri@acm.org), a Research Fellow at Harvard University's Kennedy School of Government, authors CACM's quarterly Security Watch column. See her Web site at http://www.notablesoftware.com. Peter Neumann moderates the ACM Risks Forum. See his Web site at his Web site at http://www.CSL.sri.com/neumann .

========================================================

Inside Risks 160, CACM 46, 10, October 2003

Information System Security Redux

In September 2003 we discussed risks in trusting entities that might not actually be trustworthy. And yet, people use flawed systems that may cause more security and reliability problems than they solve.

There are various reasons why untrustworthy mass-market software might be used so extensively, even if the source code is proprietary and the vendor can arbitrarily download questionable software changes without user intervention. Sometimes this is a path of least resistance, with few perceived alternatives. Or it has the appearance of saving money in the short term. In some cases it is mandated organizationally -- ostensibly to simplify procurement, administration, and maintenance, or because of a desire to remain within the monolithic mainstream. Often security, reliability, and the risks of networking are considered less important, or else There is a misplaced trust that the free market will provide a cure. But the simplest answer may be ``because it's there.'' However, irrespective of any reasons why people might want to use flawed software, in certain cases it might be wiser not to use it -- especially where the risks are considerable.

In my fourth testimony (August 2001) in five years for committees of the U.S. House of Representatives, I made the following statement -- amplifying similar statements made in previous years:

``Although there have been advances in the research community on information security, trustworthiness, and dependability, the overall situation in practice appears to continually be getting worse, relative to the increasing threats and risks -- for a variety of reasons. The information infrastructure is still fundamentally riddled with security vulnerabilities, affecting end-user systems, routers, servers, and communications; new software is typically flawed, and many old flaws still persist [as RISKS readers observe repeatedly]; worse yet, patches for residual flaws often introduce new vulnerabilities. There is much greater dependence on the Internet, for Governmental use as well as private and corporate use. Many more systems are being attached to the Internet all over the world, with ever increasing numbers of users -- some of whom have decidedly ulterior motives. Because so many systems are so easily interconnectable, the opportunities for exploiting vulnerabilities and the ubiquity of the sources of threats are also increased. Furthermore, even supposedly stand-alone systems are often vulnerable. Consequently, the risks are increasing faster than the amelioration of those risks.''

The situation seems still worse in 2003, especially in mass-market software. The continuing flurry of viruses, worms, and system crashes raises the level of inconvenience to users and institutions. The incessant flow of identified vulnerability reports and the further existence of flaws that are not publicly known suggest serious problems. The continual needs for installing thousands of patches in mass-market software (and the iterative problems they sometimes cause) suggest that we are not converging. Putting the blame on inadequate system administration seems fatuous. The August 2003 exploitations of Microsoft problems (the Blaster worm and the SoBig virus) are further examples of endemic problems in vulnerable systems that can be exploited. Unfortunately, too many people seem to be oblivious to the underlying security problems.

Suggestions that we need to raise the bar may be countered with the argument that past attacks have not really been serious, and we have never had any pervasive disasters of information system security, so why should we worry? However, it is precisely because past events have been relatively benign (compared with what they could done) that there should be greater concern. Furthermore, a general overemphasis on short-term costs allows long-term concerns to be ignored.

The Free Software / Open Source movements have been touted as possible alternatives to the inflexibilities of closed-source proprietary code. Indeed, GNU-Linux/BSD Unix variants are gaining considerable credibility, and are seemingly less susceptible to malware attacks. However, by itself, availability of source code is not a panacea, and sound software engineering is still essential. Even if an entire system has been subjected to extremely rigorous open evaluation and stringent operational controls, that may not be enough to ensure adequate behavior.

Many users have grown accustomed to flaky software, perhaps because they do not have to meet critical requirements (as in nuclear power control, power distribution, and flight and air-traffic control) and suffer no liability for disasters. Perhaps it is time to follow the adage of ``Just Say No'' to bad software that is seriously unsecurable, and to demand that software development be dramatically improved.

Neumann moderates the ACM Risks Forum (www.risks.org). See www.CSL.sri.com/neumann) for extensive background for this topic -- including Congressional testimonies.

========================================================

Inside Risks 159, CACM 46, 9, September 2003

Risks in Trusting Untrustworthiness

The Internet provides ample opportunity for proving the age-old truism, ``There's a sucker born every minute.'' Carnival-style swindles and other confidence games once limited to in-person encounters are now proliferating electronically, world-wide, at low cost and effort. A blatantly obvious example is the so-called Nigerian-style scam that requests use of one's bank account to move money; hoping for a proffered generous commission, the the suckers are then separated from their assets. It is astounding that people still fall for such obvious frauds.

There are countless other kinds of scams, stings, and misrepresentations. Spam e-mail offering bogus goods and services opens up new avenues for fraud and identity theft. On-line activities are emerging with glaring opportunities for swindles, manipulations, and assorted malfeasance, such as on-line auctions (with various irregularities include nondelivery and secondary criminality), Internet gambling (especially off-shore), and fraudulent Web sites (for example, with deceptive URLs creating the appearance of legitimacy). We have previously noted here that electronic voting systems present a significant risk --- especially for use over the Internet. With independent accountability seriously lacking today, e-voting can be likened to using an off-shore gambling site not subject to any regulation. Any of these and other situations could result in inordinate risks, such as financial ruin, blackmail, compromised democracy, or even loss of life. But it is perhaps less astounding that people fall for such schemes, particularly when the technology superficially appears genuine.

We tend to trust certain third-party relationships, with banks, telephone companies, airlines, and other service providers whose employees have in some way earned our trust, collectively or individually. But what about untrustworthy third parties? Some computer-based applications rely critically on the putative integrity and noncompromisibility of automated trusted third parties, with little if any easily demonstrated human accountability. Examples include digital-certificate authorities, cryptographic servers, surveillance facilities, sensitive databases for law enforcement, credit-information bureaus, and the like. With increasingly appealing short-term cost incentives for pervasive use of outsourcing, the need for trustworthiness of third-party institutions becomes even more important. However, security, privacy, and accountability often seem to be ignored in efforts to save money.

Is placing trust in off-shore enterprises inherently riskier than using domestic services? Not necessarily. Corruption and inattention to detail are world-wide problems. The deciding factor here is perhaps the extent to which comprehensive oversight can be maintained.

Is domestic legislation enough? Of course not. Any legislation should not be overly simplistic; for example, it should avoid seeking solely technological fixes or purely legislative solutions to deeper problems. Besides, serious complexities arise from the fact that such problems are international in scope and demand international cooperation.

Is there a role for liability (for flagrant behavior) and differential insurance rates --- for example, based on how well a purveyor is living up to what is expected of it? Yes. Such measures have significant potential, although they will be strongly resisted in many quarters.

So, how can we provide some meaningful assurance that critical entities (direct parties, third parties, or otherwise) are sufficiently trustworthy? Ideally, institutions providing, controlling, managing, and monitoring potentially riskful operations should be decoupled from other operations, eschewing conflicts of interest, and subjected to rigorous independent oversight. Enronitis and collusion must be avoided, even if it means that the costs are greater. Furthermore, the people involved need altruism, sufficient foresight to anticipate the risks, and a commitment to effectively combat those risks. At the very least, their backgrounds should be free of criminal convictions and other activities that would cast serious suspicions on their trustworthiness. In addition, we need legislators able to see beyond the simplistic and palliative, to approaches that address the real problems. Above all, we desperately need a populace that is more aware of the risks and the needs outlined above.

This column should not be news to most of you. Overall, there are many risks that must be addressed. The old Latin expression ``Caveat emptor'' (Let the buyer beware) seems quite timely today. Ultimately, it all comes down to ``Sed quis custodiet ipsos custodes?'' (But who is watching the watchers?)

Peter Neumann moderates the ACM Risks Forum (www.risks.org).

========================================================

Inside Risks 158, CACM 46, 8, August 2003

Spam Wars

In the June 1997 edition of this column, Peter G. Neumann and I asked if you were being ``flooded'' with spam. At the time, spam was already annoying, but as it turns out the real torrent hadn't even begun! Fast-forward to 2003, and spam now threatens the Internet's stability and reliability -- not only of e-mail systems, but potentially of the network infrastructure itself. Spam is quite probably a greater actual threat to the stability of the Internet today than the theoretical risk of Net-based terrorism.

Estimates are that typical Internet users' inbound e-mail may now be about 50% spam. Highly visible e-mail addresses are pounded even harder. A couple of years ago spam likely accounted for something less than 10% of overall e-mail received. The trend-line is alarming to say the least.

We will drown in spam unless solutions can be found. Organizations ranging from the Federal Trade Commission (which belatedly wants more anti-spam powers) to the American Marketing Association (who worry that their members' e-mail marketing messages are ``misconstrued'' as spam), as well as other public and private organizations, tend to propose generally simplistic spam-cure patent medicines.

Yet, most of the hodgepodge of ``quick fix'' spam control ideas seem unlikely to significantly stem spam's flow, and in many cases may do more harm than good.

Legislation to explicitly outlaw spamming is certainly necessary, but tends to be of limited usefulness except against big and obvious spammers, an issue further complicated by spam's global and easily obfuscated reach. Poorly written anti-spam laws might actually have the perverse effect of legitimizing gigantic amounts of ``unsolicited bulk e-mail'' -- that is, spam!

The crooks using spam to hawk fake bodily enhancement products, illegal cable TV boxes, and Nigerian free-money frauds (to name but a few) are unlikely to be concerned about legal strictures against spam.

Common spam filtering programs are usually of the ``dirt under the rug'' variety. They categorize and/or delete spam messages after they've been received by systems, but by then much of the bandwidth and processing costs of the spam have already been wasted. The false-positive rate with such programs is also a major problem -- important nonspam e-mail is frequently identified as spam and relegated unseen to the bit-bucket.

Other ad hoc technical measures against spam can have negative consequences of their own. ISPs increasingly engage in severe restrictions (such as preventing subscribers from running their own secure mail servers) that hobble users, don't significantly dent the overall spam flow, and can also inflict collateral damage to innocent sites.

Technical anti-spam fads such as ``challenge-response'' threaten to tangle users' e-mail and legitimate Internet mailing lists into knots, while actually increasing the flow of spam-related traffic due to bounced and misdirected spam challenges.

Today's Internet e-mail systems (basically defined more than two decades ago) are inadequate to deal with the e-mail environment we now face, in terms of spam and other critical problems such as security, reliability, and authentication. It's time for fundamental change, rather than Band-Aid, piecemeal-reactive approaches.

There are various possible evolutionary routes towards practical, sustainable, and significantly fundamental structural changes to Internet e-mail that could be implemented in phased approaches to avoid unreasonable disruption of existing e-mail systems during a transition period.

Peter and I have proposed one such path for discussion, which we've dubbed ``Tripoli'' (for Triple-E, Enhanced E-Mail Environment). Tripoli focuses on vesting e-mail control decisions with users (especially the recipients of e-mail), rather than ISPs or governments.

Tripoli postulates a token-based authentication system to provide for flexible spam controls, along with intrinsic encryption and other security functions, while still providing the option of permission-based ``anonymous'' e-mail communications. Importantly, we believe that the Tripoli framework deals appropriately with the free-speech and other concerns that we expressed in our earlier column regarding anti-spam policy issues. We hope Tripoli will provide a useful and continuing contribution to the ongoing debates over e-mail futures. (Please see http://www.pfir.org/tripoli-announce for details.)

Unless we get started now on the onerous but necessary task of fundamentally redesigning Internet e-mail in a sustainable manner, we will find that our electronic mail nightmare has just begun.

Lauren Weinstein (lauren@pfir.org) is co-founder of People For Internet Responsibility (www.pfir.org). He is moderator of the Privacy Forum (http://www.vortex.com).

========================================================

Inside Risks 157, CACM 46, 7, July 2003

How Secure Is Secure Web Browsing?

Security is of particular importance when sensitive information is sent through the WWW. Web users must rely on the security of the browser's Secure Socket Layer (SSL) protocol. Although the closed-padlock icon in a browser window depicts a secure connection, it does not imply a totally risk-free secure connection. Whenever the padlock is snapped or a security related message pops up, you should be alerted and scrutinize the security of that connection.

During the handshake of a secure connection, the server sends a public-key certificate to identify itself. You assume you have a secure connection to the entity identified in the certificate, but that entity may not be who you think it is. So, what is the critical issue in verifying a server certificate? It is in the root CA's (certification authority) self-signed certificate that the verification starts. We trust root CAs (assuming that they don't issue certificates to copycat server) because our browser developer does. An initial list of root CA certificates comes with browsers. Depending on their trust in browser developers, users may assume that all root CAs that come with browsers are robust. However, authenticity is an important concern for other root CA certificates installed after the browser. An attacker can introduce bogus certificates for installation automated via a VB script. The client sees only a final approval screen that may easily be ignored by pressing the ``yes'' button.

Let's consider the following possible scenario. Suppose you've connected to your bank, www.xyzbank.com. Using network spoofing techniques, an attacker reroutes this traffic to its counterfeit site, and imitates a well-known root CA as the issuer for a fake certificate created for xyzbank. The attacker creates a second imitation certificate: a self-signed root CA certificate for the same well-known root CA. When these imitation certificates are used for a secure connection, you, as a client, will see a warning saying that the root CA is not to be trusted. Taking a closer look at the certificate details is of no help, even harmful, because your favorite root CA seems to be the issuer. You might easily prefer to continue and maybe install the imitation certificate assuming that there is a bug in your system. Because the victim will see the well-known root CA's name on the final approval screen, he/she would probably buy into the con.

The only authentication guarantee provided by a closed padlock is that the URL in the certificate is the same as the one in the address bar of your browser. A closed padlock does not indicate the server's commercial identity; browsers tell nothing about the certificate it just used to snap the padlock. You have to examine the certificate details to ascertain commercial identity. For example, when you connect to www.delta.com, you can't be certain you're connected to Delta Airlines just by the closed padlock; you have to scan the certificate details by clicking on the padlock. If www.delta.com was, say, Delta Foods, again, you would have seen a closed padlock even if the Web page looks like the airline's.

Certificate examination highlights the dilemma of server identification: the certificate contains the formal name and URL, but the average user needs to see something easily recognizable from previous experience such as the brand name, logo, or current slogan.

Furthermore, to take advantage of URL control, you always have to be aware of the URL you're browsing by checking the address bar. Some secure applications pop up browser windows with the address bars and toolbars removed, so as to restrict the customers to just the buttons provided. In other cases, address bars exist, but due to the copious information in the address bar, the address bar is scrolled left and the URL part is not visible without scrolling right.

A closed/open padlock indicates whether the just completed transfer was secured or not; it doesn't give any security information about the next connection, which might be password transfer by clicking on the ``sign-in'' button. Therefore, whether you enter your password in a secured or an unsecured Web page, that password may go unencrypted. In either case, you should examine the source code of the current Web page to see if the next connection is secured or not.

Secure Web browsing needs a careful and questioning user. Checking the certificate details and controlling the root certificate store definitely helps. Root certificate installations should be avoided. Also, keep your eyes on the address bar. In general, don't bury your head in the sand just by trusting a padlock icon.

Albert Levi (levi@sabanciuniv.edu) is an assistant professor of computer science and engineering at Sabanci University, Istanbul, Turkey.

========================================================

Inside Risks 156, CACM 46, 6, June 2003

Reflections on Trusting Trust Revisited

Security is often described as a weak-link phenomenon. Ken Thompson in his 1983 Turing Award Lecture [1] described how a compiler could be modified to plant a Trojan horse into the system's login authentication program so that it would accept a known password. In addition, the C compiler could be altered to propagate this change when it was recompiled from its (unmodified) source code. The system Thompson described was seriously compromised and could never be trusted: even a recompilation from clean source code would yield a Trojaned compiler and login program.

Twenty years later we find efforts such as the Trusted Computing Group (the retooled Trusted Computing Platform Alliance, a 190-company industry work group), Intel's LaGrande, and Microsoft's NGSCB (Next Generation Secure Computing Base, previously known as Palladium) aiming to create secure systems from the ground up [2]. ``Trusted Computing'' platforms include specialized hardware or a processor that can monitor a system's boot process ensuring that the computer is based on appropriately certified hardware and software. After verifying the machine's hardware state and firmware, the platform can check that the operating system is certified, then load it and transfer control to it. The operating system (for example Microsoft's NGSCB) can similarly verify that only secure untampered applications are loaded and executed---no more doctored C compilers or unauthorized descramblers. Thus, a TC platform can be used to rigorously enforce third-party-mandated security policies such as those needed for digital rights management (DRM) and mandatory access control [3].

Given our nearly unbroken track record of failed security technologies, we should view claims regarding a system's trustworthiness with skepticism. Recently a group managed to run Linux on a Microsoft XBox---without any hardware modifications [4]. The XBox, in common with many other game consoles, mobile phones, and even printer cartridges, can be thought as an instance of a special purpose TC platform. Although based on commodity hardware, the XBox is designed in a way that allows only certified applications (games) to run, thus protecting the licensing revenue stream of its vendor. Earlier attempts to run unauthorized software (such as the Linux kernel) on it required hardware modifications, a prospect that will not be realistic once TC features are part of the CPU (as might be the case with Intel's LaGrande design). The recent attack modifies the saved data of a particular game in a way that renders the trusted game into an untrusted agent that can then be used to boot Linux.

The two attacks, set apart by twenty years, share an interesting parallel structure. Thompson showed us that one can not trust an application's security policy by examining its source code if the platform's compiler (and presumably also its execution environment) were not trusted. The recent XBox attack demonstrated that one can not trust a platform's security policy if the applications running on it can not be trusted. The moral of the XBox attack is that implementing on a TC platform a robust DRM, or mandatory access control, or even a more sinister security policy involving outright censorship will not be easy. It is not enough to certify the hardware and have a secure operating system; even a single carelessly written but certified application can be enough to undermine a system's security policy. As an example, a media player could be tricked into saving encrypted content in an unprotected format by exploiting a buffer overflow in its (unrelated) GUI customization (so-called skin) code. Capability machines built in the 1970s used strong typing and a finer granularity object classification and access control schema that would prevent such an attack. However, as Needham and Wilkes concluded from their work on the CAP system, the operating system was too complex and therefore hard to trust and maintain [5]. Those of us who distrust the centralized control over our data and programs that TC platforms and operating systems may enforce can rest assured that the war for total control of computing devices can not be won.

1. Ken L. Thompson. Reflections on trusting trust. Ken L. Thompson. Reflections on trusting trust. Communications of the ACM, 27(8):761-763, August 1984, 27(8):761-763, August 1984.

2. Steven J. Vaughan-Nichols. How trustworthy is trusted computing? Computer 36(3):18-20, March 2003.

3. Ross Anderson. Cryptography and Competition Policy---Issues with `Trusted Computing'. Workshop on Economics and Information Security, May 29-30, University of Maryland. Online http://www.cl.cam.ac.uk/ftp/users/rja14/tcpa.pdf. Current April 2003.

4. Rob Malda Linux Running on Xbox Without Modchip! Online http://slashdot.org/article.pl?sid=03/03/30/1337234. Current April 2003.

5. Maurice V. Wilkes and Roger M. Needham. The Cambridge CAP Computer and its Operating System. Elsevier, London 1978.

Diomidis Spinellis is an Assistant Professor in the Department of Management Science and Technology at the Athens University of Economics and Business and author of the book Code Reading (Addison-Wesley, 2003).

========================================================

Inside Risks 155, CACM 46, 5, May 2003

E-Epistemology and Misinformation

The problems of on-line misinformation seem to be worsening, because of the growth of the Internet and our ever increasing dependence on on-line systems. Information technology is a double-edged sword --- perhaps even moreso than many other technologies. In the hands of enlightened individuals, institutions, and governments, its use can be enormously beneficial. In other hands, it can be detrimental. Unfortunately, the dichotomy is often in the eye of the beholder, perhaps depending on one's objectives (e.g., personal financial gains, corporate profits, global economic well-being, privacy, environmental concerns, etc.).

Given a collection of on-line information, many people behave as if it is inherently authentic and accurate. This myth applies not only to Web sites, but also to many types of special-purpose databases, such as those found in law enforcement, motor vehicle departments, medicine, insurance, social security, credit information, and homeland security. We have seen many cases in which misinformation (e.g., false flight data, erroneous medical records, undeleted acquittals, or tampered files) has resulted in very serious consequences.

Although an individual can occasionally observe that personal information about one's self is incorrect, more typically such erroneous information is hidden from the individual in question, possibly in multiple but different inaccurate versions. Overall, it is usually impossible for one to ensure that all such instances are correct. Furthermore, it is much more difficult to determine whether or not on-line information about anything else is authoritative. Worse yet, the volume of questionable information is growing at an extraordinary rate, and attempts to update substantive misinformation often have little effect -- especially with the persistence of incorrect cached versions.

We rely increasingly on the Internet for many purposes, including education and enlightenment, irrespective of whether the sources are accurate. Oft-repeated overly simplistic sound-bite mantras seem to be popular. Furthermore, some people seem eager to waste time and energy that could be better spent elsewhere -- or to drop out. There is a tendency for entrenched positions to remain fixed. Are we losing our ability to listen openly to other views and engage in constructive thought? Another problem involves the inaccessibility of vital information. We seem to have evolved into a mentality of ``If it is not on the Internet, it does not exist.'' Even though there are many more data bytes available today than ever before, search engines typically find fewer than 5% of the Web pages, almost none of the database-driven dynamic Web pages, and very little of what is in most public libraries. Copyright restrictions and proprietary claims further limit what is available. For example, the ACM digital library is accessible only to those ACM members who pay to subscribe. Furthermore, overzealous filtering blocks many authoritative sources of information. Are our educations and information gathering suffering from a lowest-common-denominator process?

The propagation of misinformation has long been a problem in conventional print and broadcast media, but represents another problem that is exacerbated by the speed and bandwidth of the Internet. In general, widely held beliefs in supposedly valid information tend to take on lives of their own as urban myths; they tend to be trusted far beyond what is reasonable, even in the presence of well-based demonstrations of their invalidity.

In the face of such rampant misinformation, the truth can be difficult to accept, partly because it can be so difficult to ascertain, partly because it can seem so starkly inconsistent with popular misinformation, and partly because people want to believe in simple answers. Thus, we are revisiting classical problems that might now be considered as E-Epistemology, involving the nature and fundamentals of on-line knowledge -- especially with reference to its limits and validity. However, there are some possible remedies, such as epistemic educational processes that teach us how to evaluate information objectively. For Web sites, this might entail examining who are the sponsors, what affiliations are implied, where the information comes from, whether multiple seemingly reinforcing items all stem from the same incorrect source, whether purported Web site security and privacy claims are actually justified, and so on.

Peter Neumann moderates the ACM Risks Forum (risks.org). His Web site contains an archival index to many relevant cases http://www.csl.sri.com/neumann/illustrative.html. Many thanks to David Parnas and Rob Kling for their constructive feedback on this column.

========================================================

Inside Risks 154, CACM 46, 4, April 2003

On Sapphire and Type-Safe Languages

Beginning Saturday 25 January 2003 around 12:30am EST, a distributed denial-of-service attack spread rapidly throughout the global Internet. Within 10 minutes, most of the vulnerable hosts on the Internet were infected. By morning, Bank of America customers could not withdraw money from 13,000 ATMs. Continental Airline's website was off-line, forcing manual check-in. Normally heavy Internet trading on the South Korean stock market vanished. Many other websites and Internet services were also rendered inaccessible by the Sapphire (or Slammer) worm responsible for the attack.

Sapphire is a 376-byte worm that infects Microsoft SQL Server 2000 hosts via the SQL Resolution Service running on UDP port 1434. It attempts to spread itself rapidly to other hosts. The worm does no damage to the infected machine: it does not modify disk files, alter or inspect database contents, or interrupt database execution. It merely probes for other SQL Servers to infect, by generating random IP addresses and sending UDP packets to port 1434 on those addresses. Since most of these IP addresses are not local, the resulting flood of packets saturates the host's connection to the Internet.

That such a small worm could so effectively disrupt so many servers so rapidly was somewhat surprising. The CodeRed, ILoveYou, and Nimda worms were all much bigger than the single Sapphire packet and took much longer to propagate. That such a simple attack was possible was no surprise at all. Sapphire exploits a buffer overflow vulnerability in SQL Server 2000 for which CERT issued a security advisory and Microsoft issued a patch six months earlier. When the Resolution Server receives a packet of type 04, it uses data from the packet to build a registry key in a fixed-size buffer on the stack. The unpatched code performs no length checks on the registry key it constructs. If the received packet is long enough, the constructed registry key overflows the stack-allocated buffer and overwrites the current function's return address, which follows the buffer in memory. The Sapphire worm consists of a single overly-long packet that causes this return address to be overwritten with the address in the buffer where the worm's code resides. The worm code begins executing when the function returns.

Buffer overflow bugs of this nature are a common source of security vulnerabilities in programs written in languages like C and C++. They also arise frequently in ordinary program development in such languages, where they cause memory corruption that leads to erratic program behavior, application crashes, and machine crashes. They can be difficult to debug because the resulting program behavior usually cannot be explained abstractly in terms of functions, variables, expressions, and statements, but must be understood at the machine level in terms of addresses and bytes. In turning a buffer overflow bug into a security breach, an adversary exploits this abstraction gap to corrupt memory in a carefully calculated way.

Some modern programming languages (e.g., Ada, C#, Common Lisp, Eiffel, Java, Modula 3, Scheme, and Standard ML) prevent this failure of abstraction. Such a language is said to be ``type-safe'' because its implementation ensures, via some combination of compile- and run-time checks, that the value a variable takes on or a function is passed always matches the language's notion of the variable's or function's type. When a type violation arising from a programming error is about to occur, such as an access to the 257th element of a 256-element buffer, the language implementation either terminates the program or raises a language-level exception.

Type safety makes program development and debugging easier by making program behavior more understandable. More importantly in today's networked world, type safety prevents an adversary from turning a type violation into a security breach. While an adversary might be able to provide inputs that trigger a run-time check, memory corruption cannot occur. There is no way the adversary can cause a buffer to overflow and be reinterpreted as a sequence of machine instructions. (For type safety to prevent security breaches, the capability that some compilers provide to turn off run-time checks must not be used.)

Type safety is not a panacea for security. Other kinds of bugs besides type violations can lead to security problems. Even the termination of an application due to a type violation can result in denial of service. But type-safe languages make it much more difficult for an adversary to turn a type violation into a more serious security breach. Type-safe languages provide an important line of defense in developing applications safe for today's networked world.

Andrew Wright (akwright@acm.org) is research staff at a large networking company.

========================================================

Inside Risks 153, CACM 46, 3, Mar 2003

Risks of Total Surveillance

The U.S. Public Policy committee of ACM (USACM) is concerned that the proposed Total Information Awareness (TIA) Program, sponsored by the Defense Advanced Research Projects Agency, will fail to achieve its stated goal of ``countering terrorism through prevention''. Further, we believe that the vast amount of information and misinformation collected by any system resulting from this program is likely to be misused to the detriment of many innocent American citizens.

Because of serious security, privacy, and personal risks associated with the development of any vast database surveillance system, we recommend a rigorous, independent review of TIA. Such a review should include an examination of the technical feasibility and practical reality of the entire program.

Security Risks. The state of the art in computer system design is such that that any systems resulting from TIA are unlikely to be able to preserve integrity and keep data out of unauthorized hands, whether they are operated by governmental or commercial organizations. Frequent reports of successful hacker break-ins and insider misuse of supposedly secure systems and the pervasive existence of software flaws constitute evidence that we are unable to make these systems adequately secure, and suggest that the likelihood of a trustworthy database system emerging from this effort is vanishingly small.

The databases proposed by TIA would also increase the risk of identity theft by providing a wealth of personal information to anyone accessing the databases, including terrorists masquerading as others. Recent compromises involving about 500,000 military-relevant medical files and 30,000 credit histories are harbingers of what may be in store.

Privacy Risks. The need for oversight and control is especially great when aggregation and analysis of personal information is done without the knowledge or consent of the people being monitored. It is misleading to suggest that ``privacy enhancing technologies'' within TIA can somehow protect people's privacy, because by definition surveillance compromises privacy. Furthermore, the secrecy inherent in TIA implies that citizens could not verify that information about them is accurate and shielded from misuse. Worse yet would be the resulting lack of protection against harassment or blackmail by individuals who have inappropriately obtained access to an individual's information, or by government agencies that misuse their authority.

Personal Risks. TIA would combine automated data-mining with statistical analysis, thereby resulting in some number of false positives -- risking incorrectly naming someone as a potential terrorist. As the entire population would be subjected to TIA surveillance, even a very small percentage of false positives would result in a large number of law-abiding Americans being mistakenly labeled.

The existence of TIA would impact the behavior of real terrorists and law-abiding individuals. Real terrorists are likely to go to great lengths to make certain that their behavior is statistically normal; ordinary people are likely to avoid perfectly lawful behavior out of fear of being labeled UnAmerican.

To summarize, we appreciate that the stated goal of TIA is to fund research on new technologies and algorithms that could be used in a surveillance system in the service of eliminating terrorist acts. However, we are extremely concerned that the program has been initiated (and some projects already funded) apparently without independent oversight and without sufficient thought being given to real constraints -- technical, legal, economic, and ethical -- on project scope, development, field testing, deployment, and use. Consequently, the deployment of TIA, as currently conceived, would create new risks while providing only the appearance of increasing homeland security.

There are important steps that the government can take now to increase our security without creating a massive surveillance program that has the likelihood of doing more harm than good. Federal, state, and local governments already have information systems in place that could play major roles with highly focused terrorist spotting. However, many of these information systems are only partly functional and/or being ineffectively used. An example is the computer system run by the Federal Bureau of Alcohol, Tobacco and Firearms which, according to The New York Times, was unable to link bullets fired in three sniper shootings in Maryland and Georgia in September 2002. Serious improvements in the use of current operational systems could significantly enhance homeland security without creating the major risks noted here.

Barbara Simons and Eugene H. Spafford are Co-Chairs of USACM. This article is drawn from a USACM letter to Congress: www.acm.org/usacm/Letters/tia_final.html. The ACM Public Policy Office can be reached at 1-202-478-6312.

========================================================

Inside Risks 152. CACM 46, 2, Feb 2003

Gambling on System Accountability

Because of rampant security vulnerabilities, ever-present risks of misuse by insiders, and possibilities for penetrations by outsiders, there are many needs for comprehensive computer system accountability -- that is, the ability to know definitively what is transpiring, particularly during and after accidents and intentional misuse. Unfortunately, security typically focuses overly on confidentiality, with integrity, availability, strong authentication, authorization, correctness, and accountability dragging way behind. In this column, we consider the potential importance of the design, implementation, and operation of policies and mechanisms for accountability that themselves resist being compromised -- especially by knowledgeable trusted insiders. We illustrate this by considering the situation surrounding the recent Pick-Six betting scam involving the Breeders' Cup horse race.

A $3-million Pick-Six payoff over six races ending with the high-stakes Breeders' Cup race seemed rather suspicious, because the Pick-Six winner also picked many consolation bets of five winners. Subsequent investigation showed that an unusual combination bet had been placed by telephone from an off-track betting (OTB) site in Catskill, New York. The results of the first four races had been chosen exactly (including two long-shots of 13-to-1 and 26-to-1), and the bets on the remaining two races covered every possible combination.

Autotote's software is used by most U.S. off-track horse-race betting sites. Because the Autotote OTB system transmits bets to the central system only after the completion of the fourth race in the Pick-Six cycle, it was concluded that the ``winner'' had placed a combination bet of w,x,y,z,*,* (for an arbitrary choice of horses w, x, y, and z, with a wild-card [*] of multiple bets over all possible horses in the last two races); then, after the results of the first four races were known (let's define them as A, B, C, D), but before the data transfer occurred, someone with access to the OTB system changed w,x,y,z to A,B,C,D. This resulted not only in the Pick-Six winner, but also in multiple consolation winners.

Accountability? Unfortunately, there is no bet-specific audit trail on telephoned OTB bets, although a spokesman for Autotote had insisted that it was ``absolutely impossible'' to hack into the system! So, you might ask, had anything like this happened previously? Indeed, there had been a previous Pick-Six payoff from the same OTB site (in Catskill, NY), and a similar earlier case in a Pick-Four. Furthermore, all of the participants in these instances were fraternity buddies from their days at Drexel University, and one of them was already under suspicion. It was also determined that they had forged tickets and collected on yet-unclaimed winning bets. The ``someone'' noted above was a former Autotote employee, who has pleaded guilty to one count of conspiracy to commit wire and computer fraud and one count of money laundering. The other two participants have also admitted their guilt.

In betting systems and financial systems generally, an inherent need exists for rigorous accountability. Many other applications previously discussed in this space also illustrate the criticality of integrity and accountability. For example, fully electronic voting systems are an example of ``self-auditing'' products that, due to their anonymity requirements, require vigilant oversight and independent accountability rather than the almost total lack of assurance that they provide today. (``Trust us,'' the vendors say.) Also, mounting privacy concerns (including the proposed Total Information Awareness effort) are another huge problem area. Unfortunately, although access controls and database accountability might help sometimes to identify the perpetrators of violations, many privacy invasions involve untraceable human actions outside of computer systems.

Several lessons are evident. In many critical applications, risks of misuse by people with insider knowledge are widely ignored; so are the risks of outsiders who can easily become insiders, because of the lack of adequate internal security. System designs that seriously ignore accountability are particularly at risk, because of the difficulties of detecting and tracing misuse. Where they exist, audit trails must be strongly tamper resistant, or else they are themselves subject to manipulation. Physical traceability, paper trails, and truly independent, unbiased, objective, and honest security audits by experts can also be helpful. Proprietary closed-source software systems is inherently suspect without meaningful accountability. In short, noncompromisible accountability can often be invaluable -- although it presents serious opportunities for invasions of privacy that must also be addressed.

See Neumann's Web site for background (neumann@csl.sri.com). Also see Rebecca T. Mercuri's article ``On Auditing Audit Trails'' in the January 2003 issue of CACM, pp. 17--20, and her Web site at http://www.notablesoftware.com.

========================================================

Inside Risks 151, CACM 46, 1, Jan 2003

The Mindset of Dependability

Computer software is legendary for the time and cost overruns producing it, and for its fragility after it is written. The U.S. government failed trying to procure dependable software for the IRS and the FAA, and the UK government was recently accused of wasting more than a billion pounds on failed or overdue information technology contracts. Perhaps only 25% of major software projects work out well. Home computer users are also accustomed to crashes. Why are computer systems so unreliable and difficult?

By contrast, the Japanese Shinkansen trains are a remarkable testimony to reliability and safety. Since they started in 1964, carrying millions of people per year, no passenger has been killed as a result of a collision, derailment, or other railway accident. Not only are the Shinkansen safe, they are also reliable. The average Shinkansen train arrives within 24 seconds of schedule. What can we learn from this?

At one level, there are details of railway construction. The Shinkansen track is laid with heavier rail and closer-spaced cross-ties than a new line in Australia that will carry trains of twice the weight.

At another level, safety benefits from Japanese culture. Any visitor can tell you that Japan is an extremely clean country; the Shinkansen tracks and stations are litter-free. The worst fire ever on the London Underground (King's Cross, 1987) started in debris under an escalator; cleanliness is not just cosmetic.

But historically, Japan was not renowned for railway safety. As recently as the early 1960s, just before the Shinkansen opened, two accidents near Tokyo each killed more than 100 people. And yet safety has now become routine. The culture of safety and dependability has been learned there; it could be learned elsewhere.

But CACM is neither a railway engineering journal nor a journal of cultural history. What should we learn about computers?

The Japanese did not do a cost-benefit analysis on safety. Nobody sat in the Shinkansen design office and thought about how to trade off cutting construction costs against the number of people that would be killed. In the computer context, we often distribute the costs of unreliable software over a great many users, who do not easily aggregate their frustrations into economic impact. NIST recently estimated that software bugs cost the US economy $60 billion per year. Lower testing costs, more features, and shorter time to market are easier to quantify than the benefits of various elements of dependability such as safety, security, and reliability -- and may be viewed as more important by the development managers. If we care about having dependable systems, then we have to be sure that safety, security, and reliability are primary requirements whenever they are needed. These are not things that can be patched in like an extra button in an interface. Today, vendors act as if people want more features and low prices first, and dependability later.

How can we achieve a culture of dependability? When buying a ticket to a symphony orchestra, people do not anticipate some particular percentage of wrong notes. Nobody thinks that some level of spelling errors in CACM is suitable in exchange for faster editing or student discounts. Yet we routinely accept basic undependability in computer systems.

We have understood for a generation that having a small, terse, and limited system kernel greatly improves reliability. Yet we still see manufacturers resorting to special-purpose bypasses to make their particular program run faster or get around some blockage, with kernels swelling to tens of millions of lines of code. We still see complexity winning over simplicity.

How do we persuade manufacturers that security must be a priority? First, we have to believe it as users. People who routinely accept downloads from almost any site and use mailers that enable executable code attachments to send 5-word ASCII strings wouldn't seem to care much about security or privacy. We need a culture change by purchasers as well as by developers. Perhaps the increased threat of cyberterrorism will reverse the trend of even security-conscious agencies to buy commercial off-the-shelf software without recognizing its risks; I hope it does so without any actual horror stories. Perhaps the recognition that simpler and more dependable systems can result in lower system administration costs, faster and fewer reboots, and lower training costs will help change the customer culture. If we can persuade manufacturers that more dependable software will pay off, and that adding more features won't enhance dependability, we might reverse a decades-long trend to greater vulnerability and lesser reliability.

Michael Lesk is known for some Unix utilities (Lex, tbl and uucp) and for research in information retrieval. He is the author of "Practical Digital Libraries" (Morgan Kaufmann, 1997) and currently works for the Internet Archive.

========================================================

Inside Risks 150, CACM 45, 12, Dec 2002

Why Security Standards Sometimes Fail

Security experts have long been saying that secure systems, and especially security standards, need to be designed through an open process, allowing review by anyone. Unfortunately, even openly designed standards sometimes result in flawed cryptographic systems. A recent example is the IEEE 802.11 wireless LAN standard, in which several serious cryptographic failures were found (see [1,2,3]), after millions of flawed hardware devices were sold.

Finding a cryptographic design flaw in an approved standard is bad news -- especially after systems using it are in wide-spread use. Such a flaw is typically very costly to fix. And, ironically, once a flawed system is widely deployed, future fixed versions of the system will almost certainly have a backward-compatibility mode, making them vulnerable as well. Cryptanalyzing the standard before it is ratified is clearly better for society and better for vendors. But is it better for the cryptanalyst? Unfortunately, we shall see that the answer is sometimes ``No''.

Cryptanalysts are usually scientists, who make their own choices about which problems to work on. Furthermore, scientific success is measured by publications. Publishing a high-visibility scientific paper in a respected scholarly journal or conference proceedings can help establish academic fame, fortune, and tenure. So, consider a cryptanalyst, Carol, who is looking for a project to work on. Would she want to get involved in a standardization effort?

Working on a standard has its own set of challenges. A standards body involves many parties with conflicting agendas, many of them powerful corporations. Furthermore, a standard is not measured by excellence or novelty. It should be a working design that is an acceptable compromise between the interests of all the parties involved. In short, a standards body is not an environment that encourages scientific discourse. Finally, even supposedly open standards bodies sometimes have onerous requirements, which may discourage scientists from participating.

Suppose that despite the challenges, Carol does get involved, and finds a cryptographic flaw in the standard's draft. Would this advance her scientific career? Unfortunately, not by much. Firstly, it may be difficult for her to get the standards body to take action, because doing so might conflict with the interests of other parties. Secondly, Carol can expect very little credit for her contribution. A standard typically has no authors, and only the standard's editors are personally recognized. If Carol tries to publish a paper describing her discovery, it will surely be rejected by any respectable scientific venue: Every standard goes through drafts, many of them faulty; so, why should a specific flaw in an early draft be interesting? Finally, if the standard ends up not being used, then her work (indeed, the work of the whole standards body) would go to waste.

Now consider what would happen if Carol finds the same flaw after the standard has been ratified, and after technology based on it is in wide-spread use. As an individual, she has much more to gain. Her work has obvious technical impact, because, by choice, the standard is already in use. She can certainly author a paper about her findings: Publishing it in a top-notch scientific venue would be relatively easy, because of the public interest. Furthermore, security vulnerabilities are considered news-worthy outside of scientific circles: Reporting services for such discoveries (such as BugTraq and CERT) have very wide readership, and stories are occasionally even reported by the general media. Such publicity is an effective way to cause Fortune-500 corporations to fix their products. All this excitement can make Carol into a star in her field.

We see that for an individual scientist, cryptanalyzing an established standard is, potentially, much more rewarding than working to ensure that the standard is secure in the first place. Luckily for society, there are reasons why many security standards do better than IEEE 802.11. Standardization is altruistic volunteer work for many participants, and this includes cryptanalysts. Also, cryptanalysts working in corporate research labs may be well motivated to contribute to a standard. But the basic conflict between the public good and the individual scientist's interests is a cause for concern.

[1] W.A. Arbaugh, N. Shankar, and Y.C.J. Wan, Your 802.11 wireless network has no clothes, IEEE Conference on Wireless LAN's and Home Networks, 2001.

[2] N. Borisov, I. Goldberg, and D. Wagner, Intercepting mobile communications: The insecurity of 802.11, 7th ACM Conference on Mobile Computing and Networking, 2001.

[3] S. Fluhrer, I. Mantin, and A. Shamir, Weaknesses in the key scheduling algorithm of RC4, 8th Workshop on Selected Areas in Cryptography, 2001.

Avishai Wool, yash@eng.tau.ac.il, is a Senior Lecturer in the Dept. of Electrical Engineering Systems, Tel Aviv University, Ramat Aviv 69978, ISRAEL http://www.eng.tau.ac.il/~yash

========================================================

Inside Risks 149, CACM 45, 11, Nov 2002

Florida 2002: Sluggish Systems, Vanishing Votes

Following the 2000 Presidential election debacle in Florida, government officials promised sweeping reforms that would prevent such chaos from reoccurring. Indeed, the Florida election code was extensively revised, punchcard systems were outlawed, and over $125 million was spent statewide on new voting equipment and training of voters and election administrators. What could possibly go wrong? Apparently enough calamity to cause Governor Jeb Bush to declare a state of emergency, extending the voting session by two hours for the September 10, 2002 primary election. Yet events earlier in the year should have provided sufficient forewarning of difficulties.

Broward County purchased new touchscreen voting machines, manufactured by Election Systems & Software (ES&S), but back in February the Associated Press reported that "more than two-thirds of the first shipment had defects and will have to be repaired." The ES&S devices in Broward and Miami-Dade were those at polling places in September that failed to open on time, in part because workers had been told that the machines would take about two minutes to boot up. Instead, most took around 10 minutes, but those outfitted for the visually impaired took an astonishing 23 minutes. Although Broward Board of Elections Commissioner Miriam Oliphant and her pollworkers were later blamed by the Governor for many of the September primary woes, the fact remains that these sluggish voting systems were certified for use by the state's examiners as well as by testing agencies overseen by the National Association of State Election Directors.

In March 2002, problems with Sequoia voting systems purchased by Palm Beach County surfaced in two local city council elections. In the city of Wellington, a run-off election involved only one race with only two candidates. The final vote tally was 1263 to 1259, but 78 ballots were not recorded by the touchscreen machines. Elections Supervisor Theresa LePore explained that people simply chose to come to the polls and not cast a vote for anyone, but this seems unlikely, and it is more probable that the machines failed to record votes that were cast.

The other contested Palm Beach election was in Boca Raton, where former mayor Emil Danciu came in third with an 8% undervote. His suspicions regarding possible lost votes stemmed from low numbers reported in his home precinct, where he was expected to do well. During court proceedings, it was revealed that Sequoia had sold the systems under trade secret protection, making it a third degree felony for Supervisor LePore if any details regarding the specification or internal functioning of the devices were revealed. Circuit Court Judge John Wessel granted Danciu a walk-inspection of the voting equipment, where it was discovered that the pre-election testing circumvented the ballot-face and the touchscreen was used only to cast one vote for each candidate listed first in every race. Because Danciu appeared third in his race, there is no test data that can reveal whether or not the machines would properly activate and record votes cast for him. (In the Wellington election, the losing candidate appeared second, so his position was also untested.) Further disconcerting information included the fact that the voting machines are reprogrammable at the firmware level via a portal on each device, and also that at the end of the election they are frozen in a mode where one can not perform vote casting, so a functional post-test is precluded.

Difficulties in Florida's September 2002 primary were not limited to the touchscreen systems. In Union County, the optical scanning system had been erroneously programmed to print out only Republican party results, requiring a hand-count of some 2700 ballots. At least with the paper ballots, an independent tally was possible. Over in Miami-Dade, reported undervotes of as much as 48% in some precincts in the Gubernatorial race caused Janet Reno to demand that a recount be performed. Here however, election officials reconstructed some supposedly missing votes by extracting dubiously recorded data from the touchscreen machines!

Florida's experience may be replicated as communities rush to adopt flawed voting products and will inadvertently squander billions of dollars in public funds. National standards for design, construction and testing have lagged behind, while Voting Rights Act initiatives have stalled in Congress. Only a lengthy moratorium on new purchases of voting equipment, until these issues have truly been sorted out, can hope to restore sanity and confidence in democratic elections.

Rebecca Mercuri (mercuri@acm.org), a professor of Computer Science at Bryn Mawr College, PA, is an expert on electronic voting systems. Her informative Web site on this subject can be found at: http://www.notablesoftware.com/evote.html

========================================================

Inside Risks 148, CACM 45, 10, October 2002

Secure Systems Conundrum

By definition, a secure system enforces some policy it is given. Such a policy might, for example, prevent your confidential files from being revealed or might notify the copyright holder every time you play an MP3 file. The former protects you as an individual; the latter enables new means of charging for electronically distributed intellectual property. Both might be seen as improvements over the status quo. Yet whether secure systems are in practice attractive really depends on two questions:

What range of policies can the system enforce?
Who chooses what policies the system enforces?

Automated policy enforcement mechanisms are incapable of showing good taste, resolving ambiguity, or taking into account the broader social and political context in which a computer system exists. So formulating as a policy something that accurately reproduces our intents is likely to be impossible, and we must endure policies that conservatively disallow actions they shouldn't. An example well known to Inside Risks readers involves system policies that disallow copying CDs containing music or software even though such copying is permitted according to the ``fair use'' provisions of copyright law. In general, intent is difficult to formulate precisely as a policy that can be enforced with a secure system---witness what happens in writing laws, which too often forbid things society didn't intend or allow things it did intend to forbid.

The question of ``Who chooses what policies are enforced?'' is tantamount to deciding who controls the computer system. On special-purpose computers (e.g., cellphones and set-top cable modems), the enforcement of policies imposed by others has not seemed offensive. Software on these devices is, for example, regularly updated and device usage monitored without user consent (or knowledge). But enforce a policy to restrict what happens on a desktop computing system, and that system might no longer be general purpose. No surprise, then, that the Trusted Computing Platform Alliance (TCPA) and other efforts concerned with hardware and operating system support for secure computing systems are controversial. The surprise is that technical details are only a small part of the picture.

The world today is one in which computer users are either unwilling or unable to implement non-trivial security policies on their desktop computers. Do you set all those file protection bits and check digital certificates for expiration? Most often not, so the policies enforced by secure systems will likely come from elsewhere. We would hope that these policies are designed with our individual and collective best interests in mind, and we might wonder what forces will cause that to happen. The law and the market seem the likely candidates.

The law arguably is not up to that task. Courts are having difficulty applying our current laws to cyberspace---witness the debate associated with interpreting copyright's ``fair use''. Moreover, digital rights management is but one class of policies our secure systems might be enforcing. New laws might be the answer, but then recent U.S. (and some EU) experiences do not bode well for the public good.

Perhaps the market could provide the incentives. However, this presumes a user or owner is free to choose which policy is enforced on a given computer. It also presumes that the market is open to would-be policy providers. Neither is guaranteed, and there are good reasons why neither might hold. The producer of a secure system has an incentive to provide a policy that prevents other policies from being added and other producers' software from being run.

Even if the computer owner were completely free to choose among policies, a digital content provider will likely require certain policies to be present on any computer accessing their content. The free choice then becomes one of choosing between desired content and desirable policy---not much of a choice.

Insecure systems today allow users to circumvent copyright restrictions, license agreements, and the law. Sometimes this circumvention is done in ignorance; sometimes it is done in protest; but sometimes it is done because the policy being enforced is clumsy and forbids something it shouldn't. In short, circumventing policy enforcement serves as a much needed relief valve for clumsy policies.

Without a doubt, deploying secure systems has risks. Individuals are unlikely to be better off with secure systems unless the way has been prepared with careful attention to who controls the policies these systems enforce and what values those policies reflect. And if the so-called secure systems have vulnerabilities---as software systems so often do---malevolent users will still be able to do things they shouldn't, whereas ordinary users will have lost their means to compensate for clumsy policies.

Fred B. Schneider is a professor and director of the Information Assurance Institute at Cornell University.

========================================================

Inside Risks 147, CACM 45, 9, September 2002

Risks of Digital Rights Management

Digital rights management (DRM) is an attempt to provide ``remote control'' over digital content. The required level of protection goes beyond secure delivery of the bits -- restrictions on the use of the content must be maintained after it has been delivered. The buzzword for this is ``persistent protection.''

For example, a digital book can be delivered over the Internet using standard cryptographic techniques. But if the recipient can save the book in an unrestricted form, he can then freely redistribute a perfect copy to a large percentage of the population of earth. This reality has led publishers to forego the potentially lucrative sale of digital books and has had a similar chilling effect on the legitimate distribution of other types of digital content. Robust DRM would, among other things, enable copyright holders to take full advantage of the Internet without having to rely on the honor system. (Recall Stephen King's experiment with an on-line book.)

Effective DRM would have other far-reaching implications. For example, armed with strong DRM, an individual could maintain tight control over his personal data, thus providing for a measure of online privacy and confidentiality that is currently lacking. Some companies even claim that their proprietary DRM system provides unbreakable persistent protection transitively and indefinitely, in some cases even with remote rights revocation.

Most DRM companies emphasize their robust cryptography, often implying that this is enough to ensure security. One company even maintains it is the only one with export permissions for its 256-bit crypto (stronger than most of its competition). While cryptography is an essential part of DRM, it can do little to ensure the higher level of persistent protection required. At some point, cryptographic keys will be in the possession of the legitimate recipient, who also happens to be the most likely attacker of the persistent protection. Though it has its own set of risks, cryptography is the easy part of any comprehensive DRM solution.

Then what technology can be brought to bear on the persistent protection part of the DRM equation? Unfortunately, there is little useful design information available on implemented systems, although there is ample senseless marketing hype. In the field of security, experience has taught that full disclosure is essential. For example, cryptographers do not trust a cryptosystem until it has been publicly vetted and subject to intense scrutiny by the cryptographic community. This reluctance to accept cryptographic algorithms at face value comes from the long list of ``secure'' algorithms that have been broken. In DRM there is, as yet, no such imperative to make the design of systems---even in a general form---available for public scrutiny. At the very least, this suggests that the level of security provided by such systems is suspect, since those making the security claims have a financial interest in boosting their perceived level of security.

The track record of fielded DRM systems is also not reassuring. For example, Adobe eBooks was easily broken, although Adobe made no real efforts at protection. Although Microsoft's MS-DRM security went a little further, it too offered little challenge to a persistent attacker. Of course, the simple reverse engineering required to break such schemes inspired the Digital Millennium Copyright Act (DMCA) prosecution of Dmitry Sklyarov and his employer, ElcomSoft. (Proposed legislation could result in life imprisonment for similar acts.)

The DRM market has been estimated at $3.5 billion by 2005. Not surprisingly, a large number of companies are vying for a slice of this enormous pie. Unfortunately, the technology behind current DRM systems is little more than glorified security by obscurity. But this awkward reality has not prevented companies from making strong claims about the security of their products -- claims that cannot be supported either by their known design features or by the real world performance of comparable systems.

The future of DRM appears to lie in the direction of tamper-resistant hardware, which promises to be a far more effective solution. Ironically, such an approach threatens to move the ``remote controllability'' from users to third parties, carrying its own set of risks. Regardless, the current state of digital rights management technology falls far short of what is required to deliver on the promise of DRM. The risk today lies in not recognizing this reality.

Mark Stamp (mstamp1@earthlink.net) spent 2 years designing a DRM system for MediaSnap Inc. He is now an independent consultant in Silicon Valley. His paper, Digital Rights Management: The technology behind the hype http://home.earthlink.net/~mstamp1/papers/DRMpaper.pdf, includes many references.

========================================================

Inside Risks 146, CACM 45, 8, August 2002

Risks in Features vs. Assurance

Essentially all commercial computer systems development and deployment have been driven by concerns for time-to-market, novel features, and cost -- with little if any concern for assurance, reliability, or the avoidance of system security vulnerabilities in networked environments. Retrofitted products for the new connected world usually expose new vulnerabilities, because the environment changes as a result. Security features of an existing product may not adequately address new risks, because new security policies take effect. New features introduce unforeseen interactions between various components, invalidating previous assurance.

Software and systems currently are related to risks under contract law rather than any more demanding liability laws. The contracts are typically extremely inequitable, with purchasers assuming all liabilities, despite the practical impossibility of their assessing the security, reliability, or survivability characteristics of the products they are buying. Indeed, most software products come with an anti-warranty: the producer warrants nothing, and customers assume all liabilities.

There is a serious lack of understanding among developers and development managers that security and survivability are different from features. Self-promoted and self-assigned ``security experts'' (often without a comprehensive understanding of security issues) often lead to security features that are promised, but poorly conceived and poorly understood. It has been to the industry's advantage to position ``security'' as a feature that is added to other systems and computing complexes, rather than primarily a characteristic of thoughtful architecture, careful design, and meticulous engineering, coding, testing, and operation. Security does not result from modules that are added after-the-fact; it must be engineered in from the beginning. The CERT/CC database contains numerous vulnerabilities such as buffer overflows and password sniffing that are often consequences of basic system architecture and design, some of which cannot easily be retrofitted. Products are shipped with many features, but assurance is at best paid only lip-service as part of the vendor's marketing campaign.

Lack of commercial preference regarding security favors feature-laden software and frequent shipping schedules; too often, it is more important to ship a product with promised features in the commercial world. Although this may seem to be the failure or ignorance of the industry, these features are often requested by customers who don't necessarily have an in-depth understanding of security, but have security vaguely in mind. If a vendor can't deliver in time or doesn't offer a feature for sound security reasons, the customer finds another vendor that can. The industry is not interested in research and development without payoffs. As long as the customer takes the risks, there is much reduced incentive, and a great incentive to offer ``nifty'' features, even if these features increase the vulnerability to compromise. Evaluation of a product against a Common Criteria protection profile (for example) is not in the list of most customers.

The feature-dominated production ignores security experts' warnings, which become the first sacrifice in crunch time in development organizations -- justified by the motto ship happens. Similar sacrifice is carried out by customers demanding functionality in a short time.

Although gaining more interest, cryptography, computer security, and survivability are not widespread. Security issues covered in most operating system and software engineering courses can be improved. It may not be possible to expand the existing courses and squeeze more concepts within the same time frame. Instead, separate computer-security courses might be added as is already done by some universities. But, one way or the other, security and software engineering need to be thoroughly integrated into the curriculum.

Inadequate testing of features and their myriad interactions generally relegates testing to a final screen. Hardware designers have long implemented design-for-testability rule sets and supporting integral test hardware (which may occupy more that 5% of a chip). Testing engineers should be involved at the inception of development to make sure that issues relating to testability and reliability are properly addressed.

The software and systems industry has been allowed to develop without substantial legal oversight, under the assumption that its customers were sophisticated and could manage their risk exposure appropriately. Unfortunately, even sophisticated customers cannot know their security exposure. Under such conditions, liability law may be held to override unjust contract disclaimers. If the industry will not clean up its act, it must expect the tort bar to do so.

Dr. Acar is a senior software engineer at Novell, Inc.: tacar@novell.com Dr. Michener is a consulting engineer at Enterprises, Inc.: jrmichener@ieee.org

========================================================

Inside Risks 145, CACM 45, 7, July 2002

Risks: Beyond the Computer Industry

As computer technologies increasingly invade everyday products, the RISKS of the traditional computer business must be revisited by each new industry, usually through failures. Issues include reliability of code, protection against component failure, security of data, privacy and security, safety, maintenance, and upkeep. There is one issue that affects all of these topics: ease of use. Poor usability leads to high support costs, high error rates, and increased injuries.

Consider the automobile, which is certainly a popular target for new technology to assist driving, enhance entertainment, and facilitate business activities. Usually driving does not require full concentration, but situations that require full attention typically arise without warning. What might be a minor secondary task under normal driving conditions can suddenly become life-threatening.

In modern cars the number of controls in front of the driver has proliferated to an unacceptable extent. BMW addressed the complexity issue with their new iDrive Controller, available in the 7 series sedan. Their solution was to replace most of the controls, knobs and displays of the dashboard with a single knob and display screen. BMW states that ``this user-friendly interface offers quick access to over 700 settings.''

When one control does multiple operations, it requires a complex menu structure and choice of modes, which in turn promotes mode errors and other sources of error. It is best to have dedicated controls for critical functions, even at the expense of more buttons and knobs. Unfortunately, there is a design tradeoff between simplicity in appearance and simplicity in use. This is a dangerous design trap. Alas, consumers (and organizations) make purchase decisions based on appearances more than reality, so this is a fundamental conflict. But BMW did not have to choose between one knob and display screen or 720 separate controls: there are alternative designs between these extremes.

BMW's user interface has been soundly trashed in the press. Let us hope they pay heed and hire professionals from the Human-Computer Interaction community (e.g., www.acm.org/sigchi) to help redesign their approach, from the initial assumptions upwards: this cannot be fixed with a simple patch or some new graphics.

A very different problem is that faced by hotels. Business travelers expect high-speed Internet access, but the technologies of Internet connection make configuration overly difficult. Internet connections require setting numerous parameters. Worse, these change from location to location, ISP to ISP. My personal experience is that the installations seldom work completely at first. Although once connected it is possible to read email from POP servers, it is usually impossible to send without multiple telephone calls to service providers to get the SMTP information. SMTP was not designed with security in mind, so most ISPs will not send mail from foreign sites, forcing the traveler to negotiate the morass of unknown ISP providers from hotel to hotel. It is time to advance from the current SMTP toward a new standard that allows one setting to work from any location, much as POP servers now allow.

Security is a major issue. A large number of intermediaries have arisen to increase security, including software firewalls, proxy servers and VPNs. Most travelers and hotel staff are insufficiently knowledgeable to navigate through these roadblocks. And everyone who changes their settings, successfully or not, faces the daunting task of resetting them afterwards.

Yet another problem area is the proliferation of services on telephone systems. Twenty years ago I suggested that the only solution was more dedicated buttons plus display screens to guide the operations in simple language. We now have more buttons and screens, but simple language still eludes many design teams, probably because the writing is seldom done by professional technical writers. Cellphones complicate the story: as the number of functions increases, size and power constraints leave little room for more buttons or larger screens.

As computer technologies migrate to other industries, ACM faces a growing challenge to promulgate appropriate human-centered development processes. More and more of the RISKS from technology come from deficient consideration of people, organizations, and cultures. ACM has taken small steps toward changing the balance. But as computers pervade the fabric of every human activity, more emphasis is required. Otherwise, the existing known RISKS will simply proliferate beyond imagination.

Donald A. Norman (norman@nngroup.com) is professor of Computer Science at Northwestern University and co-founder and principal of the Nielsen Norman group. He is author of The Invisible Computer.

========================================================

Inside Risks 144, CACM 45, 6, June 2002

Free Speech Online and Offline

Esther Dyson argued that as the world will never be perfect, whether online or offline, it is foolish to expect higher standards on the Internet than we accept in `real life'. Legislators are now turning this argument around, and arguing that they have to restrict traditional offline freedoms in order to regulate cyberspace.

A shocking example is an export-control bill currently in Britain's parliament. The government version would enable the government to impose licensing restrictions on collaborations between scientists in the UK and elsewhere; to take powers to review and suppress scientific papers prior to publication; and even to license foreign students in British universities. By a large majority, the House of Lords amended it to exclude material in the public domain and information exchanged in the normal course of academic teaching and research. It has now gone back to the House of Commons, where ministers say they will amend it back again. This fight could go on for weeks.

During the late 1990s, arms-export regulations prevented US nationals making cryptographic software available on their Web pages, or emailing it abroad. Phil Zimmermann, the author of PGP, was investigated by a Grand Jury for letting it `escape' to the Internet. The law was ridiculed by students wearing T-shirts printed with encryption source code (`Warning: this T-shirt is a munition!'), and challenged in the courts as an affront to free speech.

For government, there was a risk that crypto software would escape. For liberty, there was a risk we ignored at the time: that the bad policy would escape. Although the Clinton administration later abandoned its approach as unworkable, that did not stop other governments trying to ape it. After Tony Blair was elected in 1997, he tried to take Britain down the American path; after much protest and many battles, the current bill is the result. His attempt to have a law with no embarrassing loopholes has resulted in one that is absolutely draconian. For example, someone accidentally learning the wrong type of secret can be prevented from ever leaving the UK. (The Lords amended the Bill to remove this unpleasantness; the government says it will reinstate it.) While this particular fight is mainly a matter for Brits, it is an example of a wider and worrying trend -- toxic overspill from attempts to regulate the Internet.

There are many more examples. In the USA, Hollywood's anxiety about digital copying led to the Digital Millennium Copyright Act. This gives special status to mechanisms enforcing copyright claims: circumvention is now an offense. So, manufacturers are now bundling copyright protection with even more objectionable mechanisms, such as accessory control. For example, one games-console manufacturer builds into its memory cartridges a chip that performs some copyright control functions but whose main purpose appears to be preventing other manufacturers from producing compatible devices. There is no obvious way to reconcile the tension between competition and public policies on copyright.

Meanwhile, worries about cybercrime are leading to a Europe-wide arrest warrant that overturns the time-honored principle of dual criminality, i.e., you can be extradited from one country to another only if there is prima facie evidence that you've committed a crime according to the laws of both countries. Now Germany has strict hate-speech laws (`Mein Kampf' is a banned book), while Britain does not. Right now, I could put an excerpt from that book on my website in the UK (or the USA) but not in Germany. But the new arrest warrant would allow the German police to extradite me from Britain, for an offense that doesn't exist in British law. Thus, free-speech rights online may be reduced to the lowest common denominator among the signatory nations.

Why do we get so many bad laws about information? Many of them have to do (in some broad sense) with risks: with the perceived vulnerability of the Internet to hackers, bomb makers, credit-card thieves, pornographers, and other undesirables. There is massive hype from the computer security industry; when people get fed up with hearing about hackers, the risk changes to `cyberterrorism'. There are few or no balancing voices, as the interests of almost everyone involved in the security industry (vendors, government agencies, regulators, researchers) lie in talking up the threats. Journalists like the scare stories more than the rebuttals. Politicians and bureaucrats use them to build empires. After the .com boom, we are seeing the .gov version; and there's no sign of it bursting any time soon.

We need better ways of dealing with risks realistically at the political level. Does that mean simply educating the public about risks, or do we need something else too?

Ross Anderson heads the security group in the Computer Laboratory at Cambridge University in the UK http://www.cl.cam.ac.uk/users/rja14; Ross.Anderson@cl.cam.edu.uk

========================================================

Inside Risks 143, CACM 45, 5, May 2002

Risks of Inaction

Scientists and technologists create a variety of impressions in the eyes of society at large, some positive and others negative. In the latter category is the perception (often clearly a mischaracterization) that many individuals in these occupations are not involved with society in positive ways, making them easy to target for many of society's ills.

It's not difficult to see how this simplistic stereotype developed. We technically-oriented folks can easily become so focused on the science and machines that we willingly leave most aspects of the deployment and use of our labors to others, who often don't solicit our advice -- or who may even actively disdain it.

In the broad scope of technology over the centuries, there have been many innovators who lived to have second thoughts about their creations. From the Gatling gun to nuclear bombs and DNA science, the complex nature of the real world can alter inventions and systems in ways that their creators might never have imagined.

It of course would be unrealistic and unwise for us to expect or receive total control over the ways in which society uses the systems we place into its collective hands. However, it is also unreasonable for the technical and scientific minds behind these systems to take passive and detached roles in the decision-making processes relating to the uses of their works.

Within the computer science and software arenas, an array of current issues would be well served by our own direct and sustained inputs. The continuing controversies over the already-enacted Digital Millennium Copyright Act (DMCA) is one obvious example. Even more ominously, the newly proposed Consumer Broadband and Digital Television Promotion Act (CBDTPA), formerly known as the Security Systems Standards and Certification Act (SSSCA), is a draconian measure; it would greatly impact the ways in which our technologies will be exploited, controlled, and in some cases severely hobbled. We never planned for digital systems to create a war between the entertainment industry, the computer industry, and consumers, but in many ways that's what we're now seeing.

Controversies are raging over a vast range of Internet-related issues, from the nuts and bolts of technology to the influence of politics. Concerns about ICANN (the Internet Corporation for Assigned Names and Numbers) -- the ersatz overseer of the Net -- have been rising to a fever pitch.

Throughout all of these areas and many more, critical decisions relating to technology are frequently being made by politicians, corporate executives, and others with limited technical understanding -- frequently without any meaningful technological inputs other than those from paid lobbyists with their own selfish agendas.

The technical and scientific communities do have associations and other groups ostensibly representing their points of view to government and others. But all too often the pronouncements of such groups seem timid and not particularly ``street-savvy'' in their approaches. Fears are often voiced about sounding too un-academic or expressing viewpoints on ethical matters rather than on technology or science itself, even when there is a clear interrelationship between these elements. Meanwhile, the lobbyists, who have the financial resources and what passes for a straight-talking style, have the ears of government firmly at their disposal.

Computers and related digital technologies have become underpinnings of our modern world, and in many ways are no less fundamental than electricity or plumbing. However, it can be devilishly difficult to explain their complex effects clearly and convincingly to the powers-that-be and the world at large.

As individuals, most of us care deeply about many of these issues -- but that is not enough. We must begin taking greater responsibility for the manners in which the fruits of our labors are used. We need to take on significantly more activist roles, and should accept no less from the professional associations and other groups that represent us. If we do not take these steps, we will have ceded any rights to complain.

Lauren Weinstein (lauren@privacy.org) is co-founder of People For Internet Responsibility http:www.pfir.org. He is moderator of the Privacy Forum http://www.vortex.com and a member of the ACM Committee on Computers and Public Policy.

========================================================

Inside Risks 142, CACM 45, 4, April 2002

Digital Evidence

Those of you concerned with privacy issues and identity theft will be familiar with the concept of dumpster diving. Trash often reveals the dealings of an individual or a corporation. The risks of revealing private information through the trash has led to a boom in the sale of paper shredders (and awareness of the risks of reassembling shredded documents). However, how many of us take the same diligent steps with our digital information?

The recovery of digital documents in the Enron case and the use of e-mail in the Microsoft anti-trust case have brought these concerns into the fore. For example, we are all more aware of the risks inherent in the efficient (``lazy'') method of deleting files used by modern operating systems, where files are `forgotten about' rather than actually removed from the drive.

There will certainly be an increase in the sales of `wiper' software following this increase in awareness, but that's not the end of the story. Overwriting data merely raises the bar on the sophistication required of the forensic examiner. To ensure reliable data storage, the tracks on hard-drive platters are wider than the influence of the heads, with a gap (albeit small) between tracks. Thus, even after wiper software has been applied, there may still be ghosts of the original data, just partially obscured.

So, what more can we do? Clearly, we are in a tradeoff between the cost to the user and the cost to the investigator. To take the far extreme, we would take a hammer to the drive and melt down the resulting fragments, but this is infeasible without a large budget for disks.

One could booby-trap the computer, such that if a certain action isn't taken at boot time, the disk is harmed in some way. Forensics investigators are mindful of this, however, and take care to examine disks in a manner that does not tamper with the evidence. If we're open to custom drives, we could push the booby-trap into the drive hardware, causing it to fail when hooked up to investigative hardware (or, more cunningly, produce a false image of a file-system containing merely innocent data).

Another approach is to consider file recovery as a fait accompli and ensure that the recovered data is not available as evidence. Encryption clearly has a role to play here. An encrypting file-system built into your operating system can be helpful, but may provide only a false sense of security -- unless you have adequate assurance of its cryptanalytic strength (which is likely to be weakened if there is common structure to your data) and the strength of the underlying operating systems. Per-file encryption with a plurality of keys might help, but that begs the question of key management and key storage.

One could consider possible key escrow, backdoors, and poorly implemented cryptography software to be below your paranoia threshold. Another useful step can be secret sharing (A. Shamir, "How to Share a Secret", Comm.ACM 22, 11, 612--613, November 1979). Spread your data in fragments around the network such that k of the n fragments are required to be co-located to decipher the original file. In a carefully designed system, any k-1 fragments yield no useful insight into the contents of the file; k and n can be tuned according to the paranoia required, including the placement of no more than k-1 within the jurisdiction of the investigating agency.

Clearly, there are a number of steps we can take to push the evidence as far as possible beyond the reach of those who might use it to incriminate us. But one question not often raised in this topic is why should we bother? Given the lack of strong authentication in most computing systems, it's not beyond reasonable doubt that the files in question are not even yours.

Furthermore, there are many risks of trusting recovered digital evidence, given the ease with which digital documents can be fraudulently created, modified, or accidentally altered, or their time stamps manipulated. Corroboration by independent sources of evidence is usually required to establish a case, even for non-digital evidence, although when all of these corroborating sources of evidence are digital, the risks remain. See, for example, discussion of the potential holes in evidence in the case of the Rome Labs Intrusion in 1994 (Peter Sommer, "Intrusion Detection Systems as Evidence", BCS Legal Affairs Committee, March 2000.

Inside Risks 141, CACM 45, 3, March 2002

Risks of Linear Thinking

For over half a century we have classified research on a scale from basic to applied. Basic research is a quest for fundamental understanding without regard to potential utility. Applied research is technology development that solves near-term problems. These two models have different diffusion times from research result to practice -- often 20-50 years for basic research and 2-3 years for applied. Because the return on investment of basic research is so far in the future, the Federal government is the main sponsor and university faculty are the main investigators.

For over a generation we have classified software development on a scale from technology- centered to human-centered. Technology-centered work is focused on advancing software technology with new functions, algorithms, protocols, and efficiencies. Human-centered work is focused on making software more useful to those paying for or using it.

These two one-dimensional (linear) scales create false dichotomies, obscure fundamental issues, and encourage tensions that hurt research and software development.

Most of our academic departments place high value on basic and technology-centered work. Faculty who do applied or human-centered projects often find themselves disadvantaged for tenure and promotion and occasionally the object of scorn. Most eventually toe the line or leave the academy. (See National Research Council, Academic careers for experimental computer scientists, NRC Press, 1994.) The resulting bias prevents us from valuing and teaching the full range of vital software development topics. Many of the risks discussed in this forum over the years will never be fully addressed as long as this bias persists.

In 1997, Donald Stokes (Pasteur's Quadrant: Basic Science and Technological Innovation, Brookings Institution, 1997 http://www.brook.edu) put the research issue into a new light. He traced the conceptual problem back to Vannevar Bush, who in 1945 coined the term basic research, characterized it as the pacemaker of technological process, and claimed that in mixed settings applied research will eventually drive out basic. Bush thus put the goals of understanding and use into opposition, a belief that is at odds with the actual experience of science. Stokes proposes that we examine research in two dimensions, not one:
* Inspired by considerations of use?
* Quest for fundamental understanding?

He names the (yes,yes) quadrant Pasteur's, the (no,yes) quadrant Bohr's, and the (yes,no) quadrant Edison's. He did not name the (no,no) quadrant, although some will recognize this quadrant as the home of much junk science.

Those who favor applied research call for greater emphasis on Pasteur's+Edison's quadrants, and those who favor basic, on Bohr's+Pasteur's. In fact, most of the basic-versus-applied protaganists will, if shown the diagram of four quadrants, agree that these three correspond to vital sectors of research, none of which is inherently superior to the others.

A similar model can be applied to software development. Here the common belief is that the attention of the designer can either be focused on the technology itself or on the user, or somewhere in between. Michael Dertouzos (The Unfinished Revolution, Harper Collins, 2001) recently documented 15 chronic design flaws in software and said that they will be eliminated only when we learn human-centered design, design that seeks software that serves people and does not debase or subvert them. Dertouzos called for his fellow academics to teach human-centered design and not to scorn software developers who interact closely with their customers. Some critics incorrectly concluded that he therefore also supported reducing attention to the world of software technology. However, we can view software development in two dimensions, rather than one:
* Inspired by considerations of utility and value?
* Seeks advancement of software technology?

Three of these quadrants correspond to important software development sectors:

(yes,yes) -- projects to create new technologies in close collaboration with their customers (examples: MIT Multics, AT&T Unix, Xerox PARC Alto, IBM System R, World Wide Web).

(yes,no) -- projects to employ existing knowledge to solve human problems (examples: Harlan Mills' work, CHI, much application development).

(no,yes) -- projects to create new software technologies for their intrinsic interest (examples: many university research projects)

The final (no,no) quadrant is the home of many projects purely for the amusement of the developer. Many software developers will agree that the first three quadrants are all important and that none is inherently superior to the others. Perhaps this two-dimensional interpretation will help unstick our thinking about software development.

Peter Denning (pjd@cs.gmu.edu) has contributed to ACM for many years in many capacities. Jim Horning (horning@acm.org) has been involved in computing research for more than 30 years, and is presently at InterTrust Technologies.

========================================================

Inside Risks 140, CACM 45, 2, February 2002

The Homograph Attack

[NOTE: Choose an appropriate Cyrillic character set for your browser for this column only, if your browser does not recognize the russian for gazeta.ru and the russian c and o in microsoft.]

Oldtimers remember slashes (/) through zeros [or through the letter O where there was no difference] in program listings to avoid confusing them with the letter O. This has long been obsoleted by advances in editing tools and font differentiation. However, the underlying problem of character resemblance remains, and has now emerged as a security problem.

Let us begin with a risks case. On April 7, 2000, an anonymous site published a bogus story intimating that the company PairGain Technologies (NASDAQ:PAIR) was about to be acquired for approximately twice its market value. The site employed the look and feel of the Bloomberg news service, and thus appeared quite authentic to unsuspecting users. A message containing a link to the story was simultaneously posted to the Yahoo message board dedicated to PairGain. The link referred to the phony site by its numerical IP address rather than by name, and thus obscured its true identity. Many readers were convinced by the Bloomberg look and feel, and accepted the story at face value despite its suspicious address. As a result, PairGain stock first jumped 31%, and then fell drastically, incurring severe losses to investors. A variant of this hoax might have used a domain named BL00MBERG.com, with zeros r