COMPUTER-RELATED RISKS
Excerpts on Computer Calendar-Clock Problems
Peter G. Neumann
Computer Science Laboratory, SRI International
Menlo Park, California 94025-3493
415-859-2375, neumann@csl.sri.com
©Peter G. Neumann, 1995,1997,2000
This version created January 16, 2000.
This is an excerpt from the first edition of Computer-Related Risks, Addison-Wesley, 1995, with the inclusion of considerable new material to accompany the fourth printing of the book. Because the new material is arriving at such a rapid rate, I will endeavor to keep this section up to date and available on-line, rather than revise the book itself.
In this section, we consider time- and date-related problems. To emphasize the intrinsic nature of calendar-clock problems and how they depend on computer representations of dates and times, we use (in this section only) a mathematically natural format for clock-related dates: year-month-day. This format illustrates the natural numerical order in which carries propagate (upward) from seconds to minutes to hours to days to months to years, and it serves to underlie the nature of those cases in which carries and overflows are not properly handled. Note that the American representation of month-day-year and the European representation of day-month-year are both mathematically unnatural, although the latter is at least logically ordered. In addition, there are often ambiguities between and within the two forms, as in 11/12/98, which could be November 12 or 11 December; furthermore, the year could be 1898 or 1998, for example. We can expect many new ambiguities when we reach 1/1/00 and 1/2/00.
We consider first a few miscellaneous problems related to dependence on computer clocks, and then consider specifically problems arising with dates and times, with the year 2000, and with leap-years.
Dependence on remote clocks. In Colorado Springs, one child was killed and another was injured at a traffic crossing; the computer controlling the street crossing did not properly receive the time transmitted by the atomic clock in Boulder, which affected the system's ability to vary the controls according to the school schedule. In all, 22 school crossings were affected (SEN 14, 2).
SRI's Computer Science Laboratory computer system once used a then-common eleven-clock averaging algorithm to reset the local clock automatically on reboot after a crash. Unfortunately, at the moment of reboot, a clock at the University of Maryland was off by 12 years, based on which the CSL clock was initialized to be off by 15 months. (Yes, the new algorithms now discard extreme values, and rely on systems with more dependable clocks.)
Byzantine clocks. Algorithms for making a reliable clock out of less reliable clocks are potentially nontrivial. The old three-clock algorithms (such as taking the median of three values) break down if one clock can report different values to its neighbors at any one time -- whether maliciously or accidentally. In that case, four clocks are both necessary and sufficient. Furthermore, if k clocks can be potentially untrustworthy, then n = 3k+1 clocks are required to provide a reliable clock that can withstand Byzantine faults in any k clocks [1, 2] - that is, in which completely arbitrary behavior can occur in at most k clocks.
Year ambiguities. In 1992, Mary Bandar received an invitation to attend a kindergarten in Winona, Minnesota, along with others born in '88. However, Mary was 104 at the time. The person making the database query had typed 1988, but the software kept only the last two digits (as noted by Ed Ravin in SEN 18, 3, A-3).
G.C. Blodgett's auto-insurance rate tripled when he turned 101; he was the computer program's first driver over 100, and his age was interpreted as 1, which fit into the program's definition of a teenager -- namely, someone under 20 (noted by Lee Breisacher in SEN 12, 1, 19).
In Section 2.9 we observed the case of the almost-100-year-old man whose white-blood-cell count was interpreted as normal because the program interpreted his birthyear (input 89) as 1989, rather than as 1889 (SEN 15, 2). This kind of problem will certainly be exacerbated as we approach and pass the turn of the century, and as more people become centenarians.
We next consider problems relating specifically to calendar-clock arithmetic.
Overflows. The number 32,768 = 215 has caused all sorts of grief that resulted from the overflow of a 16-bit word. A Washington, D.C., hospital computer system collapsed on 1989 Sep 19, 215 days after 1900 Jan 01, forcing a lengthy period of manual operation. Brian Randell reported that the University of Newcastle upon Tyne, England, had a Michigan Terminal System (MTS) that crashed on 1989 Nov 16, 215 days after 1900 Mar 01. Five hours later, MTS installations on the U.S. east coast died, and so on across the country, an example of a genuine (but unintentional) distributed time bomb.
Don Stokes noted that Tandem CLX clocks struck out on 1992 Nov 1 at 3 p.m. in New Zealand, normally the first time zone to be hit, where the program deficiency attacked Westpac's automatic teller machines and electronic funds transfer point-of-sale terminals (EFTPOSs). For each of the next 4 hours thereafter, similar problems appeared in succeeding time zones, until a microcode bug could be identified and a fix reported. Thus, the difficulties were overcome before they could reach Europe. In some cases, the date was converted back to December 1983, although the bug affected different applications in different ways. Some locations in later time zones avoided 3 p.m. by shifting past it; others set their clocks back 2 years. Fortunately, the day was a Sunday, which decreased the effect.
The linux term program, which allows simultaneous multiple sessions over a single modem dial-up connection, died worldwide on 1993 Oct 26. The cause of an overflow was an integer variable defined as int rather than unsigned int. 1
Year-end roundup. The Pennsylvania Wild Card Lotto computer system failed on 1990 Jan 01. The winners of the lottery could not be determined until the software had been patched, 3 days later.
Summer Time, and the livin' is queasy. The U.S. House of Representatives passed a bill in April 1989 that would have delayed the end of Pacific daylight time in presidential election years until after the November election, with the intent of narrowing the gap between when the predictions from the rest of the continental United States start pouring in and when the polls close in California. The bill was never signed, but the option of "US/Pacific-New" was inserted into Unix systems to be used in the event it passed. As a consequence, several administrators on the west coast chose that option, and their systems failed to make the conversion to Pacific standard time on 1992 Oct 26 00:00 (SEN 18, 1). The end-April 1993 cutover to summer time in Germany resulted in a steel production line allowing molten ingots to cool for 1 hour less than intended. To simplify programming, a German steel producer derived its internal computer clock readings from the Braunschweig radio time signal, which went from 1:59 a.m. to 3:00 a.m. in 1 minute. When the process controller thought the cooling time had expired (1 hour too early), his resulting actions splattered still-molten steel, damaging part of the facility. 2
On the same night, the Bavarian police computer system stopped working at 3:00 a.m., blocking access to all of its databases. 3
Arithmetic errors. John Knight found an item in the October 1990 Continental Airlines magazine, while flying home from the 1990 Las Cruces safety workshop. The note described how the airline rented aircraft by the entire day, even if the plane was used only for a few hours. The billing for rentals was consistently 1 day too little, because the billing program merely subtracted the begin and end dates. This example exhibits the classical off-by-one error, which is a common type of programming mistake.
Nonportable software. The National Airspace Package, a program developed in the United States for modeling controlled airspace, failed to work when it was tried in the United Kingdom -- the program had ignored longitudes east of Greenwich. 4
As the year 2000 approaches, the risks of calendar-clock problems looms large whenever two-digit year fields are used. The number 99 is larger than 00, not smaller, and we can expect all sorts of computer calendar date-time arithmetic to fail whenever the relative order of dates is considered. For example, COBOL programs use a two-digit year field, and COBOL programmers are increasingly scarce. Consequently, some folks are in panic, whereas others have a while longer to plan ahead (MS-DOS bellies up on 2048 Jan 01 and the programming language Ada has a time_of year field that is exhausted after 2099). Some folks believe they are really immune - such as users of Java, which runs out of dates in the year 292271023. As noted in Section *, some systems have already run out or will soon (Tandem CLX; Apollo workstations exhaust their time fields on November 2, 1997; the Global Positioning Satellite GPS on August 21, 1999; Ed Ravin noted the Fujitsu model SRS-1050 ISDN display phones had their clocks stop at 1994 Sep 30 11:59 PM, some later.) 5
Pundits are creating estimates of how much it will cost to fix all of the software that is expected to die, beginning at the transition on midnight from 1999 Dec 31 to 2000 Jan 1. A figure of $300 to $600 billion (thousand million) has often been quoted as the Gartner Group's estimate of the overall worldwide cost to prepare for the Year 2000, and that estimate has recently been revised upward to over a trillion dollars. $30 billion was cited as the cost to the U.S. Government, with the prognostication that 30% of the systems would not be fixed in time. Consumers Power Co. in Michigan estimated that their upgrade (begun in 1993) would cost up to $45 million. The average Fortune 500 company was expected to spend $100 million. 6
Some effects were already being felt at the end of the 1990s, as systems were unable to handle expiration dates into the the 2000s. Scot E. Wilcoxon noted that a Minneapolis newspaper pointed out that five-year planning programs were already at risk in 1995. John Cavanaugh recalled seeing a Computerworld article in 1975, when some programs that did projections 25 years ahead started failing.7 In the United Kingdom, the Department of Social Security in 1996 postponed the ability of divorcing couples to split their pensions until the year 2000, because of the effects on the computer databases.8
Some lawyers are drooling over the expected lawsuits. Some hucksters are selling easy solutions. There is even a report of a Year-2000 Shark who was scamming businesses by offering to fix credit-card systems that allegedly would not work on a card with a year-2000 expiration date.9
Each leap-year or leap-second seems to bring a new set of old problems.
Leaping forward. John Knight reported that a Shuttle launch scheduled to cross the end-of-year 1989 was delayed, to avoid the software risks of both the year-end rollover and a leap-second correction that took place at the same time.
Not-making ends meat! Shortly after 1988 Feb 29, the Xtra supermarket was fined $1000 for having meat around 1 day too long, because the computer program did not make the adjustment for leap-year.
Leap-day 1992. The date of 1992 Feb 29 brought its own collection of problems. The following episodes are worth relating, all of which stem from 1992 and were described in SEN 17, 2, 10-12.
Paul Eggert's contribution for International Software Calendar Bug Day observed that Prime Computer's MAGSAV failed at midnight on leap-day. However, Prime's 800 number is not answered on Saturdays, so they probably did get as many complaints as might have occurred on a weekday. G.M. Lack noted that MAGSAV probably failed on leap-day, because it tried to increment the year by one to set a tape label expiration date, and the resulting nonexistent date 1993 Feb 29 threw it for a loop.
Jaap Akkerhuis reported that Imail caused systems to crash worldwide on leap-day 1992, because the mail handlers did not recognize the date.
Roger H. Goun reported that the PC MS-DOS mail
system UUPC/extended hung the PC on leap-day 1992.
Drew Derbyshire, the author of UUPC, traced the problem to a bug in the mktime() library function in Borland C++ 2.0, which converts a time to calendar format. Drew demonstrated that mktime() will hang a PC on leap-day, and reported the problem to Borland. As distributed, UUPC is compiled with Borland C++ 2.0, though source code is available for do-it-yourselfers.... Drew tried to warn UUPC users by mail after discovering the problem on Saturday. Ironically, many did not get the message until Sunday or Monday, when they found their PCs hung in uupoll.
Rhys Weatherley noted that a Windows 3.0 newsreader using Borland C++ 2.0 locked up, due to a bug in mktime converting to Unix date/time formats, although the problem may have been due to the run-time library.
Douglas W. Jones noted that all liquor licenses in Iowa expired on the 1992 Feb 28, and the new licenses were not in force until 1992 Mar 1. The state announced that this gap was due to a computer error and promised not to enforce the law on leap-day for establishments caught in the interim. (I suppose there might have been a headline somewhere such as "A glitch in time shaves fine.")
Tsutomu Shimomura parked on leap-day at the San Diego off-airport parking lot, and was given a time-in ticket dated 1992 Feb 30; returning on 1992 Mar 6, he was presented with a demand for $3771 (for 342 days @ $11/day, and 9 hours @ $1/hour). It is intriguing to contemplate why the computer program used 1991 Mar 30 as the time-in; it apparently kept the 30, but propagated a carry from Feb to Mar!
It ain't over 'til its over. A leap-year bug in an ATM program did not hit until midnight on 1992 Dec 31, causing several thousand ASB regional bank customer transactions to be rejected. Each magnetic stripe was corrupted during the period from midnight to 10 a.m., and anyone trying a second transaction was blocked.10 The same phenomenon also was reported for 1500 TSB regional bank customers in Taranaki, from midnight until after noon.11 Both banks used National Cash Register (NCR) ATM systems. This case was another example of New Zealand bringing in the new day first -- serving as the king's taster for clock problems (Conrad Bullock, in SEN 18, 2, 11) -- as in the Tandem Westpac case noted previously. NCR got a fix out fairly quickly, limiting the effect further west.
Leap-day 1996 in New Zealand. A computer glitch at the New Zealand Aluminium Smelter plant at Tiwai Point in New Zealand (South Island) at midnight on New Year's Eve 1996 left a repair bill of more than NZ$1 million. Production in all the smelting potlines ground to a halt at midnight, when the computers unexpectedly all shut down. General manager David Brewer said the failure was traced to a faulty computer software program that failed to account for 1996 being a leap year: the computer was not programmed to handle the 366th day of the year. The same problem occurred two hours later at Comalco's Bell Bay smelter, in Tasmania, Australia. (New Zealand is two hours ahead of Tasmania.) Both smelters use the same program, which was written by Comalco computer staff. Before the Tiwai problem could be fixed that afternoon, five cells had over-heated and were damaged beyond repair. Mr. Brewer estimated the replacement cost at more than NZ$1 million.12
Many of these problems stem from the absence of a requirement that the system be able to continue to work in the more distant future, and from the presence of short-sighted programming practice.
As elsewhere, there is a serious lack of system sense and consistent software practice with respect to date and time arithmetic. Obvious problems created by limitations on leap-year and date-time representations tend not to be anticipated adequately. Perhaps no one ever dreamed that FORTRAN and COBOL would still be around into the next century. The lessons of this section seem to be widely ignored. New systems continue to fall victim to old and well-known problems.
You might think that the leap-day problems would be anticipated adequately by now. Getting clock arithmetic correct might seem to be a conceptually simple task -- which is perhaps why it is not taken seriously enough. But even if earlier leap-year problems were caught in older systems, they continue to recur in newer systems. So, every 4 years, we encounter new problems, giving us 4 more years to develop new software with old leap-year bugs, and perhaps even to find some creative new bugs!13
Many computer programmers are concerned about the coming millenium, and speculations on what might happen are rampant. There are many lurking problems, some of which are suggested here.
The maximum-field-value problem would seem to be obvious enough that it would have been better anticipated; however, it keeps recurring. Distributed system clocks are a potential source of serious difficulties, as illustrated by the synchronization problem that caused postponement of the first Shuttle launch (Section 2.2). Defensive design is particularly appropriate.
Planning for the future is always important. It is high time that we looked far ahead. We still have time to think about the millenial problems, analyzing existing programs and developing standard algorithms that can help us to avoid alarming consequences. Events on 2000 Jan 01 should indeed be interesting, particularly because 2000 will be a leap-year, following the rule that 100-multiples are not leap-years -- except for the 400-multiples. Because there were no clock and date programs in 1900, problems caused by this particular rule may not arise until 2100 (or 2400).
From the Communications of the ACM, vol. 41, no. 9, September 1998:
Somewhere in the wide spectrum from doomsday hype to total disdain lie the realities of the Year-2000 problem. Some computer systems and infrastructures will be OK, but others could have major impact on our lives. We won't know until it happens. At any rate, here is a summary of where we stand with 16 months left.
Many departments and agencies of the U.S. Government are lagging badly in their efforts to fix their critical computers. The Departments of Transportation, Defense, State, Energy, and Health and Human Services are particularly conspicuous at the bottom of Congressman Stephen Horn's report card. The Social Security Administration seems to be doing better - although its checks are issued by the Treasury Department, whose compliance efforts Horn labelled "dismal."
The critical national infrastructures discussed in our January and June 1998 columns are increasingly dependent on information systems and the Internet. Public utilities are of concern, particularly among smaller companies. Aviation is potentially at risk, with its archaic air-traffic control systems. Railway transportation is also at risk. There are predictions that the U.S. railroad system will fail; nationwide control is now highly centralized, and manual backup systems for communications, switching and power have all been discarded. Financial systems are reportedly in better shape - perhaps because the risks are more tangible.
Many smaller corporations are slow in responding, hoping that someone else will take care of the problem. Developers of many application software packages and indeed some operating systems are also slow. Replacing old legacy systems with new systems is no guarantee, as some newer systems are also noncompliant. In addition, although some systems may survive 1 Jan 2000, they may fail on 29 Feb 2000 or 1 Mar 2000 or 1 Jan 2001, or some other date. Also, even if a system tests out perfectly when dates are advanced to 1 Jan 2000, there are risks that it will not work in conjunction with other systems when that date actually arrives. Even more insidious, some systems that tested successfully with Y2K-crossing dates subsequently collapsed when the dates were set back to their correct values, because of the backward discontinuity!
Estimates are widely heard about the cost of analysis, prevention, and repair exceeding one trillion dollars. Other estimates suggest that the legal costs could reach the same rather astounding level - perhaps merely reflecting the extent to which we have become a litigious society. There is a risk that some of the fly-by-night Y2K companies will pack up their tents and vanish immediately after New Year's Eve 1999, to avoid lawsuits. There are also some efforts to put caps on liability, in some cases as an incentive to share information.
Although a few hucksters are hawking quick fixes, there are in general no easy answers. There are also very serious risks to national, corporate, and personal well-being associated with letting other people fix your software - with rampant opportunities for Trojan horses, sloppy fixes, and theft of proprietary code. Considerable Y2K efforts are being done abroad.
Ultimately, Y2K is an international problem with a particularly nonnegotiable deadline and ever increasing interdependence on unpredictable entities. Reports from many other countries are not encouraging. Overall, any nation or organization that is not aggressively pursuing its Y2K preparedness is potentially at risk. Also, where pirated software abounds (as in China and Russia), official fixes may not be accessible.
One of the strangest risks of all is that even if all of the anticipatory preventive measures were to work perfectly beyond everyone's expectations, engendering no adverse Y2K effects, the media hype and general paranoia could nevertheless result in massive panic and hoarding, including banks running out of cash reserves.
The Y2K problem is the result of bad software engineering practice and a serious lack of foresight. Y2K has been largely ignored until recently, despite having been recognized long ago: the 1965 Multics system design used a 71-bit microsecond calendar-clock. (Java does even better, running until the year 292271023.) Innovative solutions often stay out of the mainstream unless they are performance related. For example, Multics contributed some major advances that would be timely today in other systems, although Ken Thompson carried some of those concepts into Unix.
Effects of noncompliant systems have the potential of propagating to other systems, as we have seen here before. Local testing is not adequate, and pervasive testing is often impossible. There is little room for complacency in the remaining months. Oddly, the Y2K problem is relatively simple compared to the ubiquitous security and software engineering problems - which seem less pressing because there is no fixed doomsdate. Perhaps when 2000 has past, we will be able to focus on the deeper problems.
The following list is excerpted from my complete list of illustrative risks (ftp://ftp.csl.sri.com/pub/illustrative.PS or illustrative.pdf). References are given to Software Engineering Notes (S vol number:page) and the on-line Risks Forum Digest (R vol issue). Searchable RISKS archives exist at http://catless.ncl.ac.uk/Risks/; all back issues are also available for anonymous ftp at ftp.sri.com/risks/.
In this list, the following code of descriptors is used to characterize each case.
! = Loss of life/lives; * = Potentially life-critical; $ = Loss of resources
V = Survivability problem in the face of a wide range of adversities
S = Security/integrity/misuse problem; P = Privacy/rights abuse or concern
H = Intentional misuse (e.g., user-administrator-operator-penetrator)
h = Accidental misuse or other inadvertence
I = Insider; O = Outsider; A = Inadequate authentication or access controls
d = System development problems
e = Improper maintenance/upgrade. (H,h,i,f,d,e involve human foibles.)
f = Flaws in system concept, requirements, design, hardware or software
implementation
i = Misinterpretation/confusion/human errors at a man-system interface
m = Hardware malfunction attributable to system deficiencies, electronic or
other interference, the physical environment, acts of God, etc.
+ = Beneficial; - = problematic with none of the above categories
..... Y2K Manifestations as we headed toward 1/1/00
f Discussion of date and century roll-over problems: Fujitsu SRS-1050 ISDN display phones fail on two-digit month (10); 1401 one-character year field; COBOL improvements; IBM 360 (S 20 2:13)
f Year 2000? Don't forget 1752 and 30 February 1712; two references (S 20 5:11)
$f Estimated costs of 1999-to-2000 date fix (S 21 2:16)
$f When the Clock Strikes 2000: U.S. Fed Govt cost $30 billion? Only 70% to be fixed in time? (S 21 5:18)
f$ More Y2K problems: Visa credit-card expiration-date problem and legal liabilities (R 18 63,74,75) $ Lawyers look forward to the year 2000 (S 22 2:23)
f$$ Lots more on Y2K (R 18 74-80,82,83-84,87-88), including the unforesightful use of "12/99" as an out-of-band date flag (R 18 88); more on Y2K and other date/calendar/daylight arithmetic problems (R 19 02,03,06,08-12); costs to reprogram in UK (R 19 07); Y2K cost estimates worldwide now up to $600 billion (R 19 10)
Vf More on Y2K: sliding window approach, year-2069 problem (R 19 13); (R 19 14); year 2106 and 2038 problems in Unix (R 19 15); year 65,536, leap seconds, UTC vs TAI (R 19 16); DEC OpenVMS expired on 19 May 1997 (R 19 18); clock synchronization (R 19 18); Java Y2K problem arises in the year 292271023 (R 19 21); Incidentally, Multics 1965 design used a 71-bit date-time field lasting long after Y2K
$f Y2K problem blocks UK divorcing couples from splitting pensions (S 22 1:19)
? Millenium fears lead to Virgin Birth insurance policies (in addition to alien impregnation policies) (S 22 1:19)
SH Nasty scam exploiting Y2K authorization expirations (R 18 68)
fde Y2K: Tcl 8.0 bytecode compiler Y2K risks; 00-38 now 2000-2038 (R 19 35-37); Y2K and C (R 19 37-38,40); non-Y2K problems with Java Date classes (R 19 38)
Vfe DoD Global Command and Control System (GCCS) fails Y2K test (R 19 38)
$ Y2K lawsuit: Produce Palace International sues Tec-America (R 19 29)
- American Megatrends: "Year 2000 compliance means that the internal BIOS date and time clock will continue above the date 1999. It will not reset itself after 1999 to the date of 1980. It will continue to the date of 2099 before resetting to 1980." (R 19 60)
H Y2K spam scam: jobs offered with no experience required (R 19 46)
- Ottawa firm registers "Y2K" as trademark (R 19 47)
f Freecell degenerates on erroneous date: Y2K and random numbers (R 19 48)
f Potential risks of backup and recovery after Y2K (R 19 55)
Vf PDP-11 Y2K leap-year bug with German clock board (R 19 56)
f Risks of testing Y2K by setting clocks ahead (R 19 56)
fe IRS Y2K fix threatens 1,000 taxpayers erroneously; IRS needs to check at least 62M lines of source code for Y2K (R 19 57)
$f More on Y2K lawsuits: North Carolina contemplating suing computer industry (R 19 57); California legislation proposed to limit Y2K liability (R 19 59)
f Canned-goods rejected with Y2K expiration dates (R 19 47,48)
f Miniature Enthusiasts with Y2K expiration dates deleted from address DB (R 19 57)
f Canadian guaranteed investment certificates with Y2K maturity vanish from DB (R 19 58)
e Euro changeover makes Y2K bug look easy! (S 23 4:23, R 19 69)
fh Gore congratulates 71-year-old Senator on birth of twins (S 23 4:23, R 19 64)
$ffffe... As of March 1998, only 35% of Federal Agency computer systems checked for Y2K compliance, 3,500 systems remain (R 19 64); IRS to spend $1B (R 19 68); Effects on the aviation industry (R 19 64); Financial risks (R 19 69); Y2K in Britain (R 19 64); Y2K in China (R 19 65); Australian simulated results of effects on public health and public infrastructure (R 19 71)
f Testing bugs that result from trying to test Y2K compliance, particularly when setting dates back to their correct value! (R 19 71)
f Report that only 1/3 of popular Microsoft apps are Y2K compliant (R 19 68) with further clarifications (R 19 69-70)
f Leap years: MS Excel 6.0 Office 95 version and 7.0 Office 97 version believe 1900 is a leap year (R 19 64)
Vf Summer time: in Britain (R 19 64), voicemail backup system fails (R 19 67); in Germany, Deutsche Telekom adjusted clocks twice in Lübeck (R 19 65)
f Year 2100 problems (AMI BIOS, R 19 60), IBM PCs and Network Time Protocol balk at 2100-Feb-29 (R 19 61,62);
? Fable of Y2K and 1979 Toyotas: shutdown if 00 in year field? (R 19 69,71); Computer insists on cataloguing Chateau Margaux 1900 as Ch. Margaux 2000 (R 19 67); Y2K and tombstones (R 19 61); Eagle Talons alleged Y2K problem (R 19 68) bogus (R 19 69)
- Need for contingency plans, not just questionable remediation (R 19 85,88-89)
fh$, etc. CIA worries about Y2K as an opportunity for hostile intent (R 19 84); Senate considers need for martial law after Y2K breakdowns (R 19 78); Potential Y2K railroad problems, with no more manual backups (R 19 84); Y2K risks to world shipping (R 19 82); Senate Y2K committee suspects power grid could collapse (R 19 82); Wells Fargo study shows millions of small firms at risk (R 19 77); Y2K insurance and financial risks (R 19 79); Swedish corporate insurance explicitly excludes Y2K (R 19 78); Y2K problem in Swedish personal identification numbers (R 19 74); Microsoft Y2K (non)compliance; Excel believes in 29 Feb 2000 and 29 Feb 1900, for Lotus 1-2-3 compatibility (R 19 73); More on Y2K forced-upgrade strategies (R 19 73); 102-yr old gets a birthday card for 2-yr olds (R 19 73)
f$ SIR-C processor Y2K problem in shuttle imaging (R 19 81) and its economic implications (R 19 83)
- Y2K priorities delayed security upgrades at bombed U.S. embassies (R 19 93)
+ UK Railtrack (former BritishRail) has no safety-critical computer systems, because of past underfunding! Y2K preparations simplified by delaying upgrades! (R 19 90)
f Sloppy date handling in Perl scripts (R 19 88)
+ Wall Street test simulation gives good marks to 29 brokerage firms (R 19 89)
$ No UK Y2K insurance for household electrical items (R 19 89); Canadian insurance not likely to pay off; also another 100+ year anomaly, from Rob Slade! (R 20 03)
+ Canadian RCMP blocks vacations to ensure Y2K emergency coverage (R 20 02); Wisconsin National Guard mobilizing (R 20 03)
- White House calm, DoD nervous about Y2K (R 19 90)
f Win98 date problem detected (R 19 91) occurs in 5-second window when booting around midnight (R 19 92)
f Internet Explorer 4.0 instructions on how to bypass firewalls! (R 20 01-02)
f Y2K risk in Javascript cookies despite 4-digit standard (R 20 01)
Vf Consignment of corned beef with intended expiry 2001 rejected as too old: appears as 1901 (R 19 92); cf. the leap-day Xtra supermarket meat problem, below! (There's more than meats the eye.)
$SPH Business Software Alliance finds 1400 unlicensed software copies in Los Angeles Unified School District, valued at $5M? (R 19 92)
$ Clothing retailer sues for cost of non-Y2K-compliant 1991 system (R 19 94)
Vf$ Product Palace settles with Tec America over Y2K-noncompliant software: entire system crashed on single 00 credit-card (R 19 96)
fe Saga of another Y2K bug, after being "fixed", a letter dated 3 Aug 2098 sent mistakenly to half-million recipients (R 19 95)
+ Y2K problem resolved that had threatened the production of scotch whiskey (R 20 03)
*$ Y2K-related panic may be more serious than Y2K computer problems (R 20 11)
fH DoD Special Weapons Agency falsely claimed successful Y2K tests on 3 or 5 critical systems (R 20 10)
$(f?) Hospital spends $700K on new digital nuclear medicine machine because vendor would not certify Y2K compliance of well functioning analog machine (R 20 10)
f New Vancouver Hospital pathology system default misses updates to patient files for many months (R 20 23); CORRECTION (R 20 30)
*f UK MoD admitted that Rapier anti-aircraft missile was not Y2K compliant (R 20 13)
The number of new reported cases seems to be rapidly expanding as the day approaches! Perhaps that means more flaws are being found (and repaired?)?
- 400-year-old time machine (in Liverpool museum) to suffer from millennium bug: time runs out at year 2000! (R 19 79,81)
+ Memo on Y to K conversion: Januark, Februark, ... (S 20 23:, R 20 21)
*$ Y2K-related panic may be more serious than Y2K computer problems (R 20 11)
fH DoD Special Weapons Agency falsely claimed successful Y2K tests on 3 or 5 critical systems (R 20 10)
$(f?) Hospital spends $700K on new digital nuclear medicine machine because vendor would not certify Y2K compliance of well functioning analog machine (R 20 10)
*f UK MoD admitted that Rapier anti-aircraft missile was not Y2K compliant (R 20 13)
$f 1 Jan 1999: Y2K hits Singapore and Swedish taxi meters (R 20 15)
f Windows/Visual C++ daylight saving cutover one week early on 1 Apr 2001 (no fooling!), affecting 95,98,NT (R 20 15-16)
ef Enator AdeEko Y2K update turns Malmo Sweden seriously disrupts city's bill paying (R 20 18)
+? China contemplating making all airline executives fly on Y2K boundary (R 20 17)
fm Store Baelt Bridge not Y2K-safe (R 20 22-23)
h 2,000 Texans get false overdraft notes from Bank One in Y2K test (R 20 13)
h Y2K "fix" test results in traffic offenses dated 2097 (R 20 21)
$h PSE&G Y2K test of billing program results in false billing (R 20 23)
- As of early 1999, GAO report says U.S. states lagging in Y2K readiness (R 20 20); CIA predicts serious Y2K problems around the globe (R 20 23)
*h VPA's Peach Bottom nuclear-power Y2K-check crashed monitoring systems (R 20 24)
fh Boston bank's Y2K problems blamed on IE5, but apparently not! (R 20 25)
+? Sri Lankan Banks to close on 31 Dec 1999 for Y2K tests (R 20 25)
SH Scam using Y2K bank problems as bait
+ Standards needed now for Y10K? (not April Foolery) (R 20 30)
f Nottingham weather images dated "FEB 28 2000, 2330" and "FEB 28 2000, 2400": Y2K leap-testing? (R 20 42)
Vf Y2K test knocks out Fiji's telecommunications (R 20 43)
f Downloading Y2K fixes to Internet Explorer leads to clock problem (R 20 42)
*f Y2K test sends sewage flowing in Los Angeles (R 20 46)
S Another Y2K scam (R 20 51)
Vfe* London Electricity Y2K upgrade left 2000 customers without power for days (R 20 54)
+P Canadian govt recommends encrypted e-mail (R 20 54)
? Y2K in China (R 20 55); Indonesia: wait to see what happens in New Zealand and fix it quickly (R 20 58); Iraq decides to wait and see on Y2K oil disruption; concerns that oil nations are not ready (R 20 62)
f$ Northwest Metrology stung by Y2K bug (R 20 56)
f? Unix needs 10th decimal digit for timestamp on 9 Sep 2001; risks of format problems? (R 20 58)
f Bank switch to 4-digit years blows up on 1 Oct 1999, with 10/01/1999 truncated to 0/01/1999 (R 20 59)
SHf FBI warns some Y2K fixes may be suspect (R 20 61); general concerns over Trojan horses in Y2K-remediated code
fe NT, SP5, SP6 Y2K problems (R 20 62)
f Maine year-2000 vehicles classified as "horseless carriages" from 1900 (R 20 63-65)
f Cornell University registration system welcomes students to the spring 1900 semester (R 20 64)
$? Businesses could owe millions for Y2K sliding-window fix if 1998 patent holds up, despite this being an old technique (used at least in the 1960s) (R 20 65)
$ Microsoft Y2K liability claim: "... Microsoft does not warrant or make any representations regarding the use or the results of the use of any Microsoft Year 2000 statement in terms of its correctness, accuracy, reliability, or otherwise." (R 20 65)
$efh Irish telephone network upgrade failed, backup failed, caused domino propagation in Dublin; independent cell-phone system failure; outage brings Y2K fears of lack of disaster recovery (R 20 66-67)
efh $.5M fire-station fire blamed on Y2K computer fix; breaker disabled due to Y2K incompatibility (R 20 66)
f UK Railtrack online timetable information has errors for for holiday weekend schedules Xmas 1999 and NewYear 2000 (R 20 67)
efh IEEE standard Y2K compliance attained by rendering software unusable (R 20 67)
SH Flagrant antisocial behavior of Y2K virus competitions promotions (R 20 68)
SH Y2K-related viruses: Worm.Mypic (R 2067), W95.Babylonia and others (R 20 69)
fm Y2K test takes out all power in German Department of Justice, 11 Dec 1999 (R 20 69)
$H Y2K fears lead Philippine man to withdraw his life savings, then robbed of everything (R 20 69)
fff Australian Y2K readiness news page clock sticks at 31 Dec 1999 23:56:15, then 15 Dec 1999 00:23; New Zealand airport Web site update time-stamped 1 Jan 100; in Y2K test, Henderson NZ clock flashed "GAME OVER" at midnight; 20,000 UK credit-card machines incapable of coping four days before Y2K, with settlement date in 2000; Pentagon DefenseLINK Y2K info site accidentally disabled; Oakland CA 911 system not Y2K compliant, prioritizes earliest calls (seemingly from 1900); glitch with NIST's Automated Computer Time Service; Wells Fargo CD renewal notices dated 1900; many digital certificates expire with Y2K because old browsers could not accept 2000; date field called Shirley harder to detect; risks of last-minute FAA HOCSR patch; more on risks of leap-second corrections at Y2K (R 20 71)
..... Y2K Manifestations at Y2K
On the whole, Y2K came and went without major immediately noted problems. Predictions still abound for deferred problems. But there were many reported Y2K weirdnesses, itemized below.
There are a lot of lessons that should have been learned from the entire process. My preliminary conclusions are not surprising: there has been a pervasive lack of foresight, generally lousy software engineering (especially in application software), overemphasis on the remediation process without deeper understanding, obliviousness to the risks of remediation by potentially untrustworthy third parties (with various Trojan horses discovered in remediated code), and so on. A retrospective consideration of the Y2K problem and lessons that should be learned is in my Inside Risks column in the March 2000 CACM, which is also on-line at http://www.csl.sri.com/neumann/insiderisks.html .
f Y2K dates: Numerous cases of 1 Jan 100, 1 Jan 19100, Jan 1 2100. An Australian online media news gateway had 3 Jan 3900 on 2 Jan 2000, while appple.com and happypuppy.com should get a prize for year 20100, which beat out the U.S. Naval Observatory calendar with the year 19,000 and others with 19100. Amazon announced a Sonic Youth CD would be available on 10 Oct 2011. Startrekcontinuum.com noted the next Voyager episode would air on 1 Jan 1900. *The New York Times* Website said 1 Jan 1900. Compaq sites said it was 2 Jan on 1 Jan. Several counts of time until Y2K went negative in funny ways. An old 486 PC reset its clock to 4 Jan 1980. The atomic clock at UK's NPL read 31 Dec 1999 27:39 UTC at 2:39am GMT (off by an hour, at that). Various Web sites were hacked. www.2600.com had a humorous spoof. Toronto abandoned its on-line bus information service at midnight because it was not Y2K compliant. (R 20 72)
feh A Pentagon computer system processing satellite intelligence data lost its capabilities at midnight GMT, for 2.5 hours, due to preventive human mistake (R 20 72); data from 5 satellites was reduced to a trickle for several days (R 20 75)
f Automated New Zealand radio station repeats 31 Dec 1999 11pm news hourly, due to 99 > 00; Nokia phone not Y2K compliant?; effects on mobile and phone nets; more on cost of Y2K fixes vs. preventive measures; Filemaker Pro; Word Perfect 5.1 and medical transcription; lots more on bad arithmetic date programs, including Javascript problem; X-10 controller; New York Times correcting 102-year-old issue number glitch; nuclear-power glitches; Win95 Y2K bug?; California DMV snafu; ftp date problems; Talking Clock; count-down to Y2K programs go negative (R 20 73)
f Y2K: repeated billings result from uninstalled fix; Bills for 100 years back interest; Sprint PCS network problems at Y2K; MKS Toolkit Y2K glitch: next backup 9 Jun 2005!; Barbara's Cereal expires July 1900; driver's license expires in year 1000!; NTSB website has Y2K test data mixed in with real data; Bogus message in live service for Quicken 2000; With stepped-up Y2K wariness, NAI WebShield blocks RISKS issues (R 20 74)
..... Leap-Year Problems:
f Clock problems - Leap Day, end of century, etc. (S 13 2)
f 1988 leap-year: Xtra supermarket fined $1000 for one-day overage on meat due to program skipping 29 Feb
f 1992 leap-year problems: 29 Feb invalid but 1 Mar gets correct day; Prime's MAGSAV fails, probably because one-year expiration date 29 Feb 93 invalid; Imail dies worldwide; UUPOLL on MS-DOS due to bug in Borland C++ 2.0; Windows 3.0 locks up on mktime call; glitches in watches; Iowa state liquor licenses expired on 28-Feb, new ones started 1 Mar; leap day waivered. (S 17 2)
f Airport parking bill for $3771 at $11/day with time-in 30 Feb 92 (S 17 2)
$Vf 1992 leap-year-end clock bug blocks ATM machines on 1 Jan 1993 (S 18 2:11)
f Many more calendar, date, and time problems - particularly surrounding Leap-Day 1996 and 1/1/00; Arizona lottery downed, insurance policy problem, leap-year algorithms, Excel 5.0, WIN95, and lots more; Persian gulf support problem; the business of fixing the year 2000 problem The length of the tropical year at present is about 365.24219 days. The present algorithm (not if divisible by 100 unless divisible by 400) works out to 365.2425 days, with an error of three days every 10,000 years. Expect a closer approximation in another few thousand years. (S 21 4:15-16)
$Vf Leap-Year software bug at Tiwai Pt aluminum smelter halts potlines, costs NZ$1M (S 22 4:29, R 18 74)
f
time_t offset from 1900 in C led to leap-year mistake on 2000
in Plan 9 (R 20 31)
..... Summer Time (and the livin' is queasy):
@$fe GTE Sprint billing errors from botched daylight savings cutover (S 11 5)
@Vhf Daylight savings time changeover halts train for an hour (S 15 3)
f Hawaii not on daylight time; off-island program messes up rush-hour (S 17 3)
i Some UNIX systems missed daylight savings end (US/Pacific-New) (S 18 1:5)
f More daylight savings time problems (R 18 02-05)
fi Windows 95 daylight saving confusion in Sweden (R 18 50)
*f Summer-time cutovers splatter molten ingots, down police system (S 18 3:A4)
f Daylight-savings: falling back 1997 - VCRs, Interac ATMs, Win95 (R 19 43,44)
f New remote-synch radio clock blows daylight savings changeover (R 20 03)
f Another VCR Summer Time screwup (R 20 29)
!h Terrorist bombing botched due to daylight time difference between Israel and Palestine (R 20 58)
..... Other Calendar-Clock Problems:
Vf GPS will roll over to 6 Jan 1980 at the end of 21 August 1999 [after 1024 weeks of counting] (R 18 24); More on older GPS receivers with 10-bit week-counter rollover on 21 Aug 1999 (R 19 73); Pioneer recalling GPS receivers (R 19 80); British Civil Aviation notice on GPS receiver rollover on 22 August 1999 after 1024 weeks (R 20 07); GPS clock rollover affected Tokyo taxicabs (R 20 55), the yacht Tam-o-Shanter (R 20 55), and some DoD weapons systems (R 20 62). Pioneer adapted or replaced 210,000 of 270,000 GPS receivers (R 20 55)
$h Human input error on year causes $49-million error for NJ food stamps (S 24 4:, R 20 28)
Vf Swedish passport system and Swedish Giroguide both fail on "99" (R 20 14)
fh Y2K-like problems include stop-codes such as 9999 (9 Apr 1999 is the 99th day of "99"), 99999999, etc. (R 20 14)
f$ 9/9/99 was mostly a non-event, although it resulted in a non-critical medical app failure (R 20 55) and an accidental deposit of $160K (R 20 60)
hm Japanese MARS rail-ticket system crashed due to customers wanting tickets bearing an 11/11/11 11:11 time stamp on 11 Nov 1999, year Heisei 11 of the current emporer (R 20 65)
Vf Swiss hospital computers crash on 1/1/1999 (R 20 16)
f Quicken'99 divide-by-zero bug on Jan 1999 dates in Auto category (R 20 16)
f Clock-setting algorithm gets wrong time; other clock problems (S 11 2)
f Hidden horrible bug in Grapevine mail system lurks for 5 years (S 12 1)
f Democratic bug in AppleLink in Chile, reserved word "General" (S 15 3)
f Apollo workstation date bug coming soon (S 22 4:30, R 18 78)
Vf Windows 95 will crash in 2038 (R 18 84)
f Microsoft Outlook e-mail Word problem (R 19 23)
f Thanksgiving misplaced in Microsoft Outlook 97: better check your calendar! (S 23 3:24, R 19 46:47)
f Microsoft Outlook 98 reschedules Memorial Day 1999 (R 20 30) and 2 UK bank holidays (R 20 32); not Y4.501K compatible (R 20 31)
fi Design flaw in MS Outlook/Word save procedure? (S 23 3:26, R 19 55)
f Outlook Express date parsing problem: 2099 by mistake, but displayed as 1919 (R 20 24)
fi Confusion of Microsoft Outlook shifting times with timezones (R 20 11-12); Windows 95 changes date without confirmation (R 20 13)
V Multics crashes on Bernie Greenberg's 45th birthday; Bernie never anticipated Multics would still be running! (S 20 5:11)
ehi Dartmouth Time Sharing System: Beware the Ides of March, 1970s (S 21 2:16)
Vf Misdeclared variable type overflows term
program on 26 Oct 1993 (S 19 1:4)
Vf Microcode bug downs Tandem CLX clocks at 3pm 1-Nov-1992; detected in New Zealand/FarEast, fix available before it could hit Europe, US (S 18 1:5)
Vf Every MTS shuts down: 2**15 days from 1 Mar 1900 to 16 Nov 1989 (S 15 1) @*f 100 hospital computer systems die; 2**15 days after 1 Jan 1900 (S 14 6)
f National Semi chip flaw persisted, 1987-1990; skips a day (S 16 1)
Vf NOS/BE clock failed after 2 years when the system 1st was up 24 days! (S 16 2)
VSf Security bug hung Tandem systems worldwide 27Aug91, 4:22pm local (S 16 4)
f Errant `timed' propagates effect, wrong date then skips 2.8 years (S 17 2)
f AT&T PC date problem in AT&T 6300; 5-year max lifetime assumed! (S 17 2)
Vf MOSS graphics systems crash on 15 Jul 1993 worldwide (S 18 4:3)
f Ball Aging Analysis SW clock bug prevents plotting of new data (S 19 2:2)
ehi A glitch in time shaves U.S. Naval Observatory (S 21 2:16, errata R 17 65)
hh Erroneous bank-clock coincidence puts wrong photo on Crimestoppers (S 21 2:16)
@M Clocks leap forward gradually. Power line interference! (S 16 2)
Vf Hospital computer crashes every midnight at midnight until 00:15 (R 19 25)
mfe AOL off line for two hours 29 Oct 1997 (R 19 44); AOL e-mail outage due to software, 3 Nov 1997 (R 19 45); more AOL e-mail outages, 18 Nov 1997, Internet outages 19 Nov (S 23 3:24, R 19 47,49)
Vf Windows 95, Windows 95 OEM Service Release, Windows 98 hang after 49.7 days (= 232 milliseconds, Vtdapi.vxd problem) (R 20 24)
f Overzealousness problem in Access on location-dependent date interpretation (R 20 31-32)