Microsoft outage brings "blue screen of death" to computer systems across the Bay Area
A massive Microsoft Windows failure that crashed computers around the world early Friday morning has caused flight cancellations and delays and impeded court and government functions throughout the Bay Area.
Sometime after 2 a.m. Pacific Time on Friday, a worldwide cyber security company called CrowdStrike initiated a software update to a product called Falcon, which caused countless client computers and servers running Windows to crash.
"It's called the 'blue screen of death,'" said Levent Ertaul, a cybersecurity expert and chair of the Computer Science Department at California State University, East Bay. "Imagine suddenly all the Windows computers and servers and client computers in big companies have this screen. It's affected hospitals, airports, 911 calls, courts, schools, banks, public transport, you name it."
CrowdStrike CEO George Kurtz released a statement on social media at about 2:45 a.m. saying the failure wasn't due to a cyberattack and that the problem was "identified, isolated and a fix has been deployed."
Microsoft CEO Satya Nadella said his company is working with CrowdStrike to help customers with "guidance and support."
Throughout the morning Friday, however, Bay Area organizations and public services were reporting various degrees of interruptions as a result of the bug.
For example, a San Francisco International Airport spokesperson said that while all its systems were back online, residual impacts resulted in 87 flight cancellations and 196 flight delays.
Also, superior court services were hindered in at least two Bay Area counties, but court hearings and other services weren't canceled.
In Santa Clara County, IT crews worked overnight to try to get computers running in courtrooms, in safety and security systems and at the court clerk's public service window.
A spokesperson for Alameda County courts said people are required to attend their court appearances despite the computer meltdown, which primarily downed case management systems and dozens of individual computers.
"We are diligently working to repair our systems," Presiding Judge Thomas Nixon said. "Our dedicated staff from our office of information technology to our clerks will continue to ensure our community has access to the Courts."
Several city and county governments were also scrambling to implement system repairs Friday, including Contra Costa County, which reported problems with its business operations, remote access and library services.
"We are working to minimize those impacts as CrowdStrike resolves their broader ongoing security issues," county officials said on social media Friday morning.
The alarming glitch also hit Monterey County, which has since resolved the problem in several core systems, including at the county's main hospital.
"Natividad Medical Center experienced minor disruptions in the early morning hours of Friday, July 19th. However, these issues have been promptly resolved, and all workstations are functioning normally," county officials said in a news release. "Hospital operations continue without interruption, and patient care remains uncompromised."
City officials in Oakland said "many computers" had crashed but 911 service was operational.
Nationally, the U.S. Social Security Administration said all of its offices will remain closed Friday due to the software failure, but people can still use its phone line at (800) 772-1213 or its website at www.ssa.gov.
Ertaul said cloud computing functions and servers will likely recover before client computers, since each computer will have to be dealt with individually and in person by IT professionals.
He also said the world should ready itself for similar tech implosions in the future.
"This thing shows us two very important things, first of all how dependent we are on technology and the second point is how fragile that dependence is, those are the lessons we have to learn," Ertaul said. "Things are getting too complex and not easy to manage, and I think we should get ready for this kind of problem in the future."
Also, while the massive Windows outage wasn't the result of a cyberattack, it could lead to security vulnerabilities in large computer networks.
"Does it create a weakness in the overall security perimeter? It might and someone might exploit that downtime in security," Ertaul said.
"What struck me is how brittle our systems were," Professor Shankar Sastry of UC Berkeley told KPIX.
"In the broader scheme of things, we ought to have been able to isolate, even compartmentalize the damage. But the damage, you know, spread across the world."
Sastry said mistakes do happen, and that's the reason for "sandboxing," or testing a software update in a contained environment, especially something used on this scale.
"But having a misconfiguration which has not been caught is really mind-boggling to me," Sastry added.
"Didn't you apply the upgrade on any other machine before sending it all around the world?" asked Ertaul. "There's a wow factor. This shows us how our computing systems, as of today, are interconnected and complex. It is vulnerable. That's very important. It's vulnerable to a mistake."
It's a vulnerability now embedded in the very systems we depend on, and often take for granted - until they fail.
"That is our lives, unfortunately," Ertaul said of the situation. "We will see these things more in the future. Because things are getting more complex and things are getting more interconnected."
"The number of incidents will go up," Sastry predicted. "The severity of the incident is what we need to think about."
Sastry said the crash is a sign that tech producers have prioritized speed and convenience at the expense of resilience, and that may need to change.
"This really was, on that scale, truly a wake up call for us," Sastry said.