Investigators have reproduced the network overload linked to the Federal Aviation Administration’s November 13 nationwide departure halt in a Florida laboratory, CNBC reported November 18, citing people familiar with the investigation. They still hadn’t established exactly why two routers failed. Source
The halt to departures lasted more than four hours, and the disruption took days to unwind. Scientific American reported more than 18,000 flight cancellations and more than 28,000 delays in the aftermath. Source
What the router did—and what remains unknown
According to CNBC, the incident followed contractor L3Harris Technologies’ installation of a new router at an FAA facility in El Dorado, Arkansas. Investigators said the router began transmitting excessive data within eight minutes of installation. Transportation Secretary Sean Duffy described a surge of 1,000 connection attempts every four to five seconds that destabilized aircraft communications and radar feeds. Source
CNBC’s sources said finding the root cause is critical to determining whether other routers in the national air transportation communications network could have a similar problem. Source
Duffy and FAA Administrator Bryan Bedford have said there is no indication the routers failed because of a cyberattack, CNBC reported. Source
The work was part of a project to replace aging copper communications wiring with fiber-optic cable. Scientific American reports that Duffy said it remained unclear whether the problem involved a faulty router or the crew installing it. Source
Modernization needs more than new hardware
“Like most engineers, I feel like we’re not getting the whole story here,” George Mason University transportation engineer and former Navy fighter pilot Missy Cummings told Scientific American. She also questioned attempting a challenging upgrade on a busy Friday afternoon. Source
University of Washington computer engineering professor Ratul Mahajan criticized the limited disclosure but was less troubled by the timing. “One way to make systems resilient is for changes to not be special and to be putting processes in place so that you can do updates all the time,” he told Scientific American. Source
Leaving everything untouched isn’t a comfortable alternative, either. Northeastern University researcher Michele Polese told Scientific American that changes introduce a risk of breaking things, while equipment that vendors no longer sell or support becomes harder to repair. Source
In September, the Government Accountability Office warned that the FAA was managing thousands of modernization projects without an integrated schedule to coordinate them, according to Scientific American. Duffy raised his upgrade funding request to Congress from $30 billion to $50 billion after the ground stop. A bigger dollar figure doesn’t answer the router-level questions. Source
Ahead of Thanksgiving, Scientific American reported November 19 that the FAA said its system was stable and had paused installations of the router type involved. Source
Comments