Encoded Ventures Management LLC

11/15/2026 | Press release | Distributed by Public on 11/16/2026 01:25

TDM is so back: What Actually Broke at the FAA

On Friday afternoon the FAA stopped every departure in the country for 4 hours and 21 minutes, from 3:54 PM to 8:15 PM Eastern, and by Sunday more than 20,000 flights had been disrupted. The official explanation has moved from "a telecommunications outage" to "overloaded routers" to "1980s copper," and as someone who has spent a career investing in networking and edge infrastructure, I found the coverage frustrating: plenty of airport photos, almost nothing on what the network actually did.

So I went through the public record: the DOT Inspector General's letters, the FAA's 2023 safety review, the agency's own planning documents for the network's replacement and a set of 2017 industry white papers. There is no root-cause analysis yet and I have no inside information. But the documents describe the architecture well enough to say something useful, and they show that a very similar failure already happened in November 2009.

Three official stories in 24 hours

The account changed twice between Friday night and Saturday morning, and the three versions don't fully agree with each other.

On Friday, Transportation Secretary Sean Duffy said routers got overloaded with data and started returning inconsistent information, so the FAA shut things down, and he blamed old switches and copper. The same evening, CBS News cited multiple sources saying an update to the private network that connects air traffic control caused a reboot and then a crash, and Homeland Security Secretary Markwayne Mullin said new patches and equipment had overwhelmed the older system.

On Saturday, Duffy got specific. L3Harris, the contractor that has run this network since 2002, was switching telecom lines from copper to fiber at a remote FAA site, and a router involved in that work started flooding other routers with "over 1,000 connection attempts every four to five seconds." He said officials haven't determined whether it was installer error or a faulty router.

On Sunday the FAA named the sites: a radio communications site in El Dorado, Arkansas, where L3Harris had been deploying a new router, and a radar site in Amarillo, Texas. The agency said the flood destabilized 17 of its 22 en route centers, and local reporting says the network recovered once the Arkansas and Texas routers were disconnected from it. The same reporting says the El Dorado router was made by HPE's Juniper Networks, which is hard to square with CBS's earlier sourcing on "Cisco 600 series" routers built between 2001 and 2006. My guess is that CBS's sources were describing the legacy routers on the receiving end, if the model number is right at all (the 600 series was a DSL CPE line). L3Harris says it is supporting the investigation and has finished more than 70% of the copper-to-fiber conversion, and Duffy says both routers have gone to its Florida headquarters for diagnostic testing.

First signs of instability in radar communications showed up at 3:36 PM, 18 minutes before the ground stop.

How the network is actually built

The network is called FTI (FAA Telecommunications Infrastructure). L3Harris owns the equipment and leases the circuits, connecting over 4,400 FAA sites with more than 29,000 services, and its NOCC monitors over 100,000 devices.

The design is better than the "1980s copper" headlines suggest. Per the FAA's own FTI-2 planning paper and a 2017 ACT-IAC white paper, there's a private optical backbone of 2.5 and 10 Gbps waves leased from roughly ten carriers, landing in the 21 cities that host en route centers. Each airspace has two PoPs in carrier colocation, and the big facilities reach them over dual SONET metro rings. Remote sites (radars, radio sites, navaids, over 3,000 of them) hang off leased circuits from about 200 carriers.

The part that matters for Friday: the Operational IP network runs a dual-core architecture, where each core is a separate set of routers running a completely isolated routing protocol. The stated purpose is to stop a fault, including routing table corruption, from propagating system-wide. The FAA's June 2015 paper says the contractor had "recently" added the second domain, which reads as a response to the 2009 outage described below, and in the same document the agency worries that this level of independence may not survive a move to shared networks. None of the public documents name the protocols or the router vendors.

FTI topology reconstructed from public documents

Read it top down: a remote site's traffic has to climb through access, the backbone and one of two isolated cores before it reaches a center, which is why a single edge router reaching 17 centers is the surprising part.

One caveat: these documents date from 2015 to 2025 and the white papers are industry advisory views, not as-built records, so treat this as a reconstruction.

Yes, it's literally TDM

When officials say TDM they mean it. The FAA's FTI-2 program objectives paper put about 87% of FTI services on TDM, and the edge interfaces are what you'd expect from that era: EIA-232 serial at 2.4 or 9.6 kbps, DDS-56, T1 and fractional T1, and four-wire analog voice. Of roughly 26,000 services, about 11,000 are full-period point-to-point voice and about 1,500 are surveillance feeds.

It goes deeper than the edge. A 2017 ACT-IAC paper on the TDM-to-IP transition says the on-site equipment was designed around carrier-leased TDM circuits, and that even IP services handed off on Ethernet rode a private TDM infrastructure. So historically this was IP over TDM, not the other way around.

The copper-to-fiber migration inverts that. The voice switches and radar processors at each end still expect serial and TDM interfaces, so the new fiber or Ethernet transport needs a gateway that emulates the old circuit. The same paper lays out pseudowires as the conversion path, with SAToP or CESoPSN as the encapsulation, and flags timing as the hard part: TDM wants Stratum 1 traceable clocks, so you need SyncE or PTP, and jitter buffers to absorb packet delay variation. It also warns that over carrier Ethernet FTI can't guarantee that traceability, and recommends running its own timing instead of trusting the carrier's.

The paper calls the design "Network Adaptation," and it's deliberately a last-mile fix. The existing multiplexers, DACS and routers at the site stay, the FAA-facing handoff doesn't change, and a new stack of pseudowire, encryption and firewall devices goes in at the carrier-facing side, with 10 or 100 Mbps Ethernet to a primary and an alternate serving wire center. Its four authors come from Harris, Cisco, CenturyLink and MITRE, so I read it as close to the as-designed approach, though I can't confirm what L3Harris actually deployed.

The carriers are forcing the pace. The 2023 NAS Safety Review Team report said the phone companies no longer support the TDM lines FTI runs on, and that the FAA's own July 2023 risk assessment rated FTI at its highest risk level. As of May 2026, GAO counted 2,560 of 5,170 copper lines replaced, with completion pulled forward 21 months to December 2028. That left roughly 2,600 lines to convert in 31 months, or about 20 cutovers a week, each one inserting a new router into a domain that used to see a dumb circuit.

The same GAO report describes how that pace is being managed. In August 2025 the FAA Administrator issued a blanket waiver exempting all of the modernization programs from nearly all of the agency's acquisition management requirements, and in December 2025 Peraton came in as prime integrator (about $876 million negotiated, against a $1.5 billion ceiling) with authority to manage the existing contractors on the FAA's behalf. Congress appropriated $4.75 billion for telecommunications against an FAA estimate of $5.91 billion. GAO counted 11,389 project schedules with no integrated master schedule, and warned that schedule pressure on federal programs leads to skipped or compressed testing. None of that establishes what happened Friday, but it's the environment the cutover happened in.

We've seen this movie: November 19, 2009

Seventeen years ago, almost to the week, the same network had the same kind of failure. Per the post-mortem as reported by eWeek, Harris installed a replacement router during planned maintenance, and that router carried a route error in its configuration. The error broke route translation at the logical "bridge" between the older IP ATM network and the Operational IP backbone at the Salt Lake City hub, which blocked three-quarters of the IP routes across both networks.

The DOT Inspector General put it at about 75% of FTI traffic blocked and five hours to restore, and attributed it to network configuration and procedural errors by the contractor. The line that reads differently this weekend: the FAA and Harris acknowledged then that the risk of future outages would remain as new services were added to FTI's fiber network.

Two things were different in 2009. Radar and air-to-ground communications stayed up (it was flight plan data that broke), and the failure was a hub router, not a remote site. Friday's event was worse on the first count and stranger on the second.

My read

This section is inference, not reporting, and I could easily be wrong on the specifics.

Start with the one hard number. 1,000 connection attempts every four to five seconds is 200 to 250 per second, should be no problem to handle. It only hurts if each attempt gets punted to the route processor on hardware with little or no control-plane policing, and CPUs from the early 2000s don't have much headroom. So I think this was a control-plane storm, not a data-plane flood: one new or misconfigured node repeatedly trying to bring up sessions with routers that couldn't afford to answer.

"Connection attempts" maps most literally to session setup, and the migration design puts an encryptor and a firewall at every converted site. So my first guess is a tunnel or pseudowire session storm (IKE negotiations or pseudowire control sessions misprovisioned toward hubs), with BGP or LDP session churn (a bad peer list, a duplicate router ID, a wrong template) second. It's also exactly the kind of counter a firewall reports. An IGP adjacency flap or a Layer 2 loop during cutover, with the old copper path and the new fiber path both live, are possible but fit the wording less well.

The Amarillo detail matters here. Two sites showing the same excessive traffic looks more like a shared configuration or software problem than one bad box, though it's also possible Amarillo was a victim reacting to El Dorado and not a second source.

The symptoms fit that picture. Once route processors starve, keepalives drop, adjacencies flap and paths reconverge over and over. Circuits emulated over packets are intolerant of jitter and reordering, and sources described exactly what that looks like downstream: static on voice lines and displays that flashed or went black. A timing fault is a second candidate for the same symptoms, since emulated circuits need a Stratum 1 traceable clock, but it wouldn't explain the connection-attempt count.

The question I keep coming back to is how a radio site in southern Arkansas reached 17 centers. The dual-core design exists precisely to prevent that. Either the migration has collapsed the separation between the two routing domains somewhere, or the fault sat in a layer both cores share, such as access or aggregation gear, or a boundary where routes are translated between the legacy and new domains. That last one is exactly what failed in Salt Lake City in 2009.

If that's right, the lesson isn't "old copper." Duffy himself said the new IP switches didn't go down. The fragile part is the seam between the network being retired and the one replacing it, and the FAA is crossing that seam about 20 times a week.

What I'd ask L3Harris and the FAA

  1. Which protocol were the "connection attempts," and which devices were on the receiving end?
  2. Are the two Operational IP routing domains still isolated end to end, or do migrated sites touch both?
  3. Is there control-plane policing and session rate limiting on the legacy routers, and if not, why are new sites allowed to peer with them directly?
  4. Why was this work scheduled for a Friday afternoon, 13 days before Thanksgiving, and what did the FAA's pre-change safety assessment cover?
  5. How long did it take the NOCC to identify the offending router, and was isolating it a manual step?
  6. What changed in the cutover procedure after 2009, and was it followed? The contractor operates an FTI National Test Bed for qualifying services, so was this router's configuration qualified there before it went live?

The $50 billion ask may well be justified, and replacing TDM is not optional given the carriers are walking away from it. But more money for new gear doesn't answer any of these six questions, and the next 1,500 or so cutovers will run through the same seam. Time will tell what the root-cause report says.

-ab

Encoded Ventures Management LLC published this content on November 15, 2026, and is solely responsible for the information contained herein. Distributed via Public Technologies (PUBT), unedited and unaltered, on November 16, 2026 at 07:26 UTC. If you believe the information included in the content is inaccurate or outdated and requires editing or removal, please contact us at [email protected]