BGP in 2024

At the start of each year, it's been my habit to report on the behaviour of the Internet's inter-domain routing system during the previous 12 months, looking in some detail at some metrics from the routing system that can show the essential shape and behaviour of the underlying interconnection fabric of the Internet.

The strong growth numbers that were a constant feature of the past thirty years of the Internet's story are simply not present in recent data. Is there no more demand capacity to fuel further growth? Is the Internet losing its investment appeal and so many other signals of investor disillusion over outlandish growth predictions in technology-based services? Or has the massive transition into Content Distribution Networks (CDNs) for digital services meant a declining demand for the traditional form of content distribution over on-demand network access?

Let's look at the BGP view of 2024 and see how these larger technical and economic considerations are reflected in the behaviour of the Internet's inter-domain routing system.

The first metric of interest is the size of the routing tables. Each router must store a local database of all prefixes each routing peer announces. In addition, conventional routing design places a complete set of 'best' paths into each line card and performs a lookup into this forwarding data structure for each packet. This represents an extremely challenging silicon design problem. The larger the routing search space, the more difficult the problem!

The overall stability of the system is also considered. Processing a routing update requires several lookups into local data structures and local processing steps. Each router has a finite capacity to process updates, and once the update rate exceeds this local processing capability, the router will start to queue up unprocessed updates. In the worst case, the router will start to lag in real time, so the information a BGP speaker is propagating reflects a past local topology, not necessarily the current local topology.

If this lag continues, unprocessed updates may be dropped from the queue at some point. BGP has no inherent periodic refresh capability, so when information is dropped, the router and its neighbors fall out of sync with the network topology. At its most benign, the router will advertise 'ghost' routes where the prefix is no longer reachable, yet the out-of-sync router will continue to advertise reachability. At its worst, the router will set up a loop condition and as traffic enters the loop it will continue circulating through the loop until the packet's TTL expires. This may cause saturation of the underlying transmission system and trigger further outages which, in turn, may add to the routing load.

The two critical metrics of the Internet's routing table that we are most interested in are the size of the routing space and its level of updates or churn. Here we will concentrate on the first of these metrics, the size of the routing space, and the changes that occurred through 2024, and use this data to extrapolate forward and look at five-year projections for the routing table size in both IPv4 and IPv6.

The BGP measurement environment

In analysing long baseline data series the ideal approach is to keep as much of the local data-gathering environment as stable as possible. In this way, the changes in the collected data reflect changes in the larger environment, as distinct from changes in the local configuration of the data collection equipment.

The major measurement point used here is a BGP speaker configured within AS131072. This network generates no traffic and originates no routes in BGP. It's a passive network with a single BGP speaker logging all received BGP updates since 2007. The router is fed with a default-free BGP feed from AS4608, the APNIC network located in Australia, and AS4777, the APNIC network located in Japan, for both IPv4 and IPv6 routes.

There is also no internal routing (iBGP) component in this measurement setup. While it has been asserted at various times over the years that iBGP is a major contributor to BGP scalability concerns in BGP, the consideration here in trying to quantify this assertion is that there is no 'standard' iBGP configuration, as each network has its own rather unique configuration of Route reflectors and iBGP peers. This makes it hard to generate a 'typical' iBGP load profile, let alone analyse the general trends in iBGP update loads over time.

In this study, the scope of attention is limited to a simple eBGP configuration likely to be found as a 'stub' Autonomous System (AS) at the edge of the Internet. This AS is not an upstream for any third party, it has no transit role and does not have a large set of BGP peers. It's a simple view of the routing world that I see when I sit at the edge of the Internet. Like all BGP views, it is totally unique to this network, and every other network will see a slightly different Internet with different metrics. However, the behaviour seen by this stub network at the edge of the Internet is probably similar to most other stub networks at the edge of the Internet. While the fine details may differ, the overall picture is probably the same. This BGP view is both unique and typical at the same time.

We will use the recourses of two large route collector systems to complement this single view of the BGP network. We use Route Views, a project operated out of the University of Oregon that has enjoyed significant support from the US National Science Foundation and numerous industry entities. This route collector has been in operation continuously since 1997 and holds much of the routing history of the Internet in its archives.

We also use the Routing Information Service (RIS), operated by the RIPE NCC. This service is operated by the RIPE Network Coordination Centre, the Regional Internet Registry (RIR) for Europe, the Middle East and Central Asia. Similarly to Route Views, RIS operates 27 route collectors located across a diverse collection of locations.

Currently, 777 BGP sessions are being collected across all routing peers of these two route collector systems, with 398 route sessions providing an IPv4 routing table and 379 IPv6 sessions.

The IPv4 routing table

Measurements of the routing table size have been taken regularly since the start of 1988, roughly coinciding with the early days of the NSFNET in the United States.

Figure 1 shows a rather unique picture of the size of the routing table, as seen by all the peers of the Route Views route collector system since its inception in 1997 and incorporating earlier measurements conducted on the NSFNET.

Several events are visible in the history of the routing table, such as the bursting of the Internet bubble in 2001, the impact of the global financial crisis in 2009 and the lingering effects of the COVID-19-related shutdowns in late 2020. What is perhaps surprising is one ongoing event that is not visible in this plot. Since 2011 the supply of IPv4 addresses has been progressively constrained as the unassigned address pools of the various RIRs have been depleted. Yet there is no visible impact on the growth rate of the number of announced prefixes in the global routing system since 2011. In terms of the size of the routing table, it's as if the exhaustion of IPv4 addresses has not happened at all for the ensuing decade. It is only by 2021 that we see some tapering of the growth of the size of the IPv4 routing table.

BGP is not just a reachability protocol. Network operators can manipulate traffic paths using selective advertisement of more specific addresses, allowing BGP to be used as a traffic engineering tool. These more specific advertisements often have a restricted propagation. This is evident in Figure 1, where there is no single plot in this figure (in an entirely homogenous routing environment each BGP peer would see precisely the same routing table). There is a variance across the various peers of these route collectors, which is around 50,000 routes.

We can look at just 2024, taking the collection of RouteViews and RIS BGP peers (Figure 2).

Both Figures 1 and 2 illustrate an important principle in BGP: There is no single authoritative view of the Internet's inter-domain routing table, as all views are relative to the perspective of each BGP speaker. These figures also illustrate that at times the cause of changes in routing is not necessarily a change at the point of origination of the route, which would be visible to all BGP speakers across the entire Internet. Still, it may well be a change in transit arrangements within the interior of the network that may expose, or hide, collections of routes.

The issue of the collective management of the routing system can be seen as an example of the condition of the tragedy of the commons where the self-interest of one actor in attempting to minimize its transit service costs becomes an incremental cost in the total routing load borne by other actors. To quote the Wikipedia article on this topic: "In absence of enlightened self-interest, some form of authority or federation is needed to solve the collective action problem." This appears to be the case in the behaviour of the routing system, where there is an extensive reliance on enlightened self-interest to be conservative in one's own announcements.

The next collection of plots (Figures 3 through 12) contains some of the vital characteristics of the IPv4 BGP network from the start of 2017 to the end of 2024, using the routing data for a single BGP session, collected from AS131072.

Figure 3 shows the total number of routes in the routing table over this period. The routing table has continued to grow, largely because of the advertisement of more specifics.

The eight-year period since the start of 2016 has seen the span of addresses advertised in the routing system slowing down until 2021 (Figure 4). In 2021 we saw several large address blocks being advertised in the routing system by agencies associated with the US Department of Defence. Across 2022 and 2023 the total span of advertised IPv4 addresses declined. There was a sharp rise in the total span of address space on 12 December 2024, coinciding with the advertisement of 81,224,704 addresses (the equivalent of 4.8 /8s) by ASes operated by Amazon. Their major network, AS16509, announced a span of 154,961,152 addresses at the end of 2024, or some 4.97% of the total IPv4 announced address span, second only to the 224,851,712 addresses announced by AS749 for the US Department of Defence. The collection of Amazon ASes collectively announces a span of 157,809,408 addresses at the end of 2024.

In terms of more specific advertisements and covering aggregate advertisements (Figure 5), the number of covering aggregate announcements increased across 2024, as did the number of more specifics. Looking at the ratio of these two counts, the ratio has remained steady over 2024 (Figure 6). The average prefix size is now smaller than a /20 (Figure 7). Prefix sizes of /24, /23 and /22 now account for 83% of the entire IPv4 routing table (Figure 8).

The network's topology has remained relatively consistent, with the growth of the Internet being seen as a relatively static density of interconnectivity. The average AS-PATH length has been constant since mid-2020. (Figure 9).

Further evidence of the slowing of the growth in the IPv4 Internet can be seen in the AS count (Figure 10). The growth of the AS count started to decline in late 2020 and continued to decline in the ensuing years. This is a likely signal of network saturation in many markets. The number of transit networks was constant across 2024, which appears to be a related signal of market saturation (Figure 11).

The year-by-year summary of the IPv4 BGP network (as seen by AS 131072) over the 2021-2025 period is shown in Table 1.

Table 1 - IPv4 BGP table growth profile.

In terms of advertised prefixes, the size of the routing table grew by 53,000 entries or 6%. The number of root prefixes increased by 13,000 entries, while the number of more specific routes increased by 40,000 entries. This year (2024) saw the IPv4 routing table resume the growth metrics of 2021 and 2022.

The total span of advertised addresses increased by some 69M IPv4 /32s across the year, due to Amazon's advertising of a large pool of addresses in December 2024.

The number of routed stub ASNs (edge networks) grew by 1.6% in 2024, and the total number of ASes visible in the IPv4 network grew by 351 ASes across 2024, or 3.2%.

Let's look in more detail at the 10 networks with the highest route object net growth and the highest net route withdrawal count for the year (Table 2).

Table 2 - IPv4 advertised prefix changes, top 10 ASes.

The majority of additional route objects are more specifics. The highest growth in route objects over 2024 was AS16509, where more specifics grew by 5,922 routes. One of the more notable actions was AS28202, Rede Brasileria, which advertised the same 10 aggregates and added 1,115 more specifics.

We can also look at the total span of advertised addresses for each AS, comparing the advertised address space at the start of 2024 with that at the end of the year. This data is shown in Table 3.

Table 3 - IPv4 advertised address span, top 10 ASes.

The picture of IPv4 growth for 2024 is dominated by Amazon. AS16509 expanded from 9,471 routes spanning 45.88M addresses to 12,478 routes, spanning 154.96M addresses. The next largest was Microsoft, AS8075, growing from 52.40M addresses to 65.76M addresses. Interestingly, 7 of the top 10 entities whose advertised address span grew in 2024 are US-based. There are drops in the advertised address count from the ISP sector, including Sprintlink, EE, Softbank and Windstream.

We are likely seeing several factors at play behind these changes in the IPv4 network:

• The first is the saturation of many Internet markets across the globe so that the amount of 'green field' expansion is far lower than, say, a decade ago.

• Secondly, we are seeing considerable concentration on the service market, where the utilization level of addresses is vastly greater by both content and service publishers and end clients. The service and client numbers may be growing, but that does not necessarily imply using more IPv4 addresses or more routing table entries. NATs are fully entrenched in the IPv4 world, and in the service provider market volume economics implies that few larger providers are more efficient than a greater number of smaller providers.

• Thirdly, this concentration in the service market has been accompanied by further consolidation in the access market, particularly in mobile access networks. This consolidation of client access networks creates greater efficiencies in shared address solutions.

• The continued deployment of IPv6 cannot be ignored. Within the 10 economies with the largest span of advertised addresses (collectively, these 10 economies advertise 75% of the span of advertised IPv4 addresses) 7 of these economies are also in the 10 economies with the largest span of advertised IPv6 addresses (collectively, these same 10 economies advertise 76.5% of the span of advertised IPv6 addresses). Looking at just these 7 economies, namely the United States, China, Japan, Germany, the United Kingdom, Brazil, and Canada, they advertise 76.4% of the advertised IPv6 address span and 71% of the advertised IPv4 address span. The level of IPv6 use in all seven economies has a greater use of IPv6 deployment than the total global deployment level of 39.7% of end users.

• Finally, I suspect the most significant dissipating factor has been the rise of private CDNs. The conventional drivers for growth in the routing table were based on the product of an increasing number of networks with discrete routing policies and the need for these networks to balance incoming traffic across multiple network paths (traffic engineering using more specifics). The rapid uptake of CDNs by service platforms has meant a significantly reduced dependence on BGP transit services to deliver content to users. The public part of the Internet is rapidly shrinking to the last-mile access network, which connects directly to several CDNs. The transit part of the Internet is just not so critical these days. It is unlikely that we will move away from the extensive use of CDNs in the foreseeable future. If anything, their level of use within the Internet's service portfolio will increase further.

In various parts of the network, the number of IPv4 entries in the default-free zone is between 950,000 and 1,000,000. The net routing table growth in 2024 was some 53,000 new entries, a significant increase over the 3,000 new entries seen in 2023. A net gain of 1,400 new ASNs was seen in the IPv4 network across the year, compared to 1,100 in 2023. While 2023 saw a slowing of the IPv4 network growth, 2024 has seen the growth numbers resume to the levels last seen in 2021 and 2022.

The IPv6 BGP table data

A similar exercise has been undertaken for IPv6 routing data. As with the IPv4 network, there is diversity in the number of IPv6 routes seen at various vantage points, as shown when looking at the prefix counts advertised by all the peers of Route Views (Figure 13).

Several distinct phases in the growth trends are visible in the history of the IPv6 routing table. The period between 2004 and mid-2018 could be modelled by an exponential growth function with a doubling interval of three years. There was a sharp increase across late 2021, and again in mid-2023.

A more detailed look at the most recent four years incorporating both Route Views and RIS data (Figure 14) shows some increasing diversity between various BGP views as to what constitutes the 'complete' IPv6 route set, and the variance at the end of 2024 now spans some 25,000 prefix advertisements.

Figure 15 shows a detailed view of the routing table as seen by each Route Views and RIS peers across 2024.

The comparable profile figures for the IPv6 Internet are shown in Figures 16 through 25.

The growth of the IPv6 routing table has been relatively steady across 2024, growing by some 20,000 routes across the year (Figure 16). Routing advertisements of /48s are, by far, the most prevalent prefix size in the IPv6 routing table, and 46% of all prefixes are /48s. 75% of the IPv6 table entries comprise /48, /32, /44, and /40 prefixes (Figure 21).

RIR allocations of IPv6 addresses show a different pattern, with 74% of the 66,7854 IPv6 address allocations recorded in RIR registries being either a /32 (47%) or a /29 (24%). Only 20% of allocations are a /48. What is evident is that there is no clear correlation between an IPv6 address allocation prefix size (as used by the address registries in the address allocation process) and the advertised address prefix size. Many IPv6 address holders do not advertise their allocated IPv6 address prefix in a single routing advertisement.

Why is the IPv6 routing table being fragmented so extensively? The conventional response is that this is due to using more specific route entries to perform traffic engineering. Another possible reason is using more specifics to counter efforts of route hijacking. This latter rationale also has some credibility issues, given that it appears that most networks seem to accept a /64 prefix, and the disaggregated prefix is typically a /48, so as a countermeasure for more specific route hijacks, advertising /48's may not be all that effective.

This brings up the related topic of the minimum accepted route object size. The common convention in IPv4 is that a /24 prefix advertisement is the smallest address block that will propagate across the entire IPv4 default-free zone. More complex minimum size rules have largely fallen into disuse as address trading appears to have sliced up many of the larger address blocks into smaller sizes. If a /24 is the minimum accepted route prefix size in IPv4, what is the comparable size in IPv6?

There appears to be no common consensus position here, and the default action of many network operators appear to to have no minimum size filter. In theory, that would imply that a /128 route object would be accepted across the entire IPv6 default-free zone. A more pragmatic observation is that a /32 would be assuredly accepted by all networks, and it appears that many network operators believe that a /48 is also generally accepted. Given that a /48 is the most common prefix size in today's IPv6 network this view appears to be widespread. However, we also see prefixes smaller than a /48 in the routing table with /49, /52, /56 and /64 prefixes present in the IPv6 BGP routing table. Some 0.7% of all advertised prefixes are more specific than a /48.

The summary of the IPv6 BGP routing table profile from 2020 through to the start of 2025 is shown in Table 4. The IPv6 network growth rate is somewhat lower than previous years, with a 10% growth in routing entries, and a 4% growth in the advertised address span.

Table 4 - IPv6 BGP table growth profile.

The pressures for further expansion on the IPv6 network appear to be more idiosyncratic for each market sector and region, rather than being expressed as a more general imperative. Where there is at present scant IPv6 adoption, as is the case in most of Africa, the Middle East, Eastern and Southern Europe, and the western part of Latin America, there is no apparent sense of urgency to make the shift (Figure 26). It would appear that the Internet market is largely a saturated one and the smaller pace of network growth in those regions appears, for the moment, to be adequately accommodated in the continued use of IPv4 NATs.

The predictions

What can this data from 2024 tell us in terms of projections of the future of BGP in terms of BGP table size?

Forecasting the IPv4 BGP table

Figure 27 shows the data set for BGP from January 2017 until December 2024. This plot also shows the fit of these most recent five years of data to various growth models. The first-order differential, or the rate of growth, of the BGP routing table is shown in Figure 28. The linear average rate of growth of the routing table appears to be falling slowly from 140 to 160 additional entries per day in 2016 to around 100 per day at the start of 2024.

There are several potential models to match this data. One model is to take the five-year average daily rate of change and apply this as a continuous model for the next five years. This is a 'linear' model and takes the current dynamics of the IPv4 Internet, assuming that these dynamics will operate largely unchanged over the projection period. The second model is to look at the trend in the changes of rate of change and match this to a linear model. If the first-order differential of a data series is a linear function, then the original data can be represented as a second-order polynomial. The final model used here is to model the log of the data series as a linear model and therefore derive an exponential model for the data series. The application of these three projection models to the original data series is shown in Figure 27, and the first-order differential of the data (the daily rate of change) is shown in Figure 28.

Figure 28 appears to support the supposition of the primary data series fitting a second-order polynomial model. The rate of change of the daily change to the count of IPv4 routing objects has been declining since 2020, where the range of daily change has fallen from a band of values between 100 to 200 new entries per day to a new range that sits in a band between 50 to 100 new entries per day for the first half of 2024. This rate has risen above 100 new entries per day for the second half of 2024.

The projections of the linear and polynomial best-fit models are shown in Table 4 (the resumption of an exponential growth model appears to be highly unlikely in this late phase of the IPv4 network).

Table 5 - IPv4 BGP table size prediction.

The linear projection model is somewhat unlikely, in my opinion. The drivers for the continued growth of the IPv4 network do not appear to be clear, so the projection of continued growth of the number of IPv4 FIB entries with an annual net gain of 43,000 entries is somewhat unrealistic. The O(2) polynomial projection model predicts that this first-order differential will reach the zero point by early 2028 and then decline. It must be stressed it's just a mathematical model that fits the recent data, and nothing more.

Given that the last 'normal' year of supply of available IPv4 addresses to fuel continued growth in the IPv4 Internet was now some fifteen years ago in 2010, perhaps the more relevant question is - why has the growth of the IPv4 routing table persisted with such regularity in the ensuing fifteen years?

It should be remembered that a dual-stack Internet is not the objective at this time of transitioning the Internet to IPv6. The ultimate objective of the entire transition process is to support an IPv6-only network. An important part of the process is the protocol negotiation strategy used by dual-stack applications, where IPv6 is the preferred protocol wherever reasonably possible. In a world of ubiquitous dual-stack deployment, all applications will prefer to use IPv6, and the expectation is that in such a world the use of IPv4 would rapidly plummet.

The challenge for the past decade or more has been in attempting to predict when in time that tipping point that causes demand for IPv4 to plummet may occur.

Forecasting the IPv6 BGP table

The same technique can be used for the IPv6 routing table. Figure 29 shows the data set for BGP from January 2017 until December 2024.

The first-order differential, or the rate of growth of the IPv6 BGP routing table is shown in Figure 30. The number of additional routing entries has grown from 10 new entries per day at the start of 2012 to a peak of some 230 new entries per day in May 2021.

The first half of both 2023 and 2024 had a steady period of increase in the growth of the daily count of new entries, and this declined in the second half of the year. Current growth rates are some 50 to 100 new entries per day, which is more than half the rate of growth in the IPv4 network (Figure 30).

It appears that the period of exponential growth of the IPv6 network is over, and the best fit of a model to the recent data is either a linear model or an O(2) polynomial, as shown in Figure 31.

The projection for the IPv6 table size is shown in Table 6.

Table 6 - IPv6 BGP table size prediction.

The linear and polynomial projections in Table 6 provide a reasonable estimate of the low and high bounds of the growth of the IPv6 BGP routing table in the coming years.

If IPv6 continues to grow exponentially over the next five years, doubling every 30 months or so, then the size of the IPv6 routing table will be a little over one million entries at the start of 2030. The data from the previous two years suggests that such a level of growth is extremely unlikely, and a linear growth model is a closer fit to the recent past, and an average growth rate of 26,000 new entries per year is a better fit to this recent data.

Conclusion

These predictions for the routing system are highly uncertain. The correlation between network deployments and routing advertisements has been disrupted by the hiatus in the supply of IPv4 addresses, causing more recent deployments to make extensive use of various forms of address-sharing technologies, and making fundamental alterations to the architecture of the service model of the Internet.

While several access providers and service platforms have made significant progress in public IPv6 deployments for their respective customers, the majority of the Internet user base (some 60% of the Internet's user base) is still exclusively using IPv4 as of the end of 2024 (Figure 32).

These predictions as to the future profile of the routing environment for IPv4 and IPv6, using extrapolation from historical data, can only go so far in providing a coherent picture for the near-term future. As well as the technical issues relating to the evolution of IP technology and the IPv6 transition there are also broader factors such as the state of the global communications economy and the larger global economy. Investment in communications infrastructure, as with most other forms of infrastructure investment is not generally a short-term proposition. The major benefits tend to be realized in increased efficiency of economic production, rather than short-term windfall gains from infrastructure investment. This means that short-term expedient measures, such as a response to a global pandemic or a rapid escalation of energy prices due to regional conflict, can interrupt infrastructure investment programs.

The question behind the recent slowing of the growth in both the IPv4 and IPv6 aspects of the Internet's routing space is whether this slowdown is due to market saturation in the case of IPv4 or a dissipation of collective market impetus in the case of IPv6, or an interruption due to these short-term exogenous market factors. In the latter case, we would expect growth to resume once more when the current global market conditions dissipate, while an underlying condition of market saturation is a more permanent state.

If the concern is that the routing system is growing at a rate that is faster than our collective ability to throw available technology at it, then there is no serious cause for alarm in the current trends of growth in the routing system. There is no evidence of the imminent collapse of BGP. Far from it!

However, the size of the inter-domain routing table is only one-half of the story. The stability of the routing system is also very important, and to complete this look at the routing system in 2024 we will also need to look at the dynamic behaviour of the routing system. The profile of BGP update churn in 2024 is a topic we'll look at in detail in the next article which looks at the larger picture of addressing and routing across 2024.

The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.

Public link

Please use the above public link if you want to share this noodl on another website.