The Internet is a Global Wide Area Network (GWAN) that allows computer networks to talk to each other. This “talking” is known as IP transit, and only occurs if traffic can reach its destination through use of the TCP/IP stack. In this article, we’ll dive into what you need to know about IP transit itself. But first, let’s quickly review some of the building blocks that make TCP/IP transit possible.
IP Addresses
An Internet Protocol (IP) address allows the identification of each network system (be that host device or network device). Currently, the more widely deployed version of IP is IPv4 (32-bit), while IPv6 (128-bit) deployment is ramping up.
The IP (v4 or v6) addresses are arranged in blocks. These blocks are allocated to regional Internet registries (RIR). The RIRs allocate them to different local entities such as Internet Service Providers (ISPs), enterprises, and government agencies.
Among the block of IP addresses that can be assigned, there are two types of IP addresses spaces:
- Provider independent (PI): A block of IP addresses that are assigned by the RIR directly to the end user. The end user is in charge of having agreements with the ISPs to route the block on the Internet. The PI can be advertised through any ISP (one or more at the same time) and the customer can keep it when moving from ISP to another.
- Provider aggregatable (PA): One part of a bigger block of IP addresses assigned to an ISP. The ISP can assign sub-blocks to its customers. The PA can be advertised by the ISP owning the bigger pool. When a customer moves from one ISP to another, the PA must be changed.
Autonomous System
An Autonomous System (AS) represents a set of IP prefixes that belong to a network and are managed by a single organization. Each AS is assigned an Autonomous System Number (ASN), which is unique to the network.
Owning an ASN goes hand in hand with owning IP addresses. Owning an ASN does not make any sense without having an IP block to announce on the Internet. On the other hand, you do not need an ASN to announce your PI.
Border Gateway Protocol
Border Gateway Protocol (BGP) is the protocol that runs the Internet. It is a protocol that exchanges routing and reachability information between autonomous systems on the Internet.
Small enterprises run BGP only on the edge where they are connected to their ISPs (each ISP is connected to at least one router for redundancy purposes). This type of BGP session is called external BGP, because it runs between two different autonomous systems. The internal network can use any other routing protocol.
When a network is very large or you need to be able to manipulate the traffic inside the network, BGP can run between routers in the internal network. This type of BGP session is called internal BGP, because it runs between routers in the same AS.
AS PATH
The BGP AS path is a mandatory attribute that is present in any of the BGP updates. Whenever a prefix is sent to a neighbour in a different AS, the router adds its own AS in the AS PATH list so that all the routers receiving the prefix will know which AS the prefix crossed. Always, the originating AS is the right-most AS in the AS PATH:
When routers have multiple paths for the same destination, they need to decide which route will be installed in the routing table. The BGP best-path-selection algorithm is complex and considers many criteria to determine which route will be selected as best (in case multipath is not wanted or required). A route with a shorter AS PATH will be preferred over a route with a longer AS PATH.
Communities
A BGP community provides the mechanism to tag the routes. BGP communities can be appended or removed. A BGP community is an optional transitive BGP attribute, which means that it can traverse from one AS to another when the community policy of the AS allows it.
A routing policy can be configured so all routes matching specific BGP communities are treated identically.
There are four BGP communities:
- Internet Community: Advertise the prefix matching this community to all BGP neighbours.
- No-Advertise Community: Do not advertise this prefix to any BGP neighbours.
- No-Export Community: Do not advertise this prefix to any External BGP neighbours.
- Local-AS Community: Do not advertise this prefix outside of the sub-AS.
Each IP Transit ISP provides a set of communities that allows the traffic manipulation inside the ISP or in the relation with the peers/upstreams of the ISP. These communities usually allow setting a local route preference or performing AS-PATH prepending so that a customer route can appear less preferred.
What is IP Transit?
IP transit is a service where an Internet Service Provider (ISP) allows traffic to cross its network to reach a destination. To have access to all Internet routes, you would need to connect directly with all autonomous systems. This isn’t possible on your own. Instead, you would connect to an ISP that has a way to reach any network from the Internet.
The ISP providing IP transit service is often called “going upstream.” The traffic coming from the Internet and destined to the customer is called “going downstream.”
Tiers Of IP Transit Providers
Tier 1
Tier 1 ISPs have global reach. They peer with each other, forming a global network on which tier 2 and tier 3 ISPs (lower tiers) connect. While the tier 1 ISPs peer with each other at zero cost for traffic transfer, they charge lower tier ISPs to allow traffic to transit their networks.
Tier 2
Tier 2 ISPs have very large networks and a wide global presence (one or two continents). Tier 2 ISPs are required to buy IP transit from the higher tier. Usually, to avoid the cost associated with IP transit from tier 1, ISPs, tier 2 ISPs peer with each other at very low cost (or no cost at all) to expand their global reach.
Tier 3
Tier 3 ISPs are most commonly local providers with metropolitan or national reach. They usually buy IP transit from both tier 1 and tier 2 ISPs to minimize costs associated with the more expensive tier 1 IP transit.
{{banner-3="/design/banners"}}
Single-Homed And Multi-Homed
An IP transit customer can be considered single-homed, dual-homed, or multi-homed, depending on how many upstream options are available to them.
Single And Dual-Homed
The diagram below compares single and dual-homed transit.
Single-homed means a customer is connected only to one ISP using a single link. Dual-homed means a customer is connected only to a single ISP but with multiple links. These links can be between the same routers or different routers on both sides (the customer and ISP sides).
One big advantage to being connected to a single ISP is the fact that routing policies can be simplified and the design has fewer moving parts. Also, costs are lower due to the fact that a customer has only a link or two. Moreover, because they are from the same ISP, a discount can potentially be applied.
On the other hand, if a failure inside the ISP occurs and it is prevented from accessing the rest of the Internet (maybe its own upstream has problems), the customer is isolated from the rest of the Internet.
Single And Dual Multi-Homed
The diagram below compares single and dual multi-homed transit,
Single multi-homed means a customer is connected with a single link to two ISPs. Dual multi-homed means a customer is connected with two links to two ISPs. As with single/dual-homed, links on the customer side can be terminated on the same router or on different routers.
This type of deployment allows the highest redundancy level (when each link is terminated to different customer routers). It also comes with increased costs and complex routing policies.
In order to benefit from having two upstream ISPs, a customer must receive the full Internet table from each ISP (roughly 860k routes). This allows for more granular and complex routing policies.
IP Transit vs. Peering
As we’ve learned, IP transit means that one party must pay for access to transit across another party’s network. Peering differs by enabling two parties to exchange data, benefitting equally, and thus foregoing payments. Over the peering session, the two parties advertise only their own prefixes and their customers' prefixes. This is known as a settlement-fee agreement.
Although the peering is supposed to have zero cost, it applies only for the data exchanged. There are other costs related to colocation, routers deployed for the peering, and cross-connects used to connect the peers.
BGP Communities Continued
Each IP Transit ISP provides a set of communities that will be accepted and that allows traffic manipulation inside an ISP or in the relation with the peers/upstreams of the ISP. Any other communities, if not agreed upon prior, will be stripped from BGP updates.
Most commonly, communities allow users to set the local preference of a route or to perform AS-PATH prepending. In this way, a customer route can appear less preferred to ISPs or ISPs peers.
One example of how a BGP community can be used is shown below. The customer does not want ISP1 to send Internet traffic through the direct link. At the same time, the customer wants to keep ISP1 as a backup, in case the ISP2 link goes down.
IP Transit Peering Policy
As mentioned before, tier 1 ISPs are peering with each other on a settlement-free basis so each of them will have access to the rest of the Internet.
General Requirements For Free-Peering Relationships
- Operate a national-wide network in the countries where the peering is required.
- Operate using 100G circuits in various continents.
- Operate at least 10G ports for peering connections.
- For multi-continents peers, peering must be established in each continent in at least a couple of locations.
- Have a 24/7 NOC.
- Advertise only its customer routes.
- Register the routes with IRRs.
Requirements for Traffic
- IPv6 peering will be established together with IPv4.
- Each peering connection must have a minimum monthly load.
- The traffic must ingress and egress only through the peering points.
- When traffic exchange is unbalanced, ISPs can allow it. However, traffic exchange must not exceed a specific ratio.
Failing to meet any of the traffic requirements can result in termination of the peering agreement without any notice.
{{banner-2="/design/banners"}}
Threats to IP Transit
When customers buy IP transit service, usually there are other services that can complement the service.
Distributed Denial of Service
A Denial of Service (DOS) attack is an attempt to prevent users of a service from using that particular service. A Distributed DOS (DDOS) is when the attack comes from multiple sources. Almost every time these types of attacks occur, the sources of the attacks aren’t aware they’re performing this kind of attack. They are compromised by trojan horse programs and viruses which allow attackers to take control of machines and execute the attacks.
The DDOS protection that most IP Transit ISPs provide allows customers to be notified in case of an attack so that they can clean the traffic. The DDOS protection applies to IPv4 and IPv6, up to Layer 7.
Resource Public Key Infrastructure
Before we delve into this topic, let us first explain a few scenarios that could either inadvertently or maliciously jeopardize IP transit traffic. One such scenario is BGP flapping. This occurs when a BGP system sends an excessive number of update messages to advertise network reachability information which in turn overwhelms its BGP peers. Routing equipment vendors support “BGP damping” to delay the traffic in exchange for regained stability.
Another such scenario is a BGP leak which involves the advertisement of blocks of incorrect IP addresses across networks. While this can happen because of an unintentional misconfiguration, a similar approach is used in malicious attacks known as BGP hijacks or man-in-the-middle attacks. In such scenarios, Resource Public Key Infrastructure (RPKI) can help.
Resource Public Key Infrastructure (RPKI) is a mechanism to make the Internet more secure. Through RPKI, a customer can create a Route Origin Authorization (ROA). ROA specifies which AS can advertise the prefix. This allows the IP transit ISPs to perform route validation and drop any invalid routes, thus protecting customers against routing leaks and hijacks.
{{banner-sre="/design/banners"}}
Conclusion
If you are not a tier-1 ISP who could peer as an equal with other ISPs to freely exchange traffic with autonomous systems using the BGP routing protocol, then you would pay an ISP, as part of an IP transit arrangement, to carry your traffic to its destination. The internet traffic associated with BGP communities, that are agreed upon in advance, is carried by the ISP delivering the service, while a dual-homed architecture would provide redundancy and optimal routing.
Looking to learn more about Networking concepts? Check out our full guide.