When I was asked to write a piece about regional peering I thought it would be a quick update on the current state of affairs in the UK. Alas with all these things I realised that I need to add a little back story. Feel free to skip over the content to the end if you know all the bits…

The Internet (and what its not)

Most people know the Internet is not a single entity but rather a collective of networks that use common standards to create a single network made up of independently run and managed networks that allow their customers and end users exchange traffic and therefore create the Internet and its public face – the World Wide Web.

Peering/Transit/BGP

At the edge of each network there are a bunch of routers that communicate with other adjacent routers belonging to other networks using the Border Gateway Protocol (BGP). At a basic level each network tells the other network what it knows about its network and (depending on commercial concerns) other networks it knows about using the BGP protocol. This information is shared in the form of “routes” which define a certain block of address space and how to get to it.

This leads naturally to a quick commercial discussion about the difference between Peering and Transit (and its many variants).

Peering is the term used to describe the actual direct connection between two BGP routers and the difference between “Peering” and “Transit” is the configuration on each router. What confuses the matter is that Peering has become a short hand for “Settlement Free Interconnect” i.e. neither party pays for the privilege of sharing information and data. With a peering session the only information that is shared is about the two networks and their customers and so its non-transitive.

This rather nice diagram shows this non transitive property, if EastNet or WestNet don’t have a transit provider then they won’t be able to send traffic between themselves.

Transit is a commercial agreement where one network pays the other for access to not only their peers network and customer but also all other networks that are connected to, in the above diagram if MidNet were to sell either of the two networks transit then all three would then have a full red/green/blue routing table.

For example if MidNet sold transit to WestNet then WestNet would then get the routing information about EastNet from MidNet as part of a commercial contract, similarly EastNet would now get the routing information about WestNet from MidNet as they are now a customer of MidNet.

There are now many variants on a theme when it comes Transit (or Paid Peering) where exact details of what is shared between the two routers can be modified at a technical level to reflect the commercial terms and there are whole books dedicated to the commercial negotiation of peering relationships such as The 2014 Internet Peering Playbook: Connecting to the Core of the Internet but I may return to this subject in a future post.

Internet Exchanges

So far I’ve just talked about “connecting” between routers, in the beginning these connections were a single fibre or copper cable between the two networks where ever they shared a common location but over time a number of different organisations created a dedicated infrastructure when a single cable could be used to connect to many different networks in the same location (saving money in the process).

In the US these Network Access Points (NAPs) or Internet Exchanges (IXs) started as commercial concerns run by the owners of the Datacenter, in Europe they tended to be mutual not for profit associations owned by the networks operators or academic institutions.

[youtube http://www.youtube.com/watch?v=a5837LcDHfE?rel=0]

 

Regional Peering in the UK

Over time the existence of these IXs created massive concentrations of networks in and around a single locations, in Europe this meant London (LINX), Amsterdam (AMS-IX) and Frankfurt (DE-CIX) were the dominant centres for the exchange of traffic, not just in their own countries but across the region as a whole.

Over the last couple of years as the amounts of traffic exchanged between networks has grown inline with the increase of individual end user access speeds and demands for online content there has been a growing realisation that more local IXs are required in order to both reduce the costs of running a network and improve the end user experience. To this end many countries have spawned IXs where traffic can be exchanged in country rather being dragged halfway across the continent.

In the UK the same pattern has emerged, Manchester had an IX for many years but had until recently it withered and was doing a damn fine impression of Monty Python’s Norwegian Blue as the attraction of building directly into the Docklands Datacentres where LINX was located wiped out all comers.

In 2008 the earth moved and IXLeeds was born from a meeting of local operators who wanted to save costs on sending traffic from Leeds to London and then back again, it took a couple of years but in 2010 the first members were connected to the switch and peering outside of London was back on the move.

In late 2011 a similar meeting (in the pub of course) in Manchester pushed the button on IXManchester as a fix to the existing lack of action in the region. This was at the same time that LINX was considering the wider impact of their success on the UK Internet Infrastructure and coming to the conclusion that “something must be done” and so rather than forming a new company IXManchester was built on hardware left over from the last LINX upgrade and launched just ahead of the Olympic Games.

Since then LINX has held a number of meetings in the regions (Belfast, Edinburgh, Nottingham, Cardiff, Birmingham to name a few) to gauge the desire locally for an IX to be built following on from those IXScotland went live in November 2013. What happens next depends on the appetite of network operators in the regions to exchange traffic locally.

From Europe to the World

As mentioned earlier the US based IXs tended to be run by the datacenter operator and their interests were tied into getting people to build in their DCs only. In Europe the IXs spread into multiple DCs and so created a competitive environment where there was competition to attract different operators with different levels of service and types of offering.

Jealous of this situation Open-IX was formed to try and bring the advantages of European IXs to the rest of the world, in a twist of fate LINX have built an exchange in North Virginia (LINX NoVA) and left AMS-IX (https://nynj.ams-ix.net/ ) and DE-CIX (http://nyc.de-cix.net/) to fight over New Amsterdam York.