broadband dns Engineer engineering internet ipv6 media Net peering

Experiences of Launching a Broadband IXP in North America #peeringweek @LINX_Network

LINX Head of Marketing and Business Development Ben Hedges shares his experiences launching a broadband IXP in a Peering Week guest post.

The opportunity to co-host the 24th Euro-IX forum in the UK has come along at what is a very exciting time for LINX. It’s our 20th year and this event comes shortly after us opening two brand new IXPs; IXScotland in Edinburgh and LINX NoVA in North Virginia, USA.

With LINX NoVA being our first overseas exchange there has been a lot of attention worldwide for what we’ve been building in the States. In this blog I will look to explain the background as to why we’re doing what we’re doing and why we believe this is an important development for LINX and its members plus the peering industry as a whole.

Engineer internet peering

#LINX81 internet plumbing stats

I’m at LINX81. I go to a couple of LINX meetings a year as it’s a great place to catch up with what’s going on in the world of internet plumbing. Laymen will probably not understand the allusion to plumbing. The internet is made up of lots of “pipes”. A pipe is usually measured as a multiple of Gigs (1x, 10x, 100x) is connected to a port, one at each end of the pipe (obv).

In the internet plumbing world the bigger your pipe the bigger your bragging rights. It’s all about shifting data. Timico I’m pleased to say uses 10Gig ports at its peering exchanges – LINX and LONAP. There is only one 100Gig port at LINX and that belongs to BT. I understand there are more in the pipeline.

The internet as you may know is made up of tens of thousands of autonomous networks. The chances are your favourite website is not hosted by the same network operator that provides you with your connectivity. What your ISP does is it connects its own network to every other network on the internet which it partly does via peering exchanges.

A peering exchange is a point at which networks connect to each other for the sharing of internet traffic. Being a member of LINX brings us lots of benefits. You can get an idea of this from the LINX stats:

468 members (39 new applications in 2013 so it’s growing)
1135 connected member ports
362 member-facing 1GigE ports
631 member-facing 10GigE ports
1 member-facing 100GigE port
over 1.618 Tb/sec of peak traffic (swoon)
6.792 Tb of connected capacity (so notionally plenty of headroom)
58 member countries

Peering at LINX connects us to a large chunk of the internet at relatively low cost. More from LINX81 as it happens.

Engineer olympics peering

If you see a network engineer pat him on the back and buy him a beer – Olympics good job #LINX78

I’m at LINX78 the latest quarterly meeting of the London Internet Exchange. This meeting is particularly interesting because it comes immediately after the Olympics and its attendees represent the vast majority of UK internet access networks. In other words the people responsible for making your web browsing experience a good one during the Olympics were all here.

This community of engineers should stand up and take a bow as part of the team that made the event a total success. Whilst there will be the odd exception and glitch the network of UK plc performed incredibly well. From a personal perspective although I was on holiday I kept in touch with the office from time to time.  The level of support calls in to the Timico NOC was as we would normally expect and we got the additional network capacity planning just right which is hugely satisfying.

CEO John Souter described the “Olympic  effect” seen at LINX in the run up to the games. Since LINX77 in May the exchange has seen a 20% increase in traffic capacity growing from around 5Tbps to 6Tbps. In a single month over 60 10GigE ports were installed as part of a capacity growth that month of 800Gig (including the first 100Gig port connected by BT).

The rush was prompted by a June 19th cut-off date for new capacity needed before the 14th July Olympic change freeze at LINX.

If you need some perspective consider that the average UK broadband speed is less than 10Mbps. The 6Tbps capacity is the equivalent of over 600,000 broadband connections running flat out. It’s not really a good way of looking at it as there are many other factors that need to be considered – networks have alternative routes to the internet , broadband connections not running at capacity to name but two. However it is a testament to the efforts made by the UK network operator community to ensure that their contribution to the Olympics was a success.

Note I’m told that the Dept of Business Innovation and Skills (Vince’s lot) asked for a daily report on how the LINX network was performing – such is the critical nature of this infrastructure. LINX is going from strength to strength. The exchange currently has 431 members with 64 having joined this year (that’s up on the 49 new members for the whole of last year).

If you see a network engineer pat him on the back and buy him a beer (several beers knowing the engineers I know).

Cloud datacentre Engineer peering

Notes from London Internet Exchange (LINX), including Telecity and Datacentre Market Growth

I usually attend the quarterly meetings of the London Internet Exchange (LINX). At the risk of boring readers you do find some fascinating facts at these get togethers.

LINX has 383 members with 56 new applications in 2010. That’s huge growth. Members come from 50 countries – so despite having London in its name LINX is very much international in its orientation.

LINX has 304 10Gig ports and carries over 776Gbp/sec peak traffic – roughly the same amount of traffic as around 160,000 Standard Definition video streams or 40,000 High Def. Traffic is up 22% in the last three months!

LINX members can reach around 78% of all websites in the world through their London connections. Interestingly historically LINX traffic has been fairly smooth whereas an individual ISP will see spikes based on high profile events such as the Olympics and the Football World Cup. Now even LINX is starting to see the effect of these events. The Chilean mine rescue is one example. People watched it on TV at home and then carried on using the internet once they had arrived in the office.

At LINX71 datacentre operator Telecity have just told us that they are selling out colocation space as fast as they can build it. They currently have around 23MW in the UK with a further 21MW in build.

Mind bogglingly they say that Google has as much datacentre space in Liege in Belgium as does Telecity in the entire UK.

More interesting facts as the surface – you read them first on

Engineer internet peering

LINX new members > London grows in internet importance

The London Internet Exchange (LINX) this morning announced three new members: Hutchinson 3G, Kenya Data Networks and from the USA. The only surprise as far as Hutchinson goes is that they were not already a member.

The addition of two new international members does however serve to underline the importance of London as a peering point for the global internet community.

Business competitions

cheezy grins all round at the Timico cheesecake competition

My cake making competitions have become very popular.  The strange thing is that I get people I’ve never met before coming up to me saying that the posts I write about these competitions are amongst their favourites. Well one person did last week at the LINX meeting anyway. 

I’m not sure what that says about my other posts but in answer to this huge demand I am happy to report the winner  of the August competition – this time for the best cheesecake – is Paula Davies (no relation afaik).

Engineer internet

Is black market for IPv4 blocks imminent?

Whilst I was on holiday the IPv4 Exhaustion counter ticked down another digit to 5% or 14 /8 blocks .

Nov 16 2009 10% – dropped through 400,000,000 mark
Jan 20th 9%
Feb 25th 8%
May 10th 7%
June 2nd 6%
August 5%

Currently we seem to be using a /8 block every three weeks. With 9 blocks left before we are down to the last 5 (at which point IANA will distribute these simultaneously to the 5 Regional Internet Registries) it looks like we have 27 weeks to go to IPv4 Exhaustion.

In my book this is February 2011 and not the June date reported by the Exhaustion Counter on this blog.

broadband Engineer internet media piracy Regs video

Cisco Drives Nail in Music Industry Coffin with CRS-3?

Most people won’t have given much thought as to how their email gets from A to B or how that video arrives from YouTube.  It just comes down the broadband connection which is plugged into the router next to the phone (or somewhere like that). Right?

Well today the worlds biggest router manufacturer, Cisco, announced their latest and greatest product.  It isn’t something that you will want to plug into your phone line though because it would take up most of the living room and there wouldn’t be enough room left for the sofa.

It would also be a bit of an overkill because this router, the CRS-3, is powerful enough to handle up to 322 Terabits1 per second, which  is roughly a hundred million times faster than the average UK broadband connection speed!

Engineer internet

First 100Gbps commercially available optical network rolled out by Nortel and Verizon

Nortel yesterday announced that Verizon had implemented the first commercially available 100Gbps network on a 893km link between Paris and Frankfurt.

There are several significant (or at least  I think they are interesting) points to be made regarding this milestone.

First of all Nortel is clearly a leader in Optical technology, as it is in a number of its other areas of business. 100Gbps has been discussed at the last few meetings of the London Internet Exchange (LINX) but largely in terms of the fact that 100Gbps equipment has only been achieving 40Gbps, an interim step.

It is a crying shame that the mismanagement of the business during the earlier parts of the decade resulted in the Chapter 11 situation we now see today and the break up of the business. From Timico’s perspective this is at least focussing minds at Nortel and we have seen a significant improvement in responsiveness and keeness to get things done. Good I suppose.

The 10Gbps standard was ratified in 2002 and, doing a quick trawl the first network rollouts seem to be around 2005 –  this is the case at LINX who tend to be up there amongst the leaders. Truth be told it was probably earlier than this.

The 100Gbps standard has not yet been ratified so there are clearly commercial pressures and advantages to running with the technology for a commercial operator to push ahead with it. Historically this has been 4x the cost for 10x the throughput. So it is clear that the cost of bandwidth is going to continue on a downward trend the more people use it, which they are doing.

This is an interesting wave for ISPs and network operators (surfers) to be riding.  We have to be nimble atop the big rollers making sure that we keep our network costs down quickly enough to match the competitive pricing pressures of the market place.

Engineer internet

LINX67 15th birthday meeting – the internet continues to grow

A milestone for the London Internet Exchange with its 15th birthday meeting today at Goodenough College in London. These are seriously useful meetings of like minded souls from the internet industry.

The internet is run by geeks that work extremely hard. In fact they live and breathe technology on a 24 x 7 x 365 ¼ basis and these LINX meetings bring them together for long days of meetings and presentations but also long evenings of letting their hair down.

LINX meetings are quarterly. Since the last one in the summer the peak traffic on the LINX network has grown by an incredible 25%. If the trend continued that would mean more than doubling in a year. The traditional perceived wisdom for growth in internet usage is 50% pa which is close to what I see on the Timico ADSL network.

LINX’s growth has been fuelled by an increase in membership of around 15% this year though clearly that wouldn’t account for the proportional increase in bandwidth.

For geeks everywhere who believe that size does matters the LINX network currently has over 2.5Terrabit per second capacity. The organisation is clearly providing value for money which is why all these new members are knocking on the door.

A year ago I seem to remember a statistic that said that 65% of the internet was reachable via LINX peering. Now this has grown to 70%. This is good news for end users as it helps to keep prices down.

Interestingly 75% of new members are from outside the UK, including a fair number from Russia. The UK is becoming increasingly an important place on the internet map.

More snippets from LINX67 as I think of them. The meeting is absolutely packed.

Business internet security

European Commission forecasts 193 Billion Euro cost of cyber attacks on networks

I note that the as cyber attacks on networks become more sophisticated the EC has forecast a 10% – 20% probability that telecoms networks will suffer a major breakdown within the next 10 years.  They have also estimated a potential global cost of 193 billion Euros as a consequence of such a breakdown.

To mitigate against such a scenario the EC is establishing a Public-Private Partnership for Resilience which “will help businesses share information with public authorities to ensure that adequate and consistent levels of preventive detection, emergency and recovery measures are in place in all Member states”.

I’m all in favour of this kind of thing though somewhat sceptical about its likely efficacy.  Industry is more likely in my mind to sort out its own shop through the likes of the IETF and LINX et al.

That said I do think that Government is goingto have to become far more deeply involved than it is in the internet space.  We are seeing it starting to happen and the Digital Britain Report (final report due out in early May which is light speed as far as Government is concerned) is part of this. 

It is easy to see that the potential cost of telecom network disruption could be huge.  I don’t know what the likely cost of last week’s BT fibre break in London is going to be but  just the costs of managing customer complaints would have been significant let alone the costs of the disruption to traffic. 

I missed out on a few news items to comment on last week due to being on holiday.  It’s good to be back though 🙂 .

Business internet peering

London Internet Exchange market data – from LINX64

66% of the global routing table is carried by LINX. This means an Internet Service Provider can connect their customers  to 66% of the webservers  (is websites) in the world just by hooking up to LINX  at their Docklands locations. Using Peering Exchanges like LINX allows us to cut down on expensive internet connections.

LINX has 57 of the world’s top 100 network operators as members, including 16 of the top 20. This confirms the not-for profit organisation as one of the world’s leading peering points.

In 2008 they had 13 membership cancellations of which 8 were consolidations. There are a further 6 consolidations in the pipeline. An indication of the ongoing rationalisation of the industry.

Finally I have put a pie chart together illustrating the distribution of ports at LINX in terms of 100Meg, Gig and 10Gig Ethernet. Not shown are stats that the 10Gig ports are on the rise and the 100Meg, perhaps unsurprisingly considering the rise in internet usage, in decline.


Engineer internet peering


Timico is a member of LINX, or the London Internet Exchange. Linx is a not for profit  meeting point in London where ISPs and network operators meet to peer their traffic, ie to share their connectivity with one another.

It is a sign of the pace of growth in internet related activity that membership of LINX rose  in 2008 to 308, up by around 20% from the previous year. The peak traffic carried over the LINX network is over 400Gbps which is a lot of ADSL connections.

The LINX meetings are not only good networking opportunities but a great place to keep up with developments in internet technology. This week the subject matter includes at IPv6, DNS security and SPAM. LINX64 is sponsored by Telehouse.

I’ll post any useful material as it happens.

Business internet media piracy

ISP and Music industries meet at UK Summit

At the board room of the Performing Rights Society in London today the great and the good of the UK Music industry met with representatives from the mainstream ISP community for an open discussion on how to handle illegal P2P music downloading.

Organisations represented included UK Music,  BAC&S, PPL, PRS, MMF, MPA, MU, MCPS, MPG, Timico, ISPA, O2, Orange, AOL, Yahoo, BT, GlobalMix, LINX, Playlouder and KCom. I’m sure I’ve missed some out and you will have to work out for yourselves what some of the acronyms stand for.

I was essentially there on behalf of the Internet Service Providers’ Association to represent the smaller ISP community who have been left out of the talks up until now. Whilst the “big six” largest ISPs probably represent over 90% of the market the other ISPs, of which there are easily in excess of 300, do represent a “significant other”.

As much as anything the meeting was a “getting to know each others’ perspective” session but a few points in particular stuck in my mind.

  1. We were not allowed to discuss commercial issues and there was a lawyer sat in the corner who interrupted whenever the conversation moved towards this area – the concern being that nobody wanted the meeting to be seen as price fixing. I understand that any initiatives up until now have failed because the Music Industry can’t agree on prices that will allow ISPs to make money out of offering legal music download services. 
  2. It was suggested by yours truly that to make the whole business model work there needed to be a wholesale provider that would make it easier for smaller businesses to participate.  This wholesale provider would have sorted out the rats nest of copyright and licensing issues. Some larger ISPs had 5 corporate lawyers in a department exclusively dedicated to this area. What hope the rest of us!

There is clearly some way to go to get to a working solution although there was general agreement around the table that  everybody wanted to help.

ISPs present were asked whether P2P traffic caused problems for them on their network. I stated that typically B2B ISPs did not throttle P2P traffic  and customers were provided with a high quality experierience for which they paid a premium.

In the consumer space customers seem not prepared to pay for quality and thus in order to try and preserve a reasonable experience for “ordinary” applications such as browsing and email  it is often standard practice for ISPs to throttle P2P traffic. In fact in fairness some ISPs publish these policies on their website. This touched a nerve with one Tier 1 ISP who avoided the word throttling using, instead,  “traffic management” as a less contentious phrase.