4G test results in London – comparison of O2, EE and Vodafone on a road trip.

Competition in 4G has been a long time coming. It’s almost a year since EE launched their service and we now have the Vodafone and O2 4G networks running, at least in London. When I took part in the O2 4G trials in 2013 the results were spectacular (43Mbps in the Devonshire Arms pub off Oxford Street) if confined to a few places – O2 used 25 cell sites for these trials. The results were great partly because I doubt that there were that many people using the network given that we all had dongles and not phones. You had to have your laptop out which aside from my coverage experiment conducted from the top deck of a moving number 25 bus meant that you had to be in a static location.

Now I have three networks to play with: EE, Vodafone and O2. It would be natural to expect that having had longer to roll out their network the EE coverage would be better. However with more subscribers using the EE service would their speeds be as good as the relatively empty networks of the new kids on the block?

The 4G test tools to hand were a Samsung GalaxyS4 running O2, a Nokia Lumia 920 on Vodafone and a Huawei 4G Mobile WiFi E5776 (MiFi) loaned to be my EE. The tests were conducted over two separate trips and on each occasion I had a Twitter pal along for the ride: @flosoft and @UKTamo. We also used @UKTamo’s SGS3 LTE running on EE.

In one sense because I was using four different devices the test conditions were not going to survive academic scrutiny. However having to go to the effort of swapping SIMs every time I wanted to run a test just so that I could do like for like testing wasn’t going to be practical. What you get here therefore is a mix of experiences with some real results mixed with subjectivity. It should provide a feel for the 4G experience in London.

We started off in McDonalds in Kings Cross. Day one was not an unbridled success as for much of the day the only network I had working was EE.  Having only just provisioned them, the new 4G SIMs on the other two took a while to kick in. Before realising this I thought that maybe the S4 and Lumia 920 needed a firmware upgrade. @flosoft averaged around 5Mbps using the McDonalds WiFi to download the software for the Samsung whereas I was getting double that using the EE 4G MiFi for the Nokia. Nokia took well over half an hour to perform the actual upgrade after downloading the software but it had still finished the job before the WiFi based software download for the Samsung had ended, let alone start the installation.

This became a theme. During lunch at the Nag’s Head in Covent Garden hanging off the EE4G Huawei MiFi was a better experience than using the pub’s WiFi. This is despite the fact that my Galaxy S4 is set to backup media to Google+ when connected via WiFi. Because of this any speed testing and usage on the MiFi will have been degraded because of the background uploading yet the experience was still good. It suggests to me that as 4G becomes more ubiquitous, cost of data aside, public venues will need to upgrade their broadband service if they want people to continue their WiFi rather than a cellular service.

As an aside during the 2012 Olympics I spent a lot of time testing mobile connectivity in London and found that when walking around the cellular networks were far more useful than the hundreds of thousands of WiFi hotspots in town.

Will 4G render public WiFi networks obsolete I wonder?

Roaming around central London saw very variable results with all three networks working on 4G. Handsets would switch between 3G and 4G by just turning a corner and 4G performance when in a low signal strength area felt not to be as good as 3G in the same circumstance. In theory 4G should be no different to 3G in this respect – maybe it just needs a bit more playing with.

Sat on the Number 73 bus between Kings Cross and Oxford Circus the EE network had more consistent 4G coverage than Vodafone – see the video. EE averaged 18Mbps on this route with only a couple of results dropping below 10Mbps to 5Mbps and 8Mbps.

Following on from the Nags Head lunch experience indoor coverage seemed better than I had been expecting. When my Vodafone 4G kicked in I managed to get 65.85Mbps at the back of the Pop Up Brittain shop on Piccadilly. We saw 48.62Mbps down and 43.31Mbps upload with EE in a 2^nd Floor Office in Castle Lane near Victoria which was the best combined performance. I was getting around 10Mbps down with both O2 and Vodafone at this location.

Vodafone and O2 are sharing cell sites so where you got 4G with one you would naturally expect the other to be present. This was by and large the case though sometimes one network would have better performance than the other at these locations which might be explained by traffic volumes.

We used speedtest.net for the testing and when comparing different networks it was important to be using the same server. For EE performance at one location rose dramatically when we switched away from the Vodafone London speedtest server – no dirty work on the go here I’m sure:). It was also funny that when I stood next to Yoda between Covent Garden Station and the Piazza I got a very poor Vodafone signal – the force was obviously elsewhere unless he wasn’t the real Yoda (Vodafone uses Yoda from the Star Wars movies for advertising purposes).

The fastest download seen was 73Mbps on O2 at South Kensington tube station. Sat at the Champagne Bar in Paddington I was regularly getting 58Mbps on Vodafone – indoors again (video here). The EE MiFi in this environment didn’t perform so well. Indoors in Paddington Station might be a poor EE coverage area but my guess is that there were too many WiFi enabled devices in the area and the MiFi struggled with the noise.

Overall I didn’t see quite the same peak speeds on EE compared with O2 and Vodafone. The fact that there are far more people on the EE network would explain this. As you might expect EE did seem to have better overall coverage, though this coverage was far from ubiquitous. There seemed to be pretty good 4G from all three networks in the main tourist and commuter hotspots – Leicester Square, Piccadilly Circus (video here) and major train stations for example.

One additional data point is that I had to plug both the Nokia and Samsung phones in to charge by around 2pm after a day’s testing. I was carrying two Powergen Mobile Juice Pack 6000s especially for this purpose. Whether that tells you anything about battery life when using 4G I’m not sure considering I was hammering the phones. It probably does.

Overall it’s exciting to see three networks up and running now albeit only in London. It won’t be long before competition sees coverage improve everywhere – although it isn’t advertised I could get 4G from all three networks on the platform at Slough Railway Station.

Even the slower 4G speeds were pretty fast compared to 3G. I have to believe that with 4G the mobile networks have finally moved into the 21^st century.

4G is definitely going to drive usage. I used almost 2GBytes in two days of testing with O2 – I’m on an 8GB package. I suspect the real issue is how quickly the networks will want to drive usage/fill their capacity. They will be able to control this with pricing. However although the mobile operators are desperate to move away from selling on price I can’t see them being able to do so long term. The market will have its way…

More speed test screenshots here from O2, Vodafone and EE. Thanks to @flosoft and to @UKTamo for their able assistance especially for the photos and screenshots of the test results. Thanks to EE for the loan of the Huawei MiFi – it’s a great piece of kit. I was hoping to be able to publish a comprehensive database of the tests but unfortunately the speedtest.net app only kept a certain number of results and the Windows 8 Phone version didn’t even seem to allow you to export the data. Ah well.

Other 4G posts:

4G as a fixed broadband replacement service here.

EE 4G mobile broadband roadmap here.

Google Hangouts over 4G here.