broadband Engineer

When Things Go Wrong…

Broadband Network Operations Centre spots when a line goes down

I was in the broadband Network Operations Centre  the other day talking to the guys. We were looking at our instance of the Solar Winds network monitoring tool. Then I remembered I was on the way to see a customer later that day so I asked to see their network status.broadband

Blow me down if one of their sites wasn’t showing up as red – connectivity problem. I sauntered over to first line and to my delight one of the team was already looking at it. It was an ADSL connection. As we were watching the status reverted to green. ADSLs sometimes lose and reestablish connections on their own – that’s the vagaries of copper for you.

It’s nice when systems work like this. With ADSL we have a suite of tests we can run to see where the problem might lie. Sometimes the customer has unplugged the router for some reason. If we can’t find out the answer we spring into action depending on what we have agreed with the customer. Maybe it’s just a phone call asking him to switch the router back on. Maybe its a site visit.

Anyway I suspect that on this occasion the customer hadn’t even noticed there was a problem, but we did…

c ya

Business Cloud internet

Rolls Royce visit Timico NOC as model example – well impressed

Rolls Royce visit to the Timico NOCIt’s hugely satisfying to get contacted by people out of the blue to be asked if they can come and visit your Network Operations Centre because they have heard that it is a great reference  model.

This was the case with engineering giant Rolls Royce who are building their own NOC and wanted to come and visit ours to give them some ideas. Ten of them turned up for a look around and I am pleased to report they went away mightily impressed. If anyone else out there wants a tour just drop me a line. We are proud of our NOC and the staff that operate it are a great bunch. I am always delighted to show them off to visitors.

PS they didn’t realise they were going to be photographed – it’s a security measure 🙂

Cloud datacentre Engineer virtualisation

View from inside a Network Operations Centre #digitalbritain

the new Timico Network Operations Centre in Newark has gone liveI’m excited to say that the new NOC is now up and running as the header photo shows. The screen content is evolving at the moment but is intended to serve as a useful visual monitor of what is going on in the Timico network.

Timico NOC being populated with furniture prior to "go live"

The photo on the left shows the view of the Timico NOC from inside "manage" meeting roomroom being built. The photo on the right is the view from the “manage” meeting room which has a glass wall showcasing the facility.

The security levels applied to the NOC itself means access is restricted to NOC staff. No sales people, no customers but there is a great view from the meeting room.

The shift system currently means that some desks are unoccupied at times. However as the business grows the staff will operate a hot desk system with personal possessions kept in a movable tray.

The also use a PCoIP zero client desktop hosted on our VMware platform. When a shift worker logs off and a new person logs on from the same seat the PC instance is completely rebuilt using the new logon credentials.

The official opening of the Timico Newark datacentre and NOC is on Wednesday 18th January.

Business Cloud datacentre

Interactive network design the Newark way

Timico network engineer using new Acer Aspire R3700 interactive whiteboardI hope these guys really appreciate what we do for them at Timico.interactive white board in use at new Timico Network Operations Centre in Newark First we build them a brand new data centre. Then we put in a state of the art coffee machine closely followed by a kennel. And now they get an interactive white board to play with!!!

I’m beginning to feel like a second class citizen. I only have a kettle, a normal old fashioned whiteboard and no kennel. Wossthatallabout?

For those of you who are interested in these things, ok most of you, Acer Aspire R3700 in use at teh new Timico Network Operations Centre in NewarkBen is modelling the latest Acer Aspire R3700 interactive whiteboard. It comes with an overhead projector (ceiling) and a shiny black box (inset). I can hear those oo’s from where I am sat.

On a slightly serious note this is all about productivity. On a slightly less serious note, where do they get these part names from? The Aspire R3700!

What would a world without decimal numbers be like? The Acer Aspire RMMMDCC. There you go. Sorted. Have a good one.

PS I have sneaked in the first photo of the new videowall in action – click on the header pic to see more. It was either this or Sky Sports and the boys chose the former 🙂

Cloud datacentre Engineer

A peek inside the boardroom

connect - signage at the new Timico data centre in Newark

the new boardroom at Timico in NewarkEver wondered what goes oncoffee machine dispensing a latte at the new Timico data centre in Newark inside a board room? Important decisions? Tea or coffee? Cappucino, Americano, latte?  One sugar or two?

All of the above 🙂  Obviously a lot more gets discussed as well and I am pleased to be able to show you our new accommodation. I haven’t yet decided where to sit  – another of those decisions that will work itself out as we settle in.

Timico NOC being populated with furniture prior to "go live"The photo on the left is of the development engineering area at the new Timico data centre in NewarkNetwork Operations Centre being populated with furniture. You can see it starting to take shape in front of the video wall.

The pic on the right is part of the development engineering office. This is where all the serious service platform development will happen. Visible is Group Ops Director Calum Malcolm’s office with meeting room. To the right of that is a breakout area which will have sofas etc conducive to creative thinking (and having a nap 🙂 ! )

Read it first on !

Business Cloud datacentre

Timico Data Center Build Update – The Inside Story

Timico HQ

Progress is rapid with the new data center build. In fact I was to some extent lulled into a false sense of pace (if that’s the right way of putting it) because the photo I take every day from the same spot didn’t seem to be changing much.
rear view of new Timico data center build
Casting my photographic net on the other side of the building however gave a completely different picture and on Friday afternoon I wandered down, hard hat and all to take a look. I’ve take a few shots that give quite a good feel for what is going to be delivered in the new build. This first photo is a ground floor view of the rear.

The generators have not been delivered yet but one will go on concrete plinths either side of the central column.

front view of new Timico data centerMoving round the front there is still a big hole in the middle. They are still moving a lot of materials in. This should be covered up within the next couple of weeks.

The big central opening on the top floor will be where the Timico board room is going to be.

Moving inside we have a plant room and views of the first data hall. This will house 72 x 4KW racks using cold aisle containment and free air cooling – this should mean that for substantial parts of the year we don’t need to power the air conditioning.

Timico Data center plant roomTimico data center data hall 1

Were the facility to be exclusively used for hosting virtualised services then the capacity of each rack
can be as high as 100TeraBytes.

In reality the capacity is very much dependent on the mix of resources you have in the rack – high speed/expensive drives versus bigger. This is very much dependent on the application.

under the floor of the new Timico data centerTimico data centre data hall 2 under construction

The shot on the left here shows the flooring under construction. The floor tiles are heavy duty as the floor has to support a considerable weight of the fully loaded racks above.

You don’t wanbt to move a full rack if you can help it, at least not without the use of a fork lift truck.

first floor engineering offices in new Timico Data center NOC at Timico data center under construction

Upstairs on the first floor we have the development engineering offices (left) and the Network Operations Centre.

You can see the raised floor again in the NOC though this does not need to be as heavy duty as in the data halls on the ground floor. The end wall will be flat and covered in monitoring screens.

To finish off here are a couple of photos of me with a view of the existing Timico HQ building from the top floor of the new and one stood on the actual floor of the data hall to give some perspective.

Trefor Davies in the new Timico data center buildingTrefor Davies stood next to the raised floor of a data center hall

There are a few more photos below including ones I consider to be quite arty 🙂 if you are interested.

Business internet Regs

Review of 2009

If you have managed to keep a job in 2009 it has probably not been a bad year for you. For consumers, fuel apart, costs have by and large come down as vendors compete more aggressively in the tough market conditions. In the UK we haven’t started paying for it yet. If you have been out of work in 2009 I guess it will have been a different story.

At work Timico continued to grow both in sales and profitability. It hasn’t been easy but the year end looks as if it will be significantly up on last year.

Highlights in the year include decommissioning our last 155Mbps ATM connections to BT, followed later in the year by our 622Mbps pipes. They have been replaced by resilient Gigabit Ethernet Hostlinks.

We also set up our new Network Operations Centre in Newark and saw the successful move of the NetOps team up to Nottinghamshire from Ipswich.

One of the big success stories of the year is the growth in the high bandwidth leased line business. Uncontended (ie dedicated connectivity) leased lines are becoming more affordable and companies are increasing offloading (at least some) corporate resources into the ”cloud”. We have similarly seen a growth in our MPLS estate with some customers signing up for hundreds of connected sites.

2009 also saw some major technology introductions. ADSL2+ was introduced early in the year. The technology is capable of “up to 24Mbps” though we only quote 16Mbps to our customers – most users will not get the max performance and I think it is better to manage expectations in this way rather than have unhappy customers.

Timico was the second ISP in the country to sell Ethernet in the First Mile and have also been participants in the BT Fibre To The Cabinet  (FTTC) trials, the early stage of the much promoted £1.5Bn investment in Next Generation Access technology.

“Digital Britain” was also a much used “buzzword” during the year. It is easy for me to criticise and I realise it is a lot harder when you are making the actual decisions but I am afraid that we will look back and decide that the present Government did not do a good job on this one. The first 4 months of 2010 are going to be very important with laws being passed or not passed that will potentially adversely affect every internet user in the UK.

Don’t get me wrong though. 2010 is going to be an exciting year with lots happening. More tomorrow.

Engineer internet

Netsumo NOC helter skelter

Paid a visit to Netsumo’s Network Operations Centre today with Ben, our Head of Network Operations. We are looking around at best practice in NOC design and it is always a pleasure to spend time with Netsumo CTO Andy Davidson.

Andy is one of life’s genuinely nice guys and very knowledgeable in the Internet Networking space. Netsumo is based at state of the art office facilities at Electric Works in Sheffield and operates over 30 networks on behalf of is customers. The company represents the epitome of best practice for network operations in the UK.

Like all good technical businesses Netsumo looks after its engineers and the working environment is second to none. At centre stage of this is a four storey helter skelter that transports staff to the ground floor in seconds.

Of course we had to try it out – see photos below.

Andy tells us that the helter skelter has two functions. Firstly it is used for rapid deployment of engineers when on an urgent call out. Secondly it is used for rapid deployment of engineers when off to the pub after work. Both are laudible objectives I’m sure most will agree.

netsumo3netsumo1Pics are a little small – I uploaded them to Facebook from my mobile and that is how they turned out!netsumo2

Business internet

New Technical Operations Centre – call for ideas

Having moved the Timico Network Operations Team to Newark this summer I am looking at upgrading our facilities to provide the business with a state of the art Operations Centre. With this in mind if anyone has any inputs on how we might approach this now is the time.

If anyone out there who are proud of their own facilities and wants to invite me to visit for a look see then I will gratefully accept the invitation.

I am also happy to be approached by vendors in this space if they have something that will add value here.  This could include monitoring equipment, posh screens, ergonomically optimised workstations, electronic whiteboards or even just nice plants etc, etc.

My contact details are available here.

End User fun stuff

The Ducks Are Lined Up

We have just finished hiring more staff for the Networks Operation Team. This has taken some time because we wanted to be sure that we found the right people.

I have to say that the wait has been worth it. Timico now has a superb core of people that will drive the network forwards. I’m sure it helped that they had the new Network Operations Centre to move into but also that there is a lot of technical action here at Timico. The business is constantly developing.

We are very fortunate as a business in the people that we employ. Today I sat in at the final presentation of a group of corporate account managers who had been working on developing multi-site communications solutions proposals. The team, mentored by yours truly, did a great job (I know, I know I would say that wouldn’t I).

I guess the point is that it’s people that make the difference to a business and to our customers and I am happy to be working with such a great crowd.

The photo? Ducks, because they have to be lined up to be successful, and because you can’t always take life too seriously. 

Engineer internet security

The buzz of the Network Operations Centre

It always gives me a great buzz to sit in our NOC. It’s because when we started Timico only 4 years ago there were only four of us sat in the room of Tim Radford’s parents’ stable block (it was cheap and there was no room at the inn anyway). Now on a normal working day there are more people sat in the NOC than there were in that original room. It is a world away.

Today sat in the NOC some of the engineers were setting up a MPLS PWAN for a customer. This particular PWAN had over 80 sites – a mixture of leased lines and ADSL. In itself it isn’t a big news item. It isn’t our biggest PWAN by a long chalk. However it is another new customer and an endorsment of what we set out to achieve four years ago sat in the stable block.

It is a good feeling to be at Timico.