I’m back from my hols. The news is my hard drive has 4.5GB more data on it than it did before I went on holiday. All videos and photos. In June I added 17.5GB worth of media. That’s 54GB ytd and if I extrapolate that to the year end that means I will have added 108GB to my laptop in 2013.

I’m sure I’ve recently had this conversation on the blog but I now have less than 20GB of space left on my laptop, after cleaning it up. I can’t realistically expect to get a new laptop. Mine is relatively new and I can’t expect the business to provide more storage for my personal photos. They will have to go.

So I have to come up with a strategy here. I still want easy access to the photos but it isn’t worth forking out the small fortune that the likes of Dropbox and Google Drive will charge for that level of storage.

The charts below show my own usage growth. By the end of this year at the current rate of growth my photos will take up around 330GB of space. The annual growth rate since 2004 has been around 50% a year – roughly in line with Moores law and also in line with an EMC study of storage capacity trends from 2011. Extrapolating the number forward to 2020 I can expect to be consuming almost 2TB a year of storage with a total accumulated requirement of 5.5TB.

Remember data beyond mid 2013 is extrapolated/forecast.

Does this make sense I ask myself? Well the video formats in use by 2020 are quite likely to be approaching 8k which at 350Mbps streaming data rates will produce a storage need of 44MBps or 2.6GB a minute. That’s only 13 hours of video storage. All these are approximate calculations and do assume that I am onto 8k by 2020 but the 5.5TB by 2020 sounds very plausible to me.

It’s interesting to note that whilst I am discussing my own personal data usage here the rate of growth very much reflects what we are seeing as a business. We have all heard the term Big Data bandied around. Businesses are gathering far more data than they used to. Information is power. The decisions I am facing are therefore similar decisions to those seen by businesses of all sizes. What do I do with my data?

Let’s look at my personal choices first. Currently I back up my laptop to two separate external drives. Frankly this isn’t going to be good enough for our family going forward. If I’m taking media off my laptop I want it to always be available at a click of a button. It has to be Networked Attached Storage or NAS. Because I am going to be using it more and more for lots of different family storage needs the NAS also has to be resilient so it either needs to have two mirrored hard drives or be a multiple RAIDED box such as the Drobo box shown in the pictures inset. (Box shown has 3 out of 5 slots populated with 1TB drives giving 2TB usable capacity).

We use Drobo boxes (1st pic with front cover – 2nd without) to send to some of our cloud customers to seed their online storage. A small business will not have a fast enough internet connection to upload a couple of Terabytes say in a timely manner. If they were lucky enough to get 10Mbps upload then it would take them well over 400 hours to upload the data. Even though Timico provides that customer with free bandwidth to perform the upload that isn’t a practical proposition.

The Drobo box is going to cost me knocking on a grand even if I just put in three drives and though by my current calculations as an infrastructure it would do me for the rest of this decade, taking into account the fact that I could easily upgrade the hard drive capacities, Mrs Davies wouldn’t want me to spend that kind of money. We need a new freezer and the dishwasher is about to pack up.

So I’m going to go for a cheaper option such as a dual ReadyNAS which with a couple of mirrored 2TB hard drives will probably cost around £300. I will still want to have a separate backup for this but will in the short term stick with the 2TB SSD I currently use. With that setup I reckon I will be good until mid 2018 and could probably also upgrade the hard drives at that time. Rather than have the hard drives switched on all the time I will also probably go for a power switch that I can control over the LAN. In effect it will still be instant access.

It still makes sense to have a cloud option and I should be able to go for a cheaper solution such as Amazon Glacier which costs 1cent a month per GB. This would cost me $3 a month for 300GB, just over two quid say or about £25 a year (mixing formats :)). Amazon Glacier is much cheaper than the $20 a month that Google Drive would charge me but isn’t a comparable service. Being a “deep storage” service the recovery time from Amazon is very slow versus “immediate” from Google depending on your internet download capacity. $240 a year isn’t a practical proposition for home use. I don’t need the recovery to be fast. I just need it to work as it will be the line of last defence.

The observation that springs to mind here is that the costs discussed here are insurance policies that I have not hitherto had to pay. Our use of technology is driving the change. According to my calculations by 2020 I would be paying Amazon $55 a month for the storage based on current prices. It is reasonable to expect prices to plummet but this is obviously a nice growth market to be in.

The needs of a business are similar but different. Firstly a business will typically have a lot more people generating the data. The amount of data being generated per person however is not wildly adrift of the numbers I’ve been discussing for my own usage. There’s a good chart over at NetworkWorld that tells us the average storage need per employee ranges from 160GB for small businesses up to 190GB for mid-sized companies. You can do your own calculations for forecast storage needs based on the size of your own particular business and assuming 50% a year growth.

The other issues affecting businesses relate to skillsets (overhead), security (life and death of the business) and recovery time (revenue and opportunity cost). It all revolves around money.

The numbers I have looked at for my own home use therefore don’t copy across to business. Businesses are willing to spend more for additional security, ease of use and speed of recovery. A business may also think it important to know where their data is stored – for UK regulatory requirements for example – and want to have someone to call if something goes wrong. Personal support is not something that the big cloud providers are known for.

Check out example services here and corporate services here.