Data Centre Miniaturisation

The Advent of Data Storage Miniaturisation

Data centre miniaturisation is revolutionising the way organisations manage and deploy IT infrastructure. As the demand for faster processing, lower latency, and greater scalability increases, traditional data centres are evolving into smaller, more efficient versions of their former selves. Miniaturisation leverages advanced technologies such as high-density servers, edge computing, and virtualisation to deliver the same or enhanced performance within a significantly reduced physical footprint.

This shift is driven by several critical factors, including the rising cost of real estate, energy consumption, and the growing need for localised data processing in industries like healthcare, finance, and manufacturing. By shrinking the size of data centres, businesses can lower operational costs, reduce environmental impact, and bring computing power closer to end users.

Since I entered the IT business many decades ago storage capacities have risen and storage footprints have fallen dramatically.  The rush to the cloud and reducing costs in efficiency, data management, scalability and flexibility was the way forward.  The cloud first mantra was followed and data centres were shrunk or converted into offices and IT staff numbers reduced.

Data Storage in 2024

Here we are in 2024 and over the past decade many businesses rely on the cloud for their day-to-day operations.  During this time storage capacities have risen dramatically, costs have dropped enormously and the overall storage footprint has reduced.

For example, a Petabyte of storage in 2010 would have consumed below using the largest available drives of 3TB:

Assuming a storage array was 3U and held 16 drives the petabyte of storage would fill:

• 1.5 data racks and take up 66U
• Use 333 3TB drives
• The disk drives + chassis would consume approximately 9.3Kw/hr of electricity
• The number of 3U storage arrays would be 21 at a cost of £15,000 and a total outlay excluding support, installation and maintenance of £315,000!

For comparison here is what 1PB of storage looks like in 2024.

• 5U of racks space – Uses 13x less rack space
• Use 80 20TB drives providing 1.6PB of storage!  Drive capacities have increased a minimum of 6x or 11.6x if using 35TB drives and with 60% more disk storage space!
• The disk drives + chassis would consume approximately 940W/hr of electricity – A reduction of 1000% energy use!
• The cost today of a 1.6PB solution is around £75,000 a reduction of 400% less than the 2010 solution and includes 5 years support, installation and maintenance.

It is clear from the above that over the past decade data storage costs, space utilisation and energy use have dropped massively.

Future data centre

The data centre of today will look dramatically different.  During the next few years servers, networking and storage reside today, contained in a single room.  The data centre of tomorrow could fit in a broom cupboard with increasing storage densities and amazing performance, a question arises why put data in the cloud? If the storage cost efficiencies are now firmly swinging in having a localised data centre where access to information is far faster than any cloud service, latency is a thing of the past and users no longer need to wait for network congestion to subside.

System reliability

Along with miniaturisation the reliability of IT equipment has risen dramatically.  If a disk drive reported disk errors in 2010 the whole drive would need to be replaced, whereas ADAPT Distributed Allocation Protection Technology—is Seagate’s next-generation erasure encoding solution. It replaces traditional RAID types with a protection scheme that distributes the parity across a larger set of HDDs or SSDs. The upshots? Data protection is now available at a capacity higher than ever before—with rebuilds that are up to 95% faster than with traditional solutions.

The falling price of storage

The cost of storage is constantly falling even though the footprint remains the same.  In 1990 a 3.5” disk drive could store 100MB and cost $750 or $7.5/MB.  Since 1990 hard disks have undergone a massive technological transformation, a 3.5” hard disk in 2024 now stores 32TB’s and costs $1,400 or $0.00004/MB a reduction in price of 187,500% whilst storing 33,554,432 times more data within the same physical size.

Cloud uses

The cloud is an amazing place for creating documents, sharing files or sending emails.  But if you are a CFO of an organisation constantly looking at the financial outlay of a business and ask the IT Director for his annual budget, the cloud costs are a bit like “how long is a piece of string” and many are out of control, with cloud powered servers not being utilised efficiently or dumping data without understanding it’s real worth the cloud has become an expensive way of running a business’s IT infrastructure.

Cloud smart

Many businesses today are adopting a “cloud smart” or “hybrid cloud” strategy whereby a business reviews the computer systems and data it stores in the cloud and then determines what can be stored locally and what essential information needs to reside in the cloud.  By doing this a business can save £100,000’s annually from its budget to invest in other areas.

Data Storage as an OPEX

Miniaturisation of storage devices allows for faster and more flexible scalability. As businesses grow and generate more data, the need for expanded storage is inevitable. However, with miniaturised storage solutions, companies can scale their infrastructure more easily and at a lower cost. Instead of investing in large, cumbersome systems that require significant upfront capital, businesses can incrementally add smaller, more efficient devices to their existing systems.

Whilst the cloud provides a monthly OPEX, many companies we work with now offer the same value monthly OPEX for on-premise storage, server and networking solutions, including the flexibility to grow or shrink as business requirements dictate the need.

Performance

One of the biggest challenges in hosting cloud applications and retrieving data is performance.  It will never be as fast as a locally hosted application or data due to limitations WAN bandwidth.  We have all become accustomed to accepting a delay of a few seconds when retrieving files from the cloud, but nobody looks at the overall loss of productivity for larger businesses.  For example, a business of 5,000 employees who access one file daily with a 5 second delay costs a business 73.5 lost days of productivity per year!

Another issue is that data growth is constantly on the rise and backups are a key part of IT.  Whilst storing backups in the cloud is ideal, this also creates issues in the event of a disaster and how quickly a business can restore the information from the cloud.  For example, in a perfect world restoring 50TB of data from the cloud over a 1000Mbps link would take 111 hours assuming 100% of available WAN bandwidth.  If that data was backed up locally over a 10Gb/s LAN network, it would take 1.5 hours to restore the same 50TB’s!

The continuing advancement of technology

Over the next 10 years storage capacities will increase 4-5x from today, LTO tape a long-forgotten storage technology will in 2025 launch with a capacity of 36TB native or 90TB’s compressed, on a tape no bigger than a large box of matches, that is a perfect, as an air-gapped solution against ransomware.

With disk drives capacities of 35TB today, 100TB+ in 2030 with the largest SSD drive capacities currently at 64TB and due in 2025 an SSD drive of 120TB+ will launch.  Clearly this shows that the data centre of tomorrow will be considerably smaller than even those of today and this drives down hardware costs, energy costs and space requirements.

Summary

It is clear the cloud is here to stay but increasingly many businesses are now examining the unpredictable costs of storing data and running applications in the cloud and this is the reason businesses are questioning the viability of once again running applications and data storage locally.

We can help businesses with data in the cloud, provide viable cloud alternatives or help create a private cloud, manage data management and migrations from and to the cloud, along with high performant local storage, servers and networking.