Our digital universe keeps expanding

For the London Free Press - Jan 17, 2011 Read this on Canoe

The sheer volume of digital information that we create is fast outstripping our ability to manage it all, report warns

The sheer volume of digital information continues to rapidly increase. According to a report by IDC entitled The Digital Universe Decade - Are You Ready?, commissioned by storage vendor EMC, the projected growth of the digital universe could outpace our ability to manage it, creating new challenges and opportunities.

Every time we send an e-mail, take a digital photo, blog, upload a video or download a song, we are contributing digital content. The report uses the term "digital universe" to mean the amount of digital information created and replicated each year.

This content is growing and is expected to increase exponentially. In 2009, the digital universe grew by a staggering 62% to about 800,000 petabytes (a million gigabytes).

In 2010, the digital universe was expected to grow to 1.2 million petabytes and reach 35 trillion gigabytes by 2020. That would fill a stack of DVDs that would reach half way to Mars.

The report says that over the course of the next 20 years, the digital universe will grow by 44 times, while the personnel and investment in resources to manage it will only grow by 1.4 times. This discrepancy will have real implications for both the organizations tasked with dealing with digital content and regular users and contributors to the digital universe.

Issues that arise include the amount of physical storage needed to contain all this data. This is in part attributable to the fact that only 25% of digital content being created is unique - the other 75% consists of things such as forwarded e-mails and other copies.

And backing up all that data so it won't be lost if something goes wrong faces challenges from the sheer volume, and managing the most effective and cost-effective ways of doing that.

Individuals will use higher-capacity hard drives in their computers, external hard drives, and the cloud to store and back up their personal material.

The report suggests an increasing amount of data will be housed in the cloud. This goes beyond keeping our files or backups stored at Internet-based locations. Examples include watching on-demand Internet-based TV, such as Netflix online, instead of using DVDs, and using cloud-based software rather than installing and running it on our PCs.

Finding what we need in all this data will require continued advances in ways to manage it. That includes ways to know when to delete data, and search tools to find what we need.

The report also says that the amount of data that needs protecting will increase at even a faster rate. This includes confidential and personal information, such as financial and health data. It claims that less than 10% of the information about an individual is created by the individual - such as taking photos, using social media, sending e-mails, and getting cash from an ABM. The rest is created by others, such as credit records, surveillance photos and web-use histories.

Managing the security and privacy of all this will continue to be a challenge.