Finding room for data after 'digital big bang'

With enough digital information created last year to fill three million times all the books ever written, data storage is about…

With enough digital information created last year to fill three million times all the books ever written, data storage is about to become big business, writes John Collins

The term information technology, or IT, is often bandied about in the corporate world. But according to one leading expert, a massive change in mindset is going to be required where the emphasis is on information rather than technology.

That's because the world is facing a "digital big bang" due to the explosion in information being created by the use of digital photos, e-mails, digital music and mobile phones, not to mention websites like YouTube and Bebo, where users are encouraged to create their own content.

Chuck Hollis, corporate blogger and vice-president of technology alliances with storage technology giant EMC, says value now lies in the information rather than in the technology used to store and manipulate it.

READ MORE

EMC recently engaged market research firm IDC to carry out a study of the amount of digital information being created and stored around the world.

The study produced what Hollis calls a "Carl Sagan number" - a reference to the mind-boggling large figures the astro-physicist used to deal with.

It found that 161 exabytes - 161 billion gigabytes - of digital data was created in 2006.

The human mind has trouble grasping the scale of such large figures but, to put it more simply, that's enough information to fill 5.3 billion iPods (of the 30 gigabyte variety) or about three million times the information in all the books ever written. What's more staggering is that it was all created in just one year.

The big concern for those who have to manage and store that information is that IDC predicts that this year the amount of information created will surpass the total storage capacity currently available.

As well as being a technical challenge, this also raises a number of social issues, Hollis believes - not least who decides what is kept and what is deleted.

"Who is saving all of this information? Just look at our little disaster with the Iraq war. A hundred years from now, someone is going to want to study this. What entity is preserving all of this? What's being saved? What's being edited on the way in? There are also a lot of technical issues - what media and format are we going to write it on so that 100 years from now it can be read?"

Some statistics underline the scale of the problem the world is facing as we risk drowning in a sea of information. YouTube, a service that's only been around for two years, serves up 100 million video streams a day. The average person in Britain is caught on surveillance cameras 300 times a day.

Experts say more than a billion songs a day are shared over the internet in MP3 format. Seven million pages of new information are added to the web each day.

While the increase in the amount of data is unquestionable, there are surely questions to be asked about the quality of the information we are producing.

Do we really want to preserve shopping lists, personal blogs that haven't been updated in years and other trivial content? And how much of these billions of gigabytes of information are duplicates - digital rubbish, if you like?

"Rubbish is in the eye of the beholder," counters Hollis. "The elimination of copies assumes there is an authority that has the true copy. Who would be the preserver of the true copy and do we trust them to preserve the truth and not subjectively come back and edit it later?"

While there are numerous social ramifications of this information age - not least serious implications for education, work and relationships - it also has a fundamental impact on those whose task it is to manage the information.

IDC predicts that by 2010 almost 70 per cent of digital information will be created by individuals, but organisations will be responsible for the security, privacy, reliability, and compliance of at least 85 per cent of it.

In order to respond to that change, IT managers and chief information officers in organisations will have to start acting more like money managers, says Hollis.

"If information becomes your single most important asset as a company, IT has to play a new role," he explains.

"Where the chief financial officer (CFO) plays for money, the chief information officer (CIO) has to play for information."

As such, he believes technologists will have to set the framework for how information can be used within their organisation. Currently, most CIOs are struggling with information overload and cannot answer basic questions about information in the same way a CFO reports a com-pany's financial performance.

"Go to the CIO and ask the same questions: 'Do you know where all your information comes from? Do you know how to manage it? Do you know how to protect it? Do you know how to optimise it?' and you get a waving of hands," says Hollis.

Of course, it is in the interests of EMC to talk up the problem - large businesses are going to have to invest heavily in its storage systems to protect their valuable information.

But Hollis counters that this fundamental shift in the importance of information has led EMC to change the way it does business.

Although he doesn't say it, in many ways storage hardware - disc drives, storage area networks and tapes - has become commoditised and Hollis points out that storage now accounts for less than half of EMC's business. The bulk of revenues now comes from software and services.

"Storage is an interesting starting point because that's where information lives," says Hollis.

"It doesn't live on computers, it doesn't live on networks but it has to land somewhere and it lands on the unglamorous things called storage devices."

Hollis says EMC's role is to provide "information infrastructure that can store, protect and optimise this information, regardless of where it comes from or where it is going".

The company is betting big that customers are going to buy into this vision - in the last three years it has spent $3 billion on research and development and another $7 billion acquiring other companies that build out its portfolio of tools.

And as long as we all keep sending text messages, downloading MP3s and e-mailing our digital snaps around the world, it looks like a very good business model.