Billions of shades of grey. That’s one way to think about the phenomenon called big data, our propensity to generate and store astonishingly large amounts of information in digital formats (a zettabyte, or a billion billion megabytes, last year), and our ever more refined and revealing ways of searching and interpreting it. Computers used to be limited to looking for answers and performing tasks that were essentially either black or white. But, thanks to ever-dropping costs, and ever increasing capabilities, that’s all changed. Sophisticated computers and software are no longer confined to responding in black or white, but explore and interpret billions of shades of grey. Within all that data, most of it raw and “unstructured”, coming from mobiles, computers and other devices, sensors, webpages and more, computers search and find patterns and anomalies that can bring hidden insights.
Big data creates fantastic opportunities in areas such as the sciences, social and medical research, healthcare, government services, financial markets and customer care. We can pinpoint genes associated with specific diseases. We can improve public transport and management of electricity grids. On a more prosaic level, retailers can offer us more of what we want, thanks to tracking us through loyalty cards or our online shopping habits, offering us coupons and other perquisites. And the Government hopes big data will have a big upside for Ireland, providing new jobs in a growing area.
But big data has its less benign, even sinister, aspects. Commercial harvesting of our personal data remains controversial, often done without our full awareness or complete understanding. Inadvertently, we are turning into valuable commodities for companies that want to either sell or buy the insights our personal information brings. Such commercial abuses pale against possible misuses in government, law enforcement and intelligence spheres, where privacy and basic human rights could be threatened.
Large stores of personal data make for interesting snooping, especially when numerous databases are blended together. This can facilitate nosy privacy infringements but also, revelations and misinterpretation that can devastate lives. Under the banner of fighting terrorism and crime, many states worldwide are enacting laws that enable its agencies to gain unprecedented access to citizen data -- stored for increasingly long periods of time -- with few restrictions, and no obligation to reveal what they are doing, or why. This is the bleaker side of big data, requiring responsible national and international regulation and control. Big data offers spectacular benefits for us collectively, but we cannot allow states or commercial organisations to forget that individually, we are far more than just an aggregation of data, to be bought, sold, interpreted and exploited.
