Sowing the Future’s Harvest: Working with Big Datasets

Big data is one of those concepts that few people can envision in their mind’s eye. Personally, I try to imagine a stack of paper that could be digitized into one gigabyte of information. Then I imagine stacks of paper that would fill 100 gigs of space. Then a terabyte’s worth, a whole server, a room of servers, ten rooms of servers, and on and on. Even with all these incremental steps, it is impossible to imagine how much data truly represents a “big” dataset. If we can’t even imagine it, how can we work with it, analyze it, search it to find the exact piece of paper among all the countless digital pages that we need for that day?

The McKinsey Global Institute recently published a report on the topic of big data , in which they discussed solutions that might be used to tackle this significant challenge of the future. They provide predictions and suggestions for how both government entities and the private sector will have to adapt to daily interaction with big data. McKinsey maintains a very positive outlook on how beneficial big datasets will be to society – FileBank shares this optimism, and we aim to harness it in every aspect of our work. FileBank CEO Greg Copeland recently had this to say about datasets and McKinsey's research on them: "The data gets bigger, the images more refined and consequently larger. The technology will from this pressure grow to meet these needs. The trends and benefits of this ever grander dimension will reap benefits to those who can sift through information and understand the driving undercurrent trends. "

But what does McKinsey’s research mean for the average sized organization, like a school, a small business, hospital, local government or office? In the future we will all have to use these big datasets efficiently, but how will we do it? Without a doubt, the key is familiarity and ease of use. It is one thing for technological wizards at groundbreaking companies to adopt big data, but quite another for a local employee. Once all of these trends become mainstream, and once platforms have been developed that can parse and sort through mountains of data with a single click, then the big data revolution will have truly arrived.

At FileBank, we believe that there are and will be many solutions available to both the public and private sector for handling big data in the future. While we develop those solutions within our own organization, we believe that keeping track of industry developments will be essential to all our customers. Please continue to check our blog for these industry updates and our analysis of them.

Until next time,

Vanessa Banti

Master Archivist