Guest blog: Big data to become tomorrow's corporate winner

Staff Writer
Columbus CEO

By Steve Doherty

The latest trend in the tech industry is processing data sets so large and complex that traditional processing solutions are inadequate. The industry calls this new science, "big data" and big data solutions are about increasing speed and simplification of data systems that companies need in order to unlock the value of their data and create highly reliable value-driven decisions.

Three things are changing the way companies process data: volume, velocity, and variety. The volume of data that companies collect is increasing exponentially. In 2005, 1,200 exabytes of data were created worldwide; today that figure has increased a thousand times. Velocity refers to the accelerated data flow into and out of an organization that is driving an ever greater need to deal with the mega amounts of data in near-real time. Variety refers to the range of data types and sources that data come from, such as mobile phones, tablets, emails and social media sales, marketing, websites, as well as business data. Up to 85 percent of that data is unstructured-not just simple words or numbers that are easily categorized-which require a different architecture and technology for analysis.

With 90 percent of data collected in the world today being created in the last two years, the nature of data is vastly different than it was just five years ago. By 2020, businesses will need to manage and analyze fifty times more data than they are currently. This, coupled with the need to deliver data quicker, is putting a lot of pressure on organizations to implement the necessary infrastructure and skill base to react quickly and process it into usable business intelligence quickly enough for executives to make data-driven decisions.

The spread of mobile computing, smart phones, tablets, and hand held devices has created a 24/7-generation of data that does not meet the classical or structured data requirements of older databases. However, the recent emergence of an open source software solution called Hadoop, allows systems to work with massive amounts of structured, semi-structured (email, images, etc.), and unstructured data.

John McCure, CEO of New Albany-based Big Data Masters, confirms that companies can have affordable big data solutions. "The use of multi-core clustered processing systems allows data-engorged companies to implement big data analytics at a fraction of the cost of yesterday's technology," says McCure. "And with Hadoop, companies can retain their existing software code as their hardware and software technology evolves to take advantage of massive clusters without the massive cost."

Steve Doherty is a retired United States Air Force aviator, research scientist, telecom engineer, independent writer, and the author of two World War II covert operation's thrillers. He lives in New Albany and can be contacted at (614) 949-9909 or