Skip to main content
Big Ideas 2024 Is Here! Download Now
Mentioned Companies: DATA, HDP, IBM, NEWR, QLIK, SAP, SPLK

Big Data Analytics: Life Preservers to Companies Drowning in Data

January 26, 2015
4 min read

Data has always been critical to business decision making. Companies have devoted significant resources to expand into the digital universe, collecting, structuring, and analyzing data such as customer information, purchase orders, inventory tracking, and transactional records.

As businesses have digitized operations, the amount of data has grown faster than their ability to process and organize it. They desperately need tools to capture the digital breadcrumbs that customers leave behind.

Behavior that once happened in-store or over-the-phone is moving on-line, resulting in orders of magnitude more data. Businesses can answer many more questions about their customers. How long do customers browse before making a selection? Is there a stage in the transaction process that takes abnormally long? What items do customers buy together? Are searches for additional items going unfulfilled?

These digital breadcrumbs are valuable, but storing and analyzing them properly is challenging. Historically, the digital universe, or the amount of data created, replicated and consumed each year, closely tracked digital storage capacity. For example, in 2006 the digital universe was 1.7 times greater than annual storage production.[1] Now, it outstrips capacity by roughly 500%, as shown below.

digital universe, data storgage, splunk, hortonworks, big data, ark investment management, ARKW, webx0, arkwebx0

According to the IDC and EMC’s digital universe study, the growth of the digital universe is accelerating. As shown below, it grew by 25% at a compound annual rate from 2006 to 2014, but is projected to grow 39% annually over the next five years, increasing from 8.5 zettabytes (ZB)[2] at the end of 2015 to 44 ZB by 2020.

digital universe, data storgage, splunk, hortonworks, big data, ark investment management, ARKW, webx0, arkwebx0

Businesses are inundated with more data than they can handle, and need new tools to incorporate insights. Big data analytics and visualization companies provide solutions for the volume and variety of data being produced. Companies like New Relic [NEWR], Splunk [SPLK], Tableau [DATA], Qlik [QLIK], Hortonworks [HDP], and Cloudera provide solutions for companies trying to transform rivers of data into useful information.

From a $7B market in 2014, data analytics is poised to scale multiples higher. Of the 6.1 ZB of data generated in 2014, 22% contained valuable information but only 1% was analyzed, according to IDC and EMC, suggesting a cost of roughly $.10 per GB analyzed.[3][4]  Though prices are declining, if all valuable data were analyzed at $0.10/GB, the total addressable analytics market would measure in the hundreds of billions of dollars.

The total addressable market will depend on how fast the cost/GB analyzed declines. Companies with truly innovative and useful data platforms will have an advantage because superior technology creates a pricing moat. For example, Splunk, which specializes in capturing disparate data streams and rendering them comparable, has held pricing power well above the average cost/GB. By our calculations it generated $1.00 per GB analyzed in 2014. Interestingly, even at this price point Splunk is taking share, suggesting the high value-add of its analytics.

digital universe, data storgage, splunk, hortonworks, big data, ark investment management, ARKW, webx0, arkwebx0

Clearly Splunk, New Relic, Hortonworks, Cloudera, and even mature companies such as SAP and IBM are capitalizing on the earliest innings of the data inundation game. While companies have always collected data, and the transaction ledger has been valuable for centuries, bits now are flooding the so-called ledger book. The advantage will lie with the real-time-ledger-book makers who can transform the torrent of data into actionable knowledge.

You are leaving
By clicking below you acknowledge that you are navigating away from and will be connected to ARK Investment Management LLC manages both web domains. Please take note of ARK’s privacy policy, terms of use, and disclosures that may vary between sites.