Big Data is the buzz phrase of the hour, but behind the hype there are some real and significant changes happening in the way that businesses are dealing with the deluge of data and how the are using it to make decisions.
In a recent survey by CapGemini of predominantly C-level executives working in a range of industries, over 90% said that data was now one of the essential factors of production, equal in importance to land, capital, and labor. At the time the study was carried out, 26% believed that they had seen significant performance improvements in their industry due to advances in data analytics and the willingness of executives to utilize the insights thus garnered for decision making. They expected to see a further 41% improvement over the next three years. The expected improvement is partially attributed to increased penetration of an acceptance of big data as a decision making tool.
At present, 55% believe that Big Data is not viewed as a strategic decision making tool in the higher echelons of their industries. Many executives said that big data culture was not adequately incentivized within their organizations, an opinion which is surprisingly consistent across a range of industries, from the financial sector, energy and resources, and the health and pharmaceutical industries. However, many believe that as the techniques and technologies behind the big data trend are proven effective, a paradigm shift in attitudes is inevitable.
The survey highlighted some more serious blockers to the effective use of big data. Somewhat surprisingly, volume of data did not rank highly among the factors of concern. Of those surveyed, 85% considered the lack of an ability to properly analyse and act on data in real time to be of more importance than volume. Part of the problem is a perceived insufficiency in our current capabilities to deal with unstructured data, with 40% of executives of the opinion that it is too difficult to process with current technologies. However, there is optimism that as computing power, algorithms and systems for dealing with unstructured data, and artificial intelligence systems improve, many of these problems will become soluble.
The next most significant impediment is organizational silos, with 56% of respondents saying that compartmentalized systems are making it very difficult to fully integrate and analyse data streams from across organizations. Although vertical silos are less of a problem than they once were, horizontal silos, such as proprietary CRM and ERP systems constrain data use and limit the scope of analytics that could provide useful guidance for decision making.
Nevertheless, the survey respondents reported that big data was used for decision support 58% of the time, and for automated decision making 29% of the time. The proportions can only grow as big data becomes increasingly crucial to decision making in the coming years.
