This Eckerson Group report recommends 10 vital steps to attain success in DataOps.READ MORE
People have been writing about “Digital Transformation” for years and no one could quite decide what was meant by it. All they knew was that it was supposed to be a good thing.
Now it has become clear that digital transformation is all about taking 20th-century companies and preparing them to use data as an engine for growth and competitive advantage. And while that is a very easy sentence to write, the actual execution of a digital transformation strategy in the real world when you are the CEO, CTO or CIO of a company that existed long before “Google” was a verb, is much easier said than done.
One thing we are now confident of is that the ability to effectively use data in an agile manner in support of business decision making is critical for any digital transformation strategy. So building out a very rough (and I mean very rough so please don’t overanalyze the analogy) digital transformation equivalent of Mazlow’s Hierarchy of Needs, you get a pyramid that looks something like this.
At the base, you have the foundation built around Enterprise Data Operations and Orchestration (EDO2). This is the baseline necessary to do anything useful with the data you have. Think about it in the same way companies think about a CRM for their sales operations. You wouldn’t build a modern billion-dollar company without a system that helps you manage your customer interactions and your sales processes. In the same way, you wouldn’t have a billion-dollar organization without financial software to manage your financial processes. The same is true for data IF you want to use it effectively as a competitive differentiator. As a result, the foundation of a digital transformation strategy pyramid has to start with systems and processes that enable businesses to organize and manage data from disparate sources and process the data for delivery to analytic applications.
The next step up from the EDO2 systems are the analytic pipelines that are used to move data around in support of decision-making systems. There has been confusion in recent years that the goal of EDO2 and the subsequent pipeline development is to support pipelines that are able to support “big data”. That is one condition for success when an organization has lots of data and lots of data types, but today, it is more or less table stakes.
The more important criteria is the ability to be able to build 100s or 1000s of supportable and maintainable data pipelines quickly. It is less about the volume of data, and more about the volume of data pipelines because each pipeline represents a new business idea that is being tested and rejected, or tested and used. But, if it takes 6 months to create an analytics pipeline just to test out a new business model, that model may have already been put into production by a more nimble competitor.
It really is about agility at this level in terms of creating and operationalizing new pipelines. Note that the operationalization of pipelines is incredibly critical as most analytics projects end up failing when they attempt to go from the development sandbox into production. While there has been a significant focus on tools that are used to develop data pipelines, that turns out not to be where most of the challenge is located. Equal attention needs to be paid to the hardening of those pipelines for enterprise use.
The next level up then is the actual analytics performed by applications using those pipelines like next best offer applications in a retail setting, or BI dashboards or machine learning models used to optimize drilling in an oil field or reduce credit card fraud. The same points made above about the operationalization of pipelines apply to the use of data within these applications as well.
Ultimately, the goal of creating these data-oriented applications is to provide information that drives enhanced customer experience, improves overall business operations, or creates new insights that drive decision making. Improvements in those areas result in gains around creating new and sustainable customer advantages that result in improved business growth, which is what sits at the top of the pyramid.
In the end, the foundation built by establishing the skills, technology, and processes around great enterprise data operations and orchestration will provide the fuel that drives competitive advantage for companies in the 21st century.