This Eckerson Group report recommends 10 vital steps to attain success in DataOps.READ MORE
You’ve probably heard the fable of the blind men and the elephant. One blind man touches the elephant’s tusk and says the animal is actually a spear. Another grabs the tail and says no, it’s a snake. A third man grabs a leg and decides it’s a tree. And so on it goes, with each man drawing conclusions based on his own limited experience.
A couple of months ago, SnapLogic posted the results of a survey that suggests most organizations are similarly only seeing limited pieces of the whole, thanks to their growing problem of data silos.
The data silo definition is, of course, a group of data that isn’t accessible to the enterprise at large. It could be raw data that isn’t available to anyone, or it could be processed data that’s only accessible to a business unit, supplier or other groups. Silos are created when departments either deliberately or inadvertently withhold data from other departments. They can exist when some users have access to certain data tools that others don’t. And they can multiply with mergers and acquisitions.
The problem with siloed data is it prevents people from accessing data that could have been combined with other data to form more complete pictures of whatever insights the user is after. With data silos you end up with redundant data, extra storage costs, and uncertainty when it comes to which data is the most accurate.
The problem is as data grows, silos typically do, too. According to the survey, only 2% of organizations consider themselves to be “completely effective at data sharing.” The rest are still struggling to contend with siloed data. Those surveyed mostly blame inconsistency of systems being used (42%) as well as different data formats (38%), lack of a coordinated data strategy (37%), subpar technology integration (36%) and challenges posed by legacy technologies (36%). Whatever the reason, blocked data is unshared data. Unshared data is underutilized data. If your organization’s data is siloed, any analysis you conduct may very well be less than whole.
One of the companies we work with has existed for over a hundred years. Over time, they acquired a variety of other companies, which greatly compounded their data silo problem. While they had the capital to consolidate their data and build a modern data big data machine, they didn’t have the expertise to effectively build and deploy a working solution. And hiring that kind of talent when you are a 100-year-old company is not that easy. And so for years the organization perpetually delayed implementation of a data integration and consolidation project that if it had been completed would have allowed them to do things as seemingly simple as get a 360-degree view of their IT spend or projects much more valuable like cross channel marketing and sales campaign analyses.
But automation came to this company’s rescue. What would otherwise have been a manual effort that would have taken years to complete with people they did not have, was finished in just a few days thanks to new solutions that allowed them to automate the entire data engineering process on a big data environment using the existing traditional data warehouse skills they already had in house. Walls came down. Data was centralized. And the 100+-year-old organization is now on a path to transform itself into a digital, data-driven company with the agility of newer companies.
It’s a transformation that all organizations will face sooner than later if they are to survive and prevail given the rapid pace of change in today’s world. As part of the 2019 big data trends, will this be the “Year of the Elephant” where organizations are finally able to break down their silos and see the whole elephant instead of just the parts? The technology is there. Organizations simply have to start making the most of it.