This Eckerson Group report recommends 10 vital steps to attain success in DataOps.READ MORE
Legend has it that Amazon was going to be named “Cadabra” until people within Jeff Bezos’ inner circle began mishearing it as “cadaver.” Given that Amazon did end up killing many brick-and-mortar books, electronics, and department stores and now has its crosshairs set on groceries, healthcare and others, the original name (or its malapropism) might still have been an accurate moniker. The world is pretty much Amazon’s oyster at the moment, and if you have any reason to assume your market is safe, think again. Amazon’s goal is to sell everything to everyone, driven by a user-friendly omnichannel retail experience that excels at using big data to identify and call out a customer’s needs before he or she is even aware of them.
So Amazon is coming for your business, if not for all of our businesses. The good news? There is no reason that big data can’t be used by your organization to also become an agile industry powerhouse. The bad news? Most can’t employ the armies of highly skilled data engineers and data scientists needed to make big data work in a true production environment. But, there’s more good news still: the arrival of big data engineering automation to reduce the need for all of that expensive expertise.
Data-driven decision-making has long been hailed as the great gift big data would present across all industries and sectors. At its core was the ability for decision-makers to know more and to innovate more quickly based on their newfound big data knowledge. When business leaders saw what big data was able to do for Amazon, Google, and Facebook, creating new opportunities and ultimately setting them on new paths for growth and innovation, many thought their organizations could also use the power of big data to engage in data-driven innovation, too.
What they didn’t realize was that big data technologies like Hadoop and Spark are complicated to master and deploy in a repeatable production environment. Today, the great majority of big data projects are stuck in limbo and those that do deploy to production can take months, even years. All the time that was supposed to be spent on that world-changing data analysis instead went into manually building data engineering pipelines and workflows that were often brittle and difficult to maintain. And it’s not like the problem’s getting any easier. All the data pouring into enterprises today is growing in volume and complexity, with more variety, sources, environments, and users of data being added every day.
The reality is that most companies aren’t Amazon, Google or Facebook and they can’t attract the same level of engineering talent to build a big data analytics advantage. The only way to compete is to level the playing field by taking advantage of agile data engineering principles and new automation technologies that allow you to focus on using data to make decisions and not on building and supporting the underlying infrastructure. So even if you are a retail pharmacy, an oil exploration and production company, a 100+ year old publishing firm, an international cruise line, or some other business that is not high tech, this new level of data engineering automation now allows ALL organizations to successfully launch big data projects in days instead of months without requiring an army of expensive data engineers. But if you continue to try to manually build out your data engineering pipelines, you are going to get eaten alive as competitors who chose to automate their big data initiatives swiftly pass you by. After all, how can you compete if your business is dead?
And that brings us right back to where we started with the origin of Amazon’s name. Global organizations and start-ups alike should find the moniker Jeff Bezos eventually went with, “Amazon” just as fearsome as the one that made people think of a corpse. He ultimately named the company Amazon for two reasons: One, in the early 1990s, website listings were often alphabetical, and Jeff wanted to make sure Amazon was always seen near the top. But more importantly, the Amazon is the largest river in the world. By naming the company after something that suggests great size and scale, Jeff was already making it known that there would be no limit to what Amazon could become. It’s this vision, baked into Amazon’s DNA from the very beginning that should be scaring anyone in business today, though at least now you have a choice: automate or not automate.
Of course, Amazon will be more than happy to add you to its list of “abracadavers,” so do you really have a choice?