October 7, 2014
by Evan Powell
The third wave of operations automation – what’s the same, what’s different, and what’s next?
I’m a bit uncomfortable referencing a cultural meme to kick off a weighty subject mostly interesting to the “data center intelligencia.”1 While Ben Horowitz has made it cool to refer to rap lyrics, he is a best-selling author and a billionaire and, conversely, I am decidedly neither.
Still, when I look at the wave of automation sweeping data centers and specifically hyperscale data centers, we face some questions not dissimilar from those that confronted LL Cool J so many years ago. So maybe I’m not totally crazy when those lyrics come to mind. What’s new? What’s rehashed?
We’ll begin by rolling the tape back to an era before DevOps, before AWS, and before VMware – even then time was already too short and systems were already too complex for humans alone to be left to our own devices.
Funky drummer came first
Way before the current wave of OpenStack, DevOps, containers, IaaS, and PaaS, people were automating the heck out of their data centers.
Ben Horowitz earned his credibility in part as CEO of Opsware where they did a lot in the early 2000s to popularize the idea of run book automation. Opsware came to prominence before the era of VMware, and Horowitz reports that the company was sold in part due to the impact of VMware in his excellent book “The Hard Thing about Hard Things.”
Earlier, as is often the case in IT, you can see close parallels to overall automation in approaches to mainframe management. The mainframes themselves would do some level of auto scaling and task / batch prioritization albeit on infinitesimal workloads and working sets compared to today’s hyperscale operators.
If we ignore mainframes, the first wave of operations automations crested with Opsware and Opalis – where StackStorm’s CTO and my co-founder ran engineering (Opalis is now owned by Microsoft where it became the foundation of System Center Orchestrator) – and others like RealOps (now owned by CA). VMware then came onto the scene with an approach that basically embedded aspects of automation into their products. Then DevOps and web scale operations hit and we are now seeing an incredible explosion of automation related projects, with over twenty in OpenStack alone.
Overall, the third wave brings us back to the future with a massively greater scale and a fundamentally different architecture than we had with the first wave. The second wave, dominated by VMware, was as similar to the mainframe world as it was to either the intermediate first wave world of “open systems” or the current DevOps-centric third wave. In many of VMware’s products, just as in mainframes, the black box takes care of many common automation tasks – like auto scaling – whereas in the prior and current wave these tasks are typically handled by a control plane including one or many automation services running separately from the stack.
Stay tuned for part two where we’ll go into further detail on the third wave, including why we need a new wave. Until then, we’d love to hear your feedback. Feel free to comment below, or tweet us @Stack_Storm or @epowell101.
1 If you are a) reading this article and b) looking at footnotes – you are surely an important part of the Datacenter Intelligencia.