It's 2010 and the place where I was working at the time has video encoding software set up on a dedicated server.
The setup is done manually and takes about half a day. No one really knows the exact ins and outs and there's a heavy reliance on the goodwill of sysadmins working for the hosting provider that runs the hardware. Back and forth on tickets until we are able to get a video encoded properly, fear of making any changes that would break the encoding, actually being kept "hostage" by a snowflake server. There was an urgent need to be able to take ownership of the process; we could get better prepared for outages and we could consider scaling up as this single point of failure had proven to be a bottleneck in the past already.
Technology powers your business and the configuration of your systems plays a huge part in its business. No matter how your infrastructure is deployed – whether it's on-premises, in the cloud or hybrid – configuration makes your systems work. And proper management is needed to prevent configuration drift, system outages, and security breaches.
As a developer, foraying into sysadmin tasks was both exciting and frustrating. I felt the need to be able to "rinse and repeat". Setup scripts weren't terribly exciting as they ended up being OS (and version) specific. Same for the "documentation" that was kicking around – probably gone stale immediately after the file was last saved. Until I stumbled upon configuration management tools, at the time CFEngine, Puppet, and Chef. All of a sudden sysadmin work could be structured, reused, tested, and thrown at a vanilla server to bring it idempotently into a desired state. Amazing!
"Configuration management tools are used to repeatably and consistently manage the configuration of systems and the services that they provide entirely through code."
- Carlos Nuñezarlos Nuñez
The first part of the journey to automation was to dig out the correct mix of commands that made the server an encoding server. This was helpful so we could run the commands ourselves rather than relying on a third party. However, this still meant it was a manual process. We spent time automating the setup and after a while the configuration was coded up. We didn't have to rely anymore on tribal knowledge of the setup, nor stale documentation, nor a bunch of bash scripts. #awesome
To finish the story: not too long afterwards the dedicated server got ditched and a scalable number of cloud servers took over the video encoding, all provisioned in exactly the same manner within a few minutes: something that would have been extremely hard to achieve with the original setup.
Having described the issues around a video encoding server above, it's not too difficult to imagine a render node. How easy is it for you to destroy half your farm and rebuild those nodes from scratch? How would you feel if you were asked to set up 100 extra nodes by end of play today?
There are a number of configuration management tools out there that can help you on your automation journey: Puppet, Chef, Ansible, and Salt to name just a few. Of course, I’ve got my favorite and you'll probably have yours. Similar to programming languages and frameworks: there'll always be debates.
Have you got an interesting automation journey to share? Issues you’ve encountered? Resistance from the business you're working at? You actually don't see the point of configuration management? Are you sticking to relying on "golden images"? The good, the bad and the ugly: please share your automation journey with us!