“We must accept human error as inevitable – and design around that fact.” – Donald Berwick
In our Technology field, buzz words come and go. One day we’re discussing how databases are the new best thing to the world of Agile Development and the next we see programming languages, frameworks, and methodologies bit the dust like an old western movie.
But one unchanging aspect of technology and its journey is the People who are an irreplaceable part of the creation, the demise, and the popularity of any given technology. This modern-day-world calls for a close to perfection execution, which humans cannot always account for. How does that affect us, the developers and creators, when we are called to building perfect products?
Removing Human Error
Automation automation automation – and of course throw in some orchestration deployment and configuration management. Leaving the buzz words behind the jest of all of this, “new technology frontier”, is removing human error. This is removing the dependencies of tribal knowledge when it pertains to application and system administration job duties. Those job duties are performed in a repetitive fashion. The job duties are usually consolidated into various custom scripts, leaving a lot of those scripted actions the ability to be boxed up and reused over and over again.
For example, what steps does it take to deploy an application to a server instance?
- Download and Install the various languages and/or framework libraries the application usages.
- Download, Install, Configure the Web server that the application will use.
- Download, Install, and Configure the Database that the application will use.
- And finally Test to see if all the steps are installed and configured correctly.
Last but not least run application tests to make sure it is running as expected. Those are the highlight steps for deploying an application to a bare metal, cloud, or virtual server. With a further review, there are numerous upon numerous additional steps. Something simplistic nevertheless highly catastrophic is a possibility of a typo. Consider this case, maybe in this scenario, I meant to type
cd /var/ect/ansible; rm -rf *
but instead I forgot the cd execute command and only ran
rm -rf /
Now my whole drive is erased. Surprisingly enough this can happen quite often even though I am pretty sure the source of truth of my drive is backed up. However, as Murphy Law states, what can happen will happen. And let’s be honest, redoing previous actions that could have been avoided is a down right pain.
No Hidden/Tribal Knowledge
Let’s look back at those steps to deploy an application to an environment. There are a lot of gotchas and/or small intermediary steps involved. For instance, I only listed the high-level steps. I hand these steps to a new engineer or even a junior engineer fresh out of college. The chances are these engineers will have to ask several questions on “How to”.
These series of questions will eventually lead to someone like myself saying “just let me do it”. This is how one scenario can lead to years upon years of hidden or tribal knowledge, as one may call it. To harbor information like a fugitive on the run is getting more and more exposed in the IT/Software industry. The information should be a source of truth maintained in a repository database, that is easy and intuitive to leverage
Implementation Technology Overview
What does that source of truth entail? Can we not skip the documentation of information and go straight into the execution of the steps onto a given system? Or create scripts to reconfigure the application if there was ever a need to? Those questions have been proposed several times and solutions have been formulated several times into the form of extensive and comprehensive build tools/frameworks. These tools are used throughout the industry to solve the problem of orchestrated development, configuration automation and management.
Furthermore, DevOps tools such as: Chef, Puppet, and Ansible are well mature automation/orchestration tools. Each tool will provide you with enough architecture flexibility to virtually handle any use case you present. Let’s take a slight dive into each one of these technologies.
Puppet
Puppet is the first widely used Configuration Automation and Orchestration software dating back to its initial release in 2005. Puppet uses the Master and Slave paradigm to control X amount of machines. The Ruby language is the script language say, for executing commands in a destination environment.
The “Puppet Agents”(Slave) are modularized distinct components to be deployed to a server. This can be used for the creation of the server(ie. web server, database, application) in its destination environment. The Puppet enterprise (Master) is comprised of all the inner workings to manage, secure, and organize agents.
Puppet Architecture
Documentation
- https://puppet.com/docs/
- http://pub.agrarix.net/OpenSource/Puppet/puppetmanual.pdf
- https://www.rubydoc.info/gems/puppet/
Chef
Chef is somewhat similar to Puppet. The core language used within Chef’s abstract module components is Ruby. Chef has several layers of management for your infrastructure automation needs. The Chef workstation is the primary area for managing the various Chef components. The Chef components consist of cookbooks, recipes, and nodes. The naming convention actually follows what you would consider these nouns in cooking to be with technical context of course.
Recipes are collections of configurations for a given system, virtual, bare metal, or cloud environment. Chef calls those different environments nodes. Cookbook contains recipes and other configurations for application deployment and control mechanisms for the different Chef clients. Lastly, the whole collections of cookbooks are maintained in a Supermarket.
Chef Architecture
Documentation
- https://docs.chef.io/
- https://www.linode.com/docs/applications/configuration-management/beginners-guide-chef/
Ansible
Ansible is the newest mainstream automation/configuration management tool on the market. Therefore, Ansible uses more modern programming languages and configurations concepts and tools. Python is the programming language used in this framework. One of the modern and fastest up-and-coming template languages is YAML. YAML is programming language agnostic and is a subset of the ever so popular JSON. YAML is used within Ansible to describe an Ansible playbook.
Ansible playbook contains the steps that need to be executed on a given system. Once the Ansible playbook is intact configuration or further manipulation of the host can be executed through Ansible API – which is implemented in Python. There are several other components within Ansible technology such as modules, plugins, and inventory.
Ansible Architecture
Documentation
- https://docs.ansible.com/ansible/2.5/dev_guide/
- https://devdocs.io/ansible/
- https://geekflare.com/ansible-basics/
Reusability
After covering a couple of the Configuration Automation and Development tools on the market you can see the vast amount of flexibility available in eliminating those repeatable steps from human error. This software’s framework promotes reusable software within an organization – which is the most viable. The ability to scale your application development environment and environmental infrastructure is critical.
The learning curve may be deeper than using plain old bash scripts but the structure and integrity of using a proven tool and ease of maintenance outweigh the learning curve.
Check it for yourself, reorganize and propel your CI/CD process with Configuration Automation and Orchestrated Deployments and let me know what you think about this article in the comments below.
Raphael Weaver
Sr. Software Engineer
