InRhythm

Your partners in accelerated digital transformation

JavaScript

Configuration Automation Tools: Orchestrating Your Deployment

“We must accept human error as inevitable – and design around that fact.” – Donald Berwick

Raphael Weaver

In our Technology field, buzz words come and go. One day we’re discussing how databases are the new best thing to the world of Agile Development and the next we see programming languages, frameworks, and methodologies bit the dust like an old western movie.

But one unchanging aspect of technology and its journey is the People who are an irreplaceable part of the creation, the demise, and the popularity of any given technology. This modern-day-world calls for a close to perfection execution, which humans cannot always account for. How does that affect us, the developers and creators, when we are called to building perfect products? 

Removing Human Error

Automation automation automation – and of course throw in some orchestration deployment and configuration management. Leaving the buzz words behind the jest of all of this, “new technology frontier”, is removing human error. This is removing the dependencies of tribal knowledge when it pertains to application and system administration job duties. Those job duties are performed in a repetitive fashion. The job duties are usually consolidated into various custom scripts, leaving a lot of those scripted actions the ability to be boxed up and reused over and over again.

For example, what steps does it take to deploy an application to a server instance? 

  1. Download and Install the various languages and/or framework libraries the application usages.
  2. Download, Install, Configure the Web server that the application will use.
  3. Download, Install, and Configure the Database that the application will use. 
  4. And finally Test to see if all the steps are installed and configured correctly. 

Last but not least run application tests to make sure it is running as expected. Those are the highlight steps for deploying an application to a bare metal, cloud, or virtual server. With a further review, there are numerous upon numerous additional steps. Something simplistic nevertheless highly catastrophic is a possibility of a typo. Consider this case, maybe in this scenario, I meant to type

cd /var/ect/ansible; rm -rf *

but instead I forgot the cd execute command and only ran

rm -rf /

Now my whole drive is erased. Surprisingly enough this can happen quite often even though I am pretty sure the source of truth of my drive is backed up. However, as Murphy Law states, what can happen will happen. And let’s be honest, redoing previous actions that could have been avoided is a down right pain.

No Hidden/Tribal Knowledge

Let’s look back at those steps to deploy an application to an environment. There are a lot of gotchas and/or small intermediary steps involved. For instance, I only listed the high-level steps. I hand these steps to a new engineer or even a junior engineer fresh out of college. The chances are these engineers will have to ask several questions on  “How to”. 

These series of questions will eventually lead to someone like myself saying “just let me do it”. This is how one scenario can lead to years upon years of hidden or tribal knowledge, as one may call it.  To harbor information like a fugitive on the run is getting more and more exposed in the IT/Software industry. The information should be a source of truth maintained in a repository database, that is easy and intuitive to leverage

Implementation Technology Overview

What does that source of truth entail? Can we not skip the documentation of information and go straight into the execution of the steps onto a given system? Or create scripts to reconfigure the application if there was ever a need to? Those questions have been proposed several times and solutions have been formulated several times into the form of extensive and comprehensive build tools/frameworks. These tools are used throughout the industry to solve the problem of orchestrated development, configuration automation and management. 

Furthermore, DevOps tools such as: Chef, Puppet, and Ansible are well mature automation/orchestration tools. Each tool will provide you with enough architecture flexibility to virtually handle any use case you present. Let’s take a slight dive into each one of these technologies.

Puppet

Puppet is the first widely used Configuration Automation and Orchestration software dating back to its initial release in 2005. Puppet uses the Master and Slave paradigm to control X amount of machines. The Ruby language is the script language say, for executing commands in a destination environment. 

The “Puppet Agents”(Slave) are modularized distinct components to be deployed to a server. This can be used for the creation of the server(ie. web server, database, application) in its destination environment.  The Puppet enterprise (Master) is comprised of all the inner workings to manage, secure, and organize agents.

Puppet Architecture

Documentation 

Chef

Chef is somewhat similar to Puppet. The core language used within Chef’s abstract module components is Ruby. Chef has several layers of management for your infrastructure automation needs. The Chef workstation is the primary area for managing the various Chef components. The Chef components consist of cookbooks, recipes, and nodes. The naming convention actually follows what you would consider these nouns in cooking to be with technical context of course. 

Recipes are collections of configurations for a given system, virtual, bare metal, or cloud environment. Chef calls those different environments nodes. Cookbook contains recipes and other configurations for application deployment and control mechanisms for the different Chef clients. Lastly, the whole collections of cookbooks are maintained in a Supermarket.

Chef Architecture

Documentation

Ansible

Ansible is the newest mainstream automation/configuration management tool on the market. Therefore, Ansible uses more modern programming languages and configurations concepts and tools. Python is the programming language used in this framework. One of the modern and fastest up-and-coming template languages is YAML. YAML is programming language agnostic and is a subset of the ever so popular JSON. YAML is used within Ansible to describe an Ansible playbook. 

Ansible playbook contains the steps that need to be executed on a given system. Once the Ansible playbook is intact configuration or further manipulation of the host can be executed through Ansible API – which is implemented in Python. There are several other components within Ansible technology such as modules, plugins, and inventory. 

Ansible Architecture

Documentation

Reusability

After covering a couple of the Configuration Automation and Development tools on the market you can see the vast amount of flexibility available in eliminating those repeatable steps from human error. This software’s framework promotes reusable software within an organization – which is the most viable. The ability to scale your application development environment and environmental infrastructure is critical. 

The learning curve may be deeper than using plain old bash scripts but the structure and integrity of using a proven tool and ease of maintenance outweigh the learning curve.

Check it for yourself, reorganize and propel your CI/CD process with Configuration Automation and Orchestrated Deployments and let me know what you think about this article in the comments below.

Raphael Weaver

Sr. Software Engineer

InRhythm’s Cloud Engineering Digest: Cloud, scale, microservices… and Java is there for you.

In this new digest you’ll find latest news, updates and recommendations about cloud-native architecture, patterns and anti-patterns in distributed systems and microservices.

It’s official: Apache Kafka 2.3 has been released!

Micronaut 1.1 GA has been released. You can find more details here.

Apache Dubbo, a Java-Based RPC Framework, Graduates to Top-Level Project

Project Loom: Lightweight Java threads

GraalVM 19.1: Compiling Faster

Java 13 Enters Feature Freeze and Ramp down

AWS Control Tower: The easiest way to set up and govern a new, secure multi-account AWS environment

Read:

5 Principles for Cloud-native Architecture: What it is and how to master it

API Gateways and Service Meshes: Opening the Door to Application Modernization

Patterns in Distributed Systems

Microservices adoption anti-patterns

Tell us what you think! What are you reading now? Let us know in the comments below or @GetInRhythm on Twitter

How to Be a Better Engineer ?: InRhythm’s Learning and Growth Newsletter

 

February 11th: How to Be a Better Engineer

InRhythm’s mission, in the work we do and the people we hire, is acutely focused on learning and growth. Through the development of InRhythmU, our internal professional growth initiative, we’re developing programs to ensure that our engineers reach the level of technical ability required by our clients. Being a great engineer involves more than keeping up with tech trends or mastering a framework, however.

We want to create bar-raisers who take our client relationships to the next level. The real goal is to train engineers who can approach engineering problems strategically and make good choices in a world of infinite options. As we develop this system of skill building and growth, a few key questions emerge. How do junior engineers learn to become senior engineers? How much technical ability should a UI designer aim for? Should every member of the team understand user experience design? Database design? What does “full-stack engineer” really mean? And what’s our role in building the next generation of software?

This month we dive into these issues, led by senior InRhythm engineer Brian Olore, whose article “Becoming a Senior Engineer” clearly lays out some tips, tricks, and time-tested strategies that will help you raise that bar.

Tell us what you think separates good engineers from great ones—or shout out any great engineers you know (we’re hiring!)—@GetInRhythm or here on the InRhythmU blog.

Thanks and Keep Growing,

Gunjan Doshi
CEO, InRhythm

Becoming a Senior Engineer
(4 min. read)
“Looking to take your engineering career to the next level? These five tips will set you on the path to greater things and make you a standout in the field. As we all know, it’s not easy to be exceptional in this already-competitive field, but the reward for real leadership is high, and if you’re truly passionate about your craft, these tips—including mentorship and open source contributions—will be something to get excited about.”

What We’re Reading Around the Web

The Great Divide
(14 min. read)
CSS-Tricks
“In this provocative article, Chris Coyer lays out why he thinks there’s a rift growing in the development community. The fault line? JavaScript. On one side are those who believe a ‘front-end engineer’ must be skilled with the language, while the other side feels ‘cosmically far away from the articles and conversations around accessibility, semantic markup, CSS possibilities, UX considerations, and UI polish, among others.’ Coyer quips: ‘Two front-end developers are sitting at a bar. They have nothing to talk about.’ False dichotomy? Industry impasse? You decide.”

Designing for the Web Ought to Mean Making HTML and CSS
(4 min. read)
Signal V. Noise
“Et tu, JavaScript? ‘The industry is making it too hard to work directly with the web’ says Ruby On Rails creator, best-selling author, and racecar driver(!) David Heinemeier Hansson. Never one to shy away from controversy, DHH (as he is known) calls out the industry’for polluting our thinking with over-engineered frameworks and toolkits while ignoring the raw value of “native” web technologies like HTML and CSS (and JavaScript) and the opinions of designers who work with them: ‘The towering demands inherent in certain ways of working with JavaScript are rightfully scaring some designers off from implementing their ideas at all. That’s a travesty.’ Is the soft bigotry of low expectations degrading the quality of our teams? Is knowledge of JavaScript non-negotiable for ‘front-end engineers’?”

HTML, CSS and Our Vanishing Industry Entry Points
(6 min. read)
Rachel Andrew
“It ain’t easy bein’ green. The leadership team at InRhythm spends a great deal of time improving the way we attract, train, and retain talent. Many of our hires are relatively new to this industry; Rachel Andrew has been thinking about what those new entries into this field face, worried that we’re excluding many people unnecessarily, especially those without formal training. What about those who learn ‘on the job,’ view source and copy, or futz around with a GitHub repo and learn on their own? Aren’t ambitious people to be valued? Didn’t the web grow so rapidly and profitably specifically because barriers to entry didn’t exist? ‘If we make it so that you have to understand programming to even start, then we take something open and enabling, and place it back in the hands of those who are already privileged.’ There may be implications here for society, and freedom; Andrew’s larger point is worth considering.”

What Happened to the Internet?
(11 min. read)
thinkgrowth.org
“Here’s to the crazy ones, the misfits, the rebels, the troublemakers. We’ve been told that technology companies value those who ‘Think Different.’ Is that still true on the Internet? Jared Sumner reminds us of a time when the web was ‘weird and fun,’ and we liked it! Is coding now a privilege, instead of a right? Wasn’t this technology supposed to be a great equalizer, rather than an opportunity to be monopolized by tech giants? Following this line of thinking and going deeper, Meghan Keaney Anderson asks us whether ‘what was once the closest thing we could imagine to a meritocracy more closely resembles an oligarchy ruled by a few powerful gatekeepers.’ Food for thought.”

Debugging Node without restarting processes

This post is another by InRhythm’s own Carl Vitullo. For the full, original post check out “Debugging Node without restarting processes” on hackernoon. Make sure to follow him on Twitter and in Reactiflux!

I’m typically a frontend developer, but every now and then I find myself writing or maintaining some backend code. One of the most significant drawbacks I’ve encountered when switching contexts to Node is the lack of the Chrome Developer Tools. Not having them really highlights how hard I lean on them during my day to day in web development. Luckily, there are options for enabling them, and they’ve gotten much more stable and usable in recent times. Node has a built in debug mode that allows you to connect to the DevTools, and there’s a package called node-inspector that connects automatically.

It’s worth noting that versions of Node < 8 use a now-legacy Debugger API. Node 8 introduces the Inspector API, which better integrates with existing developer tools.

There’s one common theme that I’ve encountered when using these methods: they must be invoked when starting the node process. The other day, I found myself with a process in an odd state that I’ve had trouble reproducing, and I didn’t want to risk losing it by restarting the process to enable the inspector.

However, I found a solution — no less, a solution from the official Node docs.

A Node.js process started without inspect can also be instructed to start listening for debugging messages by signaling it with SIGUSR1 (on Linux and OS X).

This only applies to unix based OSes (sorry Windows users), but it saved my bacon in this case. The kill command in unix may be ominously named, but it can also be used to send arbitrary signals to a running process. man killtells me that I can do so using the syntax, kill -signal_name pid. The list of signal names can be enumerated with kill -l, shown below.

$ kill -l
hup int quit ill trap abrt emt fpe kill bus segv sys pipe alrm term urg
stop tstp cont chld ttin ttou io xcpu xfsz vtalrm prof winch info usr1 usr2

By default, kill sends an int, or an interrupt signal, which is equivalent to hitting ctrl-c in a terminal window. There’s a lot of depth to process signals that I won’t get into (I encourage you to explore them!), but towards the end of the list is usr1. This is the SIGUSR1 that the node docs are referring to, so now I just need a pid, or process ID, to send it to. I can find that by using ps and grep to narrow the list of all processing running on my system

$ ps | grep node
9670 ttys000 0:01.04 node /snip/.bin/concurrently npm run watch:server npm run watch:client
9673 ttys000 0:00.46 node /snip/.bin/concurrently npm run watch:server-files npm run watch:dist
9674 ttys000 0:33.02 node /snip/.bin/webpack — watch
9677 ttys000 0:00.36 node /snip/.bin/concurrently npm run build:snip — — watch
9678 ttys000 0:01.65 node /snip/.bin/nodemon — delay 2 — watch dist ./dist/src/server.js
9713 ttys000 0:01.00 /usr/local/bin/node ./dist/src/server.js
9736 ttys003 0:00.00 grep — color=auto node

My output is a little noisy due to a complex build toolchain that spawns many processes. But I see down towards the bottom the right process: node ./dist/src/server.js, with a pid of 9713.

Now I know the signal name is usr1 and the pid is 9713, so I need to run.

$ kill -usr1 9713

It runs with no output, but I check the logs of my node process and see

Debugger listening on ws://127.0.0.1:9229/ad014904-c9be-4288–82da-bdd47be8283b
For help see https://nodejs.org/en/docs/inspector

I can open chrome://inspect, and I immediately see my inspect target.

I click “inspect”, and I’m rewarded with a Chrome DevTools window in the context of my node process! I can use the profiler to audit performance, use the source tab to add break points and inspect the running code, and use the console to view logs or modify the variables in the current scope, just like I would on the web.