InRhythm

Your partners in accelerated digital transformation

Newsletters

Creating Robust Test Automation for Microservices

Any and all projects that a software engineer joins will come in one of two forms: greenfield or legacy codebases. In the majority of cases, projects will fall into the realm of the legacy repositories. As a software engineer, it is our responsibility to be able to strategically navigate our way through either type of project by looking objectively at the opportunities to improve the code base, lower the cognitive load for software engineering and make a determination to advise on better design strategies. But, chances are, there is a problem. Before architecture or design refactors can be taken its best to take a pulse on the health of your platform End to End (E2E). The reason being, lurking in a new or existing platform is likely a common ailment of a modern microservices approach – the inability to test the platform E2E across microservices that are, by design, commonly engineered by different teams over time.

I recently worked on a greenfield platform and nearly 6 months into the project, one of the most critical parts of the project had fallen several months behind from a quality assurance perspective. It was no longer possible for QA to catch up, nor was it possible for QA to engineer and execute E2E testing to complete common user journeys throughout the enterprise system. To solve this conundrum, E2E data generation tools needed to be created so that the QA team could keep upbuilding and testing every scenario and edge case. As I see it, there are three main requirements for an E2E account and data generation tool. The tool should:

1) Create test accounts with mock data for each microservice

2) Link those accounts between up and downs stream microservices

3) Provide easy to access APIs that are self-documenting 

Using a tool like Swagger, QA can use the API description for REST API, i.e. OpenAPI Specification (formerly Swagger Specification) to view the available endpoints and operations to create accounts, generate test data, authenticate, authorize and “connect the microservices”.

By creating tools for E2E testing, our QA team was able to eliminate the hassle of trying to figure out which upstream and downstream microservices needed to be called to ensure that the required accounts and data were available and set up properly to ensure a successful test of all scenarios i.e. based upon the variety of different data types, user permissions, user information and also covering the negative test cases. The QA team was able to catch up and write their entire suite of test scenarios generating the matching accounts and data to satisfy those requirements. The net result of having built an E2E test generation tool was automated tests could be produced exponentially quicker and the tests themselves are more resilient to failure. 

Even though the microservices pattern continues to gain traction, developing E2E testing tools that generate accounts and test data across an enterprise platform will likely still remain a pain point. That having been said, the purpose of this article is to raise awareness. Software Engineering Leadership has an opportunity to step up and ensure that the enterprise system that they look after is healthy E2E. I can’t think of a better way to maintain a healthy system than to ensure accounts and data in the lower environments actually work and unblock testing end-to-end. 

James Woods, Web Engineering Practice Lead

Configuration Automation Tools: Orchestrating Your Deployment

“We must accept human error as inevitable – and design around that fact.” – Donald Berwick

Raphael Weaver

In our Technology field, buzz words come and go. One day we’re discussing how databases are the new best thing to the world of Agile Development and the next we see programming languages, frameworks, and methodologies bit the dust like an old western movie.

But one unchanging aspect of technology and its journey is the People who are an irreplaceable part of the creation, the demise, and the popularity of any given technology. This modern-day-world calls for a close to perfection execution, which humans cannot always account for. How does that affect us, the developers and creators, when we are called to building perfect products? 

Removing Human Error

Automation automation automation – and of course throw in some orchestration deployment and configuration management. Leaving the buzz words behind the jest of all of this, “new technology frontier”, is removing human error. This is removing the dependencies of tribal knowledge when it pertains to application and system administration job duties. Those job duties are performed in a repetitive fashion. The job duties are usually consolidated into various custom scripts, leaving a lot of those scripted actions the ability to be boxed up and reused over and over again.

For example, what steps does it take to deploy an application to a server instance? 

  1. Download and Install the various languages and/or framework libraries the application usages.
  2. Download, Install, Configure the Web server that the application will use.
  3. Download, Install, and Configure the Database that the application will use. 
  4. And finally Test to see if all the steps are installed and configured correctly. 

Last but not least run application tests to make sure it is running as expected. Those are the highlight steps for deploying an application to a bare metal, cloud, or virtual server. With a further review, there are numerous upon numerous additional steps. Something simplistic nevertheless highly catastrophic is a possibility of a typo. Consider this case, maybe in this scenario, I meant to type

cd /var/ect/ansible; rm -rf *

but instead I forgot the cd execute command and only ran

rm -rf /

Now my whole drive is erased. Surprisingly enough this can happen quite often even though I am pretty sure the source of truth of my drive is backed up. However, as Murphy Law states, what can happen will happen. And let’s be honest, redoing previous actions that could have been avoided is a down right pain.

No Hidden/Tribal Knowledge

Let’s look back at those steps to deploy an application to an environment. There are a lot of gotchas and/or small intermediary steps involved. For instance, I only listed the high-level steps. I hand these steps to a new engineer or even a junior engineer fresh out of college. The chances are these engineers will have to ask several questions on  “How to”. 

These series of questions will eventually lead to someone like myself saying “just let me do it”. This is how one scenario can lead to years upon years of hidden or tribal knowledge, as one may call it.  To harbor information like a fugitive on the run is getting more and more exposed in the IT/Software industry. The information should be a source of truth maintained in a repository database, that is easy and intuitive to leverage

Implementation Technology Overview

What does that source of truth entail? Can we not skip the documentation of information and go straight into the execution of the steps onto a given system? Or create scripts to reconfigure the application if there was ever a need to? Those questions have been proposed several times and solutions have been formulated several times into the form of extensive and comprehensive build tools/frameworks. These tools are used throughout the industry to solve the problem of orchestrated development, configuration automation and management. 

Furthermore, DevOps tools such as: Chef, Puppet, and Ansible are well mature automation/orchestration tools. Each tool will provide you with enough architecture flexibility to virtually handle any use case you present. Let’s take a slight dive into each one of these technologies.

Puppet

Puppet is the first widely used Configuration Automation and Orchestration software dating back to its initial release in 2005. Puppet uses the Master and Slave paradigm to control X amount of machines. The Ruby language is the script language say, for executing commands in a destination environment. 

The “Puppet Agents”(Slave) are modularized distinct components to be deployed to a server. This can be used for the creation of the server(ie. web server, database, application) in its destination environment.  The Puppet enterprise (Master) is comprised of all the inner workings to manage, secure, and organize agents.

Puppet Architecture

Documentation 

Chef

Chef is somewhat similar to Puppet. The core language used within Chef’s abstract module components is Ruby. Chef has several layers of management for your infrastructure automation needs. The Chef workstation is the primary area for managing the various Chef components. The Chef components consist of cookbooks, recipes, and nodes. The naming convention actually follows what you would consider these nouns in cooking to be with technical context of course. 

Recipes are collections of configurations for a given system, virtual, bare metal, or cloud environment. Chef calls those different environments nodes. Cookbook contains recipes and other configurations for application deployment and control mechanisms for the different Chef clients. Lastly, the whole collections of cookbooks are maintained in a Supermarket.

Chef Architecture

Documentation

Ansible

Ansible is the newest mainstream automation/configuration management tool on the market. Therefore, Ansible uses more modern programming languages and configurations concepts and tools. Python is the programming language used in this framework. One of the modern and fastest up-and-coming template languages is YAML. YAML is programming language agnostic and is a subset of the ever so popular JSON. YAML is used within Ansible to describe an Ansible playbook. 

Ansible playbook contains the steps that need to be executed on a given system. Once the Ansible playbook is intact configuration or further manipulation of the host can be executed through Ansible API – which is implemented in Python. There are several other components within Ansible technology such as modules, plugins, and inventory. 

Ansible Architecture

Documentation

Reusability

After covering a couple of the Configuration Automation and Development tools on the market you can see the vast amount of flexibility available in eliminating those repeatable steps from human error. This software’s framework promotes reusable software within an organization – which is the most viable. The ability to scale your application development environment and environmental infrastructure is critical. 

The learning curve may be deeper than using plain old bash scripts but the structure and integrity of using a proven tool and ease of maintenance outweigh the learning curve.

Check it for yourself, reorganize and propel your CI/CD process with Configuration Automation and Orchestrated Deployments and let me know what you think about this article in the comments below.

Raphael Weaver

Sr. Software Engineer

Change? What Change? Everything Always Goes to Plan

As software engineers, we are conditioned to brace for change. The very nature of our work requires that we be ready for it. Yet, at our cores, we don’t want there to be any changes. We don’t surrender to the inevitable changes that are going to be required after we submit content for a code review. Instead, we steel ourselves against it, and gear up for battle with a mindset of resistance. We don’t want our QA teams to find any bugs because we’ll then have to make changes to the code we’ve already written. 

The experience, however, is like the Grinch who does everything that he can to stop Christmas from coming. Of course, the holiday arrives anyway. Similarly, despite our best efforts to plan and design an agile effort which builds change in, we’re still not entirely prepared to embrace those changes when they do come. And they will. They always do.

Agile Value #4: responding to change over following a plan” speaks to this behavioral response. It’s all about our attitude. Are we prepared for change and willing to accept that there will be change (as there is for every software engineering project)? Or are we preparing for a fight that we are inevitably going to lose? 

Agile craftsmanship requires that we be nimble in our designs and in our approach. The code we write must stand up to meet the quality demands of the project and not interfere with other parts of the project, but it has to be nimble, too. Anyone at any time in the future may need to rework that code or replace it altogether. If it’s rigid and inflexible, the effort to update it (or replace it) will be that much more intense. Working sprint after sprint is already intense enough – we don’t need to add more stress to our day. In contrast, we need to take measured steps to reduce it.

One of the best ways to manage this stress and to reduce our anxiety is to prepare for change. Embracing a mindset of flexibility prepares us for the change that is coming. If our response is calm and collected, the changes requested will be received just like any new project instead of what may feel like a personal attack on our abilities.

Approaching any project with the expectation that it will likely end differently than it began requires proactively building a roadmap that is designed to be nimble and responsive by anticipating potential changes. Getting ahead of industry trends and predicted needs requires that we are constantly in the mode of collecting feedback from clients, tracking the market and soliciting input from other agile experts. Scenario planning is another way to help with preparedness. This is the only way that we can evolve as a trusted colleague and partner to the clients we collaborate with.

Christopher Okhravi, an authority on agile software development, offers an insightful perspective on the application of Agile Value #4. He reminds us that, “We should never forget that they [the clients] are the ones asking for the system. And they are the ones who will ultimately use it.” Another factor that we need to remember is that rigid planning and process are the antithesis of the freedom derived from an agile approach. As each project unfolds, we learn more about the needs of that project and gain new insights on how to best deliver the code needed to execute the project. However, as we do so, we tend to drift further from the original plan which no longer reflects our reality and this creates tension for everyone involved. 

So, what is the best way forward? Embrace Agile Value #4. We must plan for the worst but hope for the best because change is inevitable. We must be nimble – and stay that way. Change is a necessary element of what we do. We can design our product roadmaps to be responsive and proactive or we can choose to be reactive and struggle to keep up. Getting ahead of industry needs and trends helps us prepare for the changes that will come. Being open to continuous feedback from our clients, our managers, our peers and our colleagues helps us evolve into a trusted co-worker and business partner. 

“Change always comes bearing gifts.”  ~Price Pritchett

Robert Morrell

Practice Lead, Java Cloud

What We’re Reading Around the Web

Agile Manifesto 4/4 – Responding To Change Over Following A Plan by Christopher Okhravi
YouTube
“We should never forget that they [the clients] are the ones asking for the system. And they are the ones who will ultimately use it.”

Manage Stress
Healthfinder
“To be agile, you need to be able to ask, ‘Is this agile?’”

Applying Agile Management Value 4: Responding to Change Over Following a Plan
Dummies
“Unfortunately, traditional project management approaches attempt to wrestle the change monster to the ground and pin it down so it goes out for the count.”

Getting Started With Agile: Responding to Change Over Following a Plan
Pivotal Tracker
“As project execution unfolds, the team learns more about what needs that resulting product will fill as well as how that product can best be built. As this new reality emerges, the team struggles to keep their project aligned to the original plan, which likely no longer reflects the team’s new reality.”


Change Is The Only Constant

A true Agile Craftsman Mindset understand that change is the only Constant

As product developers, we take pride in the “big picture” of the function of the software that we are designing and implementing. Delivering a solution for our clients, and their customers, takes diligence, long hours and impeccable planning. However, even the best-made plans can go awry. How we deal with those plans differentiates the inflexible resistors from the agile craftsmen.

“Agile Value #4: responding to change over following a plan” speaks to this behavioral response. Optimists hope for the best but plan for the worst. Pessimists expect the worst but plan for the best. And flexible realists exhibit behaviors that are the best of both understanding that being nimble and prepared for change is the right mindset. Framing everything that you do with the strong possibility that things will likely change and be different that you planned for cues up your brain to be prepared for change. Hence, you enable yourself to successfully adapt to whatever the new situation is.


Beyond one’s mindset, a critical aspect of practicing Agile Value #4 relates to planning and design. Approaching any project with the expectation that it will likely end differently than it began requires proactively building a roadmap that is designed to be nimble and responsive by anticipating potential changes. Getting ahead of industry trends and predicted needs requires that we are constantly in the mode of collecting feedback from clients, tracking the market and soliciting input from other agile experts. Scenario planning is another way to help with preparedness. This is the only way that we can evolve as a trusted partner. As agile craftsmen, our clients expect us to be inherently nimble, ready for change, willing to adapt and to plan for adjustments so that we can still track to a set schedule and deliver the quality product we are contracted to produce.


Understanding that change is a natural aspect of a project is one thing. Being prepared for it with a thoughtful plan and design that anticipates where and when those changes are most likely to happen is quite another thing altogether. Here, experience can make all the difference. The client relationship is another factor, one that we’ve discussed in a previous newsletter, ”Customers are Much, Much More Than Signed Contracts”. When a client and vendor embark on a path towards partnership, versus a client-vendor relationship, both sides enter the agreement and project understanding that it’s best to expect the unexpected and be prepared to flex. Sometimes, the client may need to be flexible. Other times, the contracted agile partner may need to adjust as the situation changes for the client.
If change is the only constant, and Agile Value #4 requires that we plan for and accept change versus resist it, we need to each ask ourselves if we are agile and flexible. Or, are we rigid and linear?

Thanks and Keep Growing,

Gunjan Doshi

CEO, InRhythm

What We’re Reading Around the Web

Agile Manifesto 4/4 – Responding To Change Over Following A Plan by Christopher Okhravi
YouTube
“We should never forget that they [the clients] are the ones asking for the system. And they are the ones who will ultimately use it.”

Manage Stress
Healthfinder
“To be agile, you need to be able to ask, ‘Is this agile?’”

Applying Agile Management Value 4: Responding to Change Over Following a Plan
Dummies
“Unfortunately, traditional project management approaches attempt to wrestle the change monster to the ground and pin it down so it goes out for the count.”

Getting Started With Agile: Responding to Change Over Following a Plan
Pivotal Tracker
“As project execution unfolds, the team learns more about what needs that resulting product will fill as well as how that product can best be built. As this new reality emerges, the team struggles to keep their project aligned to the original plan, which likely no longer reflects the team’s new reality.”


Customers are Much, Much More Than Signed Contracts

December 3rd: Customer Collaboration Over Contract Negotiation

Like most things, how you see something is a matter of perspective. The lens that you look through is shaped by experience, environment, who you’re influenced by and the attitude you take towards life and the given topic in particular.

Applying a customer-centric lens to branding, product development and marketing seems like an obvious thing to do, however, it wasn’t until the 1960s when Lester Wunderman urged companies to do so. Moreover, it wasn’t until the turn of the last century when customers took control with their collective new ability to dictate a brand’s narrative via the internet and dawn of social media. So, this concept of customer-centricity, in practice, is actually a modern way of thinking.

“Agile Value #3: customer collaboration over contract negotiation” is an example of a customer-centric approach. However, the core component of the directive is essentially the same. It’s all about communication.

That doesn’t mean unilateral outreach where you as the brand are sending messages, postcards, coupons or holiday catalogues. What it does mean is creating a forum for open dialogue in the spirit of true communication where conversations are bilateral. Doing so ensures that pain points can be discussed until they are understood and then resolved.

As agile product developers, the responsibility of maintaining a health dialogue with our customers is up to us. All too often, I have heard software engineers citing that communication with a client is the responsibility of the sales and marketing team. Not so!

We’re the people who are typically onsite with the customer. It is up to us as the agile craftsmen to deliver our best work to our clients so that their light can shine brightly. Clients look to us as their trusted advisors who are there to help them meet their needs, deliver quality work on time and help ensure our client’s success with their customers. Losing sight of the value of wearing a customer-centric lens compromises our ability to deliver our best for our clients. As soon as we we make it about ourselves and not our clients, the work environment will become more challenging and potentially even toxic.

The concept of collaboration over contract negotiations delineates the difference of being regarded as a vendor versus as a partner. Here at InRhythm, we know which side we aim to be on. Approaching software engineering with agile methodology requires that we are constantly communicating with our clients, assessing their needs and anticipating their needs even before our clients realize that things have shifted.

This brings us back to perspective. If we communicate regularly, with transparency, and deliver quality work in a timely manner, our clients will view us with the lens of partnership. Conversely, if we view our clients with the lens of a signed contract and the dollars tied to it, our perspective will be tarnished. As software engineers, we will struggle to deliver our work with that passion and quality that is required to fulfill the demands of an agile effort.

Agile Value #3 is the reminder for all of us that the lens that we view our work with is critically important. Viewing our deliverables and efforts with a client-centric perspective will positively impact how they view us. And, it can make all the difference between doing business versus being out of business.

Thanks and Keep Growing,

Gunjan Doshi

CEO, InRhythm

What We’re Reading Around the Web

The Real ROI Of Being Customer-Centric
Entrepreneur
“No business will survive long without satisfying its customers. That much should be evident to any company whether it is established or just starting out.”

100 Of The Most Customer-Centric Companies
Forbes
“Customer-centric companies live and breathe their customers and are laser-focused on providing amazing experiences.”

Customer Centricity — Marketing as customer-centric corporate management
Medium
“[The] key to success: a more radical focus on humans. Genuine customer centricity requires to rethink all functions and levels.”

6 Ways to Build a Customer-Centric Culture
Harvard Business Review
“To successfully implement a customer-centric strategy and operating model, a company must have a culture that aligns with them — and leaders who deliberately cultivate the necessary mindset and values in their employees.”