InRhythm

Your partners in accelerated digital transformation

Learning and Development

Creating Robust Test Automation for Microservices

Any and all projects that a software engineer joins will come in one of two forms: greenfield or legacy codebases. In the majority of cases, projects will fall into the realm of the legacy repositories. As a software engineer, it is our responsibility to be able to strategically navigate our way through either type of project by looking objectively at the opportunities to improve the code base, lower the cognitive load for software engineering and make a determination to advise on better design strategies. But, chances are, there is a problem. Before architecture or design refactors can be taken its best to take a pulse on the health of your platform End to End (E2E). The reason being, lurking in a new or existing platform is likely a common ailment of a modern microservices approach – the inability to test the platform E2E across microservices that are, by design, commonly engineered by different teams over time.

I recently worked on a greenfield platform and nearly 6 months into the project, one of the most critical parts of the project had fallen several months behind from a quality assurance perspective. It was no longer possible for QA to catch up, nor was it possible for QA to engineer and execute E2E testing to complete common user journeys throughout the enterprise system. To solve this conundrum, E2E data generation tools needed to be created so that the QA team could keep upbuilding and testing every scenario and edge case. As I see it, there are three main requirements for an E2E account and data generation tool. The tool should:

1) Create test accounts with mock data for each microservice

2) Link those accounts between up and downs stream microservices

3) Provide easy to access APIs that are self-documenting 

Using a tool like Swagger, QA can use the API description for REST API, i.e. OpenAPI Specification (formerly Swagger Specification) to view the available endpoints and operations to create accounts, generate test data, authenticate, authorize and “connect the microservices”.

By creating tools for E2E testing, our QA team was able to eliminate the hassle of trying to figure out which upstream and downstream microservices needed to be called to ensure that the required accounts and data were available and set up properly to ensure a successful test of all scenarios i.e. based upon the variety of different data types, user permissions, user information and also covering the negative test cases. The QA team was able to catch up and write their entire suite of test scenarios generating the matching accounts and data to satisfy those requirements. The net result of having built an E2E test generation tool was automated tests could be produced exponentially quicker and the tests themselves are more resilient to failure. 

Even though the microservices pattern continues to gain traction, developing E2E testing tools that generate accounts and test data across an enterprise platform will likely still remain a pain point. That having been said, the purpose of this article is to raise awareness. Software Engineering Leadership has an opportunity to step up and ensure that the enterprise system that they look after is healthy E2E. I can’t think of a better way to maintain a healthy system than to ensure accounts and data in the lower environments actually work and unblock testing end-to-end. 

James Woods, Web Engineering Practice Lead

Letting Go of the Formality

It wasn’t that long ago that suits, ties, knee-length skirts, heels and pantyhose were required wardrobe staples. In fact, many enterprise Fortune 100s upheld a formal dress code as part of their HR Policy Handbooks. Goldman Sachs, the last bastion of formality on Wall Street, finally relaxed its dress code and offered a “flexible style” option. They only did so last year. Their updated policy permitted men to choose garments other than suits. Admittedly, this made things more complicated for women but that’s a topic for another blog.

Since we’re agile practitioners here at InRhythm, it made sense to look at our own approaches to talent and acquisition. Not to mention retention. Practices and policies drive corporate culture. Here, Agile Platinum Principle One: resist formality, must serve as one of the pillars of our talent strategy.

If your company doesn’t offer a culture that’s aligned with the needs and expectations of today’s job seekers, then, as Hiring Manager or Chief People Officer, you’re going to have a really hard time. At best, you’re going to struggle and slog through your days and recruiting campaigns. At worst, you’re not going to have any people to serve as Chief over!

The link between informality and tech

Ask anyone within the technology field, or external to it, to describe or draw a person in tech, be it a software developer or a startup founder. The universal response is a male, typically Caucasian, dressed in sloppy jeans and a hoodie. Where did this level of informality originate?

Many would cite Lou Gerstner’s (former CEO of IBM) decision to roll out “casual dress.” Even the New York Times balked with a headline that read, “Black Jeans Invade Big Blue.” Shortly thereafter, Hewlett Packard (HP) introduced “Blue Sky Days” which permitted employees to wear jeans to work on Fridays. From there, dress code devolved or evolved – it depends on your perspective – into what we see today.

However, the Platinum Principal of resisting formality is not restricted to dress code. It extends into everything that we do. Formality also has implications with unconscious bias, how we greet and interact with people, the forms that we require candidates to complete and so on. Tone and style go hand in hand.

How are we resisting formality?

We are constantly taking steps to increase our awareness and connection with our prospective employees, candidates, contractors, staff and employees. Efforts are made to understand what they need so that we can craft our talent, acquisition and retention strategies around their needs and expectations versus requiring that they conform to ours. Here, we have embraced an agile culture across the company, not just in our product development efforts.

As agile craftsmen, with respect to HR recruiting and retention strategies, we have adopted an approach that is more responsive than it is prescriptive. Our applications are deliberately designed with brevity in mind to expedite the hiring process. We measure how long it takes from the time that an application is submitted to an open job posting until someone is seated in the role. A conscientious effort, with monitoring, is made to ensure that we are constantly moving our candidates through the hiring process as swiftly as they seek to do so.

Another way that may come as a bit of a surprise is the effort we make to create a relaxed culture that allows for flexibility and downtime. We understand the enormous pressure and stress that our employees feel as they do all that they can to deliver on our clients’ needs. It’s okay to push through a couple of scrums or back-to-back sprints but it’s not sustainable. Over the holidays, we relaxed with an in-house office party and gave people some bonus time. This week is Valentine’s Day: celebrate it or not, you don’t need to make it about sending flowers and chocolates to your partner. Use it as a reminder to resist formality, take some time and reach out to whomever it is that you care about, be it a parent or sibling or a friend.

Resist formality! Make strides to emulate the people and culture around you. Forcing people into a process defined by its rigor and inflexibility is the antithesis of what a company should do. Particularly a company who’s purpose is to bring agile methodologies and success to our clients.

Stu Weiser,

Director of Talent Acquisition
Image created by katemangostar – www.freepik.com

Configuration Automation Tools: Orchestrating Your Deployment

“We must accept human error as inevitable – and design around that fact.” – Donald Berwick

Raphael Weaver

In our Technology field, buzz words come and go. One day we’re discussing how databases are the new best thing to the world of Agile Development and the next we see programming languages, frameworks, and methodologies bit the dust like an old western movie.

But one unchanging aspect of technology and its journey is the People who are an irreplaceable part of the creation, the demise, and the popularity of any given technology. This modern-day-world calls for a close to perfection execution, which humans cannot always account for. How does that affect us, the developers and creators, when we are called to building perfect products? 

Removing Human Error

Automation automation automation – and of course throw in some orchestration deployment and configuration management. Leaving the buzz words behind the jest of all of this, “new technology frontier”, is removing human error. This is removing the dependencies of tribal knowledge when it pertains to application and system administration job duties. Those job duties are performed in a repetitive fashion. The job duties are usually consolidated into various custom scripts, leaving a lot of those scripted actions the ability to be boxed up and reused over and over again.

For example, what steps does it take to deploy an application to a server instance? 

  1. Download and Install the various languages and/or framework libraries the application usages.
  2. Download, Install, Configure the Web server that the application will use.
  3. Download, Install, and Configure the Database that the application will use. 
  4. And finally Test to see if all the steps are installed and configured correctly. 

Last but not least run application tests to make sure it is running as expected. Those are the highlight steps for deploying an application to a bare metal, cloud, or virtual server. With a further review, there are numerous upon numerous additional steps. Something simplistic nevertheless highly catastrophic is a possibility of a typo. Consider this case, maybe in this scenario, I meant to type

cd /var/ect/ansible; rm -rf *

but instead I forgot the cd execute command and only ran

rm -rf /

Now my whole drive is erased. Surprisingly enough this can happen quite often even though I am pretty sure the source of truth of my drive is backed up. However, as Murphy Law states, what can happen will happen. And let’s be honest, redoing previous actions that could have been avoided is a down right pain.

No Hidden/Tribal Knowledge

Let’s look back at those steps to deploy an application to an environment. There are a lot of gotchas and/or small intermediary steps involved. For instance, I only listed the high-level steps. I hand these steps to a new engineer or even a junior engineer fresh out of college. The chances are these engineers will have to ask several questions on  “How to”. 

These series of questions will eventually lead to someone like myself saying “just let me do it”. This is how one scenario can lead to years upon years of hidden or tribal knowledge, as one may call it.  To harbor information like a fugitive on the run is getting more and more exposed in the IT/Software industry. The information should be a source of truth maintained in a repository database, that is easy and intuitive to leverage

Implementation Technology Overview

What does that source of truth entail? Can we not skip the documentation of information and go straight into the execution of the steps onto a given system? Or create scripts to reconfigure the application if there was ever a need to? Those questions have been proposed several times and solutions have been formulated several times into the form of extensive and comprehensive build tools/frameworks. These tools are used throughout the industry to solve the problem of orchestrated development, configuration automation and management. 

Furthermore, DevOps tools such as: Chef, Puppet, and Ansible are well mature automation/orchestration tools. Each tool will provide you with enough architecture flexibility to virtually handle any use case you present. Let’s take a slight dive into each one of these technologies.

Puppet

Puppet is the first widely used Configuration Automation and Orchestration software dating back to its initial release in 2005. Puppet uses the Master and Slave paradigm to control X amount of machines. The Ruby language is the script language say, for executing commands in a destination environment. 

The “Puppet Agents”(Slave) are modularized distinct components to be deployed to a server. This can be used for the creation of the server(ie. web server, database, application) in its destination environment.  The Puppet enterprise (Master) is comprised of all the inner workings to manage, secure, and organize agents.

Puppet Architecture

Documentation 

Chef

Chef is somewhat similar to Puppet. The core language used within Chef’s abstract module components is Ruby. Chef has several layers of management for your infrastructure automation needs. The Chef workstation is the primary area for managing the various Chef components. The Chef components consist of cookbooks, recipes, and nodes. The naming convention actually follows what you would consider these nouns in cooking to be with technical context of course. 

Recipes are collections of configurations for a given system, virtual, bare metal, or cloud environment. Chef calls those different environments nodes. Cookbook contains recipes and other configurations for application deployment and control mechanisms for the different Chef clients. Lastly, the whole collections of cookbooks are maintained in a Supermarket.

Chef Architecture

Documentation

Ansible

Ansible is the newest mainstream automation/configuration management tool on the market. Therefore, Ansible uses more modern programming languages and configurations concepts and tools. Python is the programming language used in this framework. One of the modern and fastest up-and-coming template languages is YAML. YAML is programming language agnostic and is a subset of the ever so popular JSON. YAML is used within Ansible to describe an Ansible playbook. 

Ansible playbook contains the steps that need to be executed on a given system. Once the Ansible playbook is intact configuration or further manipulation of the host can be executed through Ansible API – which is implemented in Python. There are several other components within Ansible technology such as modules, plugins, and inventory. 

Ansible Architecture

Documentation

Reusability

After covering a couple of the Configuration Automation and Development tools on the market you can see the vast amount of flexibility available in eliminating those repeatable steps from human error. This software’s framework promotes reusable software within an organization – which is the most viable. The ability to scale your application development environment and environmental infrastructure is critical. 

The learning curve may be deeper than using plain old bash scripts but the structure and integrity of using a proven tool and ease of maintenance outweigh the learning curve.

Check it for yourself, reorganize and propel your CI/CD process with Configuration Automation and Orchestrated Deployments and let me know what you think about this article in the comments below.

Raphael Weaver

Sr. Software Engineer

Unleashing The Potential Of Your Offsite Team

Thriving vs Surviving

Are You Unleashing Your Team’s Potential? Or, are you an engineering Practice Lead or Manager who’s unconsciously capping your team’s potential? While there may be the odd diabolical leader who intents on deliberately holding his or her team back, I like to think that doing so is not the norm. Leaders of agile craftsmen want their teams to flourish and fly through each sprint. Doing so will require creating an environment and work culture that fosters improvement and unleashes a team’s potential.

With a new year and decade here upon us, the idea of making resolutions to catalyze changes isn’t new. It’s a centuries-old tradition that has begun to fall out of favor. Given that < 8%  of us make it through the year, performance and motivation coaches suggest that we set goals for the year instead of resolutions. By setting a goal, we know which direction we’re headed for and it allows us to “slip back” or go off-course occasionally yet still track towards the goal unlike resolutions, which do not offer the same latitude. 

Here at InRhythm, we’re looking forward to achieving continued success in 2020. With increased efforts to build and empower our 10x teams and to better enable our clients, we have set several key goals for the year ahead.

2020 Goals

  1. Continue to put our customers first and make every effort towards 100% customer satisfaction
  2. Foster a culture that truly embodies the Agile Manifesto
  3. Grow our footprint in new markets and industries 

As the Practice Lead , I’m conscious of how well my team is performing and how the toll of being onsite at clients’ workspaces affects them. I’m also aware that I need to give the team space to work together on a sprint and to figure things out on their own. When they hit an obstacle or begin struggling with forward progress, that’s the time to jump in. Until then, leaders must encourage their teams to experiment and err, support them, and back them when the choices they’ve made may not be the ones anticipated by their employer but they’re exactly the right ones needed for the client.

In addition to giving your team some latitude and space so that they can creatively solve problems without you helicoptering over them, you also need to think about giving them recognition. Of course, nobody wants to celebrate repeated failure, but, if a software developer made an informed decision to try to do something in a new way and everyone could extract lessons learned from that effort, then s/he/they should be celebrated for making a bold move and trying to do something different. It should go without saying that the successes also need to be celebrated. Recognition motivates teams to try harder.

Efforts in the opposite direction will have – no surprise – the opposite effect. For example, a lack of recognition for doing well or finger-pointing when there is an error will demotivate teams and start to put restraints on their ability (and desire) to flourish. Powering up 10x teams requires motivation and support so that members can dig deep and unlock their potential. Motivation should be a weekly or monthly focus and not reserved for the one day each year when companies formally recognize their employees, clients or management team. 

Here at InRhythm, we’re encouraging more dialogue between members and practice leaders and added the role of a liaison designed to facilitate open and honest discussion. We’re creating a culture where it’s safe to respectfully voice an opinion and to try new things that could make a difference for our clients. It’s also about accountability. If a team member come forward and voices a concern or a need that does not get addressed with an effort towards closed-loop communication, that member’s potential becomes thwarted. 

Have you made 2020 all about your team’s potential? Are you subconsciously thwarting it or actively boosting it? Be the practice leader that helps teams thrive – not just survive.

Change? What Change? Everything Always Goes to Plan

As software engineers, we are conditioned to brace for change. The very nature of our work requires that we be ready for it. Yet, at our cores, we don’t want there to be any changes. We don’t surrender to the inevitable changes that are going to be required after we submit content for a code review. Instead, we steel ourselves against it, and gear up for battle with a mindset of resistance. We don’t want our QA teams to find any bugs because we’ll then have to make changes to the code we’ve already written. 

The experience, however, is like the Grinch who does everything that he can to stop Christmas from coming. Of course, the holiday arrives anyway. Similarly, despite our best efforts to plan and design an agile effort which builds change in, we’re still not entirely prepared to embrace those changes when they do come. And they will. They always do.

Agile Value #4: responding to change over following a plan” speaks to this behavioral response. It’s all about our attitude. Are we prepared for change and willing to accept that there will be change (as there is for every software engineering project)? Or are we preparing for a fight that we are inevitably going to lose? 

Agile craftsmanship requires that we be nimble in our designs and in our approach. The code we write must stand up to meet the quality demands of the project and not interfere with other parts of the project, but it has to be nimble, too. Anyone at any time in the future may need to rework that code or replace it altogether. If it’s rigid and inflexible, the effort to update it (or replace it) will be that much more intense. Working sprint after sprint is already intense enough – we don’t need to add more stress to our day. In contrast, we need to take measured steps to reduce it.

One of the best ways to manage this stress and to reduce our anxiety is to prepare for change. Embracing a mindset of flexibility prepares us for the change that is coming. If our response is calm and collected, the changes requested will be received just like any new project instead of what may feel like a personal attack on our abilities.

Approaching any project with the expectation that it will likely end differently than it began requires proactively building a roadmap that is designed to be nimble and responsive by anticipating potential changes. Getting ahead of industry trends and predicted needs requires that we are constantly in the mode of collecting feedback from clients, tracking the market and soliciting input from other agile experts. Scenario planning is another way to help with preparedness. This is the only way that we can evolve as a trusted colleague and partner to the clients we collaborate with.

Christopher Okhravi, an authority on agile software development, offers an insightful perspective on the application of Agile Value #4. He reminds us that, “We should never forget that they [the clients] are the ones asking for the system. And they are the ones who will ultimately use it.” Another factor that we need to remember is that rigid planning and process are the antithesis of the freedom derived from an agile approach. As each project unfolds, we learn more about the needs of that project and gain new insights on how to best deliver the code needed to execute the project. However, as we do so, we tend to drift further from the original plan which no longer reflects our reality and this creates tension for everyone involved. 

So, what is the best way forward? Embrace Agile Value #4. We must plan for the worst but hope for the best because change is inevitable. We must be nimble – and stay that way. Change is a necessary element of what we do. We can design our product roadmaps to be responsive and proactive or we can choose to be reactive and struggle to keep up. Getting ahead of industry needs and trends helps us prepare for the changes that will come. Being open to continuous feedback from our clients, our managers, our peers and our colleagues helps us evolve into a trusted co-worker and business partner. 

“Change always comes bearing gifts.”  ~Price Pritchett

Robert Morrell

Practice Lead, Java Cloud

What We’re Reading Around the Web

Agile Manifesto 4/4 – Responding To Change Over Following A Plan by Christopher Okhravi
YouTube
“We should never forget that they [the clients] are the ones asking for the system. And they are the ones who will ultimately use it.”

Manage Stress
Healthfinder
“To be agile, you need to be able to ask, ‘Is this agile?’”

Applying Agile Management Value 4: Responding to Change Over Following a Plan
Dummies
“Unfortunately, traditional project management approaches attempt to wrestle the change monster to the ground and pin it down so it goes out for the count.”

Getting Started With Agile: Responding to Change Over Following a Plan
Pivotal Tracker
“As project execution unfolds, the team learns more about what needs that resulting product will fill as well as how that product can best be built. As this new reality emerges, the team struggles to keep their project aligned to the original plan, which likely no longer reflects the team’s new reality.”