InRhythm

Your partners in accelerated digital transformation

Web Engineering

Lightning Talk: Creating a Proxy with Node + Express

tldr;

I created a Node and Express proxy that rams together two websites / apps. If you are ever asked to develop code for a site you don’t have access to, this could save the day.

  1. Download the code:
    https://github.com/mattbillard/code-collider-v2
  2. Watch the video:

My most recent hobby project involved building a Node and Express proxy to ram two websites or apps together. I dubbed it “The Code Collider.” 

The idea of the Code Collider was born about two years ago when I was given a rather difficult coding challenge at a client. Another department had a website and we were being asked to iframe it. As you might know, iframes have a lot of “interesting nuances” as they might politely be called, one of which is that if a user right-clicks on a link and chooses “Open in new tab,” they will break out of the parent’s iframe and go directly to the iframed child site. Of all things, I was asked to write some code to prevent this and provide my solution to the other team. There was just one catch: I had no way of running the other team’s site in my local environment and thus (or so I initially thought) no easy way to test my code as I wrote it. 

This situation comes up more often than you’d think. There are several reasons a developer might not be able to run a site or app in their local environment. Perhaps it’s complex to set up, involves a lot of permissions, or the other team just can’t be bothered to spare the time to help you out. Regardless of the reason, I did eventually strike upon a novel way to solve the problem.

After what felt like a meandering odyssey of Stack Overflow posts, I managed to cobble together a Node and Express project that took the other team’s site, piped it through a proxy, injected my script into it, and served it to my browser. Even though my innovation worked and I was able to deliver the final code requested, there was just one thing that bothered me: I always had the nagging feeling that I had only gotten the proxy to work through a mixture of stubborn persistence and blind luck. I resolved to rebuild it from scratch in my free time one day, piece by piece, actually understanding how the solution worked – and that as it turns out is exactly what I did.

The Architecture

Here’s a diagram of how it works:

The browser makes a request to our Node Express proxy server where we then have three scenarios:

  1. If the user requested an HTML page, we need to combine the page from website 1 and 2. The proxy first asks website 1 and then website 2 for its HTML. It then modifies the two HTML files, combines them, and returns the result to the browser. (Details on how this works below.)
  2. The HTML page will then request the CSS, JavaScript, and other assets it requires. These requests will again go through the proxy which will pass on the requests. If website 1 has the asset, great, the proxy will return it to the browser. 
  3. If website 1 does not have the asset, the proxy will then ask website 2 and return it to the browser.

Here are screenshots of the result:

In this example, we’ll use the real InRhythm.com website as the target into which we’ll inject some local code (in this case a basic Create React App project). The final result on the right is an actual screenshot of the 2 websites living together in the same browser window. Now this is pretty exciting; the target website and the code that you are developing could be anything!

How it Works

  1. As mentioned above, website 1 and 2’s HTML are combined. This involves a few steps. Webpages can’t have 2 doctype, html, head, or body tags, so we’ll use some regex to strip those. Now that website 2’s HTML is ready, we’ll inject it before website 1’s closing </body> tag. 
  1. And here is the code that shows our modifications to website 1’s HTML. It shows a few things.

    Firstly, many websites have full ‘absolute URLs’ for their links. They look like this: https://www.inrhythm.com/who-we-are/
    The problem is if the user clicks on this, they’ll be taken away from our proxy and go to the target website. We’ll solve this by removing all www.something.com bit and keep just the bit after the slash.

    Secondly, we show injecting the CSS discussed above to remove backgrounds and allow clicks to pass through website 2 to website 1. (Keep in mind this will probably be slightly different depending on the two sites you are combining.)

    Thirdly, we show injecting the HTML from website 2 we modified from above before the closing </body> tag.

Gotchas to Avoid

As mentioned above, it took a lot of research as well as trial and error to solve all the technical hurdles this project presented. Here are some of the main ones below.

  1. Websites usually compress or “Gzip” their content. Normally this is a great thing. It means less data is transferred and websites load more quickly. This is not going to work for our case however. You can’t parse, manipulate, and modify HTML if it looks like gibberish. The solution was actually quite simple: as it turns out, there’s a header you can send with your request to ask the server not to Gzip anything.
  1. There’s another header we need to send however. Because we’re using a proxy, all requests are going to have the header ‘host’ set to ‘localhost’. Now this is probably not a problem for most sites, but to the target server, this doesn’t look like a very normal request, and indeed, I did find some websites responded oddly and returned pages that looked nothing like I expected. The solution again was quite simple, just modify one of the headers of our request. 
  1. Now as you’ve probably gathered from above, we’ve modified our requests quite a lot, and as a result this is going to cause the browser to do some odd things. The solution to this problem is to delete the ‘content-length’ header before the proxy sends your browser any response. This will stop the browser from truncating the response and removing all the hard work we did. 
  1. Finally, I’ll mention one other problem I solved. If you are combining sites that use https, the proxy might complain that the SSL certificates don’t match what it’s expecting. Turns out it’s rather easy to relax this with the following code: 

Conclusion

That’s it. Feel free to download the code, follow the instructions in the readme, and open your browser to localhost:8000 . You should be able to see two web sites rammed together. 

How Customized Usability Testing Benefits You and Your User

Software development is all about accurately capturing what the user wants. More accuracy is always better, of course. But what if you could have more accuracy while also saving costs? Why wouldn’t you want that?

Traditional software development was to actually write the software, get your users to test it, have their inevitable change requests come in, change your software, test again, etc. It’s more labor intensive that way (thus more costly), but more importantly it’s less accurate, though that might seem counterintuitive. When you deliver something the user does not want, you did not accurately give them their desired solution. By accurately capturing what the user wants earlier in the process, developers can more accurately engineer the solution later.

For example, let’s say you are mocking up new screens in wireframes on Figma. It’s much easier to change a wireframe than to change code, and for the user it’s basically the same thing. They see the UI changes, get to make their change requests, and you get to implement those change requests in Figma, not in code. 

But usability testing takes this a step further. By using mood boards you can get on the same page as your user early in the dev process.  Using rapid prototyping lets you quickly iterate through various ideas in the early, cheaper stages of the project. And then wireframing, whether you are using Figma, Sketch or XD files, lets your users see how the app will appear. All of this can be done during the early, cheaper stages, and why wouldn’t you take advantage of that with the great tools we now have? Usability testing gives you more accuracy at a lower cost. Can’t beat that!

Letting Go of the Formality

It wasn’t that long ago that suits, ties, knee-length skirts, heels and pantyhose were required wardrobe staples. In fact, many enterprise Fortune 100s upheld a formal dress code as part of their HR Policy Handbooks. Goldman Sachs, the last bastion of formality on Wall Street, finally relaxed its dress code and offered a “flexible style” option. They only did so last year. Their updated policy permitted men to choose garments other than suits. Admittedly, this made things more complicated for women but that’s a topic for another blog.

Since we’re agile practitioners here at InRhythm, it made sense to look at our own approaches to talent and acquisition. Not to mention retention. Practices and policies drive corporate culture. Here, Agile Platinum Principle One: resist formality, must serve as one of the pillars of our talent strategy.

If your company doesn’t offer a culture that’s aligned with the needs and expectations of today’s job seekers, then, as Hiring Manager or Chief People Officer, you’re going to have a really hard time. At best, you’re going to struggle and slog through your days and recruiting campaigns. At worst, you’re not going to have any people to serve as Chief over!

The link between informality and tech

Ask anyone within the technology field, or external to it, to describe or draw a person in tech, be it a software developer or a startup founder. The universal response is a male, typically Caucasian, dressed in sloppy jeans and a hoodie. Where did this level of informality originate?

Many would cite Lou Gerstner’s (former CEO of IBM) decision to roll out “casual dress.” Even the New York Times balked with a headline that read, “Black Jeans Invade Big Blue.” Shortly thereafter, Hewlett Packard (HP) introduced “Blue Sky Days” which permitted employees to wear jeans to work on Fridays. From there, dress code devolved or evolved – it depends on your perspective – into what we see today.

However, the Platinum Principal of resisting formality is not restricted to dress code. It extends into everything that we do. Formality also has implications with unconscious bias, how we greet and interact with people, the forms that we require candidates to complete and so on. Tone and style go hand in hand.

How are we resisting formality?

We are constantly taking steps to increase our awareness and connection with our prospective employees, candidates, contractors, staff and employees. Efforts are made to understand what they need so that we can craft our talent, acquisition and retention strategies around their needs and expectations versus requiring that they conform to ours. Here, we have embraced an agile culture across the company, not just in our product development efforts.

As agile craftsmen, with respect to HR recruiting and retention strategies, we have adopted an approach that is more responsive than it is prescriptive. Our applications are deliberately designed with brevity in mind to expedite the hiring process. We measure how long it takes from the time that an application is submitted to an open job posting until someone is seated in the role. A conscientious effort, with monitoring, is made to ensure that we are constantly moving our candidates through the hiring process as swiftly as they seek to do so.

Another way that may come as a bit of a surprise is the effort we make to create a relaxed culture that allows for flexibility and downtime. We understand the enormous pressure and stress that our employees feel as they do all that they can to deliver on our clients’ needs. It’s okay to push through a couple of scrums or back-to-back sprints but it’s not sustainable. Over the holidays, we relaxed with an in-house office party and gave people some bonus time. This week is Valentine’s Day: celebrate it or not, you don’t need to make it about sending flowers and chocolates to your partner. Use it as a reminder to resist formality, take some time and reach out to whomever it is that you care about, be it a parent or sibling or a friend.

Resist formality! Make strides to emulate the people and culture around you. Forcing people into a process defined by its rigor and inflexibility is the antithesis of what a company should do. Particularly a company who’s purpose is to bring agile methodologies and success to our clients.

Stu Weiser,

Director of Talent Acquisition
Image created by katemangostar – www.freepik.com

Unleashing The Potential Of Your Offsite Team

Thriving vs Surviving

Are You Unleashing Your Team’s Potential? Or, are you an engineering Practice Lead or Manager who’s unconsciously capping your team’s potential? While there may be the odd diabolical leader who intents on deliberately holding his or her team back, I like to think that doing so is not the norm. Leaders of agile craftsmen want their teams to flourish and fly through each sprint. Doing so will require creating an environment and work culture that fosters improvement and unleashes a team’s potential.

With a new year and decade here upon us, the idea of making resolutions to catalyze changes isn’t new. It’s a centuries-old tradition that has begun to fall out of favor. Given that < 8%  of us make it through the year, performance and motivation coaches suggest that we set goals for the year instead of resolutions. By setting a goal, we know which direction we’re headed for and it allows us to “slip back” or go off-course occasionally yet still track towards the goal unlike resolutions, which do not offer the same latitude. 

Here at InRhythm, we’re looking forward to achieving continued success in 2020. With increased efforts to build and empower our 10x teams and to better enable our clients, we have set several key goals for the year ahead.

2020 Goals

  1. Continue to put our customers first and make every effort towards 100% customer satisfaction
  2. Foster a culture that truly embodies the Agile Manifesto
  3. Grow our footprint in new markets and industries 

As the Practice Lead , I’m conscious of how well my team is performing and how the toll of being onsite at clients’ workspaces affects them. I’m also aware that I need to give the team space to work together on a sprint and to figure things out on their own. When they hit an obstacle or begin struggling with forward progress, that’s the time to jump in. Until then, leaders must encourage their teams to experiment and err, support them, and back them when the choices they’ve made may not be the ones anticipated by their employer but they’re exactly the right ones needed for the client.

In addition to giving your team some latitude and space so that they can creatively solve problems without you helicoptering over them, you also need to think about giving them recognition. Of course, nobody wants to celebrate repeated failure, but, if a software developer made an informed decision to try to do something in a new way and everyone could extract lessons learned from that effort, then s/he/they should be celebrated for making a bold move and trying to do something different. It should go without saying that the successes also need to be celebrated. Recognition motivates teams to try harder.

Efforts in the opposite direction will have – no surprise – the opposite effect. For example, a lack of recognition for doing well or finger-pointing when there is an error will demotivate teams and start to put restraints on their ability (and desire) to flourish. Powering up 10x teams requires motivation and support so that members can dig deep and unlock their potential. Motivation should be a weekly or monthly focus and not reserved for the one day each year when companies formally recognize their employees, clients or management team. 

Here at InRhythm, we’re encouraging more dialogue between members and practice leaders and added the role of a liaison designed to facilitate open and honest discussion. We’re creating a culture where it’s safe to respectfully voice an opinion and to try new things that could make a difference for our clients. It’s also about accountability. If a team member come forward and voices a concern or a need that does not get addressed with an effort towards closed-loop communication, that member’s potential becomes thwarted. 

Have you made 2020 all about your team’s potential? Are you subconsciously thwarting it or actively boosting it? Be the practice leader that helps teams thrive – not just survive.

The Importance of Placing Individuals Above Processes and Tools

Empathy should be regarded as a super-power. Not enough people have it and the workplace – indeed, the entire world – would be a better place if more people did have it. When you have project deadlines or you’re in the midst of an intense all-hands scrum, it’s easy to forget about the people factor. That is, there are people working with you and for you who are missing celebrations, concerts or the opportunity to just chill out at home because their effort is critical to your product development effort.

But you’re on a tight timeline. You have key deliverables. Your client has worked backwards from the launch and expects that you will deliver a bug-free software solution by the date circled on the calendar. As the Practice Lead, you may be luckier than most in your role if you have a dedicated team cranked up on caffeine and working late into the evening – every evening, in fact. This is great for deadlines but it can have far-reaching negative consequences if this pace is expected to  be sustained.

The Agile Value #1 requires that we value individuals and interactions over processes and tools. Yet you’re conflicted. Project deadlines and deliverables are what they are and you’ve explained them to your team. Missing a due date and delaying a launch is not going to happen on your watch. Processes for coding, testing, documenting must happen as prescribed, regardless of the yeoman’s effort required to get it done. Your team understands this, right? Surely you told each team member that they were doing great work when you met last year for their performance review?

Maybe they do. Or maybe they don’t. Agile product development places enormous demands on the people doing the work. Both the team members and their practice leads have to grapple with the relentless pressure and the temptation to skip part of the process to move things along faster. 

When you’re building high velocity teams, you need to make daily investments in your people, processes and infrastructure. Take any and every opportunity to express empathy, understanding and gratitude for the work that your team, and each individual on your team, is doing. Expressing appreciation once per year during a performance review will likely end up becoming a one-and-done experience because that employee likely won’t be around this time next year.

There are simple ways to bolster your team’s productivity and motivate them to stay on track with respect to processes. The easiest way is to say, “Thank you.” If you want a high velocity team who is loyal, dedicated to the needs of your customers and proud of the work they do, then say what you mean and mean what you say. Be sincere. 

To help ensure that your agile engineers remain “your” agile engineers, go one step further and acknowledge the efforts of individuals in front of others. People are human and their efforts, sacrifices and basic needs must be openly acknowledged. As software engineers in a hot market, we have the flexibility to go wherever we like towards the goal of being part of a community that values our membership. Agile leaders must reinforce appreciation of these “extra efforts” through grace, gratitude and occasionally, through special dispensation. Catering dinner (that’s not pizza) for the team every once in a while is a good expression of gratitude. This is how to build trust and long-term relationships, as well as high velocity, agile teams. 

Staying strong on execution and adhering to process is critical for success. Acknowledging the personal efforts that it took to enable that success must be acknowledged. Remember, as per agile Value #1 –  if you put people first, ahead of processes and tools, you will have created a space that fosters a high performing team’s success.