Coming to grips with DevOps metrics

In my team we have been considering ways to monitor our own performance, and finding some ways to contextualise our ongoing process and quality improvements. Like many other teams, we’ve landed on the DORA metrics as a good way of doing this.  These four key metrics are an easy way to understand what adjectives like “maintainable”, “reliable”, and “efficient” mean in practice when applied to software and teams, and the provide a way of comparing team performance across teams and over time.

Last week I facilitated a session at XTC, where we discussed the new product development framework from Basecamp, Shape Up, led by Thomas Ankorn.

It was a really interesting discussion with people exploring the ideas openly, guided by questions posed by Thomas to get the conversation started.

I’ve summarised the points that came up in the discussion, reconstructed from memory and the collected post-its of scribbles at the end of the evening.

1. What problems does this solve?

We started by looking at the positives of the framework, trying to identify strengths that would help mitigate problems we have encountered in our own teams.

DISCLAIMER: This was not my idea - I picked it up from Jane Nicholson at an XTC event, who was introduced to it by Jess Gilbert (who in turn, I am told, got it from someone else). This post is just explaining why I believe it can be a useful exercise, not any truly original thinking!

One of the things that is really important to me is that my team and I keep learning at work. As such, fostering a learning environment is kinda key. I think we do that pretty well at Haplo – we have hired nearly exclusively early-career developers to our tech team for several years now, and one of the key points most hires mention as to why they join is that they feel it’s a good place to learn quickly, with the requisite support early on in their career.

It is upsettingly easy to work hard without being productive. The Lean Startup includes a quote I really liked:

“[People] feel that a good day is one in which they did their job well all day.”

The point being that this doesn’t account for whether you are doing the right work. This is a really common trap to fall in to. I know I have often worked really hard on something and produced code I’m very happy with, only to talk to my boss or the client and find that nobody else seems to be nearly as pleased with what I’ve produced as I am.

A theme of the last few books I’ve read has been user testing. Specifically, that it is completely unreasonable to suggest that you could produce a system that will work smoothly in production without first getting several actual human users unfamiliar with your system, from your clients or in a related industry, to road-test the system.

I think anyone can see that this is a good idea. It’s far too easy to make assumptions about the users’ requirements or use cases. Without user testing software would rarely solve the problems it was designed for.