Accelerate, the science of DevOps

Reading Time: 4 minutes


DISCLAIMER: this article is the transcription of an internal talk.
You can find the presentation here.
In Everli we have recurring internal moments that we use for sharing knowledge.
We are going to integrate the key points of the book we’re reviewing as we move forward and then we’ll share our learnings and outcomes.

Today I’d like to talk about one of the most interesting and educational books that I’ve recently read.

This book is Accelerate, the Science of Dev Ops and I liked it a lot for mainly three reasons:

  • it investigates the connection between software performance and business performance, a topic I always found very interesting and I consider crucial
  • it is highly practical and not too much philosophical: there’s a bunch of concrete actions and suggestions to implement
  • there’s a lot of science behind it: the rigorous mathematical approach has been one of the most appreciated aspects by the community

If I have to simplify, this could be the statement that properly summarizes the main message of the book:

“High IT performance correlates with strong business performance, helping to boost productivity, profitability and market share.”

which is quite a bold statement, if you think about it, right?
Well, reading the book we learn that there’s even more, something stronger than just correlation, but we’ll get there shortly.

Accelerate was published for the first time two years ago (2018) and it has since become one of the must-reads for the entire industry.
It was the result of four years of research work, where authors gathered thousands of data points from thousands of different organizations, in a very broad set of industries (software startup, healthcare, finance, government, …).

The key findings of this book are many, sometimes surprising (I think I’ve raised my eyebrows multiple times) and sometimes also counter-intuitive:

  • an organization’s ability to make software positively impact profitability, productivity and market share
  • we can measure software development and software delivery, and we can do this in a statistically meaningful way
  • high performing organizations are better in doing this and they do significantly better if compared to medium or low performing ones
  • throughput and stability don’t fight against each other, conversely they move together
  • culture and technical practices do matter

The authors discovered also that 24 different capabilities drive improvements in software delivery; you can consider these capabilities as the leverages that every organization can use to improve themselves, enhance the IT performance, hence the business achievements. These capabilities are grouped into five different clusters (Continuous Delivery, Architecture, Product and Process, Lean Management and Monitoring and Cultural); you can find them listed at the end of the post.

But there’s one problem: we are talking about software performance, and we know that measuring performance in the software domain is hard.
This is because we usually make a couple of mistakes: we usually focus on output rather than outcomes (think about using lines of code or even velocity for this, we all agree that it’s pretty much stupid) but also because sometimes we focus on individuals while we should look at a higher level.
This means that the correct way of measuring is by focusing on outcomes rather than output and looking at the global/team perspective rather than on a personal one.

How can we measure performance then? Well, luckily the authors did the work for us and they provide a list of simple metrics that are suitable for the job:

  1. Lead time: the time it takes to go from local env to production env
  2. Deployment Frequency: or, if you can measure it, batch size
  3. Mean Time To Restore: how quickly can the service be restored?
  4. Change Fail Percentage: what percentage of changes to production (software and infra) fail?

The researchers applied cluster analysis in all four years of their investigation and found that, when it comes to software delivery performance, every company can be easily grouped into three well-defined categories; they also realized that the four abovementioned KPIs are perfect classifiers; ultimately each group is significantly different from the others.

Here you can see the results for 2016:

I think I can hear your doubts: you might be asking whether to trust this data. I mean, they used surveys, right? People can cheat, people maybe can misinterpret, etc…
Well, there’s an entire (and quite interesting) chapter on the science behind this book, but shortly I can say that surveys are the best tools in these cases because you wouldn’t have had direct access to measurement, every company may have (or don’t have) their the measurement in place, …
If you ask questions through unbiased and structured sentences (using Likert questions, for example), it works.

Moreover, every construct (this type of sentence) has been statistically validated against discriminant validity, convergent validity and reliability; three out of the four above-defined KPIs respect these statistical checks.
Anyhow, Change Fail Rate, even if does not pass this statistic check, strongly correlates with software delivery performance.

I think now we can tackle the most important question: does it matter? Does IT High Performance really matter in business?

…Yes, it does: it turns out it totally correlates not just with business performance but also with non-commercial goals!
The surprising thing here is that there is a stronger link between these elements because it’s not just correlation, but causation too.
This means that IT performance can predict performance on profitability, market share and productivity, but also the quality of the goods and services, on customers’ and employees’ satisfaction.

“Whatever the mission, how a technology organization performs can predict overall performance.”

So, don’t outsource software that is core for your company; invest money, time and resources in strategic software, people, tools and try to implement these 24 key capabilities: it’s worth it.

Accelerate Key Capabilities

Continuous Delivery

  • Use version control for all production artifacts
  • Automate your deployment process
  • Implement continuous integration
  • Use trunk-based development methods
  • Implement test automation
  • Support test data management
  • Shift left on security
  • Implement continuous delivery

Architecture

  • Use a loosely coupled architecture
  • Architect for empowered teams

Product And Process

  • Gather and implement customer feedback
  • Make the flow of work visible through value stream
  • Work in small batches
  • Foster and enable team experimentation

Lean Management and Monitoring

  • Have a lightweight change approval process
  • Monitor across application and infrastructure to inform business decisions
  • Check system health proactively
  • Improve processes and manage work with work-in-process (WIP) limit
  • Visualize work to monitor quality and communicate throughout the team

Culture

  • Support a generative culture (as outlined by Westrum)
  • Encourage and support learning
  • Support and facilitate collaboration among teams
  • Provide resources and tools that make work meaningful
  • Support or embody transformational leadership

Author: @marcorisi (CTO)

Leave a Reply

Your email address will not be published.