Enabling Microservice Success

Managing Technical, Organizational, and Cultural Challenges

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

Sarah Wells

Enabling Microservice Success

by Sarah Wells

Copyright © 2024 O’Reilly Media. All rights reserved.

Printed in the United States of America.

Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.

O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com.

Acquisitions Editor: Melissa Duffield

Development Editor: Jill Leonard

Production Editor: Clare Laylock

Interior Designer: David Futato

Cover Designer: Karen Montgomery

Illustrator: Kate Dullea

January 2024: First Edition

Revision History for the Early Release

2022-10-21: First Release

2023-01-23: Second Release

2023-05-19: Third Release

See http://oreilly.com/catalog/errata.csp?isbn=9781098130794 for release details.

The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Enabling Microservice Success, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc.

The views expressed in this work are those of the author and do not represent the publisher’s views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

978-1-098-13081-7

[TO COME]

Preface

Microservice architectures can be a very effective approach to speeding up delivery of value to your organization and customers. If you get it right.

Get it wrong and you can end up with a complex mess that makes operation and maintenance very hard and leaves you with small teams trying to support lots of services, some of which they’ve never touched.

Adopting microservices goes beyond selecting an architectural approach. To be successful at doing microservices, you need to make cultural and organizational changes. You have to move towards autonomous, empowered teams.

That means that many things that used to be someone else’s concern are now the responsibility of engineering teams. You need to think beyond system design, architecture and implementation. That includes considering how you will build systems that you can successfully operate in production, and how to maintain and manage them for the long term. You need to understand distributed systems architecture and are likely to be more hands-on with at least some parts of your infrastructure.

This book will help you with all of that. It gives practical advice on how to adopt a microservices architecture and how to make sure it still works for you once you are several years in, maintaining and sustaining your systems as they mature.

Why I Wrote This Book

The focus of this book is how to benefit from microservices for the long term. I want you to avoid getting several years in and looking around to find lots of accidental complexity, with developers spending their time on things that aren’t providing business value. If you’re already at that point, I want to help you tackle that.

I built my first microservices in 2013 and I was still there at the same organization building and operating the same systems 8 years later. That means I’ve seen the problems, tried to solve them, and been there long enough to know whether those solutions actually worked.

During that time I’ve worked in product development, operations and incident management, and engineering enablement. That’s given me a wide perspective on how to approach microservices. My aim is to help you get to a point where you are using a microservices architecture successfully and sustainably.

Who Should Read This Book

This book is for engineers and technical leaders who are moving to microservices and wondering what that means for all the techniques and processes they currently use. And those already using microservices who are struggling with the complexity and want to learn how other people have successfully met some of those challenges.

I assume that the reader is familiar with the fundamental concepts of software development and architecture but I don’t assume you already have experience with microservice architecture.

I won’t spend a lot of time on the details of specific technologies, or stepping through how to do things. There are already books that cover these things, and I will recommend them at the relevant points. This book will focus on practical advice but won’t be specific to any one technology, instead focusing on the principles that will help you decide what tools you need.

If you’re new to this architectural style, Chapter 1 is an introductory chapter where I cover an overview of what microservices are, the advantages and disadvantages, and the enabling technologies that helped them become widely adopted. If you already feel you have a grasp of that, you can skip that chapter and start with Chapter 2.

Navigating This Book

This book has three parts. The first part sets the context - what are microservices, what does success look like, and are microservices the right architectural pattern for you?

The second part focuses on the organizational and culture challenges, because they are critical for getting everything else right. That includes finding the right boundaries between systems, aligning your teams to match, and putting the things in place that will allow those teams to be autonomous.

The third part digs into the practicalities of building and operating microservices. Each chapter covers techniques for getting the most out of microservices, explaining where they require a different approach to the monolith, and drawing on nearly a decade of experience. I also talk about what to do if you didn’t get things right to start with: how can you make things better?

Conventions Used in This Book

The following typographical conventions are used in this book:

Italic

Indicates new terms, URLs, email addresses, filenames, and file extensions.

Constant width

Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords.

Constant width bold

Shows commands or other text that should be typed literally by the user.

Constant width italic

Shows text that should be replaced with user-supplied values or by values determined by context.

Tip

This element signifies a tip or suggestion.

Note

This element signifies a general note.

Warning

This element indicates a warning or caution.

O’Reilly Online Learning

Note

For more than 40 years, O’Reilly Media has provided technology and business training, knowledge, and insight to help companies succeed.

Our unique network of experts and innovators share their knowledge and expertise through books, articles, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in-depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, visit https://oreilly.com.

How to Contact Us

Please address comments and questions concerning this book to the publisher:

O’Reilly Media, Inc.

1005 Gravenstein Highway North

Sebastopol, CA 95472

800-889-8969 (in the United States or Canada)

707-829-7019 (international or local)

707-829-0104 (fax)

support@oreilly.com

https://www.oreilly.com/about/contact.html

We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://www.oreilly.com/catalog/9781098130794.

For news and information about our books and courses, visit https://oreilly.com.

Find us on LinkedIn: https://linkedin.com/company/oreilly-media

Follow us on Twitter: https://twitter.com/oreillymedia

Watch us on YouTube: https://www.youtube.com/oreillymedia

Chapter 1. Understanding Microservices

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 1st chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at jleonard@oreilly.com.

If you are new to microservices, this chapter will give you a solid grounding on what they are, where they shine, and where they present challenges. I’ll also cover the ecosystem that tends to go along with microservices—​the technologies that enable this architecture.

Let’s start by defining what microservices are.

Microservices are independently releasable services that are modeled around a business domain.¹

Modeling microservices around a business domain gives a closer alignment between business and IT, and means that most changes are within a microservice, so your team has complete control over making that change. In other words, costly coordination can be avoided.

Having independently releasable services means that as soon as a change is ready, you can release it. Typically, this happens multiple times a day.

The separation between these services gives teams more options: they can be more flexible in terms of the technology used, they can build services with different levels of robustness, they can scale them independently. This flexibility also makes change easier, allowing engineers to solve problems as they hit them.

All of these aspects mean microservices give teams the capability to move fast.

Defining the Microservice Architectural Style

Let’s go further than a definition of what microservices are, and look at a definition of the microservices architectural style. A microservice architecture is made up of lots of microservices, communicating over the network, meaning this is a distributed architecture.

Here’s what James Lewis and Martin Fowler wrote in their 2014 article that set out to define the then-new way of architecting software systems:

The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms…​. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.

Let’s extract the key elements of that definition by James and Martin:

a suite of services

each running in its own process

communicating with lightweight mechanisms

built around business capabilities

independently deployable

small

with a bare minimum of centralized management

heterogeneous (may be written in different programming languages or use different data storage technologies)

Let’s now dig a bit deeper into each of them in turn.

A Suite of Services

For a microservices architecture, rather than deploying all the code for your system as a single monolithic executable, you split your code across multiple services that can be deployed independently. As mentioned above, this allows you to release code for just one service, as soon as it’s ready, and gives you flexibility in how you build each service.

Each Running In Its Own Process

A microservices architecture is a distributed architecture. Each microservice runs in its own process, which means that what would be a method or function call within a single process in a monolith now goes over the network. This provides a clear boundary, making it hard to accidentally couple parts of the system together. However, calls over the network are much more likely to fail than those that are in-process. You can’t assume that the service you are calling will always be accessible and ready to serve your requests as quickly as you might like.

Communicating With Lightweight Mechanisms

A principle of the microservices architectural style is to keep the communication between services as simple as possible, in contrast to earlier service-oriented architectures (described later in this chapter) where much of the complexity lived in the messaging layer.

Keeping communication simple means your services should either make direct calls to other services, for example over HTTPS, or pass messages using a lightweight message bus. The advantages here are that you don’t need a deep understanding of a complicated specification for message format, ² and you keep the business logic in one place—​the service—​rather than having some of that logic in a shared messaging layer where changing the logic will require coordination between teams, meaning it will take longer to make those changes.

Languages generally have the ability to make HTTPS calls built in (you can of course use other protocols, for example gRPC) and it is an easy thing to debug—​for GET requests you can make a call using a browser.

Keep all the complexity within your services. They should understand what types of message they need to send or receive.

Built Around Business Capabilities

Teams working on a microservices architecture should own a business domain end to end, from the UI or API layer through the business logic and on to the database.

Most software changes happen within a particular business domain, so as long as you find the right boundaries (discussed further in Chapter 4), you should find that teams within a microservice architecture can work fairly independently. This avoids the coordination overhead for planning work that you see in an architecture where frontend, business logic and data are all owned by different teams.

Note

Including the UI within the microservice is the ideal, although I’ve found it’s rarely the case. In general there is still a distinction between the backend code that handles business logic and talks to the data layer and frontend code that displays information to the customer. A good API between the two and the use of microfrontends rather than a monolithic frontend application are important here.

Independently Deployable

A microservice should have its own build and deployment pipeline, making it possible to release just that service. It should have well-defined endpoints, for example APIs, meaning you should know where you have a change that could impact people outside your team.

Provided that the microservice is highly cohesive, i.e. you found the right boundaries between services (see Chapter 4 for more on this), most changes should be internal to the service, because they are specific to the business domain. You should be able to release a new version of a microservice as soon as it’s ready, without needing to release other services or coordinate with anyone outside your own team. As a result you naturally move to smaller changes, released often.

To be able to benefit from this, you need to automate your build and deployment pipeline: you can’t afford deployment to be a manual process as it will be too much of an overhead. Taking an hour to do a manual deployment once a week might be OK, but taking an hour to do a manual deployment 10 times a day is not a good use of anyone’s time. This makes investing in automation crucial.

Independently deployable also means independently scalable. If you have some parts of your system that struggle under load, you can spin up more instances of just that microservice. With a monolith, bottlenecks that constrain throughput generally mean you have to scale the whole thing.

Independently deployable also means that you can make and roll out changes to your architecture service-by-service, so that for example changing a programming language version no longer has to be done as a big bang change for the whole system.

“Small”

Microservices are smaller, because you have multiple services each implementing a specific business capability, rather than having one deployable that includes all business capabilities.

How small exactly though is a matter for discussion—​and I think finding the right level of granularity is one of the challenges of this architectural style (see “Finding the Right Level of Granularity”).

With a Bare Minimum of Centralized Management

Many aspects of microservices act as a decentralizing force. Assigning ownership of particular domains to individual teams, keeping the business logic within the service, and the ability to more easily use different technologies all move an organization away from centralized management.

To really benefit from microservices, teams will need to take on responsibility for things that used to be handled centrally—​because coordinating with another team would slow them down too much. For example, they will probably do their own releases, and support their systems when something goes wrong.

Note

Moving to microservices doesn’t mean teams will all have to do 24/7 support and carry pagers. My view though is that if you are making frequent changes to a system, there will be some problems caused by that code rather than the underlying infrastructure that only you and your team will be able to quickly fix. I will talk about this in a lot of detail in Chapter 7 because this was one of the biggest changes we faced at the Financial Time (FT) and caused a fair amount of concern to teams.

This can also include high level decisions about the technologies you use, although different organizations tackle this differently. Personally, I feel there is less space for a central team mandating all the tech that people will use—​and more space for teams to make a case for having specific needs that require something else.

Heterogeneous

With independently deployable microservices, you have the option to make different choices for different services. Maybe you want to write the code for your website in Node.js, but use python for data processing.

Similarly, you can use different types of data store, depending on how you need to access that data. In the content publishing team at the FT, we stored articles in a document store and these were generally retrieved by unique ID as entire documents. We stored the relationships between people, organizations, content and topics in a graph database, because that supported the kinds of queries we needed to make: for example, retrieving the 10 most recent articles about Google, or all the articles written by a particular author. We could also focus on the data each system cared about: the graph didn’t need the full content of an article to be stored in it, for example.

Warning

It’s worth saying though that while you can take this polyglot approach, you need to consider the increase in complexity as you add each new thing. I will talk about this much more throughout the book. In general, proceed, but proceed with caution, weighing up whether the benefit of using something that is a better match for your specific needs outweighs that increase of complexity.

Forerunners and Alternatives

Architectural choices involve a trade-off. It’s a question of looking at the strengths and weaknesses of a particular approach and comparing them to what matters most to your business.

In Chapter 3 I’ll talk about how you can assess whether microservices are the right trade-off for you.

For now, I’ll briefly cover some of the architectural alternatives then some of the advantages and disadvantages of microservices. The aim isn’t to give a comprehensive assessment of these, it’s more about setting the scene. What were microservices replacing? And what other replacements could you opt for? That means I also need to talk about the technologies and processes that are commonly adopted alongside microservices, because they maximize the advantages and minimize the disadvantages.

The Monolith

I want to talk first about the architectural approach that we generally compare microservices with, both because it was widely used before microservices took off, and because it is still generally the first style of architecture used for a system: the monolith.

Note

I’ll talk about this more in Chapter 3, but for small teams the cost of adopting microservices is generally not yet worth it. A monolith should be your first choice!

A monolith is a software system where all the code is deployed together, for many different business capabilities.

There will likely be some structure within the code to help developers navigate it, for example having different packages for different business functionality, but generally, the code is packaged up, tested and released together.

While we say monolith, in fact it’s common to have multiple tiers in this architecture. Often that is three: one for data, one for business logic, and one for UI. Since communication between the tiers may happen over the network, and monoliths may have multiple instances in different availability zones or regions, monoliths are likely to be at least a little bit distributed.

Figure 1-1 shows the kind of diagram interviewers regularly used to ask me to draw on a whiteboard. There are three tiers, and each tier has specialist teams with specific skills working on it. Pretty much any change to business functionality involves changes in each of the tiers, meaning communication and coordination between those teams.

Figure 1-1. The Monolith

Because all the code lives together, it is easy for one team to make a change to some code and find it unexpectedly impacts some other business functionality. Also, it is easy to reuse code for different use cases without thinking about the implications. For example, if two teams working in different domains both have the concept of “Account”, they should probably model it differently, but in a monolith that may not happen.

Also, a release can be a significant event, because of the time it takes to do. You have to run all the tests to be sure there are no accidental impacts of a change. This tends to mean fewer releases, and more change in each release.

On the plus side, a monolith is simpler to understand and to operate than a more distributed system. Most calls are within process; no network issues to trip you up. The architecture is simple to draw, and it doesn’t change rapidly so you can likely trust the architecture diagram, something I’ve learned isn’t necessarily the case for a microservices architecture. If something goes wrong, you can jump on a box and tail logs. A monolith is absolutely a valid architectural choice for many organizations and most startups keep a monolith until they scale up to a size where there are too many people working on it and tests are starting to take a long time to run.³

I want to note that most organizations have more than one monolith. As an example, when the Financial Times was using monolithic architectures, we had multiple monoliths. Among them:

the editorial content management system

the website, including the publishing flow

membership and subscriptions

These monoliths were generally integrated through direct and custom-coded integrations.

Modular Monoliths

There are ways to reduce accidental coupling and speed up releases without giving up on the monolith. One way is to separate the code within a monolith into logical modules tied to business domains. This leads to what is known as a modular monolith.

Here, the code still lives in a single repository and is deployed as a single deployment unit, through a single build and deployment pipeline. However, the code is split logically into components that map to different domains, and the boundaries between those domains are carefully managed.

The logical split should reduce the likelihood that a change made by one team breaks some other feature. However, it can be difficult to catch accidental blurring of the boundaries. This could be down to inexperience or a lack of onboarding, so that people don’t recognise they are crossing a boundary. And teams under pressure may decide to deliberately cross the boundary as a form of technical debt. If they are able to pay back the debt quickly, this may be a trade-off worth making, but the danger is that you find your modular monolith is a lot more coupled than you were planning for.

Releases can still take a long time, for example if they run the whole test suite each time. There are a couple of approaches here. You can run a specific subset of tests for changes in a particular module, which speeds up the release—​but may not catch problems where there is coupling you didn’t know about.

Alternatively, you can accept that the release takes a while but have mechanisms in place so that by the time your code is merged to main and is going to be released you have a lot of confidence that it will pass your pipeline—​for example by running a full set of tests on a branch for that release before you merge it. Some companies batch up a few code changes together to avoid having multiple releases going through the pipeline at the same time. Using canary releases where the code is only live on a small subset of your instances also makes it easier to reverse out changes that have an unexpected impact (I’ll talk a lot more about canary releases in Chapter 9).

A modular monolith is a good approach to take if you are starting to see issues with your monolithic architecture. Best case, it will solve them. Worst case, it will help you find the boundaries where you can extract services if you want to move to a microservices architecture.

In Chapter 3 I give a detailed case study of how Shopify have used a modular monolith approach, so go there for more detail on this.

Service-Oriented Architecture

Integrations between monoliths used to require point-to-point custom integrations, meaning that if two systems needed to get access to membership information, they would both have to create an integration.

Service-oriented architecture (SOA) was a response to this. In SOA, teams create services that provide specific business functionality: for example, retrieving information about a reader’s subscription status. They register these services centrally, and any team needing access can find and make use of them. These services might be a thin wrapper around a legacy system: the benefit is to simplify interactions and reduce duplication of effort and code.

SOA emerged in the late 1990s, but it really took off in the early 2000s with the arrival of web service standards, and in particular SOAP - simple object access protocol (an XML-based message protocol). Often, SOA implementations of the time relied on a centralized software component (the Enterprise Service Bus or ESB) to keep track of the services, perform any necessary transformations and route messages to the right place.

My experience with SOA was that this middleware could be a bottleneck as it had a lot of logic in it: every application needed to use and configure the ESB and changes made for one purpose could impact others. In fact, sometimes the ESB would include business logic which would mean deploying the app and an ESB patch in lock step. I also found that communication protocols like SOAP were fiddly to work with and I spent quite a lot of time managing changes to schemas.

I see microservices as a development of SOA, and a development that relied on other changes to happen in the tech world. When people were getting started with SOA, in general we were setting up our own servers, manually. Our data was in relational databases with many tables. And our release process was slow and also mostly manual. None of those things are still the case.

The Microservices Ecosystem

Microservices are an evolution of SOA, made possible by new technologies and new ways of working that have become available over the last decade or so.

These changes include the types of infrastructure we can run our applications on: with the availability of new deployment technologies such as containers and orchestration, serverless and Platform-as-a-Service (PaaS) as we have moved to the cloud. They include the benefits of automation, both for provisioning of that infrastructure and for the deployment of code. They also include changes in how we work, with the rise of DevOps as an approach to building and operating our systems and a shift from monitoring to the broader concept of observability.

These are enabling technologies and approaches for microservices. Meaning that without these it would be hard to do microservices, and I think impossible to be successful at them. If you read this chapter and you don’t have these enabling technologies in place, they would be a good place to focus your earliest efforts.

Together, these enabling technologies—​new deployment options in the cloud, automation, DevOps and observability—​allow a Cloud Native approach: building applications that are designed to make the most of the cloud, rather than being a lift and shift of a monolith running in a data center.

Cloud Native is about speed and scale. Can you move fast, and scale when you need to? Microservices are are a good fit here.

Let’s dig into these enabling technologies.

Infrastructure as Code

When I first joined the FT, in 2011, it was to build the first Content API, giving access to the FT’s articles and images for internal teams and specific third parties. For this new project, we needed a server, and it took 6 months to buy it, build it, rack it, configure it, set up DNS, etc. The whole process was manual.

Over the next few years, the FT invested heavily in technologies to speed this process up. First of all, we set up a private cloud in our data centers.

The US National Institute of Standards and Technology (NIST) defines cloud as access to a pool of computing resources (servers, storage, networks, services etc) that can be rapidly provisioned and made available with minimal overhead.⁴

The FT built an Infrastructure-as-a-Service (IaaS) platform so that a new virtual machine (VM) could be spun up on demand and an application deployed to it, rather than requiring someone to buy and set up a new physical server and then configure everything required.

Once you can spin up VMs, it makes sense to automate the process so that it can be done in minutes. This has the happy additional benefit of ensuring a level of consistency in the VMs you spin up, because you use the same server image template for all of them.

Servers, however, have a tendency to become more different to each other over time (“configuration drift”). That might be because people have made manual ad-hoc changes, or because you changed that server image template, so new servers will be different to old ones. This inconsistency can lead to unexpected behavior and instability.

Infrastructure as code is the solution to this. As you might suspect, this is about defining your infrastructure in code and, crucially, continuously rerunning that code so that your infrastructure remains consistent.

As Kief Morris writes in Infrastructure as Code:⁵

Infrastructure as Code is an approach to infrastructure automation based on practices from software development. It emphasizes consistent, repeatable routines for provisioning and changing systems and their configuration. You make changes to code, then use automation to test and apply those changes to your systems.

Because the infrastructure configuration is code, it is held in source control, making it easy to see what has changed and who made that change, and to go back to the state at a particular point of time if necessary, for example if something went wrong.

Because the process of making a change is automated, you can make sure that you create an audit log that shows the changes and who applied them: great for security.

Infrastructure as code means we can create servers, provision them, update them and tear them down through running software commands, and the results are the same every time. In a microservices architecture, these are things we do frequently, which is why infrastructure as code is important.

Continuous Delivery

Releasing code to our hand-built server in 2011 was also a very manual process. The steps were laid out in an excel spreadsheet and there were more than 50 of them. Because it was manual, it was error prone.

It was also slow, taking hours, and since while it was happening journalists couldn’t publish any content, we couldn’t do it during normal working hours. This meant we released our code to production on a Saturday morning,⁶ and no more frequently than once a month.

You cannot successfully do microservices unless you automate the process for releasing code. But that’s not all. You also need to be able to release changes with negligible downtime so you can do that at any time. And finally, you need to speed up the time spent on testing, through a focus on automated tests that don’t require complex setup or a shared staging environment.

You need to be doing continuous delivery (see Figure 1-2).

Note

This book is only going to touch briefly on continuous delivery.

For in-depth coverage, see Jez Humble and Dave Farley’s Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation (Upper Saddle River, NJ: Addison-Wesley, 2010)

Figure 1-2. The Continuous Delivery Cycle

Continuous delivery is about continuously releasing small changes, through an automated build and deploy pipeline that incorporates automated testing.

It’s hard to work in small batches—​one of the key principles of continuous delivery—​without a loosely coupled architecture; specifically, one where you can change part of the system and test just those changes.

It’s also hard to benefit from a move to microservices unless you are doing continuous delivery!

At the FT, moving to continuous delivery and adopting microservices for the content publishing platform took us from 12 releases a year to around 2,500. That’s around 10 releases per working day—​about 200 times as often.

The Public Cloud

The FTmay have started with a private cloud, but it soon moved to take advantage of the public cloud, using AWS. Public cloud means someone else owns the hardware.

I remember the concerns in the early 2010s about what it meant to run your code on machines owned by someone else—​what would happen if they upped the price? was it safe to keep our data there? could they go bust? Over time, as more people moved to the public cloud, we got more comfortable about the risks. And there are significant advantages.

Firstly, you no longer have to buy, build and manage the underlying resources. This can save you money, although you do have to make sure you keep on top of the bill, because provisioning on demand can mean a lot of developers provisioning servers that are more than big enough “to avoid issues”, and provisioning things they then forget about. This also changes the cost model of buying servers from CAPEX (capital expenditure: buying lots of servers up front) to OPEX (operational expenditure: hiring machines when you need them). It is worth talking to your finance department before making this switch because they may have a strong opinion on whether this is a good thing!

This will definitely save you effort. If you run your own private cloud in a data center, you are still having to buy servers and network kit and pay all the associated support and maintenance costs, including an internal ops team to support that infrastructure. You have to do the patching and upgrading, including responding to security issues. With the public cloud, the providers handle this for you.

With private cloud, you still have to do capacity planning to make sure there is an underlying physical server for the new VMs someone needs. With a public cloud, you don’t have that constraint. You will rarely if ever be unable to provision a VM when you need it.

Additionally, the public cloud providers do a lot more than supplying elastic compute. They offer a lot of value-added services. You can spin up a new database, a queue, an API gateway. These are things you need in a microservice architecture and it’s a lot quicker and easier to use a managed service from your cloud provider than it is to set this up yourself. And because this is quick and easy, you can try out alternatives to see whether they offer a better solution for your particular use case.

In the content platform team at the FT, we introduced multiple new data stores that met specific needs. The difference between installing and managing a database cluster in two regions ourselves vs using database-as-a-service options from AWS was weeks vs days of effort. It’s significant.

The combination of infrastructure as code and elastic provisioning allowed us to move to treating our servers as cattle rather than pets.⁷ This is a concept first popularized by Randy Bias, who has written up the history. I first heard it, like many others, from Adrian Cockcroft, then at Netflix.

When we hand built our servers, they were like pets. We gave them names, and lavished attention on them. They were around for a long time and could have an uptime measured in years. We formed an emotional attachment to them, and if they got sick, we’d nurse them back to health.

In the cloud, virtual machines don’t stick around for a long time. They don’t have names, they are numbered and tagged with what purpose they serve. And if something goes wrong, we won’t nurse them back to health. It’s common to terminate a server that is having problems and spin up a new one.

Public cloud mostly enables microservices through the things that you can do on it, and in particular the new deployment options that are available.

New Deployment Options

Many people will immediately think of containers and Kubernetes in the context of microservices. However, this isn’t the only way to run a microservices architecture. The FT has run microservices on Kubernetes (used for the content publishing platform) but it has also run them on Heroku (used for ft.com)⁸ and makes significant use of serverless too.

I want to take a step away and talk about what things like containers and kubernetes represent: new deployment options. Containers and orchestrators, serverless, and PaaS options all allow teams to hand off some part of the complexity of running distributed systems of small services; and reduce or make more predictable the costs of doing that.

Containers

Early on in our microservices adoption at the FT, we were running each service on its own virtual machine (VM). Virtualization lets you split up an underlying physical machine into multiple smaller virtual machines which, to the applications running on them, seem just like a normal server. This provides isolated execution environments and higher utilization of the underlying physical hardware.

However, given our tiny microservices, even with the smallest VM these services were overprovisioned, so we were spending more money than we needed to. But more significantly, each new service we set up needed multiple steps to provision, configure, and deploy. It was fiddly and a source of friction.

That made us ripe for early adoption of containers.

A container image is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries and settings.

Containerization involves a standard packaging format, a standard interface for controlling a running container, and an engine for running containers. Container images become containers at runtime, where the container engine unpacks and runs the application in a way that isolates the software from any other containers running on the same infrastructure. Because this is much more lightweight than a VM, containers are quick to start up. They are also immutable. If you want to change your application, you have to update the container image and deploy a new container.

Although we could likely have changed our previous platform to support running more than one service on a VM, that would not have been a great idea, because we would have lost that isolation between different applications. Containers are smaller, isolation and built to be stacked which made this very simple. Adopting containers reduced our AWS costs by 40%, because we now ran on 8 very large VMs rather than several hundred very small ones. It also meant fewer steps to set up a new service. We no longer had to provision a VM or set up deployment pipelines for a new service. Everything was defined in a single version-controlled configuration file.

However, we still had some challenges, because we were such early adopters that the container ecosystem wasn’t there. We had to build our own system for managing those containers. This included working out how many instances we needed, any constraints on where they should be running, whether they should be deployed sequentially, how to route requests between containers, etc.

We effectively built our own container orchestration, because that’s what made containers work for us. In general, I’m a big fan of choosing boring technology. Dan McKinley of Etsy wrote in favor of this, because boring technologies are the ones lots of people have used successfully. New and innovative technology is exciting, but the capabilities and in particular the ways they can go wrong are not likely to be well understood. Dan suggests that you limit how much innovative stuff you try to do at any one time by thinking in terms of having a few tokens you can spend: make sure you spend your innovation tokens wisely.

For my team at the FT in 2014, building our own container orchestration was an innovation token we were willing to spend. But as soon as we had alternatives we could move to, we did.

Orchestration

Container orchestrators like Kubernetes will dynamically manage a container cluster for you, handling routing of requests between services, restarting failing applications, moving services around where there are problems with CPU or memory use, and handling deployments.

Kubernetes is not itself a platform: there are a lot of things to consider beyond container management, such as service meshes, API gateways, log aggregation, etc. etc. As the container ecosystem matures, more and more tools exist to make it easier to work with containers and kubernetes. In fact, it can be a bit overwhelming. The Cloud Native Computing Foundation maintains an interactive Cloud Native Landscape that is a very helpful guide. See Figure 1-3 and note that the specific details are not important, rather gaining a sense of the scale of available tools in this area. But even with all these tools, if you opt for using Kubernetes, you are opting for building your own platform with the flexibility and complexity that comes with that.

Figure 1-3. The CNCF’s Cloud Native Landscape as of October 2022.

My feeling is that this is another place where we should lean on providers and get them to do the heavy lifting. Most public clouds provide managed Kubernetes services or you can get third parties to do the management for you. Those are the options I’d be looking at if I was adopting Kubernetes now.

However, I would also be considering whether I needed to go for a Kubernetes-based solution. Kubernetes is powerful, but complex. Cloud providers also offer their own services for managing containers, and those may provide better integration with other parts of that cloud provider’s ecosystem.⁹

People often choose Kubernetes for portability. I’m not convinced this is a good reason. Even if it is easier to move from Amazon’s Elastic Kubernetes Service to Google’s Kubernetes Engine than it is to move from Amazon’s Elastic Compute Service to Google Cloud Run, how likely are you to make that move? I’d rather make the commitment to using a particular cloud provider and use as many of the managed services they offer as deeply as possible than to spend time and effort in trying to stay relatively vendor-neutral.

Platform-as-a-Service options

An alternative is to use Platform-as-a-service (PaaS) options. Here, you deploy your applications to a platform completely run for you that can often also provide things like managed databases.

You lose some flexibility and the cost to run the application will be higher—​but you no longer need to build and maintain a platform yourself so the overall cost may be lower. Your teams can also focus on delivering business value.

When the FT started building microservices, many teams chose to use Heroku and benefitted from the ease of use and well-thought out tools. I don’t think we would necessarily make the same choice now - we haven’t seen many new features added over the last few years. While there are still many vendors offering PaaS solutions—​such as Render, fly.io, Netlify, platform.sh—​I have heard far more about people running relatively simple applications on these, rather than complicated microservice architectures.

These options are definitely worth considering, particularly for organizations that don’t have to support high workload or scaling challenges. However, for a complicated architecture, you may be better off looking at the options from public cloud providers, because you have a lot of options for what you install and run alongside applications: databases, messages queues, storage.

If you’re doing this, you should also consider whether an event-based serverless compute option is a better match for your requirements.

Serverless

Serverless is another option for building a loosely coupled architecture. Serverless allows you to build applications without thinking about the servers they run on at all.

Many of the managed services that cloud providers offer are serverless. For example, Amazon offers things like S3 for file storage, SNS for message, Aurora for data storage via PostgreSQL. In all of these cases, you don’t have to worry about scaling, backups, clustering etc., you configure the service and start using it.

Cloud providers also provide serverless compute (which is what I find people generally think about when you mention serverless). AWS Lambda is an example of Function-as-a-Service (FaaS), an event-driven model where your code is invoked when an event happens—​a file being written or a message being sent.

You can definitely consider FaaS to be a type of microservice architecture¹⁰, although you may find that you need to deploy one logical microservices as multiple functions. Sam Newman has a great in-depth discussion of this in his Building Microservices book.

What serverless options have in common is that you are charged on usage, i.e the number of requests made or the amount of storage used.

Making your choice

In general, I think you should aim to have your cloud provider do as much of what Werner Vogels of Amazon terms “undifferentiated heavy lifting” as possible. Vogels describes undifferentiated heavy lifting as “tasks that must get done but don’t provide competitive advantage. For most businesses, these tasks include things like server management, load balancing, and applying security patches.” (If you’d like to find out more, check out this interesting overview of how AWS approach building software.)

In general, you shouldn’t be running your own messaging queues or database clusters. You don’t get a lot of value from that extra work. You should use managed services wherever you can.

Whether you deploy your application code as short-lived functions or as longer-lived services or a mix of the two is less clear cut. It depends on what you’re doing and the profile of your workload. Responding to events can be a great use case for FaaS. If you have a pretty steady stream of requests to your website, maybe that will be better running as a containerized service.

At the FT, we had a mix of serverless functions and longer-lived services in many of our teams. I think that’s reasonably common.

DevOps

When the people developing the code are separate from the people who operate it, there is a mismatch of incentives. A developer wants to release their code and move on to the next feature. An ops person wants to keep systems up and running, and knows that code releases are highly correlated with things going wrong.

DevOps is about developers and operators working together, and it is a cultural change. Once you are working together, you start get more aligned. Operations engineers start to solve more problems through software engineering—​writing code and automating things. Developers start to take on more responsibility for the operation of their software, whether that’s about building observability in or responding when there are production problems. This is a good thing, because you build better systems when you are the people who might have to wake up at 3am because something has gone wrong.

Doing many small releases, as you can with microservices, makes it much less likely that each release will cause problems, and much easier to roll it back if you find it does. However, you must have those releases being done by the developer because you can’t handover to another team ten times a day. That means the developers need to be able to support that code running in production. So DevOps is essential for doing microservices successfully.

I strongly feel that DevOps is a mindset, rather than a job you do. However, in the industry it is common to see people recruiting for DevOps engineers or building a DevOps team. DevOps teams and engineers are the people who are in what Matthew Skelton and Manuel Pais in Team Topologies¹¹ would call Platform teams, who are building tools and processes to support engineers who are building a product.

Chapter 7 talks far more about how to move to “You build it, you run it”. The rest of the chapters in Part 2 also cover why this matters, with a full discussion of the different team types in Team Topologies in Chapter 4.

Observability

A microservice architecture gives you lots of places where things may have gone wrong. With a complicated web of services, built with the expectation that the system as a whole should still work even if some instances of services are unavailable, monitoring can’t necessarily tell you whether something is really broken. You can have an alert for an instance that is being upgraded, with all traffic successfully being routed around it.

Generally, it is much harder to predict what information you are going to need. Old-school monitoring and dashboards will only get you so far (as Liz Fong-Jones and Charity Majors have said, “dashboards are the scar tissue from previous incidents”, which means they may not show you what you need for this one!).

Luckily, we’ve seen the rise of new tools, focused on observability. These go beyond log aggregation and tracing (although these are both important) to a place where you can capture high-cardinality, high-dimensionality information about events. High cardinality means you have a lot of possible values for a single attribute, for example something like userID. High dimensionality means you have a lot of different key-value pairs. This means you can ask detailed questions about what has happened in your production environment, giving you a good chance of finding the solution for some fairly esoteric bugs where the problem exists only for some small subset of users or for some unusual combination of circumstances. Of course, you can only do this if the engineers were thinking about observability when they wrote the code!

Advantages of Microservices

The advantages of microservices come from two aspects. Firstly, microservices break the system up into many small parts. This means you can scale these parts separately, and you get increased resilience because a failure of one part doesn’t mean the whole system is down.

Secondly, a microservice architecture is loosely coupled: you can change one service without having to change anything else. That means you don’t have to coordinate releases between teams. It also means you can choose different technology, depending on the needs of each service. And if you want to try something new, you can do that easily. You don’t have to migrate the whole system.

Microservices focus on being as loosely coupled as possible, and achieve this by doing things like owning their own data, avoiding coupling through centralized database schemas. They prefer lightweight communication mechanisms, avoiding overly complex or smart integration technologies (for example, an Enterprise Services Bus) that can be a bottleneck for change. They are deliberately built around business capabilities, meaning most change should happen within the service or services owned by a single team. If you get those domain boundaries right, the interface for the microservice will be stable and change relatively infrequently. That means you don’t need to spend a lot of time coordinating with other teams before you can make a change. It also means that as an engineer, you don’t need to understand the whole system, just your own services and the interfaces they offer or use.

Let’s spell out these advantages in more detail.

Independently Scalable

With a monolith if you need to handle increased traffic, you need to scale the whole monolith.

With microservices, you can scale just the part of the system that is under increased load. For example, for the Financial Times, you may have a big increase in traffic to the home page when there is a major news event, but there may not be any impact on how many searches people do. With microservices you can scale just the home page.

You can also of course scale things down independently, reducing the scale of a component when it isn’t being heavily used, and this has a positive impact on both costs and sustainability.

You can also treat different parts of the system differently. For example, if one part of your system is CPU-bound and another is memory-bound, you can run them on different types of hardware.

Robust

While a function call in a monolith is much less likely to fail than a call over the network between two microservices, overall a microservices architecture is pretty robust.

If something goes badly wrong in a monolith, you lose the whole thing. If something goes badly wrong in a microservice, you have lost only part of the system. Put another way: the blast radius for something going wrong is small. For example, let’s imagine the home page of a movie purchasing web site that includes a list of personalized recommendations for you. If that service breaks, you can still see the rest of the page. You can still search for a film and buy it. And maybe the system will fallback to showing you a list of popular movies if the personalized list can’t be retrieved.

This is oversimplifying things Firstly, monoliths can be deployed to multiple machines, perhaps in different regions. It would be rare that losing one machine would take down the whole system. And secondly, robustness in a microservices architecture takes work. You need to think about what happens when things go wrong. That is covered in a lot of detail in Chapter 11.

Easy to Release Small Changes Frequently

With a monolith, even the smallest change involves deploying the entire application. That can take a while, and so it’s more likely that changes are batched together. A release feels risky and can often—​although not always—​involve downtime. This is a vicious circle, because the risk means you are less likely to do releases, making each release riskier.

Adopting microservices should allow you to make small changes to a part of the overall system with a high degree of confidence that you aren’t going to break something unexpected. And because these are separate services, you can deploy just the service that you have changed.

If something goes wrong, it’s easier to reason about it, because it’s a small, self-contained change that should be easy to roll that back. This makes releasing code into something normal, not scary.

With microservices, you should get to the point where you are releasing small changes as soon as they are ready, typically tens or even hundreds of times a day. The Financial Times did around 100 releases every day in 2021. This delivers real business value: you can quickly implement and then get real feedback on your ideas.

Accelerate by Nicole Forsgren et al. digs into what high performing technology organizations have in common and defines high performance as being about a positive impact on the productivity, profitability, and market share of your business, compared to competitors.

Their research found that high-performing organizations have a higher deployment frequency, a shorter lead time for changes, a lower change failure rate, and a shorter time to restore service when something goes wrong. Microservices help with all those metrics.

I’ll be returning to these metrics throughout the book, starting in Chapter 2.

Support Flexible Technology Choices

With a monolith, you are constrained to use a single programming language, and probably a single datastore. There may well be places in the codebase that would benefit from something different, but they have to make do. This means, for example, that data that is naturally graph-like can get squashed into a relational database.

With microservices, you can choose the right tool for your needs. Frontend services can be written in Node.js and backend services in Go. You can store an article in a document store, because you generally retrieve the whole thing. You can store metadata in a graph, because you want to be able to navigate the relationships.

What this also means is that microservices support change. You can try out a new technology in one service, and if it provides value, you can migrate other services to it. Or you can leave those other services as they are; your choice. The same applies when keeping the technology up to date: it is much less scary to upgrade the version of your programming language microservice by microservice than it is to do it for a monolith. This should help you stay closer to the latest versions.

Challenges of Microservices

Many of the challenges of microservices are because these are distributed systems. However, the number of different services and the rate of change can turn this up to 11.¹²

Pretty much the whole of Part 3 of this book deals with how things change when you are building and operating a microservices architecture. Here I will focus on things where microservices really don’t do well, rather than on things—​for example, testing—​where you need to approach things differently.

Latency

Since a call over a network takes much longer than an in-process call, if a flow goes through multiple microservices, you could have a large percentage of the processing time being those network calls.

For many systems, this isn’t a major issue, but if latency matters for your system, you should be cautious about how many network calls are involved in a particular operation. In particular, you want to avoid network calls over a long distance: (as a rough guide, a TCP packet round trip between the US and Europe will take around 150 ms—​in the same time, you could likely have done 300 roundtrips within a single data center!) This is the kind of thing that can bite you when you go into a state of partial failure. If that means having some services in Europe calling other services in the US and vice versa, you may still be up, but you could be unacceptably slow.

Estate Complexity

Flexibility to choose the right tech can lead to running a lot of different tech. This is a place where local optimization (the team wants to use a graph database because their data fits that style) clashes with global optimization (the organization already has 5 different databases being used in different teams and they all need to be patched regularly and to have mechanisms for backup and restore).

This breadth of technology being used also makes it harder to provide tools, and centralized support for development teams. Each new programming language means that libraries and documentation for shared tools have to incorporate this language, meaning this is an area where organizations commonly impose some constraints. I talk more about finding the right balance between autonomy and simplicity later in the book, particularly in Chapter 10.

Operational Complexity

We replaced our monolith with micro services so that every outage could be more like a murder mystery.¹³

@honest_update on Twitter

There are three reasons why microservices are operationally complex:

they are distributed systems

they change rapidly

they are loosely coupled

Because microservices are distributed systems, things are a bit more flaky. A call may fail because of network or DNS issues. We work around that by building resilience in (see Chapter 11), but that means the exact route a request takes can’t be predicted, and if you don’t handle failure well, you can end up with timeouts and traffic peaks. Being a distributed system also means you can’t jump on a box and tail logs. You need to have log aggregation, so you have a place to go and look at all the logs—​but if something goes really wrong, those logs may not make it to the aggregation tool. And you need to have something that allows you to trace a single request through your system, whichever services it goes through. This can be through using a specific distributed tracing tool, or as we did originally at the FT, by tagging all the logs that relate to a single request with some unique transaction id.

Additionally, things change rapidly, which means that your understanding of what the system looks like is quite likely not up to date. This is particularly true whe you have multiple teams working on a system and sharing support. If someone from the article page team is doing support and there is a problem with personalization functionality, they will have to work out how this works now by using their observability tools and looking at the codebase.

When I was working on monolithic architectures, the architecture diagram would be pretty useful. I could generally expect it to be up to date, with the right server names, ports, etc. The complexity was inside the application. For a microservices architecture it’s hard to keep a diagram up to date manually. You also really need to have different levels of architecture diagram. Perhaps the most useful is the one that I found we had the least at the FT—​one that showed the different systems owned by different teams, and the flows across those boundaries.

Finally, microservices are loosely coupled. Which is good! However, when you have a flow that goes across multiple teams, there may not be anyone who understands that whole flow. I saw this at the FT whenever we had problems between an article being published and it appearing on the home page of the website. This went through three separate teams: editorial tooling, content publishing, and the ft.com team. The first challenge was to work out where things had gone wrong.

We handled this with tools that let us look at the interfaces between the systems (and therefore the teams) so that we could see how far an update to the article had successfully got to. I will cover this in Chapter 12.

Data Consistency

microservices (n,pl): an efficient device for transforming business problems into distributed transaction problems¹⁴

@drsnooks on Twitter

Within a monolith, with a single database, you could use transactions to make sure that a single logical update either completely succeeded or completely failed.

Once you have data stored in more than one place, you can’t rely on transactions any more.

My recommendation would be to use this to design the boundaries within your system: if you have a single logical update, wherever possible, make that within a single microservice where you can commit or rollback the whole thing. That won’t always be possible. You can use the Saga pattern to apply compensating changes when part of a change fails, but that adds a level of complexity into how you build, test and run your system. It’s simpler if you have a system where you can accept eventual consistency: data might be inconsistent but should eventually converge to become consistent.

Working on the FT’s publishing platform, I benefited from two things. The first was that we were not the source of truth for articles. That source of truth was the content management system (CMS) operated by the editorial tooling team. If we lost our copy of an article, we could go back to the CMS for it. We also benefited from the idempotent nature of publishing an article. You can repeat publication of that article as many times as you want and there are no side effects. Often, we simplified our interactions with databases by:

treating clusters in different regions as completely separate

publishing to both

using monitoring to pick up where there were inconsistencies

manually fixing those inconsistencies by hitting a “publish again” button for that article.

I’ll discuss this more in Chapter 11 as this is an important aspect of building a resilient system.

Finding the Right Level of Granularity

Microservices are small. But how small?

Many of the early definitions of microservices that I came across nearly a decade ago focused on size, and it was confusing because they were quite different. Was a microservice something that was “no bigger than a few hundred lines of code”¹⁵ or something that was owned by a single "two-pizza" team? What does it mean that a microservice should “do one thing, well”?

I think it’s a bad idea to focus too heavily on the idea that these are micro services. Let me flip the question of “how small” and ask instead—​how many microservices do you need?

Different companies take a different approach here. At one extreme is a company like Monzo, which has many hundreds of microservices.¹⁶

I’ve asked the audience at a few different conferences about microservices, and generally, people have one or two orders of magnitude fewer microservices than that—​somewhere between 15 and 150. At the Financial Times (FT), we were on the higher side: we had many hundreds of microservices, and teams could have 10 to 50 microservices that they made regular changes to or needed to support.

So how do you decide where in that spectrum you should sit?

Firstly, you want your microservices small enough that they are owned by a single team. If you have multiple teams working within one business domain, you should aim to find a way to divide that domain up, so that each team has ownership of part of it. Otherwise, you need those teams to coordinate their work, which will slow them down. For the FT’s content publishing platform, we had multiple development teams and so we split that domain to match. One team was responsible for the content domain—​articles, videos, images. The other handled the metadata domain—​information about the subjects, people and organizations discussed in that content, used for example to automatically generate topic pages on the website.

You also want your microservices small enough that the people in your team can understand the whole service. In particular, a new person joining your team should be able to look at the code and quite quickly understand what the service does.

That was definitely not something I could do when I was writing code for a 70-package monolith, it took months or even years to understand that.

Microservices are easier to understand not just because there is less code though: there is also a clearly defined interface. There is generally one way into and out of the service, and if you understand that interface, you understand a lot about the service.

A nice benefit of this is the often cited ability to replace that microservice completely with little risk and not too much time taken. And I’ve seen that happen (although not that often), perhaps as a result of a service getting passed over to another team that works in a different programming language, or because of swapping one data storage solution with another.

However, you don’t want to go too small. Partly that’s because the more microservices you have, the more times you’ll need to make the same change when you need, for example, to upgrade a library. Automation can help, and you will need to automate things you do regularly—​but even if you automate a lot of that library upgrade, for example by creating pull requests for each affected service, you still need to approve/test each of those.

But also, if you go too small, you’ll find it very difficult for that microservice to own its own data and not need access to data owned by another service.

Kyle Brown and Sharir Daya of IBM cover this well in their blog post from 2020 What’s the Right Size for a Microservice?, where they discuss how going too small is a common mistake they see for teams adopting microservices. They argue that this is about a misconception that each microservice should provide one REST interface—​for example, that an Account service would only handle operations on a single account, such as Open, Close, Credit, Debit.

When you need to transfer money between accounts, you have two choices. One approach is to set up a separate Transfer Service, which will first debit from one account then credit to another. But that is a transaction where you want both the debit and credit to succeed. Distributed transactions are hard. It would be simpler to add a transfer operation to the Account service.

When you’re designing your microservices and finding the boundaries, one thing you should be guided by is transaction boundaries. When you find your handling of data starting to get complicated, consider whether combining your microservices together might be the right thing.

Similarly, if you find you always change two microservices at the same time, maybe they shouldn’t be two microservices.

This is not something you need to get right first time—​you can split or combine microservices when you realize you didn’t get the boundaries or the size right. I will discuss how to determine the right boundaries in a lot more detail in Chapter 4.

Handling Change

Upgrades and migrations are a fact of life. You will always have something that needs to be fixed up:

Maybe you built something from scratch and now there’s something you can buy in. An example: we built our own service orchestration, then moved to kubernetes when it was production-ready.

Or maybe you need to upgrade your programming language because the version you’re using is about to be deprecated

Or perhaps there is a library that needs to be upgraded because it has a security vulnerability

The key point here is that these are changes that you have to make globally, i.e. in every place where the service, programming language or library is used, and generally with some deadline—​versions going end of life, or the expiry of a license period. This is the flip side of one of the advantages of microservices. You can upgrade microservices one by one at lower risk than upgrading a monolith, but there’s always a risk that you’ll discover 50 services are still running on a version of java that has been out of support for months or even years!

There are two things that make this more painful in a microservices architecture. Firstly, if you have for example multiple data stores, you will have multiple upgrade paths. Five times as many databases can mean five times as many upgrades. Hopefully, each upgrade is simpler and takes less time but that isn’t always the case

Use Managed Services where possible

Having someone else handling the vast majority of upgrades for you is another good reason to use managed services, such as managed databases. Even here though, there will be changes that are significant enough that you need to do some work, for example where there are upgrades that are not fully backwards compatible, meaning you need to make changes to your application.

Secondly, the more microservices you have, the more places you need to make a change. You need to build things so that migrating 150 services to something new doesn’t involve weeks of work. And that means investing in automation. For example, you should aim to template your deployment pipeline so you can amend all pipelines easily if you are adding a new step into deployment (e.g. to enhance security scanning). You also need to work out how to quickly patch all the services that use a particular version of a library, particularly where there is a security issue. Maybe you can do this via automating the creation of a pull request (PR) but you still need to release all that code.

Automation can speed things up, and it can reduce errors that manual repetition can introduce, but it still requires people to invest some time in something that isn’t new feature development and even where you automate the changes made to the code, you will still want to review those changes and test them. I talk about how to approach this in Chapter 13, and it really can be a killer for teams, where they feel they are on a treadmill of uninteresting but important updates.

Require Organizational Change

You need your teams to be both autonomous and empowered. That won’t work if you still have, for example, a set of gates for people to get through to get code to production.

Really trusting your teams to do the right thing, and ensuring they understand what that right thing is, can be quite a change for many organizations.

The whole of Part 2 of this book will dig into the organizational structure and culture that make it more likely you will be successful with microservices. At the FT, that organizational change took years to complete.

Change the Developer Experience

A big surprise to me when moving to microservices was how much the things I worked on as a developer and the way I did that work changed.

The balance of where I spent my time shifted. Working on a monolith, I mostly wrote code that was deployed within the existing application. With a microservice architecture, I set up new services fairly regularly, which meant I now needed to understand a lot more about the infrastructure. Choosing our own data stores meant setting them up and understanding how they worked too.

My development cycle changed too. With the monolith, I’d write code and tests and run the unit tests in my IDE. But I’d also often start up the application and do some manual testing. I could attach a debugger to the running application and step through the code.

With microservices, we initially tried to run large parts of the system locally. Before containerization this could be fiddly to do, for example, we often found two people had chosen the same port to run a service on locally. But we could do it, because we were mostly installing our own databases and queues on EC2 instances on AWS, rather than using AWS-specific alternatives.

My view is that you should use the value-added services your cloud provider offers instead, because it hands some of the effort over to someone other than your team. It also has the happy side effect of moving you away attempting to run a complete replica of your system locally.

Running an entire system locally encourages you to use end-to-end acceptance tests, which couples your microservices together, turning them into a distributed monolith (see Chapter 9 for more on this anti-pattern), rather than focussing on unit testing and contract testing. I think you should be aiming to run small parts of your system locally, at most. And if you need to have a more complete environment, use something remote. But even here, I’m going to say minimize how much of your complete system you are running.

Note

To be clear, I see value in end-to-end tests, but I’d prefer to have small numbers of these, and ones that don’t know anything about the internal implementations of the individual microservices.

The first suite of acceptance tests we wrote for the content platform had fixtures that would set up data for each microservices involved in the publication of an article. They were hard to update, taking longer than code, and constrained the order we released changes into our staging and production environment, because changes in different microservices could mean changing the same set of acceptance tests.

We didn’t fully replicate our production environment anywhere. One reason was cost—​my team had a staging environment but we didn’t run it multi-region. Often, you won’t run as many instances. That means there is a whole class of issues around configuration and failover that you won’t catch in the staging environment. The second reason is that inevitably, other services and resources you interact with will be on a different version in staging than they are in production. At the FT, many teams didn’t have a staging environment, meaning you could at best maybe use their production environment (as long as your traffic didn’t write anything and wouldn’t overload it).

All of this meant that there were problems that could only happen in production and be diagnosed there. Proper observability tooling and the ability to quickly turn off functionality became much more important than replicating stuff locally.

In Chapter 9 I will cover why I think testing on production is a natural consequence of moving to microservice architectures and provides a great deal of value.

In Summary

A microservices architecture breaks a system into lots of independently deployable services modeled around business domains. It’s been around for nearly a decade and can work very well.

The key benefit of a microservices architecture is that it gives you the ability to move fast. You can make small changes to a part of the overall system with a high degree of confidence that you aren’t going to break something unexpected. You can release these small changes on demand, typically hundreds of times a day. This is what delivers real business value: you can quickly implement and then get real feedback on your ideas.

However, with this comes increased complexity: this is a distributed system, and there are many moving parts.

In Chapter 3 I’ll discuss how to work out whether a microservices architecture is the right approach for you. But first, in the next chapter, I’m going to talk about the things other than moving fast that show whether your software delivery organization is effective. We’ll look at how the microservice architecture measures up, and the tools, techniques and processes that will help you be successful if you do choose this approach.

¹ This is the succinct definition Sam Newman uses in his Building Microservices book, 2nd edition (Sebastopol: O’Reilly, 2022)

² I won’t be the only person who was glad to move away from SOAP for sending messages between services!

³ Too many and too long are necessarily vague. They will be different for different organizations.

⁴ I found the excellent NIST definition of cloud computing via Martin Fowler’s website. I recommend the full standards doc from NIST as a really clear explanation of a term that is often a bit…​ cloudy.

⁵ Kief Morris, Infrastructure as Code 2nd edition (Sebastopol: O’Reilly, 2020)

⁶ the FT focuses on business news, meaning weekends are the time when there might be a few hours without much new content

⁷ This is a metaphor that in my experience also works very well when explaining developments in the way we build software to non-technical folks!

⁸ I am not sure I would pick Heroku in 2022, but in 2016 this was a successful choice.

⁹ If you asked your cloud provider for their recommendation, it might well be serverless rather than containers as a starting point. Adrian Cockcroft, previously of Amazon TODO give a summary of the talk

¹⁰ As in this blog post from Surya Sreedevi Vedula and Ashish Bhalgat of Thoughtworks.

¹¹ Matthew Skelton and Manuel Pais, Team Topologies 2nd edition (Portland, OR: IT Revolution, 2019)

¹² As Nigel Tufnel says in Spinal Tap, this is definitely at least one louder.

¹³ Honest Update, @honest_update, October 7th, 2015, https://twitter.com/honest_update/status/651897353889259520

¹⁴ Al Davidson, @drsnooks, October 6th, 2015, https://twitter.com/drsnooks/status/651373314368253952

¹⁵ It’s hard to track down who first said this, but I am pretty sure I first heard this from Fred George in a talk around 2012. As a Java developer at this point, that seemed challengingly small!

¹⁶ In 2020, Matt Heath and Suhail Patel quoted 1600+ and growing, Modern Banking in 1600 Microservices, InfoQ

Chapter 2. Effective Software Delivery

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 2nd chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at jleonard@oreilly.com.

The architecture you choose generally has to deliver over several different timescales.¹

You want to see at least some benefits in the short term, because a large up-front investment with no immediate reward makes people nervous.

In the medium term, you want to reach the sweet spot where your architecture enables you to be effective. For me, effective software delivery means you can:

regularly deliver real business value

maintain appropriate service levels

adapt to change so that you are always working on the most important things

provide people with an environment where they get to spend most of their time on meaningful work

keep risk to an acceptable level

But beyond that, what about the long term? In much of my career, the long term has generally meant a big-bang rewrite and the rebranding of the system as “legacy”. This is, however, costly.

So, can you avoid having to start over?

This chapter will discuss all these aspects of becoming a high-performing and effective software delivery organization, in the specific context of microservices. I will introduce the tools, techniques and processes that will help you be successful in each aspect, and the ways you can assess—​or hopefully measure—​how well you are doing.

These are all things I’ve spent the last 5 or 6 years wrestling with, first as a Principal Engineer for the content publishing platform at the Financial Times and then as a Technical Director with a focus on engineering and operations across the organization.

You can think of this chapter as a guide. I’ll introduce the concepts I’ll be talking about in the remainder of the book—​the things that will help you succeed, that will stop your systems getting into a mess—​before we break them down in detail later. If you’re struggling with a particular aspect, feel free to skip forward to the relevant chapter for a more detailed perspective that will help you tackle that problem.

Let’s start with the benefit that comes from one of the primary goals of microservices, which is decoupling of services, and see how that helps you to regularly deliver business value, and to move at speed.

Regularly Delivering Business Value

When I first worked on the content publishing process at the Financial Times, releasing our code to production took hours to do, and the process was manual. Often, it went wrong, and recovering from that could take many hours too. I remember one release where the rollback went on overnight and I was called the following morning to check we were fully restored to the previous state.

This meant we released outside of normal working hours, and infrequently—​no more than once a month. If the release succeeded but we then had reports of issues with the running code, I had to think back to the work I was doing four to six weeks ago. We also had to consider the many changes all released at the same time: which of those changes broke the functionality?

Similarly, if we saw a change in the way users engaged with our site, it was hard to attribute it to a particular cause. This meant there was little scope for experimenting: we couldn’t easily know which change made the impact. Also, there was a reluctance to undo a change, given the amount of time it would have taken to get that change implemented and live. It’s human nature not to want to admit you spent lots of time on something that turned out not to be worthwhile!

Five years later, still working on the content publishing process, we released code on demand, many times a day—​around 200 times as often (see Figure 2-1), and note this was just for the content platform. At the end of 2021, the FT as a whole was deploying code around 100 times per working day.

Figure 2-1. Before and After Comparison of Numbers of Deployments

Additionally, when there were reports of issues, we could automatically roll back the changes in minutes. I had the full context of the change in mind because I had only just finished it. And we did roll back changes because an experiment didn’t have the impact we expected.

What I mean here by experiment is something specific. Linda Rising pointed out in her presentation Experiments: The Good, the Bad, and the Beautiful that an experiment is something that has a hypothesis, that can fail, and that has a control group. For most organizations, those things are not in place and so experiment just means try.

Although Linda was talking mostly about experimenting with process, exactly the same thing applies for experimenting with product changes.

But people do not like to invest time and effort in something and then throw it away. This is the sunk cost fallacy. To experiment, you need to be willing to fail, so you need the experiments to be small and cheap.

As discussed in Chapter 1, Accelerate identified four key metrics associated with high performing technology organizations. These metrics are often referred to as the DORA metrics because they were first identified by The DevOps Research and Assessment (DORA) group through its State of DevOps reports. The metrics are:

Deployment frequency: how often an organization successfully releases to production

Lead time for changes: how long it takes from committing code to having it go live in production

Change failure rate: the percentage of deployments into product that break things

Time to restore service: how long it takes to recover from a failure in production

High performing organizations have a higher deployment frequency, a shorter lead time for changes, a lower change failure rate, and a shorter time to restore service when something goes wrong.

The first two metrics are about how quickly you can deliver value. The last two are about how stable you are while moving fast.

So why do those first two DORA metrics matter?

High Deployment Frequency

Deployment frequency is really measuring how small your changes are, because if you are writing the same amount of code, releasing 100 times a week versus once a week means that each release is a lot smaller.

Small changes are easier to understand. This is a benefit to code reviewers, who will find it a lot easier to understand what the code is trying to achieve.

It also helps you to isolate the impact of each change. If you release a lot of code at the same time, it can be hard to work out what is having an impact: did more people click on this button because it got moved to the top of the screen, or did a seemingly unrelated change on the same page make the button stand out more?

Finally, if something goes wrong, rolling back the release will also undo every other change packaged in the same release. You may even struggle to work out which change caused the problems.

When you are deploying frequently, you can deploy each change independently, in its own release.

That means you know exactly what changed. You can measure the impact on the business and operational metrics you care about. If something goes wrong, for example you see users unexpectedly dropping out in the middle of a task, or an increase in error rates, you can easily roll back the release and undo the change.

Short Lead Time for Changes

Lead time for changes for the Accelerate authors is the time between committing code to seeing it running successfully in production. Faster is better—​but why? Because it means we get faster feedback on what we have built.

Being able to make a change with a very short lead time means that you can have the conversation about the change and on the same day, make it live.

You aren’t having to remember what happened days, weeks or months ago, which helps if something goes wrong and you need to fix it. But, the more important aspect is that a feature is also quicker and simpler to remove, if necessary.

If you want your product development to be driven by metrics, by data, then you should expect some changes to have an unexpected impact. Maybe the change actually means you do worse on the metric you were targeting!

If that happens, you need to be willing to undo the change.

Small changes and short lead times make it much more likely that you’ll choose to undo a change. Firstly, you haven’t had to invest a lot of time and effort: there aren’t that many sunk costs.

There also isn’t a whole heap of other stuff built on top of your code change in the time between you making it and seeing it in production, so reverting the change isn’t expensive. That short period between writing the code and deploying it also means you’re more likely to check what the running code looks like, because you haven’t switched context in the meantime to work on something else.

To make small independent changes and release them to production quickly, you need continuous delivery (as discussed in Chapter 1).

Continuous delivery is a lot easier to implement when your architecture is loosely coupled (this is one of the findings from the DORA research, see Chapter 4 of Accelerate which lists drivers of continuous delivery). Specifically, you need an architecture where you can change part of the system and test just those changes, without depending on running a full suite of tests in complex shared integration environments. This is what allows for a fast deploy.

You also need to be able to deploy the system with negligible downtime so you can do this during normal business hours.

This is somewhere that microservices absolutely shines. When microservices work,² you can make changes to a part of the overall system with a high degree of confidence that you aren’t going to break something unexpected. And you can test those changes with low level tests, such as unit tests.

Note

Testing in a microservices architecture looks a lot different—​see Chapter 9 for a discussion of why we need more automated testing and also need to get more comfortable with doing more of our testing in production, among other things.

You also need a team that is autonomous and empowered, meaning they can complete their work and test and deploy their code without communication or coordination with people outside their team. This normally means the team will be cross-functional, so that they can handle all the necessary steps to get code released within the team. See Chapter 5 for more on this.

And finally, you need to know when a change has had an unexpected impact, which means you need insight into your code as it runs in production. This is observability, and is discussed in Chapter 12.

Running Experiments

Once you have continuous delivery, and are releasing small changes frequently, what else do you need in place to be able to experiment?

Let me illustrate this with an example from the FT where the target was to increase the engagement for film reviews on ft.com.

To run an experiment, you need to:

Come up with a hypothesis: For example, “If we add review scores to the page listing film reviews, we will increase engagement with individual film reviews”

Work out what the measure is: Such as, “The calculated engagement score will increase”. At the FT, engagement is a well-understood metric made up of a combination of time on page, frequency of visits and how recently someone has visited

Segment your audience so that you have at least a test vs. a control group. The FT website had an A/B testing framework built in (A/B testing is discussed in Chapter 9)

Run the test for long enough to be able to have a statistically significant impact on that measure

Stop, do the analysis, and if the hypothesis was proven wrong, turn off the feature.

What actually happened for this experiment is that people were less likely to go and read the reviews. The metric went in the wrong direction.

If this had been on the old FT website, from when I first joined—​the one where there was only a release every month or so, and where changes took a long time—​I don’t think we would have been able to work out whether this was the thing that was having the impact. Also, I don’t think we would have reverted the change, even if we had tied it to the feature. By the time we measured it, we would likely have made a lot of other changes on the codebase, so it might have been difficult to revert. But also, this is where the sunk cost fallacy comes in. People do not like to admit they made the wrong bet and invested time and effort into something only to remove it.

But this was on the new FT website, meaning the change was small and hadn’t taken a lot of time. The code got released as soon as it was finished, the experiment was run, and when it didn’t have the impact we expected, the code change was reverted: the review scores were removed from the page listing film reviews.

If you run experiments that never fail, you aren’t experimenting. People form incorrect hypotheses all the time. What you are looking to achieve is the ability to realise you were wrong more quickly and at a lower cost!

Separating Deploying Code from Releasing Functionality

There is a bit of a wrinkle in all of this, which is that often, you need to run an A/B test for a fairly long time; certainly for longer than the time between deployments.

You could aim to schedule development work in a way that means changes that impact the same functionality are kept apart, so there is time to run each experiment. But I think that once you are in the world of microservices and autonomous teams, this would be a bad idea. It would couple things together and slow you down: the changes may not even be made by the same team.

Instead, separating the deployment of code from the release of functionality can help. This is done through the use of feature flags. A feature flag is an if-else statement that you put around the new functionality you are adding. Different segments of your audience have the flag set to a different value, and so will see different behavior.

Feature flags (discussed in detail in Chapter 9) can be built in house or added through third party commercial tools. They are beneficial because they can be turned on whenever you like. Which means that the code can be deployed when it is ready—​no coordination needed. Feature flags can be used for more than just A/B testing of course: they are useful for any situation where you want to turn functionality on or off separately from deploying code. That includes:

coordinating feature rollouts that involve changes to more than one service

turning on features after appropriate training for users

operational control such as turning off the sending of scheduled emails over holiday periods

The simplest way to avoid clashes between different A/B tests is to only ever run one test at a time. However, in a company that sees the value of this sort of testing, I suspect you will quickly fill your available pipeline of tests, and then you need a way to establish which tests can run in parallel without interfering. If you are using feature flags, you will already have moved this decision away from the process of writing and committing code, which means you have avoided coupling work together too much.

Handling Work That Goes Across Team Boundaries

In Chapter 4, we’ll talk about organizational structure and how you need to find the natural domains within your organization and align teams to those. The aim is to do this so the boundaries are fairly stable, and at places where most work sits within a domain and can be tackled by a single team or aligned teams.

There will however be changes that affect flows across the boundaries you have defined. These generally come from company-level or engineering-wide objectives. You want to be able to work on these changes in a way that regularly delivers value, and lets you continue to move at speed.

An example that affected companies across the world, but particularly in the EU, is the General Data Protection Regulation (GDPR) legislation. Compliance with this was a huge piece of work that touched many different parts of the typical software estate.

Migrations of technology, for example where you are changing a vendor or upgrading a common piece of technology, impact multiple teams and require a lot of coordination.

Where you need to make changes in areas owned by several different autonomous teams, you have two challenges. Firstly, can you get all those teams to schedule the work? And secondly, can you make those changes separately? Maybe one change needs to come before the other, but you should be aiming to avoid a painful process of coordination and lockstep releases.

There are several things to consider here.

Firstly, you need to make sure that all the teams that need to work on this change are committed to do the work and have a shared understanding of the timeline. This is about high level prioritization.

In my experience, you should be very wary about building something when the other team hasn’t committed to their side of the work (or to working with you in a “feature team"--a team spun up just to deliver this feature). It could never be used, and you could be doing something more useful with your time.

At the FT, we used objectives and key results (OKRs) to define our goals each quarter and measure how well we did on those goals (the objectives are the goals, and should be significant, concrete and clearly defined, and the key results are 3-5 measurable success criteria for each objective). That worked well for this kind of shared understanding. Whatever your organization’s process for planning where people are going to be spending their time, make sure that this big piece of work is reflected within it.

Personally, I learned to seek either a department level OKR related to any big piece of work, or an OKR for each group or team that needed to work on it, before scheduling it for my team. Sometimes, that would be a promise that there would be an OKR in the following quarter. You have to assess that based on experience within your own organization—​do things change a lot, or are these kinds of predictions generally good to rely on?

Next, you need to work out the best ways to work together. Team Topologies, by Matthew Skelton and Manuel Pais, focuses on teams as the fundamental means of delivering value within a software organization. They identify different types of team and I will cover this in Chapter 5.

Team Topologies also suggests ways of interaction between teams of various types, each of which minimizes coupling and coordination—​the things that slow you down:

The three interaction modes are:

Collaboration: two teams working together, for example via a feature team that exists until the work is completed and then disbands. You do, however, need to be quite intentional about working out what will happen with the things this feature team builds once it disbands. Which teams will own them in the long term?

X-as-a-service: one team provides a service that the other consumes. An example here could be where there is a new type of content being published. The web team and the API team can collaborate to define the boundary between the domains, in this case an API. The API team can then add the new API once it is ready, and the web team can start calling it sometime later (there is no need to synchronize these changes)

Facilitation: where one team helps and mentors another team. Even a cross-functional team can’t contain every skill they may ever need. That’s where enabling teams come in—​these teams have particular expertise. For example, maybe they know all about Content Delivery Networks (CDNs). If a product team is adding in Web Application Firewall (WAF) rules, the CDN team can facilitate that process.

These ways of working are discussed in much more detail in Chapter 5. The key thing is to make sure you have a shared understanding of how you will work together on this particular piece of work.

Another thing to consider is how you will provide ongoing support for the new feature that crosses multiple teams. Will you be able to tell that it is broken? Will you be able to locate where the change happened that broke it?

When we started this section, we led off with a discussion on boundaries and domains. Contract testing, where a team consuming a service writes a test that documents what it expects from the service, can help here as it allows you to put something in place at the boundaries between domains. For example, if the web team is calling an API, they can specify that they need a universally unique identifier (UUID) field to be returned; they can specify that they expect a minimum of two items in a list of content.

Contract testing as a forcing function

Maybe the web team shouldn’t expect a minimum of two items in a list of content! Putting contract testing in place acts as a useful forcing function for visiting your assumptions and thinking about what your code will do when those assumptions aren’t met.

Contract testing runs when the service being called is being rebuilt and tested. This means that if that service breaks the contract, they should know before they push that code live.

I hope this section has shown that microservices can really help you to delivery business value through frequent code deployments.

But there is more to a successful architecture than being able to deploy code hundreds of times a day (although that is a very good start). Let’s talk about those other aspects that matter.

Adapting to Changing Priorities

The next aspect of effectiveness I want to talk about is how you adapt to change, and in particular, change in where you need to focus your efforts.

When moving to a microservices architecture you need to find the boundaries between domains, to separate out subsystems. Let’s assume for the moment that you have done that (we will return to that particular challenge in Chapter 4) and that those boundaries are allowing your teams to work autonomously. Let’s also assume that your services are owned by specific teams (I will argue in Chapter 8 for the importance of this).

That still leaves you with a challenge. Let’s say you have two teams: A and B. Both have a backlog and are making changes, but in fact, the work planned for A is much more likely to bring in significant revenue. How do you get more people working on A’s backlog?

Things change in organizations all the time. Different people join, with different views of what is important. But also, the industries we work in change. New possibilities arise. Perhaps new legislation is enacted and you need to react to it. Ultimately, you need to be flexible.

In general, the move to microservices has come hand-in-hand with an associated move from project-based to product-based working. Projects are about delivery of a particular piece of work, which is considered finished at the end of the project. But of course, someone still needs to maintain software even if the project team has disbanded.

Product-based working fits with the aligning of a team to a domain. They own the domain, the outcomes, and the software they build, long term. They can decide how to best serve their customers, rather than following a plan defined outside the team.

This allows longer term work on something, but there is still the question: what happens when you achieve a big goal, for example you complete the rebuild of something?

You can maintain a team of the same size but it’s likely that the work they do will be smaller and less impactful. If there isn’t an obvious next phase of that work, would it be better to focus some of those people onto something else that badly needs to be built or rebuilt?

To do this, you need to make it relatively easy to change the organizational structure to have more people working on one area of functionality and less on another, either permanently or in the short term. That is where the idea that microservice architectures give you flexibility to use whatever language, database, or tooling you want starts to come off the rails.

It’s good for both developers and the organization for people to move around occasionally—​because people and teams get to learn new things—​but it’s much more difficult to move if things are very different in the new team. You may have to learn how to work with a new team, and new process, and new tech. This may not be tech that you already know—​or like. Learning new things can be fun, but not when you feel you are starting from scratch.

Changing programming language can be particularly painful in my experience. It’s not particularly about whether someone can pick up a different language, although there are definitely some transitions that are easier for people to make given the fundamental properties of the language. It’s more about whether someone has invested time in building expertise in their language of choice, and whether they feel this is going to move them in a direction they don’t want to go, for their career and for future opportunities. If you are a Java developer, maybe you don’t want to move to a programming language that is far less widely used, because it may limit the job opportunities available for you.

If you want people to be able to move around the organization, you will find it easier if you have some consistency in how things are done. This includes tools, processes, and languages. Many organizations who have successfully moved to a microservices architecture—​companies like Spotify, Skyscanner and Monzo—​have created what Daniel Bryant calls “a Heroku-like CLI for their developers, so that a command like “create new microservice” spins up some scaffolding, plugs into CI, plugs into pipelines, plugs into observability”.³

One thing I won’t dig into here, although it is important, is the challenge of moving people from one domain to another. Where people have built up their domain knowledge—​and in an autonomous empowered team, I expect people to develop a fairly deep understanding of the domain, moving beyond the feature factory--they may resist a move. I will only comment that it’s better to have willing volunteers than to force people. It can be facilitated if people get the chance to learn something new or to fill out something they need to do to get promoted, or even if there is an explicit promotion opportunity: “please apply for this senior level role!”

Somewhat related to this is where there is a completely new opportunity and you need to move people into that area rather than recruiting a completely new team. I see that as a special case of this prioritization. There are additional challenges in this case of setting up tools, processes, etc., particularly if your organization doesn’t have a lot of standardization in place.

Engineering leaders have a responsibility to make it easy for our product leadership to make this kind of big cross-team prioritization call that relates to company-level objectives. It’s much easier to choose which items on a team’s backlog get done than it is to say that this isn’t the right backlog to be working on: if the tech provides a barrier, it is even less likely that this call will be made.

Maintaining Appropriate Service Levels

An effective organisation needs to run software effectively, not just build it effectively.

Things go wrong in any production system. When they do, what kind of impact is there, and how easily can you restore service? Can you tell when something important is broken, fix it quickly and with the minimum of stress, and avoid cascading failures? Do you have service levels in place so you know how urgent it is to fix this problem?

Microservice architectures are unlikely to be something that a separate team can fully support. Application developers will likely be doing a lot more thinking about how to maintain service levels, and will be involved in at least some of the production incidents that happen: they are simply the only people with the relevant knowledge to fix a problem.

One of the advantages of a microservices-based architecture over a monolith is that the blast radius of a failure is smaller. If you release a bad code change to one service, there should not be an impact on a service that doesn’t rely on it either directly or indirectly.

What that means is that people can probably still use your system, but maybe not all parts of it. Maybe they can browse and put things into a shopping basket but they can’t buy those items. Maybe they can read the news on your website but they can’t take out a new subscription.

The flip side is that microservices-based architectures move the operational complexity out of the services and into the connections between them, and often when things go wrong there, it is harder to work out what is going on and to fix it. You may not even know what services call yours!

Because the system is distributed, things go wrong more often—​for example, a call fails because the instance being called is no longer there, or something times out. We can and should build resilience into the architecture, but that means we need to take care not to alert for an intermittent failure that has already been dealt with through a retry, and we need to be careful that we don’t end up with a cascade of failures as all our services send a bunch of retries through to a service just getting started up. Often, when things go badly wrong for a microservices system, it’s because lots of small things all combined to create a storm of retries or alerts.

When a Release Goes Wrong

This is where the other two DORA metrics come in: change failure rate and time to restore service. These are about how stable you are when moving fast.

If you are releasing small changes, you will understand what the impact should be when they go live. You should see a much lower rate of failed changes—​and that’s backed up by the research in Accelerate.

Note

This is a much lower rate of failed changes, but the absolute number will likely go up. However, 5 out of 12 releases failing with the monolith has a very different to impact to say 20 out of 2500 releases failing with microservices, precisely because those 20 failures are easy to spot and affect far less of the system.

But time to restore service isn’t only about reverting a change. Sometimes, things go wrong and there is no obvious change to revert.

That can happen with configuration changes that don’t live in code: for example, AWS or API gateway keys. If a key is rotated but the developer doing it misses a place where it is used, you find functionality breaks.

There are two key things that allow you to maintain a high level of service, will making lots more changes. Firstly, can you quickly work out that something has an unexpected impact. This is about knowing what metrics matter, and being able to tell when they are impacted—​whether these are business or operational metrics. This is discussed further in Chapter 12.

The second thing is to be able to quickly recover when you have a failed change. Here, the same things that allow you to roll out changes quickly also let you restore service quickly, whether that is by rolling back a change or by rolling out a fix. You use the same automated pipelines to fix that you do to release the code in the first place.

Knowing When Something Important Is Broken

Can you identify that something important is broken? With microservices, you can have a lot of “noise” from alerts.

Partly, you get a lot of alerts firing because you have a lot more individual services and a lot more monitoring checks that might result in an alert. 100 services means 100 times as many checks.

But also, you have a distributed system. Calls made over http and lookups using DNS are more likely to have issues than a call between functions in a single process in your application.

There is a classic paper by Richard I Cook—​How complex systems fail. It’s not specifically about software systems—​the paper mentions transportation, healthcare, power generation—​but the points in it apply for microservice architectures as well. Complex systems have multiple levels of defense against failure, which generally work. It normally requires multiple failures at the same time for an incident to occur. But this also means that the normal operational state is to run with failures happening.

Distributed systems are complex systems. As Charity Majors says, “Distributed systems are never “up”; they exist in a constant state of partially degraded service. Accept failure, design for resiliency, protect and shrink the critical path”.⁴

So, we build distributed systems differently (this will be the focus of Chapter 8).

We build microservices to be robust, through redundancy: multiple instances of a service, in different availability zones and maybe also in different regions.

What this means though is that you may have alerts going off for an instance of a service, but the other instances are fine and your system as a whole is working as expected and there is nothing you need to do: the system will fix itself. Those alerts do not indicate an action you need to take.

On the flip side, you can also find in distributed systems that a subtle underlying fault, rather than a total failure, has left you with a “gray failure”. This happens when at least one user of the system sees it as unhealthy, but your own internal monitoring is unaware that there is a problem.⁵

Understanding that there is a real impact on some functionality you care about, where you need to do something to fix it, is important here. Not all checks that fail should result in an alert.

Note

Even working out what your stakeholders would view as “real impact” can be a challenge. People don’t have an instinctive grasp of what “too slow” would mean in terms of target numbers. And also, context matters. I can tell you that being unable to publish a story for ten minutes is much more serious when there is breaking news than on a Saturday at 3am, but that kind of nuance is hard to capture.

One approach is to focus on your service level objectives (SLOs) and alert when those are not being met. An SLO is a target value or range of values, for example, that you want the search to return results “quickly”, defines as “in less than 100 milliseconds”. Choosing and agreeing on SLOs can be complicated, and I will talk about this more in Chapter 12.

At the FT, we focused on business capabilities—​the things our customers wanted to be able to do—​and aimed to monitor whether these were functional. That could be via monitoring of endpoints from outside the FT: a call to https://ft.com/ from various locations around the world is an effective test for problems with different hosted regions, and for problems with things like our CDN or DNS that were part of our ability to deliver the news.

For other functionality, e.g. publishing an article, what you care about is largely internal and is not something where you can make a request and look at the response to see if it has worked. Here, we tried an approach of writing an end-to-end test that ran as a kind of monitoring in production. In this case of publishing an article, the test would publish a known article from the newsroom content management system and then check for an updated timestamp for the same article on the website.

This was complicated to get arranged, but when in place, things like this can help you realize when you have a real problem with business functionality that spans several domains.

A related challenge is whether you can work out where to focus your attention when things do go wrong. I found when first working on a microservices architecture that the monitoring within an application of whether it could connect to the services it depended on, which worked well for a monolithic architecture, meant that I would have so many alerts when something at a low level of the stack was having problems that it was hard to find and single out the alerts for that low level service. This is due to the complicated web of dependencies, as shown in Figure 2-2, where a naive approach to monitoring could lead you into trouble. A failure at the database layer can cause alerts in the applications connecting to the database, and also in applications connecting to those applications. It can be overwhelming!

Figure 2-2. Naive monitoring can mean a lot of alerts

I talk about the types of checks and alerts you actually need as part of discussing the complexity of operating this type of system—​at great length, because I think it is super important to understand if you are adopting a microservices approach—​in Chapter 12.

Restore Some Level of Service Quickly

Can you mitigate or fix an issue within an acceptable amount of time? What “acceptable” means is going to be different for different parts of your software estate, and having a good understanding of that is your first challenge.

I like to think in terms of mitigate first, fix next, because often our first reaction as a developer is curiosity about what is happening—​but you can make things better without understanding why they are broken. Failing over to run solely from a region that isn’t alerting, or scaling up the number of instances are both things that can work on a temporary basis to improve matters while you work out what is going on. Similarly, restarting services can be effective if there is a problem with a memory leak. Building this sort of resilience into your system is covered in Chapter 8.

Sometimes, mitigation can be done by a first line team—​if there is good documentation of the service. I cover how to get decent runbooks in place in Chapter 12.

Typically, I would expect a first line team to do a couple of things: firstly, to check whether there are any recent changes in this area of the system. At the FT we created a change API that logged changes to a data store and also into a slack channel. It is not a bad idea to roll back a code change where the timing is aligned even if you don’t think it can be related. The worst that happens is that it doesn’t fix the problem, and provided you have an automated release process, you can roll it back out again.

Secondly, I’d expect this team to be able to failover from an active to a passive region, or to send all traffic to one active region where the other seems to be having problems.

Tip

Make sure you have tested that you have the capacity in a single region to serve all traffic. Otherwise there you could make things worse when the single serving region collapses under the load!

Scaling up is another option. I would be more hesitant about restarting services because if the problem is load, this could make everything worse: starting up new instances then shutting down the old ones is one way to avoid that.

Once we get beyond these options, most of which can of course be automated, you are in the realm of complicated systems where it is likely the failure is novel and unpredictable, in the sense of “not something that people predicted would happen”.⁶ At that point, you need the development team to get involved.

That means you really do need every service to be actively owned. For me, that has to be by a team, not by an individual. I cover strategies for this in Chapter 8.

And that team needs to be able to drill into what is going on with the system. This is where observability, as opposed to monitoring, is key.

Observability is about being able to look at the outputs of a system running in production to answer questions about what is happening.

Observability ultimately comes down to being able to trace an event through your system. At a minimum, that means structured logging and a unique trace ID attached to the event when it enters the system, and propagated on all calls. There is a lot more to this, which I discuss in Chapter 12.

Avoid Failure Cascades

There is a very different mindset you need when you are building services in a microservices architecture. You need to build defensively.

You are more likely to find that something you depend on is unavailable for some period of time. As an example, zero-downtime deployment of microservices means that your service can be talking to an instance that disappears as it is restarted or replaced by a new instance.

You don’t necessarily know who is going to call your service, and can’t rely on them to throttle calls to you. It’s easy to get this wrong and send a “thundering herd” to a service that is just getting back into a healthy state.

It is down to you to protect your service.

When I was running the content API teams at the FT, a developer in the web team was testing new functionality and accidentally made 30,000 requests in parallel to one of our APIs. This brought down one of our production clusters.

We failed over with minimal impact—​before we knew what was happening: mitigate then fix! But our cluster going down wasn’t the fault of that developer, it was our own problem. We had an API gateway sat in front of our service that could have been configured to throttle requests. We hadn’t set that throttle to an appropriate limit. What should have happened is that the developer’s requests were throttled, either being dropped completely or being queued and processed slowly.

You should also be a good citizen yourself. You need to have backoff and retry in place for calls you make, and you probably want to think about circuit breakers for when a downstream system is unavailable for a while: circuit breakers stop you from hammering other services. They trip when there are a number of errors in a short time span, stopping further calls until the service is recovered. They also mean you can fail fast, i.e. stop doing work if you aren’t going to make a call.

Finally, you should build your service so it starts up regardless of whether it can talk to the services it depends on, and that it shuts down gracefully. In a microservices world, and especially when running in containers, things don’t stick around for a long time.

The use of service meshes and API gateways, discussed in Chapter 8, can help a lot with this defensive approach.

It is a bit of a mindset change. When I was working on a monolith, I rarely thought about what would happen to my code once it was running in production. As we adopted microservices, development teams needed to consider this. It makes you a better engineer to have this broader focus, but I could have done with a bit more guidance on what it meant—​my hope is that this book can provide guidance to you if you are making this switch.

Spending Most of Your Time on Meaningful Work

If you can write new features and deliver them quickly, but most of your engineers are working on other things, you are not being as effective as you could be.

Effective organisations structure themselves to enable most engineers to focus on business value, but the autonomy and empowerment that microservice architectures enable can bring challenges.

As I mentioned in Chapter 1, with the shift from writing a monolithic system to a microservices-based one, the things I spent time on changed. I went from spending 90% of my time focused on development (designing, coding, testing) to more like 50%. This was because I was no longer committing code that someone else packaged up and deployed, onto a relatively stable set of servers.

Now, I would be the one who set up the service and released the code, which meant for a new microservice I would go through multiple steps before any code could go live. At a minimum, that would mean spinning up a VM (later, writing container configuration), creating a build pipeline, setting up credentials and wiring up access to other services or a database. To allow my team to support the service in production I would have to set up log aggregation and other observability tooling. Sometimes this could mean also setting up additional infrastructure for the cluster the service would run on, such as databases, IAM policies, and load balancers.

This was nearly a decade ago, and we were still learning what running a microservices-based architecture involved. Also, our central teams were still supporting existing types of architecture: they weren’t working on tooling for our new approach—​so “you build it, you run it”⁷ meant product teams like mine took on a lot, reducing the amount of time we spent on writing new features.

But if every product development team is doing this, you are wasting time solving the same problems, and you will not be releasing as much value to the business. You don’t want to have a significant portion of your team working on infrastructure and resilience rather than business features!

There was a time when I was working on our content platform, prior to the existence of kubernetes. To get the benefit of containerization, we built our own container orchestration platform. Almost half the services in our estate were there to enable us to build, configure and operate the system, rather than to provide business functionality. Kubernetes removed a lot of that complexity, but not all.

And this brings me back, again, to the idea that microservices give you flexibility to use whatever language, database, or tooling you want. You can do this, but it comes at a cost.

Autonomous empowered teams should be able to choose the right tool for the job. Mostly though, you should expect them to choose something that makes life a bit easier, something that is built and maintained by central teams.

Those central teams are what Team Topologies calls Platform and Enabling teams (see Chapter 5 for more on this). Their focus is on providing well-documented, self-service functionality for things like setting up and running servers, containers, DNS, CDNs, observability, etc.

They should be building a paved road (discussed in more detail in Chapter 6)--a set of options that work well together and make life easy for product development teams, abstracting some of the complexity of microservices and reducing the cognitive load on engineers. Crucially, a paved road is not mandatory. You can choose to go off road, but you’ll likely find that it is slower going and requires more effort.

See Figure 2-3 for an example of the kinds of things the platform and enabling teams supported at the FT.

Figure 2-3. The FT’s Tech Hub shows the types of capabilities that can be offered by platform and enabling teams

Product development teams that go off road will need to do more work, because they will be the ones supporting things out of hours, doing patching and updates, making sure that logs get shipped etc. This requires a level of governance to exist, to specify the expectations they need to meet—​the subject of Chapter 10.

Having teams in place building a paved road can really help product development teams to spend more time on valuable work. But they will still need to look after their systems, and microservices bring challenges simply because there are more of them. If you need to upgrade a library, you probably have to do it in 20 places, rather than one (monorepos can help here and this is discussed in Chapter 13). Similarly with database upgrades—​if you have five different data stores rather than one SQL database, you will probably be doing database upgrades five times as often.

Any investment in reducing this toil pays off: that can be, for example, through automation or through handing work off to others by selecting PaaS or SaaS options when you are moving off the paved road.

It’s important to enable people to spend as much time as possible on meaningful, interesting work for morale and retention. People leave when they have just spent three months upgrading dependencies.

Not Having to Start Again

Another big challenge if you want to be effective is to resist the lure of starting again.

There’s a reason job adverts mention “this is greenfield development”!

We start work on a new system and, after we get the necessary scaffolding in place, we are rolling. It’s easy to add new stuff, we have a good size team and everything is nice and shiny.

But things don’t stay that way.

At some point, the thing you were building is mostly there. By this point, you are more likely to be iterating on something that already exists than building something new. The team might also have reduced in size as people move onto other priorities.

There are some parts of your stack that no one still on the team has ever touched. They are haunted forests⁸--areas that are scary, that you go out of your way to avoid.

Decisions you made, or more likely decisions someone else made, three years ago, are turning out to make a lot of work for your team. It’s not fun any more.

With monoliths, this was normally the point where someone made the case to start again, on new technology. But that’s not supposed to happen with microservices. The promise of microservices was that if you need to rebuild part of the system, it’s simple to do that because the logic is encapsulated in a service, or maybe a few services. You build those services again and—​voila!

Unfortunately, that may not be the reality. Often, the problems are at a wider scale: you are facing an upgrade path for something used across your stack, or you need to add something in that doesn’t fit with your architecture.

Starting again is often more appealing to developers: you can use cool new tech, and maybe you can even make the case that to move fast you need to hand off the existing system to another team to maintain while you get on with fun stuff (this happens and I’ve been on the team that had to maintain the old thing).

But this doesn’t make business sense. While you build the new thing, the old thing will essentially be in maintenance mode. When the FT rebuilt its website in 2015, it cost £10million and there was essentially no improvement of the live website for nearly 2 years.⁹

You need ways to continually improve and replace parts of your architecture—​to combat entropy and tame the haunted forests.

Sam Newman recently introduced me to the concept of “habitability”. This is an idea developed by Richard P Gabriel, based on ideas from the architect Christopher Alexander,¹⁰ that we want our software to make us feel at home; to be easy to understand and to change. “Habitability” is about whether your system feels livable.

This is about finding a balance between the needs of the whole (the grand design or architecture, which won’t change often) and the needs of the parts (the inevitable changes that various parts of the software go through). Software must be habitable because it always has to change. Alexander says the same about buildings. They don’t get finished, they get modified, reduced, enlarged, improved. Piecemeal change is normal and if we can make that easy, we will probably find ourselves in a good place.

Microservices should be habitable, because with microservices, change is a first-class design consideration.¹¹

Some of the same topics we discussed about how to ensure people spend most of their time on meaningful work apply here as well. For example, consider using a paved road that someone else provides, where they do the majority of upgrading databases and migrating platforms to newer tech. Additionally, choosing PaaS or SaaS solutions where you do move off the paved road helps prevent these tasks from falling to your teams to tackle.¹²

Paving the road can also apply within your team. You can settle on particular patterns for how you do things, and agree on an approach for moving off that paved road.

It’s also important to make sure all your services are actively owned. Active ownership means you need to understand the service well enough to be able to fix it if things go wrong. It should also mean that you invest time in fixing issues as they come up—​an actively owned system shouldn’t get to the point where a dependency is out of support before you suddenly realize you have a problem.

Find the time to discover the places that cause you pain, and invest time in making things better. This can be a challenge. Often, it is hard for engineers to make the case for spending time away from feature delivery. You need to talk about these changes in terms that mean something to product owners and delivery folks. For example:

“This change will allow us to create new services in half an hour rather than 1 day, so we can spend more time on new features rather than plumbing. Also, automating it means we won’t have bugs because someone missed a manual step. This happened 3 times in the last year and took 2 person days to deal with.”

“This fixes a critical security vulnerability. Contracts worth $4 million depend on certification that will lapse if we don’t fix issues of this nature.”

Keeping Risk at an Acceptable Level

As someone who used to run an operations department, this is close to my heart. We need to consider risk to be an effective software development organization.

We cannot remove risk. Every decision we make around our architecture comes with risk. Let’s consider one example: what is the risk around having a single CDN provider?

If they have an outage, there may not be a huge amount you can do. You will likely find your website is down, and possibly even some of the internal tools you use to support your estate—​if they are all available via the public internet. This is even more likely if you are using the tools your CDN provider offers—​for example, maybe you are using the CDN to integrate with your single sign-on provider.

But the alternative also comes with risks. Having two CDN providers increases the change of something breaking, because it increases the complexity of your software estate.

And it comes with costs. If you have two CDN providers, you will be spending more money. What could you spend that money on instead?

And every change then has to be tested for both CDNs. You have to test failover between the two. And now, you can’t benefit from the custom functionality that one provider offers, or if you do, you have to work out what happens when it isn’t available.

There are similar discussions to have about many other things—​cloud vendors, DNS providers, source control, continuous integration tools.

And there are lots of types of risk—​operational, security, cost.

An “acceptable” level of risk is an evaluation that will be different for every organization. I’d suggest that you don’t want to be having to make that same evaluation in lots of independent autonomous teams, for lots of different areas of risk.

This is somewhere that governance makes a difference. But this should take the form of guardrails and insight, rather than gatekeeping and mandates.

What do I mean by that? Mostly, that you should guide people to do the right thing by making that the default. If they go against that to instead do something that is risky, make sure they understand that. That’s more likely to be successful than writing a set of standards for people to comply with. I expand on this in Chapter 10.

I don’t think any company is going to take a hands off approach for everything. Some things are too risky. At the time of writing, failing to comply with GDPR can cost up to 4% of turnover or €20 million, whichever is higher. I don’t think any team should be empowered to ignore that!

Microservices help with risk in some ways. You can more easily isolate parts of your system—​for example, those that deal with personally identifiable information (PII). However, microservices also can result in a lot more work. An example here is security. Scanning dependencies for security vulnerabilities can result in a lot of upgrade work when you have 100s of services. With something like the log4shell vulnerability of late 2021 it can take time just to realize where you may have a dependency on a vulnerable version, particularly with the widespread use of SaaS and PaaS. For your own code, you can add dependency scanning, and I will talk about that in Chapter 10.

How Microservices Measure Up

Mark Richards and Neal Ford’s First Law of Software Architecture is that “Everything is a trade-off”. Microservices make some things easier, and other things harder, but not impossible. We’ve discussed the trade offs and things that help throughout this chapter.

I’d like to finish up this chapter by summarizing how microservices measure up, in the short, medium and long term.

When you choose an architecture, you also implicitly choose particular software processes and practices. For example, microservices won’t work if you have manual provisioning and deployment processes, little testing, and an old school operations team.

This does mean that microservices can require a bit of investment to get started, depending on what engineering practices you already have in place. The investment isn’t solely in engineering practices, microservices also work best with particular cultural and organizational structures and changes to these are not something you can do in days or weeks.

However, there are ways to start benefiting from a microservices approach without a large up front investment. I’ll talk about this in Chapter 3 where I also discuss how to work out if microservices are the right approach for you.

For me, microservices shine in the medium term, at the point where you can quickly implement and then get real feedback on your ideas because you are able to release changes as soon as they are ready, typically releasing code many times a day. This delivers real business value.

But what about the long term, and the need to stop and rebuild? Well, microservices offer hope that we won’t need to do this any longer, because we should be able to continually improve and replace parts of our system. It does however still take work to avoid getting into a state where the only way out is a lot of work or a complete rewrite. I will talk about this throughout the book, but in particular, in Chapter 13 which covers keeping things up to date.

In Summary

This chapter covered a lot of ground about what is needed for effective software delivery in the context of a microservices architecture. Many of these topics will be covered in greater detail throughout the book, but for now, there are a few key takeaways that are worth repeating.

Successful architectures need to deliver over different timescales: architecture is the stuff that’s hard to change later, so you want to still be benefitting several years in.

Microservices do require a bit of investment to get started, both in terms of tools and technologies but also organization and cultural changes.

Microservices shine because they allow teams to work autonomously, delivering business value quickly. However, in the longer term, as microservice architectures mature you can end up in a mess, with small teams trying to support lots of services, overloaded with operational and maintenance challenges.

This chapter discussed those challenges and introduced the tools, techniques and processes you’ll need to have in place so that you can maintain and sustain your architecture for the long term. This is what Parts 2 and 3 focus on.

But first, in the last chapter of Part 1 we’re going to dig in to how to assess whether microservices are a good fit for you and for your organization. We’ll also discuss how to try out this architecture with a minimum of up front effort.

¹ There are of course cases where the short term is the only thing that matters—​for example if you are a startup and you don’t know that you’ll still exist in three months’ time!

² If you don’t have that high degree of confidence, then microservices likely aren’t working for you. This book will suggest some ways to tackle that.

³ As quoted in this article

⁴ This is one of many good points about what ops looks like now in Ops: It’s everyone’s job now.

⁵ Gray failure is the subject of Gray Failure: The Achilles’ Heel of Cloud-Scale Systems, and will be discussed in more detail in Chapter 8.

⁶ In Cynefin terms, these systems are complex rather than complicated and failures can very often be chaotic

⁷ Where development teams run their own code in production. Coined by Werner Vogels in 2016 to describe the approach taken at Amazon.

⁸ As discussed by John Milliken based on experience at Google and Stripe

⁹ Details from my former FT colleague Anna Shipman’s QConLondon 2022 talk, No Next Next: Fighting Entropy in Your Microservices Architecture, which talks about the ft.com team’s strategy for not having to start again.

¹⁰ An architect of buildings, rather than software

¹¹ As Mark Richards and Neal Ford note in the preface of Fundamentals of Software Architecture (Sebastopol, O’Reilly, 2020)

¹² Effectively, you still have a paved road, it’s just paved by someone external. Thanks to Benji Weber for making this point!

Chapter 3. Are Microservices Right for You?

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 3rd chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at jleonard@oreilly.com.

Microservices are now a mainstream architectural choice, but they are not the only viable option, and you shouldn’t adopt them by default. You need to take time to work out whether they are the right solution for you. That depends on you: what kind of organization are you, what technology do you have in place, and what skills do your teams have? It also depends on the problems you are facing.

The answer to “why microservices?” should be “for business reasons”. That should be true of pretty much every technology decision we make—​a decision should allow you to do something that matters to your business, whether directly, for example allowing you to introduce new functionality more quickly, or indirectly, for example through reducing risk or reducing cost.

Understanding what you are hoping to get out of adopting microservices is important because it helps you to prioritize those specific outcomes. It guides you on what measures you should be tracking, and it can tell you when you’ve “done enough”.

However, even if you have a problem that microservices can help with, you also need to look at whether you will be able to get the conditions in place to successfully adopt microservices. This is important because if you can’t make the necessary organizational and cultural changes to be successful, then you could end up in a worse position, with a more complicated architecture but without benefiting from being able to deliver better business outcomes to your organization.

Let’s start by discussing reasons you might move to microservices, before looking at the things you need to have in place to be able to make that move successfully.

Reasons to Choose Microservices

Microservices are very often a solution to an organizational problem rather than a technical one. That organizational problem is often about being able to enable large numbers of people to work on the system effectively, in parallel. It can also be about developer experience, important in speeding up delivery of value but also in recruiting and retaining people.

There are also a number of technical reasons to choose microservices (many of these already introduced in Chapter 1 as benefits of this architectural style), all of which result from the boundaries that a microservice architecture puts between different parts of the system. These boundaries mean that you can scale just part of the system, or implement auditing or compliance only where you are dealing with particular types of data. A failure can be limited to a single part, making the system more robust. Finally, you can use different technologies for part of the system, ones that are a better fit to the requirements.

Scaling the Organization

Up to a point, adding new people to a team increases the productivity of the team. Different members will bring different skills, and there is more scope for people to work on the tasks that interest them and that they can do well. So a team of five is likely to be more effective than a team of two. But what about a team of 20?

I expect your intuition is that a team of 20 wouldn’t work. And that has to do with the nature of a software development team. People on a software development team don’t work independently: they make decisions together, and communicate about the work they are doing.

That communication becomes more of an overhead the more people you have on that team. When you move for example from 5 to 6 people in a team, the number of lines of communication goes up by 50%, from 10 to 15 (see Figure 3-1).

Figure 3-1. The number of communication links in a team of six is 50% higher than for a team of five

Not every interaction in a team needs to happen 1-1, but plenty do. Think about a daily standup. If each person gives a 3 minute update, a 15 minute standup for a team of 5 becomes an hour for a team of 20.

And it’s not just about the time spent in communication. As a team gets bigger, there are more opportunities for miscommunication to occur, making it harder to maintain a common understanding of the purpose of the team, and easier to end up with people in the team who have different perspectives on how parts of the system work or what the outcome of a particular task should look like.

There is also a tendency for individuals to slack off when working in groups. This “social loafing” effect may be related to feeling less personal responsibility for the output of the team as the team gets bigger, or to a feeling that your efforts just don’t matter.

The outcome of all these aspects is that you can’t keep adding people to a team and expect that team to continue to function effectively. Jeff Bezos famously introduced a two-pizza limit for teams at Amazon: if two pizzas isn’t enough to feed a team, the team is too big. This limit (let’s assume it’s four to six people, because it’s a pretty imprecise guideline) is at least supported by research by Neil Vidmar and Richard Hackman where teams of various sizes were asked to assess after completing a task whether the team was too small or too large for the best results. The cross-over point where people were most satisfied with the size of the team was between four and five people.¹

So, as the organization scales, you split into multiple teams. However, in a monolith, these teams are making changes to the same codebase. By default, i.e. unless you actively design your monolith to prevent it, that means it’s possible for code to directly access other parts of the codebase and even the database, so that changes made by one team can have unexpected consequences for another team who were relying on something that the other team felt was an “internal detail”. Alternatively, you can have multiple teams making changes to the same shared files with the associated challenges of releasing those files: what happens when the first team to commit changes to the file isn’t ready to push the change live?²

Well, it slows you down. Accelerate’s research identified that high performing organizations have loosely coupled teams and architecture, so that things can change in one place without impacting the wider system.

Loose coupling means having separate components for different business capabilities, so that a change to that business capability happens in one place only, and is handled by a single team. Information is encapsulated inside those modules, with access only via a well-defined interface, which reduces the direct knowledge that different components have of one another.

Loosely-coupled architectures mean that teams can generally get their work done without having to coordinate with people outside their team. That doesn’t mean that there is no interaction between components: what you are aiming for is to have all access via a defined interface; to get the boundaries right so that the interface for a component is stable and changes infrequently; and that most change happens within a component owned by a single team.

Done right, a microservice architecture is loosely coupled, and I think for most people, the main reason to adopt microservices is this need to scale the organization.

Accelerate doesn’t advocate microservices though; in fact it says that what tools or technologies to use are the wrong questions to focus on. “What is important is enabling teams to make changes to their products or services without depending on other teams or systems.”³

As an alternative, also loosely coupled, you can opt to make your monolith more modular. I discuss this later in this chapter, and it can be a good option where you don’t have the conditions in place to make a success of a microservice architecture.

Developer Experience

You want your technical stack to attract good people to work for you, and to keep the good people already working for you, and the impact of this on technical decisions shouldn’t be underestimated.

People are attracted by the technology they will get to use and learn, but also by the culture of the organization. Is there space to be innovative? Will they find it easy to move on in a few years time with this tech on their resumé? You don’t want to do resumé-driven development but equally, you don’t want to make it harder to recruit or retain people.⁴ In my view, many organizations adopt technology around microservices (for example, Kubernetes) too early because of this driver.

Microservices, in my opinion, are now in the late majority phase of the technology adoption life cycle. That means for many people, it’s what they expect. Are people going to be hesitant to join an organization that has a large monolithic architecture with many teams working on it? Maybe.

Beyond the choice of technology and the signal that sends, there is the actual experience of working as a developer using this architecture. The clear separation of a microservices architecture into different domains means that it is easier for a new developer to get up to speed. They don’t need to understand the whole system, just the part they are working on. That goes for supporting the application too. It should take much less time to be able to support a small application than the whole of a monolith.

The local developer experience should be smoother than working on a monolithic application. You should not have to spin up the whole system on your laptop to work on a particular service. That means a shorter development cycle time of code, run, test, because it should take only seconds to compile and restart your application and to run the automated tests just for that service.

For many developers, the autonomy a microservices architecture gives them is also a major attraction, because it greatly reduces the amount of time they spend waiting for someone else to take an action. If you are used to doing a release once a month, being able to do a release in minutes is a significant benefit.

A caveat though. It’s easy to get microservices wrong and end up with a worse developer experience. A simple example, already mentioned in Chapter 2: when I first built microservices, we implemented an end-to-end acceptance test suite. It was very hard to maintain, because data fixtures needed to be set up in multiple services. It coupled all our services together, and it slowed down development drastically. One developer made a code change that took half an hour and took over a week to fix all the acceptance tests, because of the coupling and bad structure.

This is related to the challenge I mentioned in Chapter 1. The developer experience in microservices is different and if you don’t adapt the ways you work, you could struggle.

Keeping an eye on developer experience is a good way to make sure you get the most benefit from microservices and much of the rest of the book is about how to optimize for this.

Compliance and Security

Microservices let you minimize the parts of your system that manage particular types of data, for example payments or personally identifiable information. That can make it a lot easier to make sure you are complying with requirements like the Payment Card Industry (PCI) standards or General Data Protection Regulation (GDPR).

The teams working on those services will need expertise in these requirements, for example, understanding what they can log without concern vs places where they need to log to somewhere that most people can’t access, but other developers won’t. They are protected by the boundaries put in place between these services and the rest of the system.

Scaling For Load

With a monolith, if you need to scale, you have to scale the whole thing.

With microservices, you can scale just the parts that need to be scaled. You can also run different services on different hardware, optimizing for the characteristics of that service—​for example, whether it is IO bound or CPU bound. And finally, if you have some parts of your system that are only needed at specific times, you can scale them right down or even turn them off.

All of these options can save you money and increase performance. This kind of scaling is a key reason that companies like google and netflix use microservices (it may be less relevant for companies operating at smaller scale).

Another advantage of a microservice architecture is that the data is owned by individual services. That means each data store will contain less data, making it less likely that you will need to shard your database across multiple servers. Sharding makes things more complicated since you need to be able to route requests to the appropriate shard. Moving to microservices that each own their own data can avoid this complexity.

Increasing Robustness

As mentioned in Chapter 1, with a monolith, it’s hard to degrade gracefully. If something goes badly wrong, your whole system will be unavailable. With microservices, you can design your system so that losing one service doesn’t affect the rest of the system. This can involve failing back to a simpler solution. For example, let’s imagine an online book store. If you can’t access information that allows you to show someone a personalized list of recommendations, you can show a list of the most popular items instead.

This does take effort, because you need to decide what to do in case of partial failure. But it does mean that failure has a smaller blast radius.

Another thing that a microservices architecture allows is the ability to choose different levels of resilience for different parts of the system. With a monolith, if you need high availability, you need the whole monolith to be deployed in multiple availability zones and multiple regions. But with microservices, you can accept that some parts of the system are effectively “nice to haves” - you can reduce cost and complexity by running those systems with a lot less redundancy. So that list of personalized recommendations might be a lot less important than the ability to search for a particular book.

Increasing Flexibility

Sometimes you find a part of your system that is quite different in its requirements. With microservices, you can choose different technologies for that domain much more easily.

To expand on my example from Chapter 1, when the FT had a monolithic java app and a relational database for the content publishing platform, metadata had to be stored in that database. But metadata is a graph: an Author is linked to many Articles they have written. The Articles mention Organizations, which are linked to People—​the CEO, the Founder.

Graph databases offer much simpler support for queries that navigate that graph. Moving to a graph database for storing metadata, and a document store for storing articles, fitted the data we had and the way we wanted to interact with it much better than a relational database.

Conditions for Success

So, let’s say you have a compelling reason to adopt microservices. How can you have confidence that this will deliver the change you need?

In this section I want to go over a few questions to ask yourself about your organization and your context, before embarking on a move to microservices. If you don’t have these things in place already and can’t imagine introducing them, you could find moving to microservices a challenge.

These are all positive changes in my experience, whatever architectural style you end up with! So I encourage you to think about what it would take to introduce them in your organization. I will talk a little about change management later in the chapter, in the context specifically of how to make these changes.

Domain Understanding

The first question is, how well do you understand your domain?

If you are building something new, you probably don’t yet have a deep understanding of the domain. That means you will struggle to identify the right boundaries between parts of the system. If you divide up your system now, there’s a high chance you’ll find you need to move things around later.

This is a good reason to start with a monolith. The early phases of any project tend to mean writing new functionality, which means less contention for making changes to the same code. You should make good progress to start with.

As you work on the monolith, you will start to see code that hangs together nicely and changes at the same time. Then, when you start to need a move away from that monolith, you will have a good idea of where to split out a service.

Products Not Projects

Many companies fund development work through projects. A feature or set of features to develop is identified, a team is put together and funded for a period of time and once those features are complete, the project is “finished”. After that point, the system goes into maintenance mode, run by a maintenance team.

This doesn’t really work for a microservice architecture. The teams get to make decisions about the tech they use because they are going to support that tech in production. This is good for the quality of the system being built—​you think differently when you are the team that might have to pay the price for a particular technology decision.

What it means for the organization is that you need to move to a product-focussed approach. You form a team to build a product, and that team will own it throughout its lifetime. That doesn’t mean the team won’t change, or reduce in size. But it means that there is some continuity between the people making decisions and the people supporting the consequences.

The problem with changing from a project to a product-focused viewpoint is that this requires changes beyond the Technology teams. You need Finance, Operations, Marketing, Sales and other departments to understand and accept the change too.

There are two things that can help “sell” the change. Firstly, you should be able to make changes those departments care about more quickly. And secondly, the change from project to product should mean more flexibility to meet the needs of those departments as they understand them, rather than those departments needing to get everything they care about put into scope for a project. You need to prove this works though!

Leadership Support

Microservices can require big organizational and cultural changes that take years to complete. And microservices have a start up cost. You will move slowly while you build necessary tooling, and develop a culture of long-term ownership and support.

If your leadership aren’t bought in, there is a good chance you won’t get that time.

Does your leadership understand the problems you are trying to address by moving to this architectural style? Do they understand the trade-offs? And do they understand the costs that might be associated with this? For example, if development teams are now all supporting their own systems, you have to think about how you pay teams for doing that support, and you may have 10 times as many licences needed for operational software that handles or escalates incidents.

Think about what metrics matter to your leadership, and how you will show the positive impact of a microservices architecture on those metrics.

Teams that Want Autonomy

If you move to microservices, you need each team to want to make their own decisions. If your teams want to write code within an established framework and don’t want to think about operational concerns or infrastructure at all, moving to microservices is likely not going to work out.

You also need teams that are willing to support their services both during working hours and out of hours. And you need a plan for how that out of hours support will work.

Over time, you can shape your teams through turnover and recruiting for experience and interest in microservices, and you can support people currently in the teams to feel more comfortable with new responsibilities.

Support For That Autonomy

Do you have a change advisory board that signs off on releases to production? Can a developer spin up infrastructure or do they need to raise a ticket and wait for a central team to approve it?

A change advisory board feels like something that will make it less likely to have a problem in production, but research shows that isn’t the case. All it really does is slow things down.

Having a central team in control of the infrastructure feels like it will keep a handle on costs and reduce risk from misconfiguration. But again, waiting for that team to do something slows people down. It is more effective to provide templates that help people set things up right, and to review your estate to pick up on wasted resources.

You need a willingness to allow teams to get on with things. This means that teams that handle infrastructure, operations and other central concerns need to move to an enabling, self-service mindset where as much as possible, they remove “gates” that teams need to get through.

Some of these changes will affect what people do, day-to-day. That makes this difficult and emotional. However, if you don’t make this change, you will struggle to truly benefit from adopting microservices.

Technical Maturity

If your organization still does lots of things manually, you aren’t yet in the right place for a move to microservices. Sort out the groundwork first. This means automation.

That means, you should be able to spin up a new server in a matter of minutes. You should be able to deploy via an automated pipeline that includes automated tests rather than a manual regression test. And you should have a mature approach to monitoring the health of your servers and your applications.

These are all changes that will provide value to your organization, whether or not you move to microservices afterwards.

Note: you don’t have to introduce these things for all existing systems. You can extract a single service and start by automating deployment and introducing observability. The problem is if your organization can’t make that move, either because of culture or technology.

Managing Change

Maybe you don’t have the conditions in place to be successful—​yet. So how can you influence your organization to change?

Change management is a huge topic and is not in scope for this book, not least because I am not an expert! I do have a few comments specifically about managing a move to a microservices architecture, which I see as being as much an organizational and cultural change as a technical one, precisely because of what needs to be in place to be successful.

First up, I think it is easier to try something out for small groups and for a limited period. Find a group of like-minded people, and run a project to extract a service, or to automate server provisioning, or to introduce continuous delivery. See whether you get the benefits you were hoping for. A good reason to extract a service might be where it has different requirements, or where you want to try out something new on a small scale, such as a different database or even programming language.

Sometimes, this is about finding a way to work around processes that slow you down. At the FT, when I joined in 2011 we had a change management board and would agree up front on when code could be released to our test and then production environments. But we had a third party that built key parts of our estate, and they didn’t have to follow the same process. When our stakeholders asked why that third party could get things done more quickly, it opened up a conversation about how comfortable the stakeholders were with trusting development teams to decide when to release changes. Soon, we moved from a change management board to tracking of changes and handing responsibility to teams to choose when to do them.

Secondly, accept that not everyone will respond in the same way to a big change, such as a move to microservices and the related change in responsibilities. You will probably find that you have some keen adopters, some refusers, and some in the middle, both for whole teams and for individuals. Work with those that are keen, provide as much information and reassurance as you can for those in the middle, and accept that for some people, this will be a reason to move on. You have a chance through natural turnover and recruitment to shape your teams over time. This is an area where that leadership support is important, because your leaders are the ones who can be clear on what you are doing as an organization and why.

Sticking with a Monolithic Architecture

Unless you have compelling reasons to adopt microservices and can get the conditions in place to be successful, be wary.

There are many advantages of building a monolith, or at least of building a monolith first. They are a lot simpler to operate and to understand, for example. And you can invest in making changes to the monolith to tackle those big challenges that tend to make people look towards microservices.

Specifically, you can address the challenges of scaling your organization and with developer experience by building a modular monolith, one where the code is structured into domains with clearly-defined boundaries.

And you can address different requirements for parts of the system by splitting out parts of the monolith, without going further into a microservices approach. This will allow you to support different levels of required compliance, security, and redundancy, and to use different technologies if those are necessary.

I will talk more about both of these in the following section, but first, I want to briefly talk about how you can deal with moving to zero-downtime deployments in a monolith.

Enable Zero-Downtime Releases

The ability to do a zero-downtime release transforms your ability to deliver value, whatever the architecture. That’s because it lets you move towards releasing on demand, and that lets you start to improve on the DORA metrics: specifically, lowering the lead time between committing code and releasing it, and allowing a higher deployment frequency.

To be able to do a zero-downtime release, you need automated deployment pipelines, containing automated tests. This allows for quicker and reliably repeatable deployments. This is something you can do with your monolith.

With a monolith though, “quicker” still might not be particularly quick. Running every automated test for a large system can take a significant amount of time. You might find you have more changes happening than can each run through the pipeline in a day, in which case you will need to batch together more than one change, and have multiple changes going through the pipeline together. That does mean working out what happens to later deployments if a deployment fails. This can be complicated—​but is it more complicated than supporting a distributed system architecture?

Once you have continuous delivery working smoothly, the next thing you need is to be able to deploy code to a server that is not handling traffic (blue green deployment). Once a deployment is complete, you can move traffic over to the new version. Generally, you want to do this gradually so you can find out about a broken release before it impacts more than a small percentage of your users. Again, this is possible for a monolith as it is for microservices.

The most complicated aspect is generally the database. Keeping two databases in sync isn’t easy, and often the blue and green environments in fact share a database. This means database changes need to be backwards compatible. For example, don’t drop a table or a column until you know you do not need to back out the code change that meant it wasn’t needed anymore. This is definitely more complicated where you have a single database that contains all your data than when you have microservices that own their own data, but it is easier when each individual change is relatively small than in the days where a deployment contained weeks of schema and data changes.

Build a Modular Monolith

Microservices are not the only loosely coupled architecture. One alternative is to break your monolith up into logical modules. This is the approach take by Shopify, and has proven very successful for them.

In this model, the code still lives in a single repository and is deployed as a single deployment unit through a single build and deployment pipeline, but it is split into components that map to different domains, and the boundaries between those domains are specified carefully, although probably not enforced.

Case Study: Shopify’s Modular Monolith

Shopify started off with a Ruby on Rails monolith. This worked well for them initially, but by 2016 as Kirsten Westeinde notes in her Deconstructing the Monolith blog post, Shopify had millions of lines of code and over a thousand developers.⁵ and they were starting to hit problems.

There was nothing stopping code from calling code that was part of a different business domain, leading to a lot of coupling between different parts of the application.

That meant that changes in one place could have unexpected consequences, so that a small change could cause many tests to start failing. This also made for a steep learning curve as developers had to understand a large part of the system before they could make changes to a smaller part.

Shopify wanted to decrease the coupling between different parts of the system, reducing the density of the dependency graph and giving them confidence that changes in one part were not going to impact other parts.

This is what microservices give you, but Shopify liked many aspects of having a monolith. There is only one repository, meaning you can easily search within it, and make changes in calling code at the same time as in the code being called. There was only one build and deployment pipeline to maintain, and only one set of infrastructure. Calls between components were in process rather than over the network, making them faster and more reliable.

Rather than moving to microservices, Shopify reduced the dependencies by making the move to a modular monolith. This was a significant change—​starting with a complete refactoring of the codebase to be based around domains rather than software layers (the model, view and controller layers that a Ruby on Rails application defines by default within the app folder).

Those domains then defined public APIs, with data owned by the component only to be accessed via those APIs. Shopify developed tools to flag any access not happening through the APIs, as it shows coupling across domain boundaries, and worked to reduce this coupling.

These changes paid off. New developers could focus on the parts of the monolith they were making changes to, including running just the relevant subset of tests. Feature development time was reduced.

Another blog post, by Philip Müller,⁶ provides an update several years into this transition, noting that while they stand by their original ideas, almost all the details have changed.⁷

When the team reflected, they concluded that this was primarily a people problem rather than a technical one. They were making a fundamental change to a system actively being worked on, and they needed to change developer behavior. They found some of the things they were doing, for example definining patterns for designing component boundary interfaces, were adding friction, rather than making things easier.

The team decided to focus their energy on: * fostering grassroots efforts by giving talks, working with teams, and generally acting as evangelists * tackling the circular dependencies in their dependency graph, in particular through identifying key components like the “Shop” class that almost everything else depended on, and removing as much concrete implementation from it as possible to leave it depending on almost nothing * revisting their tooling to remove noise and make the tooling much faster to run

For me, it’s fascinating to read this blog post, because the things Shopify have learnt overlap significantly with the things the FT learnt: modularization, whether within a monolith or via microservices, involves understanding and influencing developer behavior (which I discuss in Chapter 6) and a careful application of tooling. And it takes time. While they are already reaping the benefits of their work, they don’t feel they have finished, several years into the process. That matches my feeling from the FT that moving to microservices successfully takes years. You get benefits earlier, but the full benefits come when the new approach is ingrained in the team.

One final comment on Shopify’s modular monolith is that they do in fact split out specific services too. Whether to do this is assessed based on whether

the functionality is something only some merchants need

there is a significantly different performance profile

there are significantly different security constraints

An example is the Shopify storefront. When this was rebuilt in 2019, that was done outside the monolith, because performance is such a critical aspect for this service.

Sticking with a single deployment artifact does reduce your flexibility. For example, to treat parts of the system in different ways because they have specific security or performance requirements, or to use different technologies for different parts of your system—​particularly things like different programming languages. Personally though, I’m not super convinced of the value of multiple programming languages being supported for services operating in the same part of the stack, i.e. backend services. You can still use different data stores from within the monolith, for example a graph vs a document store, and you can certainly split your monolith to satisfy different requirements, without moving to a fully distributed microservices solution.

Your challenges to successfully design a modular monolith are very similar to those to successfully design a microservice-based system. Can you identify business-focused domains where you have low coupling between the domains and high cohesion within them? Can you make that significant shift in thinking for the critical mass of developers, so that modular design becomes the natural way to approach things?

If you can’t do that, you won’t succeed, whether you are modularising the monolith or going for microservices. Architect and creator of the C4 model Simon Brown covered this well in his GOTO Berlin talk in 2018.⁸

For both modular monoliths and microservices, modularization should lead you towards developing and testing against just your part of the overall system. You shouldn’t need to run all the tests—​if you have managed to create loosely coupled components.

Once your code is ready to build and deploy, things feel pretty different between the two. Although you can use a monorepo with microservices—​and I will discuss that in Chapter 13, as an approach that can help with managing dependencies—​in my experience it is more common to have each microservice in its own repo, with its own tests. In general, a deployment will go through the build pipeline in minutes, and deployments (and releases, since releasing functionality can happen after deploying the code when you make use of feature flags and AB deployments) for different microservices should be so independent that no coordination is required. You really do deploy at will. It’s unusual to have deployments queued up on a particular service. If something goes wrong, you can generally either roll back to the previous version of the service or make a code change and roll forward to the fixed version: I saw this “roll forward” happen a lot at the Financial Times. It’s possible when the change is well understood and the fix is small, and the developer finds out there’s a problem while they are still in the codebase.⁹

With a modular monolith, you are deploying the full system each time, through a single build pipeline. Because every code change goes through the same build pipeline, you may end up grouping more than one change into a deployment. Shopify can include up to 16 pull requests (PRs) in a single deployment to production.

It’s pretty unlikely you can run all your tests for each deployment because it takes too long, even if you throw a lot of machines at the problem (Shopify at one point were running tests on more than 100 machines for each pull request), which means you need to invest in ways to run a sensible subset of the tests: the ones that touch the code being changed in this release. You do often end up with deploys running through the process at the same time—​think of this as like trains on a single track. If something goes wrong with a deployment, other trains have to wait.

There are ways to mitigate these challenges, for example by running full sets of tests against PRs ahead of merging of code. Also, by focusing on writing tests that run quickly, so that almost all your tests are unit tests rather than integration tests.

And on the plus side, teams spend a lot less time setting up and maintaining build pipelines!

Once the code is running in production, it’s unquestionably easier to understand what is happening with a modular monolith, because there are significantly fewer moving parts.

One thing to note though is that few monoliths are truly running in a non-distributed fashion, and that is because of other changes happening in how we build our systems.

Everything’s a Bit Distributed Now

With most software being delivered via a browser, it’s rare to have an application deployed on the same box it is being used from. Monoliths generally talk to their database over a network, and clients access them over the network, and there are content delivery networks (CDNs) etc in the path too. Also, the rise of the cloud and the scale of traffic systems can get means even monolithic applications are often running on many machines in parallel. As Brendan Burns says in Designing Distributed Systems: “Once upon a time, people wrote programs that ran on one machine and were also accessed from that machine. The world has changed. Now, nearly every application is a distributed system running on multiple machines and accessed by multiple users from all over the world.”¹⁰

Alongside this, many recent developments in software engineering push us towards a distributed architecture with small components. These are things that allow development teams to focus on business value, and hand other responsibilities off so they are someone else’s problem (even though it may not feel like that if you are struggling to cope with all the things you now need to know about, beyond how to write code!). Let’s explore this further.

The Rise of Cloud Native

Firstly, this is about the move to the cloud. To be “cloud native” means taking advantage of the cloud rather than just treating it as a data center someone else runs. That means using the “value added” stuff from the cloud providers, rather than treating the cloud purely as self-service compute on demand.

And lots of this value-added stuff is something that comes in small pieces, that you compose together: a queue, a datastore, functions as a service. These approaches push you towards very different ways of building a system: serverless really is a cloud-only pattern. Serverless has a lot of the same benefits and challenges as microservices because it also involves small components in a distributed system and in fact, you can just treat it as one deployment option for this type of architecture.

At the Financial Times we made use of many of the options available. We had microservices running on Heroku and AWS. One group used containers running on Kubernetes. And almost every team used AWS Lambdas, either acting as cron jobs or responding to events. It was a mix, and all of it was distributed.

I think that’s probably going to be common, wherever your autonomous teams are choosing the most appropriate tools for the job.

SaaS Makes Sense

The rise of the cloud has also enabled Software as a Service (SaaS), where companies run software for you, accessed over the internet, usually for a subscription fee.

Firstly the cloud makes it easier for those SaaS companies, as they can scale to meet demand easily. But also, my view is that companies were cautious about giving up control of their applications—​I can still remember discussions about why no one would want to run their code in the public cloud, not even that long ago! However, once people have accepted that AWS is hosting their code, it’s a smaller step to using a third party who is also running on AWS.

Using SaaS providers means you can focus on the things that are differentiators for your business. I am not interested in building my own authorization framework, but actually, I’m also not particularly interested in running one built my someone else, if instead I can hand the whole thing off to a third party. But this means that a lot of your business flows are necessarily distributed, because at least some of the steps pass out to third parties.

Once you have learnt how to handle that, the next steps towards microservices are likely less intimidating.

Recommendations

Given all the things I’ve discussed so far in this chapter, what would my advice be to an organization considering whether to move to microservices?

A microservices architecture made up on many services means you are building and operating a distributed system with a lot of moving parts, and that can be complicated. Everything you have to do—​for example, upgrading a dependency, or adding a security scanning tool into a build pipeline—​now has to be done multiple times, once for each service. And rather than having calls between functions in the same running application, you are going over the network, with all the flakiness and latency that entails.

The benefits can outweigh the costs, but only if you invest in the tools and processes you need, and make sure your organization is set up to allow you to work effectively. This is a non-trivial change: it will likely take years to complete, although you should see a positive impact much earlier than that.

Starting from Scratch

If you are building something from scratch, I would tell you to start with a monolith. Firstly, you don’t know whether you need microservices. Why pay the “microservices tax"--the work it takes to provide tools to build, deploy and operate multiple services—​until you know whether you’re building something people are going to use?¹¹

Secondly, you don’t yet have a full understanding of your domain. You are very unlikely to be able to work out the right boundaries between services until you have had a chance to see what parts of the system change together and what parts seem very independent of each other.

Finally though, you are unlikely to successfully build a microservice architecture from scratch because it’s too complex. You need to start with something simpler.

Gall’s Law applies. “A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system.”¹²

So, start with a monolith. As you grow your codebase and/or your team, look out for when you start to slow down and find you get in each other’s way. At that point, it’s time to do something to fix this. By this point, it is likely to be clear where you have parts of that monolith that hang together coherently and could be extracted out, either within the monolith or as separate services.

Replacing an Existing Monolith

I think it is essential to have a modular architecture for a modern software system once you have more than one team working on it: something with loose coupling and high cohesion, where you can release small changes regularly. Many of the benefits of a microservice architecture arise from defining component boundaries well. You will see these same benefits with a modular monolith (see Figure 3-2).

Figure 3-2. Both modularization and a move to distributed systems have value. You can do the first without the second

Building a modular monolith might work for you, and if you don’t have the necessary things in place to make a success of microservices—​for example, if your teams don’t want the responsibility of being fully autonomous or your leadership aren’t on board with what needs to happen—​I would stick there.

However, and this may not surprise you given I’m writing a book about microservices, I would definitely consider taking the next step and moving these modules into separate processes.

With a modular monolith, I think you end up with a lot of complexity in the build and deployment process. It’s hard to make sure you can release code quickly and with a high degree of confidence. It’s also much harder to maintain boundaries between modules through consensus within a monolith than it is where services are built and deployed separately. It’s too easy for developers to make decisions for the short term that break the boundaries (true Tech Debt as defined by Ward Cunningham, in that you take it on knowing that you will be paying interest on it until you go back and fix it).

It is a lot easier now to build and operate a distributed system than it was when we first started building microservices at the FT. There is over a decade of experience to draw on, and many more tools that can help. But even in 2015, you could build successful microservices-based systems. We built three at the same time at the FT!

I’d certainly start with fairly coarse grained services, to minimize the overhead of running them all and to minimize the chance that you have to deal with data changes across services. Sam Newman describes a useful pattern here: the Strangler Fig pattern.¹³ Your new system grows up around the old system, which acts as a support structure until the new system can stand on its own. You can extract a single service, see what benefits that brings, then go back and do another. Stop when you no longer get the value, or keep going until there is no monolith left.

I would also aim for the least possible amount of additional technology while in the early stages of implementing a microservice architecture. You want to minimize the up-front setup costs, and setting up a Kubernetes platform—​or learning about Kubernetes, if you choose a platform run by someone else—​is likely to be quite an investment.

If you are running in the cloud, I would aim to outsource as much effort to your cloud provider as possible. Package your application up as a container image and have those run for you.

I would also take a good look at whether serverless solutions would work for you. Adrian Cockcroft pioneered the use of microservices when he was at Netflix. Now his advice is to look at serverless first, using serverless functions to glue together all the really reliable long running services that cloud providers already offer, such as databases and message queues.

I would probably use a container-based application for anything that has a consistent workload, and anything where latency matters. I would definitely look at serverless for any code that needs to respond to an event, for example, a file being written somewhere.

Almost everything I’m going to say in this book applies to any distributed system made up of small services, whether those are long running containers or short-running serverless functions. It also feels like things are converging a bit. As an example, AWS now supports deploying lambda functions as container images, allowing you to use a consistent set of tools across both, and choose to deploy containers or functions, whichever meets your need.

In Summary

Any architectural choice is a trade off. Microservices can be complicated to build and operate, but they solve particular problems very well:

Scaling your organization while maintaining the ability to move fast

Enhancing developer experience through quicker onboarding, a quicker development cycle and a quicker release into production

Allowing you to isolate compliance and security requirements

Scaling just those parts of the system that need to be scaled

Keeping the blast radius for an issue smaller, increasing the resilience of your system

Letting you choose different technology for parts of your system, based on differing requirements, for example having a graph database for data that is graph-like

Even where you have these challenges, you should make sure you have the conditions in place to be successful with a microservice architecture:

Understanding your domain, because if you don’t yet understand it, you won’t be able to identify the right boundaries

Long term funding for products, allowing services to be owned, rather than short term funding for projects that then need to be handed over to some other team

High level support for making the change, because it’s a cultural change, not a technology one

Teams that want autonomy, and enabling teams that will support them with that, because you need people who will take responsibility for the services they build

A high level of technical maturity around automation and infrastructure as code, because you can’t set up servers and release code by hand when you’re doing this frequently

Notice that there is nothing here about particular technologies. In fact, I strongly recommend doing as little as possible setup of new technologies when getting started with microservices.

You should definitely start something new with a monolith. When you find things slowing down, you should look to make your architecture more modular, whether that is within the monolith or by extracting microservices. But the world of software is moving in the direction of distributed components within an architecture, both through the use of cloud provider managed services and SaaS options. Even a monolith likely makes calls over networks so you should get comfortable with distributed architectures.

I like the firmer boundaries you get with a microservice architecture over a modular monolith. It constrains what people can get wrong! Once you have multiple teams working on a system, microservices would be my choice, although I would start with fairly granular services.

If you do decide to go ahead with microservices—​or already have—​I have a lot of suggestions about how to make a success of that. In Part III of the book, I focus on the technological aspects. But firstly—​and I think more importantly—​in Part II, we will talk about the necessary changes in organizational structure and culture.

¹ Discussed in Richard Hackman’s book Leading Teams (Boston: Harvard Business Review Press, 2002).

² I have worked on quite a few monoliths where the architecture could best be described as a "Big Ball of Mud“: haphazard, sprawling, and the result of pressure to build new functionality without the time to structure the code “properly”.

³ From Accelerate’s chapter on Architecture

⁴ In the UK, resumé-driven development is known as CV++!

⁵ I also recommend Kirsten’s Craft Conference 2022 Deconstructing the Monolith talk

⁶ Former Staff Software Developer at Shopify. Thanks Philip for your input to this section!

⁷ Really interesting when people publicly reflect on how things have gone, I wish more people did it. I really want to know what the impact is many years in.

⁸ This link leads to both the video and the slide deck: https://gotober.com/2018/sessions/515/modular-monoliths

⁹ When my team first moved to containers, our hand-written container orchestration system did not support rolling back. If we wanted to revert code, we’d build a new release with the code reverted back to an earlier state. This was not generally a problem for us.

¹⁰ Brendan Burns, Designing Distributed Systems (Sebastapol: O’Reilly, 2018).

¹¹ This is Martin Fowler’s Design Stamina Hypothesis. Until you have built enough features to find out whether you have product market fit, who cares how well designed your system is? Once you know you have a successful system, it’s time to invest in intentional design.

¹² Of course, you can also build a simple system that doesn’t work!

¹³ First captured by Martin Fowler

Chapter 4. Conway’s Law and Finding the Right Organizational Structure

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 4th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at jleonard@oreilly.com.

Conway’s Law says that software development organizations produce software architectures that mirror their organization structure: if you have two development teams, you will end up with two subsystems.¹

I’m going to start this chapter by discussing Conway’s Law and how you can make it work for you, by building an organizational structure that matches your target architecture.

I’ll then talk more specifically about the type of organization that enables a microservice-based architecture: it should be loosely coupled, made up of autonomous teams, each with a clear area of responsibility that maps to one or more microservices.

Within an organization, teams are the fundamental units of software delivery. We’ll look at what makes for an effective team, and I’ll describe the types of team you should have in place, drawing heavily on the definitions in Matthew Skelton and Manuel Pais’ Team Topologies, which matched very closely to the types of team we had adopted at the FT as we moved to a microservices architecture.

Finally, I’ll discuss how you can find the right boundaries between teams (which will also be the boundaries of your architecture), and how to spot when those boundaries need to change. You should expect this, as the needs of your business, or the technology available, or the things you are focussed on change—​but it will also happen as you understand your domains better.

But first, let’s dig further into Conway’s Law, and the implications for your organization.

Conway’s Law

Any organization that designs a system … will produce a design whose structure is a copy of the organization’s communication structure

Melvin E. Conway

In 1968, Mel Conway published a short paper—​just 45 paragraphs long—​called How Do Committees Invent?²

The conclusion of the article, quoted above, has become known as Conway’s Law, and as Martin Fowler says, it is “Important enough to affect every system I’ve come across, and powerful enough that you’re doomed to defeat if you try to fight it”.³

Conway starts by noting that “Any system of consequence is structured from smaller subsystems which are interconnected”. You keep subdividing into smaller systems until you get to a system that is simple enough to be understood. Different teams then design these different subsystems.

Conway points out that both organizations and systems have a graph structure, made up of nodes and lines. For a system, the nodes represent subsystems and the interfaces between them, and for an organization, they represent subgroups and communication paths (since groups must talk to agree interfaces between subsystems). Conway notes that the system and the organization designing it will be homomorphic, meaning they have the same shape. The subsystems and interfaces will map to the subgroups and communication paths. Figure 4-1 shows what this looks like.

Figure 4-1. Homomorphism from System to Organization

Note

Like me, you may be more familiar with isomorphism than with homomorphism. Homomorphism means “same shape”. A group within an organization may design more than one subsystem, but ultimately the graph for the system and that for the organization will collapse down to look the same. You can map from each subsystem to a single subgroup, but you can’t necessarily map in the other direction.

Isomorphism means “equal shape” and indicates that you can follow the relationship in either direction.

The separate groups designing subsystems need to coordinate so that the subdesigns can be consolidated into a single design, and you’ll find that teams that are closer together will produce designs that are closer together too, because it’s easier for them to communicate and learn about each other’s systems.

There are a couple of interesting consequences here. Firstly, because the work gets divided out to the existing groups, any constraints you have in the organization will be represented in the systems too. This means there are some types of system design that cannot happen, because the organization can’t support them. A hierarchical organization will produce a quite different system design than one that has more of a graph-style structure.

Secondly, in splitting out subsystems, you are inevitably making design decisions already: each delegation rules out certain alternative designs. And so your organizational structure pushes your system to look a particular way.

There are two other things that Conway discusses in the paper that I find really interesting.

Firstly, that research which leads to techniques permitting more efficient communication among teams will play an extremely important role in the technology of system management. Each subsystem needs to understand what the other subsystems do so that a single overall coherent design can emerge, but communication is costly. Conway points out that “Elementary probability theory tells us that the number of possible communication paths in an organization is approximately half the square of the number of people in the organization. Even in a moderately small organization it becomes necessary to restrict communication in order that people can get some “work” done.”

Microservices (and other loosely-coupled architectures) are a tool here. They reduce the communication overhead because they hide much of the information about a system—​all the parts that other systems don’t need to know about—​behind a well-defined interface.

The second interesting point is around the “rightness” of any design. Conway points out that “It is an article of faith among experienced system designers that given any system design, someone someday will find a better one to do the same job. In other words, it is misleading and incorrect to speak of the design for a specific job, unless this is understood in the context of space, time, knowledge, and technology. The humility which this belief should impose on system designers is the only appropriate posture for those who read history or consult their memories”.

What this means is, you should expect to iterate as you develop your understanding of what your system design needs to deliver. Given Conway’s Law, that iteration is as much about organizational structure as it is about the design.

The Inverse Conway Maneuver

You can increase the likelihood of successfully building a new system if you match the design of that system to the design of your organization. You can also go a step further, and attempt the Inverse Conway Maneuver. This is where you evolve the design of your organization to make it easier to build your system the way you want to.⁴

Note

In fact, you likely need to evolve the organization and the architecture together, particularly if you already have a system in place. It’s no good changing the organization to conflict with that and hoping it will all work out. As Mathias Verraes says, “A reorganisation won’t fix a broken design.”⁵ This is where a step-by-step approach, pinching off parts of the system and parts of the organization and changing just those parts, iteratively, may help.

If you want to move to a microservices architecture, what kind of organization design do you need? I’ll tackle this in three parts. Firstly, the underlying organizational culture. Second, what effective teams look like. And finally, what types of team you need. The Inverse Conway Maneuver is for me at least as much about these things as it is about a big reorganization.

TODO title

In fact, you should be very wary about trying to reorganize your organization to success, particularly where that involves breaking up existing teams.

Reorganizations are stressful, and they impact people’s productivity for weeks or months. Often, they act to overcome people’s inertia, making it more likely that people will leave the organization.

Keep teams together where you can, or evolve them over time, don’t throw away your old org chart!

Organizational Culture

A microservice architecture is loosely coupled, so you need your teams to be loosely coupled. That means setting those teams up so that they can get their work done without needing coordination with other teams. The reward for this is the fast flow of change, delivering frequent business value—​but only if the organizational culture can let that happen. What characteristics of a culture support autonomy and a fast flow of change?

Open

In an open organization, information is shared, not hoarded. That means defaulting to open: teams should be able to read—​but by default, not modify—​code in repositories owned by other teams; they should be able to find out what those teams are working on, and how that’s going, for example through being able to see the key results and progress against those; they should be able to find enough information about APIs and interfaces to get started using them.

All of these approaches decouple the broadcasting of the information and the use of it. You want people to be able to answer their own questions without having to book a meeting or have an email conversation with you.

To become an open organization, you need to set the expectation of sharing information. No one should ever be punished for doing that, even if the news they share is bad!

As someone who used to be responsible for production support at the FT, I did everything I could to encourage people to tell us if there was a problem in production. That meant making it easy for anyone to raise an incident, asking people to do that even if they weren’t 100% sure, and ensuring a culture where everyone’s first reaction was to help, not to blame. This is also important for helping people feel comfortable with supporting their own code in production, which I will talk about in Chapter 7.

Learning

In a learning organization, people are encouraged to try out new things. This can be new technology, new processes, or new ideas for products or features. Giving teams time and space for this can pay off in many ways.

It’s important to recognize that sometimes things won’t work out. If you are early adopters of an innovative technology, you may see the benefit for your business—​or you may find it’s not production-ready, or that the company gets acquired and that product gets shut down (that has happened to me!)

Creating a culture of innovation means accepting a reasonable level of risk, and making it safe to fail.

When things go wrong, the organization should seek to learn from that, rather than looking for someone to blame. That also includes when things go wrong in production (I’ll talk about learning from incidents and blameless incident reviews in Chapter 7).

To become a learning organization, you need to treat learning as strategic. There should be budget, time, and recognition.

Note

One team I led at the FT had fortnightly “10% time” that gave people space to experiment. My colleague Sophie Caramigeas set the framework for this based on her experience running this elsewhere, with great results:

no one had to take part: if you didn’t have an itch you wanted to scratch, you could carry on with other work

you had to share what you were going to dig into ahead of time: this was to help people make a choice and focus on one thing rather than jumping around

everyone had to do a five minute show and tell the next week, with a total acceptance that sometimes this would be “I tried this and couldn’t get it working”

We got some great ideas for features, built tools that automated away pain, and found better technology options. We also quickly found out that some things were not going to work for us. One major benefit for me was that we avoided lots of discussion about whether to adopt alternatives, because people could try them out in 10% time. I consider the times where people came back and told me “X isn’t as good as what we have now” as equally valuable to the times where we found a great alternative, because we found out quickly.

Empowering

The authors of Accelerate say, “The goal is for your architecture to support the ability of teams to get their work done—​from design through to deployment—​without requiring high-band-width communication between teams.

That means you need teams to be autonomous and empowered, meaning they are able to work independently and make their own decisions.

This is about trust. The organization needs to trust that teams will make the right decisions, removing barriers between the team and production.

This doesn’t mean allowing a free-for-all. It means, for example, moving from needing sign off from a change management team for a change to go to production, to allowing teams to deploy to production while logging exactly what change happened, why and who applied it.

The people who will best understand the risk of making a particular change are the ones who are making it: the development team.

The next few chapters cover how to empower teams while maintaining the things the organization cares about, like quality, security and cost control.

Optimized for Change

The flexibility of your organization is important for a number of reasons.

As Conway pointed out in his article, “It is an article of faith among experienced system designers that given any system design, someone someday will find a better one to do the same job. In other words, it is misleading and incorrect to speak of the design for a specific job, unless this is understood in the context of space, time, knowledge, and technology. The humility which this belief should impose on system designers is the only appropriate posture for those who read history or consult their memories”.

In short, you are almost bound to get the system design wrong at first, so you should make sure your organization can handle that, which means having an organization that expects and can adapt to change. That means finding a way to reward managers for keeping their organization lean and flexible (easier said than done, since leaders are very reluctant to reduce the headcount of their organization when that will also reduce their area of influence!).

Beyond that though, modern organizations need to be able to change in response to business, technology and regulatory changes. Think about the introduction of GDPR regulations, the rise of machine language models, the move to the cloud. If you aren’t able to respond rapidly, your competitors might.

Westrum Model

Bringing all of this together, these characteristics of an organization are what the sociologist Ron Westrum describes as a “generative culture”, part of a model of different organization types he developed from research on human factors in system safety.

Westrum identified three types of organizational culture, based on the organization’s style of information processing:

Pathological (power oriented): information is hoarded for political reasons

Bureaucratic (rule oriented): information can languish because of barriers between departments

Generative (mission oriented): information flows well, and elicits prompt and appropriate responses

The research by DORA, documented in Accelerate, used survey questions based on Westrum’s model.

Westrum’s theory is that organizations with a better information flow function more effectively, and the DORA results bear that out. They show that “in organizations with a generative culture, people collaborate more effectively and there is a higher level of trust across the organization and up and down the hierarchy.”⁶

Generative culture:

shows high levels of cooperation

doesn’t punish messengers who bring bad news

shares risks through sharing responsibilities

encourages a break down of silos

doesn’t blame individuals for failure, but looks to learn from it

encourages innovation and experimentation

Individuals are “encouraged to speak up, think outside the box, and to act as fully conscious participants in a great cooperative enterprise.”⁷

You may be thinking that changing the culture of your organization is above your pay grade, but, while you will struggle to change culture without a leadership team that understands the need and supports the efforts, changing the way people work will impact your culture, and that is something we can all contribute to. Changing what people do changes the way they think.

Note

John Shook’s case study on introducing Toyota production and management systems into a US car plant, with transformational results, concludes that the way to change culture is to change what people do rather than attempting to change how they think. Accelerate found that applying specific technical and management practices—​continuous delivery and lean management—​can indeed predict a move towards a generative culture.

It is true of course that changing an organization is hard: it can take months or even years.

However, if you can introduce a practice or process that works to foster those behaviors associated with generative culture, you can kick off a positive change in culture, whatever your role.⁸

For example:

encourage people to work in cross-functional teams, and to collaborate with other teams for specific pieces of work, can increase cooperation and break down silos

blameless incident reviews can show people it is safe to share bad news, and encourage the organization to learn from mistakes

hack days, mini conferences and 10% time to work on experimentation, along with a willingness to try things out as a result, can foster a learning and innovation culture

having every team responsible for quality, reliability and security makes sure that those risks aren’t seen as the responsibility of a separate team (although having expert teams to advise and support in these aims is still a good idea—​and discussed further in the next few chapters)

Effective Teams

Teams are the fundamental unit of software delivery. This is a key point that Matthew Skelton and Manuel Pais make in Team Topologies. Effective software delivery comes from teams, not individuals, because we need to take in large amounts of information, solve complex problems and adapt rapidly to change. There is too much involved in modern software development for individuals to work effectively.

For a microservice-based architecture, what should those teams look like, how should they be grouped together, and how should they operate?

Motivated through Autonomy, Mastery and Purpose

For the kinds of work we do in software engineering teams—​creative, interesting and self-directed—​the best performing teams are those where motivation is intrinsic rather than extrinsic.

Extrinsic motivation comes from outside the team. The reward is separate to the task itself, for example, bonuses, pay rises, promotions.

Intrinsic motivation comes where someone wants to accomplish the task because that in itself provides value to them. For example, they see that it will have a positive impact.

Dan Pink’s book, Drive,⁹ is about what motivates us. He identifies three components: autonomy, mastery and purpose.

Do you have the freedom to decide what you are going to do and how? Do you have the opportunity to improve your skills and learn things? And do you feel you are working towards something worthwhile?

But what do these three motivations look like for a software development team? Essentially, an autonomous team is a team that can do all the major parts of their job without having to co-ordinate or get permission from anyone outside the team. That means being able to write, test and deploy code without needing to check in with people outside the team.

Having a cross-functional team is necessary, but not sufficient! I’ll return to autonomy in the next chapter, to discuss how an organization can enable autonomy in teams through the processes and practices they put in place.

Mastery begins with work that is neither too hard nor too easy, and depends on a continuous challenge. Each time you learn one thing, you have the opportunity to build on that.

Purpose is provided through the outcomes the team is working towards. They need to understand the shared goal, and see the value in it.

John Cutler’s blog post 12 Signs You’re Working in a Feature Factory describes what to me is the opposite type of team, ““just sitting in the factory, cranking out features, and sending them down the line.”

These are teams where:

developers work through a list of features prioritized by someone else, rather than targeting agreed outcomes, and they don’t see a connection to business outcomes

no one really knows whether a feature was “successful”. Once it ships, it doesn’t get iterated on and it is rarely acknowledged as a failure or removed

“resources” get moved from team to team and everyone is over-utilized, there is no slack

shipping features is the only measure that counts, incurring technical debt and making it hard to spend time on refactoring and improvements

This way of working is clearly demoralizing for teams, but it’s also bad for business! Teams that understand and are connected to the business goals will do a lot better.

Aligned to Business Domain

Almost always, you want your teams to be aligned to business domains, because you need the teams to be able to do the majority of their work without having to coordinate with another team. The reason for the almost there is that sometimes, there are reasons to set your boundaries according to something else, for example, technology or location. Finding the right boundaries is discussed later in this chapter.

Small

I’ve found that teams around 5-7 people in size work best. This fits with what Skelton and Pais say in Team Topologies (5-9, see p 32).

Smaller than this, and you face challenges to keep making progress when some people are out on holiday or off sick (and to support your systems, both in and out of hours). You will also find it hard to be truly cross-functional.

Larger than this, and communication within the team becomes a challenge (as I touched on in Chapter 3). Teams need to make group decisions all the time: should we use this framework? How will we tackle this feature? Which is our priority between these three things we want to do? Finding consensus when your team is large becomes difficult.

Where I’ve seen teams with 7 or more developers in, they’ve ended up dividing into subteams, whether those subteams are officially recognized or not. That can be OK, but don’t expect this to work like a single, coherent team.

I think, if you find it impossible to take your entire team out for lunch, because you can’t get a consensus on where to go, and can’t fit around a table - this is a sign your team is too big.

There will also be a challenge for leadership - once you get beyond 5 or 6, you are moving towards too many people for one person to manage effectively. I know plenty of people do manage more people, but if you want regular 1-2-1s and effective coaching AND technical leadership, that starts to be very hard.

Trust is also a factor in team size: critically important for high-functioning teams because those teams need to collaborate to deliver work. There is a limit to how much of delivering a feature can be worked on in parallel. This was identified way back in The Mythical Man-Month (Fred Brooks, 1975) with Fred Brook’s observation (now known as “Brook’s Law”) that adding more people to a late-running project just makes it later, although that hasn’t stopped this exact thing happening to me in the past!¹⁰

For software development teams, high trust within the team is a strong indicator of team performance. Google’s Project Aristotle researched the characteristics of high-performing teams and found that psychological safety—​feeling safe to take risks and be vulnerable in front of each other—​was far and away the most important dynamic within the team that they found.

Anthropologist Robin Dunbar’s research into the type and depth of relationships we can have with others shows a set of natural limits, marked by different levels of connection and trust. Dunbar says we can have a close relationship with a group of 5 people, share a level of deep trust with around 15, have a meaningful relationship with around 50 and an active contact, where we stay in touch at least once a year, with about 150 people.¹¹ This adds more support to a team size of around 5 people.

Going back to Conway’s Law, I will just note that these natural limits on team size put constraints onto the systems we design. As Team Topologies says, we need to design our systems to fit into the available cognitive load of the team. I’ll talk about this in the later parts of this chapter, on finding the right boundaries.

Cross Functional and T-shaped

To be able to take a feature through to production without input from any other team you need cross-functional teams, i.e. ones where all the necessary skills are inside the team: design, coding, testing, setting up infrastructure, deploying and operating the system.

When I first joined the FT, I spent maybe 80% of my time designing and writing code. That code would be handed off for testing by the QAs (quality assurance engineers), and then be packaged up and deployed by someone else. We also had an infrastructure engineering team who would set up servers, build pipelines etc. I might have to wait for a QA to be free to test my code, and then have to go back and fix any issues they found. I definitely had to wait for the code to be deployed as that happened at most once per month.

The move to cross-functional teams at the FT meant that engineers like me would take on infrastructure tasks like setting up servers and build pipelines. We’d also do a lot of the testing, via automated testing, and our QA teams moved to focus more on exploratory testing - good QAs overlapped a lot with business analysts in that they had a great understanding of the domain and the customers.

When we talk about cross-functional teams, it’s important to think about what skills can sensibly be in that team.

You should be able to deliver a feature to production the vast majority of the time using just the skills in your team. That means analysis, design, architecture, coding, testing and deployment with appropriate observability built in.

Given the size of a typical team, this means people have to be able to tackle more than one part of that journey to production. They don’t need to be experts in all of them: at the FT we spoke about having “T-shaped” engineers (see Figure 4-2), who have depth of experience and skill in one area, but experience across other parts of the process. This could be an infrastructure specialist who can also code, developers who can set up basic infrastructure and write automated tests, QAs who can analyze what is needed from a feature, write automated tests and deploy features to production.

Figure 4-2. Two different T-shaped engineers, with depth of experience in one area and breadth of competence across roles

The important thing is for engineers to be open to learning about skills outside their current role—​I actually like the idea of a “Broken Comb"-shaped person, where people are looking to learn skills in any area, so there may be multiple roles where they have significant expertise. This is great for maintaining flexibility within a team.

The more of a mix of interests and skills on the team, the more likely you are to find that people are able to both hone a skill and learn new skills from experts, giving them the motivation of mastery.

I’ll talk in Chapter 5 about the things an organization can do to help teams develop the full set of skills needed to be autonomous.

Strong Ownership

Autonomous teams need to have a strong level of ownership of the services they work on. Strong ownership means you make the decisions about how to structure your code, what level of testing is needed, when to release a change, and more.

Strong ownership makes it more likely that the code you write will be maintainable and supportable. If more than one team is making changes to the same part of the system, it isn’t clear who is responsible for the results of those changes. That has issues when things go wrong in production, but also for working out who needs to upgrade a dependency that’s been flagged as insecure. I’m going to talk about this a lot more in Chapter 8!

If the system is loosely coupled, needing a change from another team to achieve your aims should be fairly rare. Strong ownership means that when such a situation doesarise, you can’t just go ahead and make that change. There are lots of ways to approach this though. Collaborating—​even pairing—​on a code change is going to ensure that the team that owns the code properly understands the change and reduces the chance of unexpected effects. You can take a more asynchronous approach and have the owning team review the proposed code changes: this is most suitable for simple changes (but I’d probably argue that about most reviews via pull request!).

Strong ownership by a team does not mean strong ownership within a team. You really want the whole team to understand the systems they are building and to feel collective ownership of the code. Otherwise, you run the risk that some changes can’t be made because the “owner” is away that day.

Products Not Projects

A project-based approach to software development doesn’t allow for long-term purpose or strong ownership.

Projects run for a specified period of time with a specified outcome. After the project is complete, who owns the software?

Product-thinking assigns teams to a product with a lot more autonomy to decide what the outcome is that they are targeting. It provides that sense of purpose to the team.

A product isn’t over until the product has been decommissioned, although the number of people actually working on it will vary considerably over time.

Long Lived

Products not projects means teams need to be around for a while. They need to be long lived.

Teams should also be long lived because it takes time for a team to become effective. The people in the team need to get to know each other and develop ways of working. This is the “forming, norming, storming, performing” model.¹²

To expand:

forming - the early stages of getting to know the other team members. People are on their best behavior

storming - as real work starts, the team may show signs of conflict, over hierarchy and ways of working

norming - increasing acceptance of each other

performing - norms and roles established, the team can focus on their goals, performing well

Sustainable Cognitive Load

There is a limit to how much information anyone can maintain in their head, and the same applies for a team. Once you exceed the cognitive load of a team, they won’t be effective. Team members won’t feel they have the time to bring another member of the team up to speed on something new, so you’ll start to see bottlenecks where only one person knows about some part of their estate.

Where the team has more than one responsibility, excess cognitive load will mean lots of context switching as you move from urgent task to urgent task.

Finding the right boundaries between teams is key to keeping cognitive load sustainable and is discussed later in this chapter. No team should have to manage multiple complicated subsystems.

But cognitive load isn’t just about the features and products you are building. Team Topologies introduced the idea that total cognitive load matters for software development teams, and the discussion there goes beyond whether a team has too much work. It’s also about the types of work, both in terms of the impact on the team and the approaches you can take to mitigate overload.

There are multiple types of cognitive load:¹³

Germane: related to the task at hand. For example, understanding the domain problem space

Intrinsic: fundamental to the problem space. For example, the inherent difficulty of writing a programme in a particular language

Extrinsic: not inherent. For example, how you check in code, how you build it. It doesn’t improve your ability to solve the problem or build up your skills.

For intrinsic load, you can invest in training, set up guilds of practice and generally make it easier for people to build up their skills.

For extrinsic load, you can invest to remove it, for example through documentation and automation. Building a developer platform is one way to do that, and discussed in Chapter 5.

Reducing these types of cognitive load will allow teams to take on more of the germane cognitive load: the stuff that actually delivers business value.

High Trust and High Psychological Safety

In 2012, Google’s People Operations department carried out research into what made certain teams more effective than others.¹⁴

They found five key dynamics:

psychological safety - does the team feel safe?

dependability - can the team count on each other?

structure and clarity - do team members understand roles, plans and goals?

meaning of work - do team members feel the work is important to them?

impact of work - do team members feel the work matters?

Psychological safety means a team feels safe to take risks and be vulnerable in front of each other. This is another reason for long-lived teams, because this level of trust in each other takes time to develop.

Dependability is about being able to count on the other team members to work at a high quality and get things delivered.

These two things together tell you whether this is a team with a high level of trust in place.

The remaining three key dynamics all relate to having a clear purpose, both as a team and as a person. Those key motivational aspects from Dan Pink again!

Part of a Group

Once you get to a certain size, you will have too many teams for them all to be part of a single organization.

You can group teams together, but there is a natural size for that group. Dunbar’s research suggests that fifty people—​around 8-10 teams—​is the limit of people where we can have mutual trust. Beyond 150, we can’t even remember everyone’s capabilities.

Team Topologies suggests teams of five to eight people, in groupings of 50 to 150.

Why do we need these groups? One reason is about the flow of information. Meetings of team leads have the same constraints as meetings within a team: they won’t be effective if there are 20 people in the meeting.

The second reason is that you will probably need those groups to be able to manage some important tasks around supporting your code in production. It’s hard to maintain a 24/7 rota with 5 developers. I will talk about this more in Chapter 7 because there are challenges in navigating “You build it, you run it” while keeping team sizes down!

Optimizing for Flow

The team I’ve described in the previous section is autonomous, empowered, cross-functional and T-shaped. It should be able to work with a minimum of coordination or waiting on other teams.

But I also said that the team can’t be too large!

A five-person team can’t have expertise in all the things that matter—​such as accessibility, observability, and security—​without impacting their ability to deliver value to the business. And “full stack” has to stop somewhere! We can’t expect engineers to know everything down to the kernel.

Anything that needs to be done for every release of code, you probably need each team to be able to do. Beyond that, you don’t want all the team to have to solve the same problems, to have to build their own tools and abstractions to spin up servers and deploy code, to sort out their own observability tooling and write things to manage costs and compliance.

To be able to focus on meaningful work, to keep risk under control, product teams need to be able to look to others to make it easy for them to make progress (by reducing the amount of extrinsic cognitive load they are under) or for specific expertise. Maybe you need to add a WAF in front of your website, or you’re about to try your first serverless project. These are things that are best served by specialist teams that provide help and assistance.

Team Topologies is one of those books that really struck a chord with me: it provides powerful tools for thinking about your organization, focusing on what you can do to optimize for a fast flow of value.

The authors make the case --and I agree --that there are a limited number of fundamental types of team needed for building and running modern software systems. I recognised these from my own experience at the FT, developed in over a decade of doing devops and microservices. Team Topologies gave me the language to talk about this, and suggested places where we could go a little further in our own approach.¹⁵

There are four types of team mentioned in Team Topologies, but I’m only going to talk in detail about three of them.¹⁶

Stream-Aligned

These are the teams we’ve been talking about so far, the teams that are working on products.

A “stream” is a continuous flow of work, related to a specific business domain or organizational capability. Each team should be aligned to a different stream. This provides clarity of purpose and responsibility.

The majority of teams in your organization will be stream-aligned teams, and the purpose of the other team types is to reduce the burden on the stream-aligned teams.

Stream-aligned teams will be able to take a feature from initial exploration right through to production. To do this, they need to be able to:

design and architect

code

test

provision and deploy

ensure their applications are secure and monitored

run their code in production

Enabling

Enabling teams are specialists, skilled in particular domains, and their mission is to help stream-aligned teams to develop new capabilities. Stream-aligned teams should be focussed on delivering value to the business. They may not have time to learn about and practice new skills, but in any case, you don’t want every stream-aligned team to be doing this.

An enabling team can do the research, try things out, and then work with stream-aligned teams to help them adopt new technology or practices. They act as consultants, and like consultants, they bring in new ideas and work alongside teams for a limited period to transfer that expertise.

The enabling team should do the work to make sure any new tools comply with organizational policies or principles. The aim is to increase the autonomy of steam-aligned teams through growing their capabilities.

Examples of the kinds of technologies an enabling team may focus on include deployment, build and continuous delivery, observability, security engineering, or web components and design systems.

Enabling teams can reduce the intrinsic cognitive load that a team is under by sharing their skills and expertise.

Complicated Subsystem

Complicated subsystem teams are also specialists. The difference from enabling teams is that rather than exploring capabilities, they instead build and maintain specialist subsystems, for example that might be a system that does machine learning, or has a complicated set of algorithms.

Where many stream-aligned teams interact with this complex subsystem, it isn’t feasible to embed specialist knowledge in each of those teams, and so a separate team builds the subsystem, focussing on what those stream-aligned teams need.

Platform

The purpose of a platform team (in fact, this is almost always a group of teams and Team Topologies practitioners are moving towards using “platform group” or just “platform”) is to provide internal services that reduce the extrinsic cognitive load for stream-aligned teams, so that people don’t need to remember a set of commands or spend time finding out how to configure a database backup.

They are service providers and the services they build need to be easy to use, reliable, and fit for purpose. They should focus on creating services that stream-aligned teams can use without needed help. This means building well documented APIs and self-service tooling.

Platform teams can reduce the extrinsic cognitive load a team is under, so that people don’t need to remember a set of commands or spend time finding out how to configure a database backup.

A good platform should allow stream-aligned tools to do the right things, in the right way. It should reduce the complexity of the underlying systems without abstracting too much away, because teams may need access to the underlying functionality.

If you hide too much away, you run the risk of teams failing to adopt the platform and instead developing their own approach. I will talk about this later in the chapter where I describe the evolution of the organizational structure, including the platform, at the Financial Times.

The aim should also be to build just enough platform. Team Topologies calls this the thinnest viable platform, TVP. My experience is that this is absolutely the right instinct, and I think—​in agreement with Sam Newman—​that calling these “platform” teams tends to mean people focus on the platform.

When I look at the teams we had in the Engineering Enablement group at the FT, for many of them, I couldn’t say they were a platform team or an enabling team, because they used their expertise in particular domains to build services, to advise other teams, and to develop new capabilities.

An example: the Cloud Enablement team at the FT did all of the following:

managed the relationship with AWS and other cloud providers

set up account access for individual teams

wrote blueprints for using AWS-specific resources for common use cases, for example reading from a queue and writing to S3

wrote tools to control costs and risks, for example the “tagbot” that checked for resources that weren’t tagged appropriately and reported that to teams

They used their expertise to enable other teams to move faster—​note, they wrote blueprints that other teams could use to create their own resources, rather than creating a platform for building and deploying resources.

Just Enough Platform

Beware of building a platform when you don’t actually need it.

The aim is to reduce the cognitive load for other teams. You may be able to do that through documentation and picking the right vendors and tooling. I would start by focussing on how much you can hand off complexity to cloud providers and vendors. Not every company needs to run their own kubernetes clusters!

Finding the Right Boundaries

Once you have more than one team, you need to split up your system (since effective teams have sole ownership of the code they work on).

Really, you want to work on both the organizational design and the system architecture together, because of Conway’s Law. If you have 10 developers, do you need 3 teams or 2? It probably depends on how you can best split up the system. You are looking to align system boundaries to the capabilities of each individual team.

To do this effectively, you need a good understanding of the business, but you also need a high level of technical expertise. Ruth Malan says: “if we have managers deciding… which services will be built, by which teams, we implicitly have managers deciding on the system architecture”¹⁷

How can you decide where the boundaries are between teams?

At the highest level, that of groups, you can probably start to identify boundaries by looking at the customers, and at your stakeholders. At the Financial Times, it makes sense to have a group of developers working with the editorial department, who need tools for creating, managing, tagging and publishing different types of content. There is also a natural grouping of internal departments other than editorial who need software for their needs: HR, finance, marketing. Sometimes, though, there are multiple stakeholders for a particular area. At the FT, the group of developers working on the ft.com website need to consider editorial and marketing needs (among others) but their main focus is on FT subscribers. So the “natural” higher-level splits may be based around different stakeholders, different business capabilities, or a mix of both. They form business domains.

Below that highest level, i.e. when you are trying to split work across teams rather than across groups, there are several possible fault lines to use (Team Topologies calls these “fracture planes”).

For a microservices architecture, the most likely is to continue to use business domains. But there are other options in particular cases. For example, you may have boundaries as a result of technology constraints, team locations or compliance needs. I’ve mentioned the splits I think are most likely but there will be others, it depend on your organization. However, think carefully before choosing something other than the business domain—​are you doing it because it makes sense, or because it’s too hard to fix some other problem?

Domains

Nick Tune defines business domains as “Areas of expertise in which the business develops tools and capabilities (aka products) to support people who have a purpose (e.g. users and customers)”.

There are multiple ways you could split up any system, so a domain is basically an arbitrary boundary around some subset of concepts, for example orders and line items, or customers, or accounts.

You can generally break domains down into smaller subdomains. Our aim is to continue to break down domains until we have team-sized domains (or smaller!). Aligning a team to a specific business domain allows that team to have a clear and coherent purpose.

Getting the size right matters, because if there is too much work in the domain for the team, they will suffer from cognitive overload. If there is too little, you can get a team that is bored and lacks motivation.

You may want to assign multiple domains to a team, but be wary—​this will only work for simple domains, and even then, there is a danger that the team will effectively split into two rather than deal with the overhead of context switching.

Domain-driven design (DDD) is a well-established approach for identifying domain boundaries. It’s focussed on software design, often at a lower level than what we’re talking about now but tools like event storming, a workshop-based technique focussed on quickly modeling a business process as a series of domain events, can be really helpful in finding boundaries. DDD talks about "Bounded contexts“. These are areas within the system where people use the same language to talk about things. As soon as you find that the same term—​for example, “lead"--means different things to two stakeholders, you know that your model will need to be different, and you have identified separate domains.¹⁸

DDD may be too low level and technically focussed for discussions with key stakeholders and leadership teams. Matthew Skelton recently told me about Independent Service Heuristics (ISH) as an alternative. This provides a frame for thinking about boundaries by asking the questions: could you imagine this as something-as-a-service? Could you run this as a separate website?

The checklist of questions on the github repository for ISH provides a useful way to assess whether you have found a domain that “makes sense” to separate out; that is likely to be something you can align to a stream of work.

One question that stands out to me, because it’s a place that can cause a lot of pain if you get it wrong, is around data:

Is it possible to define clearly the input data (from other sources) that this thing needs?

Is the thing fairly independent from any data sources?

Are the sources internal (under our control, not external)?

Is the input data clean (not messy)?

Is the input data provided in a self-service way? Can the team consume the input data “as a service”?

In my view, it’s good to focus on data when finding domains, because you want to keep data that changes together in the same place, owned by the same team, to avoid distributed transactions.

Locations

Communication is such a central requirement for effective teams that anything that interferes with that will make it harder to succeed. As Martin Fowler observes, “Putting teams on separate floors of the same building is enough to significantly reduce communication. Putting teams in separate cities, and time zones, further gets in the way of regular conversation… Components developed in different time-zones needed to have a well-defined and limited interaction because their creators would not be able to talk easily” (for remote-first working, everyone is online, but time zones can still have a big impact because they make it difficult to collaborate within a team).

If you are split across different countries, cities, offices or even floors, try to make sure your domains are split along the same lines. I think this is really more about allocating the domains you identify through other fracture planes to teams based on location than on using location as a fracture plane itself.

Technologies

When I first worked in IT, with more monolithic systems, this was how we divided up work. We mostly had a three tier architecture, and three teams to match: frontend, backend and DBA teams. The frontend did not speak to the database, and the frontend team didn’t generally speak to the DBAs.

There is an obvious problem with this split, which is that the business functionality is spread across three layers, with any feature generally needing changes in all the layers, with the challenge of lining up work for three separate teams. People would find ways to avoid having to ask the other teams for a change, meaning you ended up with business logic in the wrong places. I’m so glad we don’t work like that anymore and tend to have all three areas represented within a cross-functional team!

This doesn’t mean there aren’t sometimes good reasons to divide a system up according to technology. That could be because of different performance needs, or because of third party systems you are interfacing with.

We found a natural split in the responsibilities of our Reliability Engineering team at the FT based on programming language. Websites at the FT are almost all written in Node.js, but the decision to use Prometheus as the basis of a monitoring stack nudged us to use go for several services that interfaced with it. In time, this became a fault line within the team.

Compliance

As I mentioned in Chapter 3, keeping good boundaries between parts of the system that are subject to regulatory or compliance requirements and the rest of the system can be a compelling reason to use microservices.

By separating domains in this way, you can limit how much of the system is subject to strict requirements. The team involved become experts, and they can work closely with legal and compliance stakeholders.

This kind of split makes a lot of sense to me. In practice, it tends to follow the business domains—​for example, payments functionality is both a different domain and an area with compliance needs around PCI (Payment card industry compliance requirements).

This can mesh with technology choices too. Rob Donovan and Ioana Creanga gave a fantastic talk at QCon London 2022 about how Starling Bank, a digital bank in the UK, built their own card processor for approving card payments.

The part I want to highlight here is the decision to do all their cryptographic validations—​for example, verifying the PIN supplied and checking that the transaction hasn’t been tampered with—​within a zero trust VPC, one where microservices cannot talk to each other unless access has been explicitly configured. The system is split into services in a way that minimizes the risk if someone gets access to a specific service. Sensitive information, like the PIN for a customer, is stored in services that will only respond with data if supplied with the correct access token, for example, and those services that store data are separate from the ones that use it to do verification.

Tolerance for Failure

It’s a great idea to consider the requirements around reliability etc for the capabilities you are building as part of the system, separating out the critical services from the good-to-have. That way, you are able to minimize the number of services that need to be multi region, that need to be supported 24/7, etc.

This is good for cost control: you don’t want to be running every service in two regions when some of them only really need to be in multiple availability zones. It also means the people doing out of hours support can focus on a smaller area that they need to understand.

Finally, it can allow different attitudes to the risk of a change. When you are prototyping and experimenting, you may feel more free to ship something new when the consequences of a mistake are not so high. I’m really not arguing for shipping less here, more for feeling more relaxed about changes for some areas.

As an example: the FT didn’t have code freezes when I was working there, but we would remind people when there were significant news events, for example the UK Budget or a US election, and when there were fewer people around, for example over the Christmas holiday period (which in the UK involves several public holidays and a high number of people out of office). Developers and teams could make their own assessment of risk. That might be different for changes to the homepage of the website versus an internal tool.

Frequency of Changes

The idea here is that you split the areas that change once a month from the things that change every day.

While I totally see the value in separating out parts of your system that have to go through an external approval and release process—​I’m thinking about apps and the app store—​from those where you release at will, I suspect that where you control the release process yourself, it would be better to focus on making it possible to release changes more frequently than to split off the slow parts.

Things get a lot easier if you develop a principle that code is never in an unreleasable state, by doing trunk-based development, or using very short lived branches; making use of feature flags; and generally assuming that anything you have on the main code branch could be released by anyone at any time.

Identifying When Boundaries are Wrong

Unless you already have a deep understanding of your domain, it’s likely you won’t get the boundaries right first time.

Note

Getting the boundaries wrong is particularly likely if the people deciding on those highest level boundaries are not doing that with the architecture in mind, even though from Conway’s Law they absolutely are making decisions that will be reflected in the architecture!

The first challenge is how to identify where a boundary is in the wrong place.

A good place to start is to ask people! A team with a good grasp of their purpose will be able to tell you what they own that doesn’t fit with their purpose, as well as things they really should own but don’t. You may end up with systems no one thinks they should own, or where several teams lay claim—​but at least you know about that and can make a call.

Note

With one of my teams at the FT, when we were set up, we inherited responsibility for a mixed bag of tools and 3rd party systems. We identified the core aims of the team, came up with a list of things that didn’t fit, and stuck little pictures of each system up on a wall. We then did our best to decommission them, move to SaaS options, or transfer them to other teams (with their agreement). Each success was marked with a ceremony where someone put a big cross through the picture. That felt pretty good!

Also, look for signs of pain or friction. They probably indicated there is more coupling than you expected. Are there services that you always change at the same time, that aren’t owned by the same teams? Or are there some services where releases often lead to production problems? These could be caused by a poorly-designed interface, where the internal implementation is leaking out (meaning it isn’t safe for the team to change it), but it could also be a sign that you have accidentally split a domain.

Is it hard to agree the model for certain concepts, because there isn’t clarity about what they represent? That could indicate two different domains are being combined.

And have multiple teams just gone ahead and created versions of the same thing? In my experience that can happen quite frequently when you have autonomous empowered teams, and is usually a sign of a lack of agreement over the domain boundaries.

The important thing is to realize that even if by some miracle you did get the boundaries right, they won’t stayright!

Things are going to change over time. You may need to split a domain, because it has got too big to be handled by the existing team. There may be changes in the technology available: for example, someone has created a commodity version of something your business had to implement themselves previously. Think of the impact the cloud had on how systems are built.

Also, the regulatory landscape may change, or the business may need to change strategy to make the most of new possibilities.

What I’m saying is, this is not something that gets done. It’s normal to need to move things around.

Case Study: How the Organizational Structure Evolved at the Financial Times

When I joined the FT, in 2011, we actually had two separate technology departments. Frustration at not being able to get their needs prioritized had led the product team to hire an entirely new team to rebuild the website.

This did not encourage high levels of cooperation or information flow.

Within the “main” IT department there were also several silos, with teams divided by function. We had infrastructure teams, operations teams, and development teams (which included developers, business analysts, QAs, scrum masters and product owners).

My colleagues were smart, friendly and understood the mission of the FT, but we were not empowered—​we had little input into technology choices, which were largely decided on by our architecture group, and we were a bit of a “feature factory”, as described earlier in this chapter.

While we worked in a superficially agile way, there wasn’t a lot of flexibility about what we tackled: work was funded via projects, meaning there would be an estimate based on identified a list of tasks that was “in scope”.

Infrastructure was set up by a central team and that could take months.

Developers would commit their code into source control, and once a month, package a set of changes up into a release, and that was the end of their involvement. Operations released code to production, and also did all the support for the live systems.

Developers didn’t really have to consider the way their code would run in production at all. Even monitoring of the servers was setup by another team.

Changing the Organizational Culture

The first big change we made was organizational, with the two departments combined under a single CIO. That immediately improved information flow and cooperation between teams.

Further changes to the culture did indeed, as suggested by John Shook in his case study, start with changing the things we did.

We were lucky, because we had a third party company who built the FT’s mobile applications. They did not have to follow the processes within the department, and they moved a lot faster than the rest of us. That proved to technology leadership and to our stakeholders that our processes were slowing us down and not making us more likely to release quality code.

We made changes to our processes, removing the change management board and moving the responsibility for releasing code and supporting it to our development teams.

We moved away from tracking numbers of incidents and paying bonuses based on those numbers, and that encouraged people to be open when something went wrong. We also moved incident management to be largely co-ordinated via an open Slack channel, meaning everyone could see what was going on.

We started holding hack days, encouraging people to experiment, and parts of the organization introduced 10% time for their teams. A few years later, we ran an internal tech conference for the first time, with feedback from attendees indicating that it broke down silos between teams as they heard about the cool things people were doing and the constraints they may be working under.

Changing Team Setup

We also changed our teams. This started when we moved integration engineers into each team. They would set up infrastructure for the team. Still manually at first, but now this was done in collaboration.

We ended up with our infrastructure teams changing to become platform and enabling teams. Our product development teams became fully cross-functional teams that built, tested, deployed and supported code running in production.

Teams began to be aligned with a particular domain, rather than being randomly named and all working across the whole codebase for their system. Funding was much more rarely associated with projects, mostly teams would be funded for the year to continue work on their products.

Quality became a concern for the whole team, not a separate QA function: we expected engineers to write automated tests and to care about this. Engineers also started to help with infrastructure tasks. Everyone moved towards more of a T-shaped set of skills.

Finding our Boundaries

The FT’s technology department went through a number of reorganizations as we found the right boundaries, but at the highest level, there were groups of teams associated to key stakeholders, with teams within those groups generally aligned to specific domains.

A few examples:

Customer Products: responsible for the website and apps used by customers of the FT. Key stakeholders were the FT’s editorial team, B2C marketing (selling the FT to individuals) and B2B marketing (selling the FT to businesses via group subscriptions)

Internal Products: responsible for any software systems required for FT staff to do their jobs, such as editorial tooling, email platforms, HR systems

FT Core: made up of teams supporting underlying platforms—​our subscription and billing functionality; data; and content publishing. These APIs were used by other teams at the FT and externally.

Engineering Enablement: every team that supported other engineering teams to do their job, by providing a platform or specialist expertise around security, DNS, etc.

Wrapping Up

All these changes took place over a number of years. The key changes that let us adopt microservices were, in my view:

removing barriers for releasing code to production and handing responsibility of that to teams

making teams cross-functional

allowing teams to make their own technology decisions

encouraging an open culture where we focussed on fixing problems and not on finding who to blame

I’ll return to the FT as a case study in later chapters to fill in more details!

Summary

To be successful in adopting a particular architectural style, you need to make sure you have the right kind of organization. This is because Conway’s Law shows that, in Ruth Malan’s words, “If the architecture of the system and the architecture of the organization are at odds, the architecture of the organization wins”¹⁹

For a microservices architecture, which needs to be loosely coupled, you need an organization that is loosely coupled too. It should support the ability of teams to get work done; to deliver a fast flow of changes.

Ron Westrum defined a generative culture as one where information flows well, and where individuals are “encouraged to speak up, think outside the box, and to act as fully conscious participants in a great cooperative enterprise”, and this is the type of culture you should be aiming for.

Teams that will be effective within that culture are ones that have a clear purpose and high levels of autonomy, meaning they can do most of their work without having to coordinate with other teams. They should be cross-functional, with the skills to build, release and support their own code, which means there should be 5-7 people in the team and the team should be long-lived and aligned to a product, rather than formed for a specific project.

They should be empowered to make their own decisions. Both empowerment and autonomy develop in a culture where there is trust and psychological safety.

Team Topologies identifies four team types that a modern software organization needs for fast flow of value:

stream-aligned: the majority of teams in the organization, working on streams of work aligned to a particular domain or capability

enabling: specialist teams with the expertise to help stream-aligned teams

complicated subsystem: specialists who build and maintain subsystems that require specialized knowledge

platform: teams providing internal services that reduce the extrinsic cognitive load for stream-aligned teams

Finding the boundaries between different parts of your estate—​essential in setting up those stream-aligned teams—​is important, but you likely won’t get it right first time, and you should expect to have to make changes as the surrounding context changes, for example, as new technology or new business focus areas appear.

Those boundaries are likely to be based on business domains, but could also be based around different locations, technologies, compliance requirements, tolerance for failure, or rate of change.

This chapter has discussed how to build teams that will be able to act autonomously and the kind of culture that supports that. The next chapter digs, in detail, into the processes and ways of working that allow those teams to act autonomously.

¹ Conway’s original paper is broader, talking about any kind of design and the organization producing it.

² Melvin E Conway, How Do Committees Invent? http://www.melconway.com/Home/Committees_Paper.html, published in Datamation magazine in April 1968.

³ Martin Fowler, Conway’s Law, https://martinfowler.com/bliki/ConwaysLaw.html, published October 2022

⁴ This seems to have been first mentioned by Jonny Leroy and Matt Simons from ThoughtWorks in the December 2010 issue of the Cutter IT Journal, and by 2014 it was featured in on the ThoughtWorks Tech Radar

⁵ https://verraes.net/2022/05/conways-law-vs-rigid-designs/ Conway’s Law Doesn’t Apply to Rigid Designs

⁶ Accelerate, chapter 3

⁷ Westrum R, A typology of organisational cultures

⁸ Google have more detail on how to foster a generative culture in their online documentation around DevOps capabilities

⁹ Daniel Pink, Drive: The Surprising Truth About What Motivates Us (Edinburgh: Canongate, 2009)

¹⁰ Fred Brooks, The Mythical Man-Month: Essays on Software Engineering (Boston: Addison-Wesley, 1995).

¹¹ Robin Dunbar, “Dunbar’s Number”, New Scientist, https://www.newscientist.com/definition/dunbars-number/

¹² This comes from Tuckmen’s group-development model

¹³ Characterized by psychologist John Sweller and discussed in Team Topologies

¹⁴ Five Keys to a Successful Google Team, November 2015

¹⁵ I really recommend reading Team Topologies if you have any interest in organizational structure and optimizing for a fast flow from idea to production. Matthew Skelton and Manuel Pais, Team Topologies 2nd edition (Portland, OR: IT Revolution, 2019)

¹⁶ The fourth type of team, the Complicated subsystem team, is specified as optional, and I don’t feel that I’ve seen it in action at the FT

¹⁷ As quoted in Team Topologies, for a blog post no longer online, but available via the web archive

¹⁸ Honestly, I can’t find anyone who has written a truly succinct description of a bounded context, so this is my attempt. I found Martin’s relatively short blog post https://martinfowler.com/bliki/BoundedContext.html helped me to understand this useful concept.

¹⁹ In an archived blog post

Chapter 5. How to Support Team Autonomy

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 5th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at jleonard@oreilly.com.

In the last chapter, I discussed what makes for an effective team and concluded that a high degree of autonomy is essential.

Chapter 4 focussed on how to set up a team so it will be able to be autonomous, for example, making sure that teams are long-lived, cross-functional, the right size, and given responsibility for a stream of work that fits into their cognitive capacity. This chapter focuses on how to set up your ways of working to allow that team to be autonomous. It’s no good having a team with all the right skills if you have processes that require them to go through multiple gates to get work scheduled or to get changes to production.

Every time that a piece of work goes into a queue, waiting for someone to pick it up, you are having a negative impact on the flow of work and so the flow of business value. Sometimes, this is a literal queue, for example a ticket opened that someone needs to access. Other times, it’s a metaphorical queue: maybe you’re waiting for a meeting to happen before the work can progress.

Where the queue is within a single team, you can set priorities to maximize that flow: for example, you can decide that developers do their own testing, or their own deployment to production, rather than waiting for someone else to do that. Where the queue is outside your team, you will likely find it hard to align your priorities, and that means waiting for someone else to do something. Examples from my own experience include waiting for a change advisory board (CAB) to meet once a week to OK releases to test; waiting several months for a server to be set up and configured for a new application; waiting for the monthly production release slot.

If you move to a microservices architecture but you still have these kinds of dependencies, you will struggle to get the benefits. After all, for a fast flow of value, you need your teams to be able to release a new feature to production with minimal coordination with anyone outside the team.

You need to minimize the times where you need to communicate with others in order to get your own work done.

This doesn’t mean all communication is bad! I’d encourage you to talk openly about what you’ve done and plan to do, because it means someone else in your org facing the same problem can benefit from your thinking and your work, but try to do this in a way that lets people consume the information in their own time.

Modern chat tools make it easy to communicate, but if teams assume that everyone will read every message in their channel, that’s bad news for flow and productivity. Similarly, information shared in a town hall meeting should also be shared in other ways, so that people can find it when they need it.

Information flow is a key marker of that generative Westrum culture I discussed in Chapter 4 as the way an organization should work. But the key thing is that the necessary information gets to the “right person in the right form and in the right time frame”.¹

It’s likely that mostly, that is about people being able to find the information when they actually need it. For that to work, someone will have to have made the information available, in a format that can easily be absorbed, and in a place where that person will know to look.

This easy access to the information you need is key to being able to act autonomously. This chapter digs into how to support the autonomy of teams in your organization through the ways that you work.

I’ll also talk about effective ways for teams to interact, again drawing on the ideas laid out by Matthew Skelton and Manuel Pais in their book Team Topologies that I explored a bit in Chapter 4.

And finally, we’ll discuss the responsibilities of autonomous teams. It’s not all fun and games, it’s not a free-for-all. The autonomy exists to support the business (but it also makes working a lot more enjoyable, in my experience).

Interaction Styles

More communication is not necessarily a good thing.²

Matthew Skelton and Manuel Pais, Team Topologies

Communication should happen all the time within a team. People should be collaborating: reviewing design documents and code; talking about how they are going to deliver a particular piece of work; pairing or ensemble programming. If that isn’t happening, the danger is that you develop silos within the team, where no-one understands all of the domain the team is responsible for.

The clear benefit of communication within a team can lead us to think all communication is good, but that isn’t true.

Between teams, high-bandwidth and synchronous communication should be rare, because it couples the teams together.³

The time to work closely together is when there is the most uncertainty, for example when you are in a period of discovery: trying something new, or exploring which option to take.

When you are doing more routine work, you should be aiming for lower bandwidth modes of interacting.

Team Topologies describes three types of interaction between teams and advises organizations to be very conscious of choosing an interaction style. I have found this a very useful guide when considering how two teams should work together. In general, I’d advise you to choose the one with the least requirement for synchronous interactions between teams that will still allow you to be successful.

Domain-driven design (DDD) also has a lot to say about interaction styles, documenting these as "context maps“. There is a fair degree of overlap with TT, which is reassuring! For a nice description, see Vlad Khononov’s report.⁴

Here, I’m going to talk about the Team Topologies_ categorization, while calling out the related DDD terminology.

Collaboration

Collaboration involves working closely together with another team. It requires a pretty big commitment, so you will probably do this for high value pieces of work, and usually as a way of rapidly exploring and innovating at the boundary between the two teams.

You will generally collaborate for a relatively short period of time, because working together for a longer period will blur the boundaries: you don’t want to merge the teams. It is also a fairly high-intensity way of working, with a high cognitive load. Both teams should be learning from each other.

This looks like the DDD context-mapping pattern of “Partnership”. Where you have two teams that are closely aligned, it’s possible to coordinate and adapt very flexibly: one team can notify the other that they need to make a change to their API and the two teams can work together to find the most appropriate solution.

I have seen collaboration done in the form of a feature team, where some people from two different teams join together for a particular purpose. The biggest challenges I’ve seen related to feature teams are firstly, making sure you understand who will own the things being built. It’s easy to find services that are in between and orphaned after the feature team disbands. And secondly, to make sure the team focuses on the short term gains. Don’t spend a lot of time discussing your ways of working if this is only going to be a short-term thing (this means encouraging people to go with the flow!).

Note

A form of collaboration that really worked for me at the FT was a secondment process. We did this when we were first forming teams working on enabling observability and insights (platform and enabling teams). We got a commitment from our customer teams, the stream-aligned teams, to send people to us on secondment. This was generally for 3 months, which aligned to our OKR process and was long enough for people to be able to take on a significant piece of work.

There were so many benefits of this, although the logistics could be a challenge. Firstly, we had a direct line to our customers, giving us a good idea if the things we were working on were going to be valuable and adopted. Secondly, we found that our secondees were our best advocates once they went back to their teams. They knew our tools and they would share that knowledge. They’d also propose changes and improvements.

DDD has another context-mapping pattern in this collaboration area that I suspect is most likely to work for two teams aligned in a single larger group. This is “shared kernel”. Here, the contract between the teams is defined by extracting shared code that both can use. Any changes to that shared code will apply everywhere and need to be released together to avoid inconsistencies so it is best used for code that is stable, and where the cost of duplicating the code is higher than the cost of keeping it in sync.

X-as-a-Service

This is an interaction style that minimizes the need to collaborate and focuses on supporting self-service. One team provides a service and the other team consumes it.

In general, platform teams should do most of their work in this style: building self-service tools and creating great documentation.

It is important that the team providing the service has a focus on their user. They need to support the service for the long term: no unexpected or breaking changes. They need to understand the user’s needs—​product management skills are a great asset here.

I will talk about this more in Chapter 6, which discusses engineering enablement and building a developer platform.

DDD has three patterns to use where there is a customer-supplier relationship between teams. There is normally an imbalance of power—​one team has more say over what the contract between them should look like.

“Conformist” is a “like it or lump it” pattern. The supplier provides the contract and the consumer can use it “as-is”. This is how relationships with third parties tend to work, but it can also happen internally when there are many different consumers. It becomes costly to make changes for any one consumer. There is nuance of course—​at the FT, in the Content API team, we asked consumers to ignore fields they didn’t care about, meaning we felt quite relaxed to add additional fields to an API response.

“Anti-corruption layer” is a defensive approach for a consumer, where they convert things at the boundary into something they can use. The aim is to avoid being impacted within their boundary by things they don’t care about (so, for example, they could strip out any fields they don’t use).

“Open host” is for where the consumers have the power. This is where the supplier surfaces the information in the format and language the consumer needs.

Thinking through these three options is useful when planning collaboration. I wish I’d had a conversation about these when working to provide a Content API, that clarity would have reduced the number of conversations about what the API should look like!

Facilitating

Where one team has specific expertise, they can take on a facilitating style where their aim is to help and transfer knowledge to the other team.

This is the way enabling teams will often work. They can invest the time to solve problems that are affecting multiple stream-aligned teams. Often, this relates to common functionality like security engineering tools, observability, DNS, CDNs, or cloud provider offerings. A central cloud team can follow the new announcements and make sure people are aware of improvements such as new instance types or support for different programming languages.

It’s important for enabling teams to check in with stream-aligned teams often, to find out what the real problems are, rather than making those decisions themselves. I’ve seen teams invest time in a solution they liked, but that no stream-aligned team wanted to adopt.

Separate Ways

The last choice is, of course, not to interact. There may be political reasons for this, where teams are not able or willing to collaborate, or it may be a pragmatic choice that duplication of effort, and solving the same problem in different ways, is the most cost-effective way to solve a particular problem.

Probably best to be explicit when this choice is made. In two years time, I will bet someone asks why there are two versions of the same thing. Great if you can point to a document that explains the reasons for that decision! See the section later in this chapter on Light Touch Governance for a discussion of ways to document this type of decision.

Ways of Working that Support Autonomy

You can set up a team with all the necessary skills, but they can’t be autonomous unless you align the ways you work to support that autonomy.

This means a focus on every place where people are having to wait for someone to do something, and trying to remove that gate.

It also means trusting your teams to do the right things, and helping them to know what those right things are.

Specify the Direction Not How to Get There

I’ve defined autonomy earlier in the book as “the freedom to decide what you are going to do and how”, but there are boundaries to that.

Teams don’t have complete freedom. The obvious first constraint is that they are aligned to a particular domain. You can’t have a team deciding to build something in a completely different domain, particularly if that is owned by another team!⁵

Even within their domain, there should be something providing a direction of travel. For a stream-aligned team, that is likely to come from the business or product strategy. For a platform or enabling team, there will be some constraints from the tech strategy: for example, if the tech strategy says the organization is going to be all-in on serverless, then you don’t want an autonomous team spending time on tools for upgrading servers.

Clarity is effective. As found in John Shook’s case study on successfully introducing Toyota production and management systems into a US car plant, “what changed the culture was giving employees the means by which they could successfully do their jobs. It was communication clearly to employees what their jobs were…”

If you are working on a team, you should expect some direction on what outcomes your team is targeting. But you should have the freedom on how you approach those.

Agreeing and Aligning on Outcomes

But what about the places where you need to work with another team? For example, where you can’t deliver a new feature without collaboration, or where you want to build something that needs new platform features.

You need to agree and align on outcomes. Jason Yip has a great post on “aligned autonomy”', where he identifies several challenges.

The first is the “air sandwich”. This is where there is a clear high-level vision and people understand their day-to-day work, but they don’t see how the two connect.

“We want X, you figure it out” can be too large a gap. You need people in the middle who can translate X into the context for those at the next level.

At the FT, we had a high-level goal at one point of “March to 1 million” (meaning subscribers numbers). It was down to me, as lead on the content publishing platform, to find a way to link that to our work and communicate, for example, the importance of APIs that were easy for product teams to use to build new and enticing products, alongside other concerns like accurate, timely publishing and a highly reliable software stack.

The second challenge is to make sure people hear the message they need to align to.

Too much information gets stored in a spreadsheet. Jason describes these as “information refrigerators”. You have to go and get the information from them (as opposed to “information radiators”, which share information without the receiver having to go through a lot of effort).

At the FT we used OKRs (objectives and key results) to align teams across the technology department. These were for a long time stored in spreadsheets, but there were regular meetings to review progress which I guess is slightly better. We did eventually get an OKR tool which made it much easier to link objectives and to be reminded regularly to look at progress or to keep an eye on objectives other teams were working towards.

In general, as a technical director, I would aim to agree objectives that aligned to higher level goals of the department or the business, but I would aim for my teams to define their own key results, that would then be agreed with my peers and the leadership team. And how to deliver those key results was also completely up to the teams.

One more comment about making sure people hear the message, is that while talking about goals in an All Hands meeting is good, you cannot assume everyone is there, or paying attention.

As @lenagunn (Lena Gunn) on twitter pointed out, in a now deleted tweet that really resonated with me: “one thing that happens in groups of 50+ is messages never really get to everyone, unless you make an extraordinary effort. if you say “you can request a free popsicle at any time” on email, group meetings, reminders, signs all over the place, someone still doesn’t know"

You need to repeat yourself, via different routes, until you personally are bored of saying it. This is true of a lot of things you do as a leader. Being told you are repetitive probably means you are getting it about right!

Light Touch Governance

You have to move away from processes that mean people have to ask permission or get sign off.

At the FT, when I first worked there we had a regular Change Advisory Board meeting. When I had code to release, I would need to go to the meeting, explain what the change was, and get agreement for it to go into the staging environment and then to production. When you have many people trying to make changes to the same codebase, this does make some sense: you do catch potential conflicts. But the way to fix this is to move to smaller, independent, frequent changes. To do that, we had to remove the shared staging environment, and we had to dismantle the Change Advisory Board.

We let the people with the most context about a change make the decision on when it was ready to go to production, and we let them go ahead and do that at will.

We still had a small friction there though. Every change had to be logged, along with information on how to undo the change. I think this is a very valuable practice (and will talk about it in Chapter 12), but logging the changes was being done via a GUI, so it took time. I particularly enjoyed having to enter in the timestamp for when I would start and finish the change I was about to make, then finding the start timestamp was in the past so the change got rejected.

We automated this away, creating a change API, but crucially, this wasn’t a like-for-like change. We negotiated a different approach, where rather than open a change request, make the change, then close the change request, we only logged that a change was about to be made. And we did that as part of the build and release pipeline, so it wasn’t possible to look at the changes scheduled for the day, for example.

We also removed the requirement to document how to roll things back. We could do this because the people releasing the change were the people who had written the code. They could decide whether to roll back to the previous release or create an updated release on the fly, if the change failed in some way.

There are two challenges about these sorts of changes. Firstly, for every gate on the way to production, you have people who are the gatekeepers. That could be testers, operations, security. You are removing some of their responsibilities and you need to convince them that this is an improvement, and is going to let them focus on higher value stuff.

The second is to provide guidance to engineering teams about what the expectations are around quality, security etc. They need to be able to evaluate whether code is ready to release. These guardrails and how to go about setting them up are discussed in Chapters 6 and 10.

Trust But Verify

Trust teams to take action, but verify that they did or are doing the right thing.

Some examples:

rather than getting up-front sign off to do something, audit that it took place. This can be an audit that an expected thing happened, or that an unexpected thing happened: an example at the FT was an automated alert in the main operational support channel in Slack if anyone logged in to the root AWS account. This would generally be responded to by the person making a root-level change (one of a small team of people), but meant that even that small team couldn’t do this without everyone knowing!

allow access to APIs through self-sign up, but throttle heavily, and check what people are sending in through a pre-processing validation step.

Agreeing and Aligning on Technology

Teams don’t need—​and generally don’t want—​total autonomy around the technology they choose.

Normally, the place where freedom to choose technology matters is for business differentiators. An example is where a new product or feature would be far easier to build using a new type of data store.

For most things, teams want to get to the point of solving business problems as quickly as possible.

This means there are two things you need in your organization. Firstly, you need people to be able to find out what is already available, and self-serve to use it. This is going to be discussed fully in Chapter 6, but the minimum here is to provide self-service access, document things well, and make them discoverable.

The second thing is to allow people to share where they are considering a new technology, so that the decision is informed. It’s sometimes the case that there already is another team who have used this technology, or evaluated it and found some blocking issues. There should be a forum where you can find that out.

Also, it’s often the case that one team needs something, but other teams can see they would also benefit. That means the first team is actually making decisions that will impact the whole technology organization.

That means there needs to be a way to share plans and get feedback. But this can’t be top down.

When I was first at the FT we had an architecture group who evaluated and made recommendations about which technologies we should use, e.g. “Use Kafka for messaging”, “Use this database for resilience, this one for less resilient systems”. The issues around that were the desire to limit the options: in practice, that could mean many teams compromising on something that didn’t quite meet their needs.

Note

As an example. Kafka was not, in fact, the right messaging solution for my Content publishing API team, where we had very few messages (the FT doesn’t publish vast volumes of content!). We didn’t need the scalability, or to partition topics. Kafka was complex, there was a steep learning curve. It was something the team struggled to support in production. But we had it because of the evaluation made centrally.

We moved towards something we called the Tech Governance Group (TGG), which I will talk about more in Chapter 10. In brief, it was a forum to raise a brief proposal of technology decisions that would impact other teams. It spread understanding of what was going on across the technology department, meaning fewer nasty surprises where a team found they were duplicating something some other team had already done.

The Role of the Individual Contributor

With a lot of autonomy within teams, you need someone who has a view that goes across teams and even groups; who can pick up on unnecessary duplication and decisions that are going to cause conflict down the line.

With the move to microservices at the FT, we moved from Architects to Principal Engineers. Both are individual contributor roles but the difference is that Principal Engineers, at least at the FT, are more hands on and are about more than architectural decisions, although that is a key thing for them to focus on.

As Sam Newman says in Microservices, this is about “surveying the landscape”, and the role is an enabling one, not a directive one.

It is a tough role, because influencing without being a manager is difficult, but my view is it is essential in a complex modern software environment. If you want to know more about individual contributors, I recommend Tanya Reilly’s book, The Staff Engineer’s Path,⁶ and Will Larson’s profiles of people who have reach staff-plus level.⁷

Minimum Viable Competencies

There are a set of competencies that you need in every cross-functional team, to allow them to work independently to build code and release it to production.

The precise list will depend on your organization, but I would want to make sure I had at least one person thinking about:

the product being built, whether that is a product manager or a technical leader

the architecture. Usually, this is a senior engineer or tech lead

design. This might be a designer, it might be a front end engineer making use of components and style guides put together by a central team

site reliability, i.e. focussing on building a resilient, observable service

You can support this in two ways. Firstly, by making sure there are experts within the organization who can help people develop the skills and access training.

Secondly, by being super clear on what the minimum expectations are for cross-cutting concerns such as tracing, observability, operability, and security posture. This is essential to make sure teams are good citizens of shared responsibilities, so that for example, the whole system is unsupportable because one team in the middle of a workflow chain doesn’t properly implement tracing.

There will be a few places where you have to mandate a particular approach, but in most cases, this is best handled through guardrails: guidance on what makes for a “production-ready” system. This could be around cost, operability, out of hours support agreements, etc. I will talk about this more in Chapter 6.

Learning

When you have cross-functional teams rather than teams linked to expertise (e.g. QAs), you need to find other ways to maintain and build skills in particular areas.

This means forming guilds or communities of practice, which are a shared space for particular skills, separate from team hierarchy.

It also means putting time and money aside for learning.

That could be external: for example, sending people on training, or to conferences. This is a great way to learn from the experiences of others, and to build a network of people who can answer the question “we are thinking of doing this thing, how did it work out for you?”

You can also get a lot of benefit from running things internally.

At the FT, I helped organize a Tech talks program, where we ran a mix of internal talks and ones from external invited speakers. We also set up an internal tech conference, now in its 7th year, and discussed in Internal Tech Conferences by Victoria Morgan-Smith, from the FT, and Matthew Skelton.⁸

These are great ways to influence behavior in your teams. I found it particularly effective to invite an external speaker to talk about things that mattered to me. They automatically have more authority—​even better if they are known in the industry—​and you’ll probably find people telling you “Charity said this was important” for the next year or two.⁹

Responsibilities of Autonomous Teams

“Everything broken all the time” is not the kind of autonomy we’re looking for.¹⁰

Jason Yip, Product development guiding principle blog post

It would be a mistake to think that “autonomy” means “I can do whatever I want”. The autonomy should be framed within the context of your team’s goals.

Jason Yip, in his write up of his thoughts on the Spotify model, says that autonomy should be framed as a responsibility not a benefit.

Teams should define autonomy as: “we are both authorised and expected to do what is necessary to accomplish the mission”.

What kinds of responsibilities does an autonomous team have? This will be a theme throughout this book, but let me give an overview here.

Active Ownership

Teams should actively own their code, which means doing the boring stuff. It is also a long-term commitment: code isn’t “done” until it is no longer running in production AND all the resources have been cleaned up.

One principle we had at the FT was that every service in production had to be owned. Sometimes, that meant assigning an owning team to a service that had effectively been abandoned. That sucks, but as a responsible team, you should be able to recognise that you are the best of a set of bad options for owning something!

Active ownership means:

supporting your code in production

owning decisions made by the team, even if that wasn’t you - for example, if the team decided to roll their own solution, they own that until they move off it, or manage to persuade a central team to adopt it. Example from the FT of trying to throw stuff back. Also example of changing your mind and reducing work for the team through SaaS choices etc

maintenance and patching - unless someone else is doing that for you

Communication and Cooperation

There is a lot of value in sharing information—​after all, good information flow is a hallmark of a generative organizational culture. It seems like this is in conflict with the idea that we should be working to reduce those touchpoints with other teams but the focus should be on reducing synchronous communication to only the places where it is needed.

A team should still invest in sharing information, and be open to collaborating for the places where that is the best option.

Teams should:

share what they are working on or planning to work on. I have occasionally worked with teams who developed an “ask for forgiveness not permission” approach, and sometimes the impact of that was considerable, and not positive.

look for a chance to learn from other team’s experiences. This cross pollination is highly valuable.

How you do this matters, because it needs to be easy to share and easy to find. It’s worth spending some time thinking about the frameworks to enable communication and cooperation.

Compliance with Standards

Teams have a responsibility to meet standards that the organization has. Software development is about more than writing and deploying code and the team should feel a responsibility for things like security, operability, accessibility.

The organization can help by being clear on what it means to be ready for production, by creating clear guardrails, policies and standards.

Create a “Team API”

A team should make it clear how to interact with them. They should create a “Team API”, as suggested in Team Topologies.

It should include everything another team needs to know about your team, it should be easy to find, and it should reduce the amount of routine responding to requests that your team has to do! As with programming APIs, you should focus on making it usable and useful for other teams.

In my experience, you need to include:

what the team owns, with links to the code repositories, github teams, etc

how to use the team’s services

how best to communicate with the team

the principles and ways of working the team follow

what the team is working on now, and up next

who’s in the team

At the FT, teams tended to make their own decisions about what to use for documentation. However, if I want to use a particular service, I shouldn’t have to know what team owns it to be able to find information. This is one of the places where autonomy doesn’t help you.

We invested in a couple of things that helped.

Firstly, we had our Biz Ops system registry, built by our engineering enablement teams. This listed every service, connected to a graph of other information: the team that owned the service, the groups they were part of, the communication channels to use to reach them. This allowed you to search for a service and find out what slack channel to use to talk to the owners, or to find the codebase.

We also had an API gateway and an expectation that any API being consumed by someone outside your domain should be surfaced through that gateway. This gave a central place to look for APIs. With help from the team who managed the API gateway, we also connected this into the graph of information in Biz Ops, by tagging APIs with the related system code, and having APIs owned by teams, rather than individuals.

Case Study: Supporting Autonomous Teams at the Financial Times

As described in the case study in Chapter 4, when I joined the FT, things were manual and siloed. Teams could not work autonomously. Getting things done involved a lot of hand offs between teams.

For example, we had shared staging environments and needed sign off from a change advisory board to release code into the staging environment (weekly) and then to production (monthly).

While we had a continuous integration server, it had been configured so that there were multiple sign-off steps. Another developer would need to sign this off—​but of course, could not have knowledge of everything in the release, making this a literal box ticking exercise. QAs would also have to sign off the release.

We needed to look at the processes and ways of working that we had in place so that the changes to our organizational culture and team setup could pay off.

Light touch governance

We removed the change advisory board (as already discussed in this chapter), and the highly custom-configured automated build tool that needed a chain of sign-offs. Development teams could decide when a release was ready to go out. We saw a big reduction in the number of failing releases, which convinced people that trusting teams to do the right thing was safe.

Agreeing and aligning

We encouraged teams to try out new technology, removing the central list of architect-approved software, and making the central platform optional. However, we provided guardrails that defined our expectations around quality, security, etc., and we expected teams to make their case for bringing in new technology via a lightweight process that was more about conversations than proposal writing (the Tech Governance Group, which will be discussed more in Chapter 10).

X-as-a-service

We introduced infrastructure as code and automation, and encouraged our platform and enabling teams to focus on providing self-service access wherever possible, to cut down on the need for teams to wait for something to be done. Generally that meant a move to auditing activity and flagging issues, over signing off in advance.

These changes allowed the FT to make several orders of magnitude more changes: from 12 releases of the web site a year to many more than that every day.

Summary

It isn’t enough to setup long-lived, cross-functional teams aligned to domains, unless you are willing to empower them, and remove the barriers that stop them from being autonomous.

Mostly, that is about removing the need to wait for someone else to do something. Team Topologies identifies three interaction styles for teams that need to work together:

Collaboration: working closely together with another team, reserve this for rapidly exploring and innovating at a boundary between teams

X-as-a-Service: one team provides a service, the other consumes it. Minimizes the need to collaborate and coordinate

Facilitating: a team with specific expertise aims to help and transfer knowledge to another team

Adopting these ways of interacting minimizes one team waiting for another. But what about our processes and ways of working?

As an organization, you should focus on every place where people are having to wait for someone to do something, and try to remove that. You need to trust your team to do the right thing, and to help them know what those right things are.

This requires a focus on what you want to achieve, rather than how you are going to achieve it. Governance should be light touch, and wherever possible should rely on auditing what happened rather than requiring sign off ahead of it happening.

This freedom can be exhilarating for teams, but it comes with associated responsibilities. They need to take active ownership of the things they build, to be open about what they are doing, and to ensure they comply with standards. They should make sure they are easy to work with by defining a Team API that covers how to interact with them for the best outcomes. Like other APIs, the focus should be on making something usable and useful for other teams.

Autonomous teams get to make their own choices, but those choices should be focussed on the key differentiators within their domains. In the next chapter, I am going to look at how platform and enabling teams can create a paved road for the things every team needs to do: most teams will use the road, a few will need to go off-road, which will be possible, but more work.

¹ From Westrum’s original article, available online via the BMJ at https://qualitysafety.bmj.com/content/13/suppl_2/ii22

² Quote from Team Topologies, by Matthew Skelton and Manuel Pais (Portland, OR: IT Revolution, 2019). This chapter is heavily influenced by this book and I recommend reading the original!

³ Broadcasting information in a way where people can consume it asynchronously is different. This is valuable, provided there are ways for people to find the information they need!

⁴ Vlad Khonov, What is Domain-Driven Design? (Sebastopol: O’Reilly, 2019).

⁵ However, I’ve seen this happen. It was carnage.

⁶ “Staff” or “Staff+” seem to have become shorthand for “senior individual contributor”

⁷ I hear Will’s Staff Engineering book is excellent but I have not yet read it!

⁸ Victoria Morgan-Smith and Matthew Skelton, Internal Tech Conferences (Leeds: Conflux, 2019)

⁹ Name not chosen at random at all. Having Charity Majors come to speak about observability had a noticeable and positive impact at the FT!

¹⁰ https://jchyip.medium.com/guiding-principle-autonomy-is-enabled-by-clear-intent-and-technical-excellence-e3422572d204

About the Author

Sarah Wells is a technology leader, consultant and conference speaker with a focus on microservices, engineering enablement, observability and devops. She has over 20 years experience as a developer, principal engineer, and tech director across product, platform, SRE, and devops teams.

Sarah spent over a decade at the Financial Times, leading as it transformed into a true cloud native organization, releasing code 250 times as often and embracing autonomous empowered teams.