Scaling AI Adoption in Engineering

A Practical Framework for Leaders

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

Peter Bell

Scaling AI Adoption in Engineering

by Peter Bell

Copyright © 2026 Gather.dev, Inc. All rights reserved.

Published by O’Reilly Media, Inc., 141 Stony Circle, Suite 195, Santa Rosa, CA 95401.

O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (https://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com .

Acquisitions Editor: David Michelson

Development Editor: Sarah Grey

Production Editor: Elizabeth Faerm

Copyeditor: TO COME

Proofreader: TO COME

Indexer: TO COME

Cover Designer: Susan Brown

Cover Illustrator: TO COME

Interior Designer: David Futato

Interior Illustrator: Kate Dullea

July 2026: First Edition

Revision History for the Early Release

2025-11-17: First Release

See https://oreilly.com/catalog/errata.csp?isbn=9798341666856 for release details.

The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Scaling AI Adoption in Engineering, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc.

The views expressed in this work are those of the author and do not represent the publisher’s views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

979-8-341-66681-8

[LSI]

Brief Table of Contents (Not Yet Final)

Preface (available)

Part 1: Laying the Groundwork for the AI-Powered Engineering Org (unavailable)

Chapter 1: A Tale of Two Orgs (available)

Chapter 2: AI as an Amplifier (available)

Chapter 3: What’s Your Strategy? (unavailable)

Chapter 4: Getting Approval (unavailable)

Part 2: Envisioning the AI-Powered Engineering Org (unavailable)

Chapter 5: Specifying Software (unavailable)

Chapter 6: Agentic Engineering (unavailable)

Chapter 7: Augmented Engineering (unavailable)

Chapter 8: Code Review (unavailable)

Chapter 9: Testing and Quality Assurance (unavailable)

Chapter 10: Deploying and Operating Software (unavailable)

Chapter 11: Managing Teams (unavailable)

Chapter 12: Managing Projects (unavailable)

Chapter 13: Hiring (unavailable)

Chapter 14: Onboarding (unavailable)

Chapter 15: The Future Structure of the Engineering Org (unavailable)

Part 3: Leading the AI-Powered Engineering Org (unavailable)

Chapter 16: Performing a Readiness Assessment (unavailable)

Chapter 17: Aligning the Executive Team (unavailable)

Chapter 18: Managing Stakeholder Expectations (unavailable)

Chapter 19: Resetting Employee Expectations (unavailable)

Chapter 20: Building Personal Competence (unavailable)

Chapter 21: Staffing the Initiative (unavailable)

Chapter 22: Partnering with Product (unavailable)

Chapter 23: Tool Selection and Approval (unavailable)

Chapter 24: Identifying Experiments (unavailable)

Chapter 25: Driving Adoption (unavailable)

Chapter 26: Leading in Uncertain Times (unavailable)

Chapter 27: Final Thoughts (unavailable)

Preface

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the Preface of the final book. Please note that the GitHub repo will be made active later on.

If you’d like to be actively involved in reviewing and commenting on this draft, please reach out to the editor at sgrey@oreilly.com.

If you’re a CTO, I have good and bad news for you. The bad news is that many organizations are getting no ROI from adopting AI within their engineering orgs (according to the 2025 NANDA and METR studies). The good news is that other companies are seeing a tremendous acceleration in their ability to deliver value. The best news is that the differences between the high- and low-performing orgs are now becoming clearer–and so is how to move from one category to the other.

If you are a software engineering leader (especially if you’re running an org of 100+) and you’re looking for good practices for adopting and scaling AI to accelerate how your org delivers business value, this book was written for you. The synthesis of hundreds of interviews with “CTOs at Scale” and executives from top AI companies, it is designed to provide you with a comprehensive plan for experimenting with and scaling the adoption of GenAI across your org. Recognizing that the hardest parts of developing software effectively are almost always about humans and very seldom about code, it focuses on the strategies, stakeholder engagement, and change management you’ll need to lead your org through turbulent times.

Here are some things this book is not focused on:

How LLMs work

I’ll assume you already know enough to be able to adopt them.

Vibe coding

I have a chapter on augmented (IDE-based) and agentic coding, but the hard part of adopting LLMs is scaling value across the org, not just improving individual efficiency.

Architecting GenAI systems

I’ll share some good practices on creating a shared internal AI infrastructure to improve quality, observability, and control, but this isn’t a book on architecting AI products.

Operationalizing inference

I’ll share some hard-won wisdom about hosting internal GenAI tooling, but this is not primarily a book about MLOps.

Specific GenAI tools or infrastructure

I may mention some examples of tools, and I’m busy building a whole catalog of AI dev tools, but specific recommendations change too quickly to commit to paper.

AI enabling your business offerings

This book is focused on transforming your engineering org. Repositioning your company is a wider conversation that I will only briefly touch on, in Part 1.

AI-enabled business processes

There are substantial efficiencies to be gained by leveraging AI across all business functions, but this book is focused on the subset of processes that directly support the software development lifecycle (SDLC).

The broader impact of AI on society

There is a huge cone of uncertainty as to AI’s long-term impact on society, jobs, public discourse, inequality, and–potentially–the future of humanity. I’d be delighted to discuss any or all of those over a drink.

This book is focused on the more tractable and immediate goal of helping CTOs successfully drive the adoption of AI within the SDLC. It’s aimed at helping engineering leaders navigate what might be the biggest organizational change of their careers.

Using this book

Start with Part 1. It’s a quick read and it’s designed to provide context, without which the rest of the book would be tactics without a unifying theory or strategy.

In Chapter 1, I tell a “tale of two orgs,” synthesizing hundreds of interviews into a vignette of two fictitious companies (Flatline Systems and Summit SaaS) that epitomize the two extremes between which most real companies fall when adopting AI for engineering. I then touch on some key takeaways that I’ll unpack in more detail throughout the rest of the book. In Chapter 2, I expand on the concept of AI as an amplifier, highlighting some of the organizational traits that maximize the value available by adopting LLMs.

In Chapter 3, I take a step back so we can examine your broader business context. Most engineering orgs can get value from LLMs, but some face a more urgent, existential threat from AI than others. If you’re selling website-building SaaS software, you’d better be one of the innovators in adopting AI. If you manage ski resorts, the risk of GenAI is less existential–whatever happens with LLMs, people are probably still going to be skiing 10 years from now. I provide a framework for thinking about AI adoption and “picking a lane” for your company, whether that’s as an innovator or as a more cautious “late majority” adopter.

In Chapter 4, now that we have clarity on how aggressively your organization wants to adopt GenAI for software development, I share some good practices for aligning with the security, audit, risk and legal teams to ensure you are capable of approving experiments and unblocking your team.

WIth that framework in place, in Part 2 I take some time to envision what engineering orgs might look like in the future. I share what’s working now and possible futures for the entire SDLC, from specification through coding, reviewing, testing, and deploying new features. I also look at what’s working for all of the supporting activities that go on within engineering orgs, from management to hiring to onboarding.

Finally, in Part 3, I provide detailed guidance on the practical steps required to build out an AI-powered engineering org effectively: from executive alignment, managing stakeholder expectations, and setting employee expectations to building personal competence, staffing the initiative, partnering with product, tool selection and approval, identifying experiments, driving adoption, and leading in uncertain times.

Chapter 1. A Tale of Two Orgs

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 1st chapter of the final book. Please note that the GitHub repo will be made active later on.

If you’d like to be actively involved in reviewing and commenting on this draft, please reach out to the editor at sgrey@oreilly.com.

Some companies are getting transformative results from large language models (LLMs); others are just wasting their money and frustrating their engineers. That’s the GenAI divide. In this chapter, to highlight what’s at stake, I’ve synthesized dozens of conversations with CTOs into two composite stories so we can compare and contrast two approaches to engaging with AI in the software development lifecycle (SDLC). I then pull out some high-level insights from the CTOs who are getting the best results from their AI transformations.

There is no element of these composites that I’ve not encountered in the wild.

Flatline Systems

Flatline Systems is a publicly traded AdTech company with over 800 engineers, a third of whom are contractors from a large outsourcing firm. Flatline builds low-latency, high-performance ad servers for connecting publishers to advertising opportunities in real time. Its code is mainly a collection of Java microservices, with C++ for much of the ad-server code. It has most of its apps running on public clouds (with just a few on-prem holdouts). Test coverage is spotty, manual QA is required for every release, and deployment of many of the apps still isn’t automated.

The CTO at Flatline knew he needed to show decisive action on GenAI. As soon as it was announced, he inked a deal to provide Enterprise Copilot access to an initial test group of 20 engineers to see how much value it could deliver.

The CTO was already overwhelmed with his existing roadmap, but he asked one of his newest directors to shepherd the pilot and to report back each quarter. He couldn’t pull the director off of her main project, which was mission-critical, but he added a sixth quarterly goal to make sure she’d be motivated to make the pilot a success.

At first, the director didn’t seem to understand the assignment. She wanted to completely restructure the pilot, pick teams rather than individuals, and go through a complicated opportunity-identification process. It made sense, but the CTO had already provisioned the licenses and had more urgent priorities.

The director kept on asking for more support. She wanted access to Claude and Cursor, even though she already had the Copilot licenses. She wanted to trial additional niche tools for test generation, code review, and DevOps automation. Most of the companies she wanted to work with were early-stage venture-backed companies with very little track record. It wasn’t even worth taking it to the legal team or the CISO–they’d never sign off on it.

She asked for a budget for onsite training, found some great trainers, and obtained quotes, but it was really hard to justify spending tens of thousands of dollars on training when the internet is full of blog posts and videos on how to use LLMs.

She also asked if the engineers could get extra time to deliver their projects. It really made no sense. The whole purpose of the experiment was to speed up the dev team, not to slow them down.

The worst part was that while some of the engineers seemed to enjoy using Copilot, according to their managers, there was no obvious improvement in velocity or code quality. This result was subjective, of course, because it’s really hard to measure developer productivity and the CTO could never justify the cost of implementing developer-productivity tools. However, it seemed consistent with reports suggesting that many companies get no ROI from adopting LLMs.

He didn’t fire the director for messing up the pilot, but he made it clear that she was on probation and had better impress him with her other initiatives if she wanted a satisfactory rating. She ended up leaving the company shortly afterwards. Clearly he’d made another poor hiring decision.

The CTO didn’t want to be seen as behind the curve, so despite the lack of proven value, he decided to allow engineers to request a Copilot license–as long as they could commit to at least a 10% productivity increase (to be evaluated by their team lead). Strangely, nobody seemed to care enough about AI to sign up for the licenses. In addition, when he looked at his regrettable attrition statistics later in the year, he noticed that a disproportionately high percentage of engineers from the original AI pilot had left the company. Another downside not enough people were talking about!

The CTO was always a little nervous when new case studies came out showing dramatic improvements at other orgs that were leveraging LLMs. He asked a couple of distinguished engineers he respected what was up. They explained that while LLMs are great for startups, they just don’t work at enterprise scale and with legacy code, especially in niche domains: “The training data just isn’t there. Autocomplete is never going to replace experienced engineers solving hard problems.”

The CTO still felt like he might be missing something, but the data seemed clear, and he really needed to focus on his Q4 deliverables to support two big new partnerships the sales org had brought in. He figured he could always check in again in six months to see if it was worth revisiting his AI strategy.

Summit SaaS

The engineering org at Summit SaaS had a lot going for it even before it started to experiment with LLMs. It was a series C, venture-backed dev tool company that seemed to attract the best engineers, both at its San Francisco headquarters and across the US and Europe. They had a motivated, remote-first org of 180 engineers. Summit paid 85% of top of market in San Francisco, and compensation was independent of geography, so they were able to get some particularly capable engineers outside of the Bay Area.

Summit had a mature platform team of 12 responsible for delivering the tooling required for the feature teams to ship faster. It was one of the first companies to rename its platform team a “DevEx” team, to reflect the team’s commitment to optimizing the developer experience for their customers (the engineers).

The DevEx team members weren’t religious about Test-Driven Development, but they had about 90% code coverage, ran continuous integration (CI) on all branches, used red/green automated continuous deployment (CD), and leveraged a mature feature-flagging system for running experiments in production. They also had excellent observability, monitoring, and notifications for system, application, and business metrics.

The CTO believed deeply in the potential of LLMs and was an early adopter of LLMs personally, often creating her own prompts to help with communications and planning. She created a lightweight set of AI agents for ingesting data from systems of record, powered by Markdown files, n8n, and a couple of Model Context Protocol (MCP) servers. This allowed her to automate a lot of the reporting that the data team hadn’t had the time to build out yet.

The CTO’s DevEx team had a substantial roadmap, but she believed deeply that GenAI was an investment worth making. After aligning with the board and her CEO, she asked her Director of DevEx (who ran the platform team) to make this his primary focus for the next 18 months, backfilling his role with a senior engineering manager to lead the rest of the platform org.

The director was also passionate about LLMs and had already been using them in his daily work. He started by agreeing with the CIO, the COO, and their General Counsel on some “red lines” for AI adoption: things they would not do, like allowing third-party cloud-hosted models to train on their company data. Then, to get some hands-on experience, he vibe-coded a lightweight internal tool for managing AI pilot requests across the company. The executive team committed to an SLA of three business days for reviewing and approving or (transparently) denying requests. There were no ROI requirements for experiments costing under $10k a month in licensing fees; engineers just had to avoid the red lines to manage critical business risk.

They assembled a cross-functional AI adoption team that met weekly to share what they were learning. Within engineering, the director created a simple system to allow anyone to propose potential use cases for intensive AI experimentation, with a clear and transparent rubric for prioritizing those larger investments. He also hired one full-time technical project manager (PM)/trainer, a former developer relations engineer, to manage the logistics of the pilots, track what they learned, unblock the teams, and organize regular comms and activities to help the team to share what was working.

They also provided their entire engineering org with access to multiple general-purpose coding tools for both augmented (IDE-based) and agentic engineering. Over the next 18 months, they also ran experiments with niche tools for performing code and security reviews, automating DevOps requests, modernizing some of their older apps, and generating tests and documentation. They experimented with multiple tools in each category and allowed different teams to run experiments in parallel.

The CTO made it clear that they were going to be an AI-forward engineering org, and that a 10 to 15% decline in short-term productivity was acceptable–providing that it accelerated AI learning and adoption across the engineering org. During Summit’s annual hack week, she asked everyone to focus on leveraging AI in their submissions–including the business ops teams, who vibe-coded some amazing prototypes for backend reporting and some customer-requested features. She also encouraged her teams to take a little slack time to play with LLMs more intensely.

She spoke frequently at conferences about AI adoption and retooled her hiring process to require engineers to use LLMs in the interview process. She wanted to attract the engineers who were most excited about adopting AI in their work.

A number of the experimental projects failed, and she noticed slightly higher levels of regrettable attrition than usual. In exit interviews, the engineers often said that they didn’t want to become “engineering managers to a bunch of agents.” But her AI adoption team coalesced around a few successful tools that they rolled out across the org. They also started to build out a company-wide context repository to hold everything: business goals, go-to-market strategies, customer-service conversations, engineering experiments, and automatically generated documentation for the app’s capabilities.

In partnership with the product org, the team also started to experiment with agents for streamlining the process of going from objectives and key results (OKRs) to experiments to product requirements documents (PRDs) to tickets to code. Often the agents would get carried away, but over time they found that 30 to 40% of their app’s features could be coded agentically, with a human reviewing and cleaning up that code. The rest could be accelerated by 30 to 80% by leveraging augmented engineering within an IDE. They also started to get much more rigorous about the format of PRDs and started committing them to a GitHub repo. They noticed a direct correlation between the specificity of their PRDs and the quality of the resulting agentically generated code.

They also experimented with using LLMs to track engineer engagement, support 1:1 meetings, bring more data to performance reviews, and automate a lot of the project management across the company. They still had a great Technical Project Manager, but her role evolved from administrative to consultative, allowing her to focus much more on alignment and prioritization and much less on reporting and documentation.

The CTO was clear that “LLMs right now are as bad as they are ever going to be” and that they still had a long way to go in learning how to use them effectively. But with some clear wins, broad adoption, and a team excited to leverage the latest tooling, she was excited to continue trialing the latest tools, with the goal of maximizing her team’s ability to deliver business value in a relaxed but productive manner.

Where is your org?

Most companies’ AI adoption is somewhere in between Flatline Systems and Summit SaaS, but the two extremes help to highlight some key commonalities in companies that are getting the best ROI from their GenAI initiatives.

AI isn’t something you can just read about or delegate. If you’re not personally using AI daily, you won’t be able to lead your org successfully through your AI transition. You should be experimenting with LLM-based tools for presentations (like gamma.app), building lightweight context about your org in Markdown files, and running Claude Code subagents to automate regular reporting. You should find a little time to futz with MCP servers, and you should start to think about how you’d need to persist and flow data within your org to improve productivity not just by 10 to 30%, but 10x.

AI requires investment. Change management is a full-time job that requires funding and support, and your team may be a little less productive as they build the skills required to leverage AI effectively. Give them time to experiment, while still requiring that the code they choose to commit must be clean and well architected, no matter how they create it.

You’re going to need an executive with platform/DevOps experience within the engineering org who is focused on driving your AI experimentation and adoption. They should prioritize enabling an efficient flow of work from specification to production, and they should have enough experience and credibility within the org that they’ll be able to get everyone–from individual contributors to executives (think the CISO and the audit and legal teams)–to engage with their proposals.

You also need to think about incentives. Saying “I need a proven ROI from every token used” creates very different incentives than saying “I’ll promote the people who are burning the most tokens.” In the early experimentation phase, the latter will accelerate your learning–even though not every experiment will prove fruitful.

You need to get out of your team’s way. If you don’t allow them to experiment with all the tools that are safe to try, they’ll either hide their usage (“shadow AI”) or move to a company that allows them to run experiments more quickly and with less bureaucracy.

You need to rethink hiring, too. Using AI isn’t cheating: it’s the job. As a CTO, the way you position AI use in the hiring process sends a very clear signal about the kind of company you want to be. Do you want to attract or deter engineers who are excited to augment their capabilities with LLMs?

Most importantly, you need to focus on the fundamentals. It turns out that AI is an amplifier; the more efficiently your engineering org operates without LLMs, the more value they will get from adopting them.

Chapter 2. AI as an Amplifier

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 2nd chapter of the final book. Please note that the GitHub repo will be made active later on.

If you’d like to be actively involved in reviewing and commenting on this draft, please reach out to the editor at sgrey@oreilly.com.

The teams that are getting the greatest value out of AI are the ones that already have the strongest culture, the clearest customer focus, the most robust engineering practices, and the slickest developer experience. In this chapter I highlight some of the consistent elements I’ve seen in the companies getting the best ROI from adopting LLMs.

In Part 3 of the book, “Building the AI-Powered Engineering Org,” I’ll dig a lot more deeply into identifying your organizational maturity in each of these areas and improving your processes and practices. Think of this as a high-level cheat sheet for reviewing how ready your organization is for AI-based acceleration.

Culture and Trust

AI adoption is much easier when you have a trusting relationship with your team. Change is scary–especially when it’s not your idea. AI is likely to transform the composition and nature of roles across product and engineering. Without a supportive culture, your team is unlikely to take the risks necessary to succeed with AI.

None of us knows exactly how AI is going to impact the nature of programming jobs. You may not be able to make promises about what the future will hold. But if your team knows that they can trust you to be honest and transparent and that you care about their wellbeing and careers, they are much more likely to feel safe when experimenting with new ways of working.

The nature of experiments is that many of them don’t work out. We’re still learning the right way to leverage LLMs to accelerate the SDLC. It’s important to decouple the success of experiments from performance reviews, bonuses, and career development. If you don’t, nobody will be willing to risk taking on experimental projects.

While it’s important that you provide a strong mandate for adopting AI, you also need to work hard to solicit tactical advice and experience from the people who are actually using the tools. Good practices for leveraging AI are not intuitive and are most often discovered through hands-on experimentation. It’s the individual contributors who are most likely to identify how to get the most value out of LLMs. Make it safe for them to tell you when your plans don’t make sense, so you can course-correct quickly.

AI is moving so fast that keeping up with it requires a lot of passion and commitment. I’m not talking about a “996” culture, where you ask your team to work 12 hours a day (9am to 9pm), 6 days a week. But when they are at work, they need to bring their curiosity, their love of learning, and a commitment to continual experimentation and improvement.

The value that LLMs can provide is directly correlated to the comprehensiveness and accuracy of the context available to them. More than ever, it’s becoming critical to avoid relying on “water-cooler conversations” or even emails and private DMs for communication, and to instead use transparent, persistent communication mechanisms like wikis and public Slack channels for any information that is not fundamentally sensitive. It’s the only way that all of your teammates–human and agentic–can know what’s going on.

Customer-Centered Collaboration

The goal of the engineering org is to solve customer problems creatively. Classic Lean and Agile patterns are incredibly important when it comes to leveraging LLMs to that end.

Increasingly, the value of an engineer will come from their ability to specify solutions to business problems that are rigorous, architecturally sound, and maintainable. There will be far fewer “code monkey” roles, where engineers are simply tasked with implementing a spec and don’t need to care about the business context.

You’ll need small, self-sufficient teams capable of specifying, designing, developing, testing, and deploying features to production. If you don’t have those, any improvements in coding time will be dwarfed by the queueing time incurred by coordinating schedules across multiple teams.

The smaller your pull requests and the less context they require, the less likely it is that your LLMs will start hallucinating API endpoints and functions that don’t exist. Small, customer-valued slices of work allow for easier review, greater velocity, and quicker learning as you see which features actually move the needle on your business goals.

Engineering Excellence

The more rigorous your engineering practices are, the more you’ll be able to trust your LLMs to generate code.

The great thing about Test-Driven Development is that it’s a design methodology, not just a testing strategy. It forces you to think through even the simplest possible behavioral changes you want to make–and to define success before you start implementing those changes. That is also a great pattern to use with LLMs to make sure you’re aligned on the outcomes before they start to work on an implementation. While it doesn’t need to be used for all new features, TDD can be a great methodology for pairing with an LLM. You and the LLM agree on exactly what behaviors you want to provide and then you validate that it’s delivering them.

LLMs will make mistakes (as will humans). But if you provide them a quick, automated method to validate success (that is, running a test suite), they’re much more likely to be able to self-correct without intervention. It’s also much easier to catch regressions when you have good test coverage and an easy way to fire off alerts and report on any errors encountered.

LLMs can accelerate the delivery of business value. Couple that speed with a focus on small, thinly sliced user stories, and it’s likely that, as you adopt AI, you will be deploying to production much more frequently. It’s important to automate your CD pipeline to keep up with the accelerating pace of feature delivery.

Everyone is testing in production, too, whether it’s testing with real-world load or testing the edge cases that only come up with real users and real data. It’s important to have robust feature-flagging tooling to decouple deployment from release. This allows you to roll out new features to users incrementally to ensure that they’re working as expected.

It’s always been important to have system, application, and business-level reporting and notifications, so that if a CPU is spiking, an application is timing out, or your number of active users drops by 50% in a minute, your team gets notified and can resolve the issue quickly. With a faster pace of change and a higher potential for errors, it’s easier and safer to go fast with LLMs when you have the guardrails in place to identify and notify you of problematic deltas in your telemetry.

Architecting for resilience is another great strategy. Futzing, game days, chaos engineering: there are lots of ways of describing the pattern, but at its heart it’s about managing the blast radius of failures. If your payment provider goes down, can people still add to their carts? If your shopping cart is broken, can they at least view the products? While architecting resilience into your applications isn’t essential to getting the best out of LLMs, it does provide defense in depth as you introduce one more variable that could break your applications.

Developer Experience and Enablement

It’s critical to be thoughtful about the developer experience, both individually and in aggregate. Companies with a strong DevEx function are much more likely to be able to support their engineers in adopting LLMs successfully.

The great thing about having a platform team is that it gives you an obvious leverage point for driving your LLM initiatives within engineering. The platform team is already in the business of accelerating your feature teams’ work, so LLMs just become one more tool they can use to help their customers (your feature teams) go faster.

Having a DevEx team is about more than just rebranding your platform org. It’s a clear signal that the job of a platform team is to facilitate the feature teams’ experience of developing software. The engineers are the platform team’s customers, and maintaining a customer focus really helps the platform team to be thoughtful when proposing, implementing, and maintaining tooling for developers–whether that’s a new internal developer portal (IDP) or a set of agentic coding tools.

Measurement is another key element of engineering success. If you want to improve your ability to deliver business value quickly without burning out your team, you’ll need to invest in engineering efficiency metrics so you have a baseline for comparison–whether that’s DORA, SPACE, DX Core 4, or your own unique mix.

AI is not a silver bullet or a quick win, but it can both create an impetus for change and accelerate the process of adopting new processes and practices.

Change, not tools

If you take just one thing away from this book, it should be that AI adoption is a change-management problem, not a tooling-selection problem. AI is going to fundamentally change the nature of your engineering organization. It’s not like moving to the cloud or adopting a developer portal–in fact, the closest analogy might be managing through an acquisition.

You’re getting acquired!

When a company is acquired, both the tech stack and the engineering org are audited and lots of hard decisions need to be made. Is this the right structure? Are we duplicating functions? Who do we keep in the same role and who do we promote, move, or let go?

To say the quiet part out loud: we’re going to have to make all of those decisions over the next few years as AI transforms the SDLC.

Some of this is simply a continuation of trends that were already in play. Ever since Meta’s Mark Zuckerberg introduced us to “the year of efficiency” in 2023, engineering orgs have seen a thinning of middle management, a focus on “player coaches,” an expanding span of control, and higher performance demands on everyone.

There is still a lot of uncertainty and it’s too soon to be making big moves, but your team knows that eventually, if LLMs fulfill their promise of greater efficiency, you’re going to have to choose between an accelerated roadmap or a lower headcount. It’s not obvious which you’ll end up going with.

Creating a change-management strategy

AI tooling is new, but change management has been around for a long time, and there are classic techniques that can help you to steady and support your org and create an environment where people can share and adopt good practices effectively.

A commitment to change

You need to make it clear that you are already committed to this transformation. The goal of starting with experiments is to identify how to proceed with LLMs, not whether to do so.

Transparent communications

Be honest that you don’t know how this will impact the size or the structure of the org, but that you’re committed to helping your team to build the skills that will allow them to thrive as augmented engineers, at your company or (eventually) somewhere else.

Identify and empower champions

Empower members of your team to experiment with AI tooling and support them in doing so. Make sure you have someone responsible for quickly unblocking any obstacles to running your AI experiments.

Listen to the team

As part of your commitment to psychological safety, make sure there are channels where the engineers can be honest about what is and isn’t working with their experiments, so you can understand what’s happening and course-correct quickly.

Celebrate learnings

Make it safe to experiment by celebrating what you’re learning across the org–even if what you learned was that a particular approach isn’t effective for your use case!

Document progress

Get a member of your platform team to create a tool that synthesizes and summarizes your AI experiments and learnings. This can provide a library and a state of play and generate regular updates for those within and outside of the engineering org.

Consistent rituals

Schedule weekly AI demos, monthly roundups, and regular office hours with domain experts to support your team in figuring out how to leverage LLMs.

Structured training

Hold lunch-and-learns, group classes, and one-on-one mentoring sessions. For an org of 100 or more, you will need at least one full-time trainer focused solely on the logistics of creating and structuring resources and experiences to help your whole team get better at managing AI context and efficiently decomposing tasks.

Regular communications

Keep beating the drum: regular communications should be top of mind for everyone. At the very least, share a weekly progress summary. Take the time to include a short personal experience, experiment, or insight each week, so you’re setting a tone of continual learning and engagement with AI. Just block half an hour on your calendar every Friday morning right now, so you always have the time to ship it each week.

Summary

AI is an amplifier. The higher performing your engineering org is, the more benefit you will gain from leveraging LLMs. AI adoption is primarily about change management, not tool selection. AI adoption is not free: it’s an investment that may take time to pay off.

I’ve touched on a lot of topics here, and I’ll spend most of Part 3 of the book providing a lot more advice and information on how to effectively enable AI in your org.

But before you start enabling, you need strategic alignment. Chapter 3 looks at how you can review and improve your company’s AI plans to ensure they are consistent and appropriate for the business you’re in and the team and the executive mandate you have.

About the Author

Peter Bell has been building CTO communities for 15 years. He hosts the O’Reilly CTO Hour, facilitates the executive summits for CNCF at KubeCon, and is the founder and CTO of Gather.dev, the global learning community for senior engineering leaders. He is also the founder and CTO of Dev Tool Collective, a community of technical founders of dev, data, and platform tool companies. He has spent most of the last 15 years helping engineering leaders to share and adopt good practices and bridging the gap between new developer tooling and the expectations of the executives looking to adopt them.