Designing AI Interfaces

Design Principles for Creative and Autonomous AI

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

Louise Macfadyen

Designing AI Interfaces

by Louise Macfadyen

Copyright © 2026 Louise Macfadyen. All rights reserved.

Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 9547.

O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (https://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com.

Acquisitions Editor: Nicole Butterfield

Development Editor: Corbin Collins

Production Editor: Aleeya Rahman

Cover Designer: Karen Montgomery

Interior Designer: David Futato

Interior Illustrator: Kate Dullea

May 2026: First Edition

Revision History for the Early Release

2025-07-09: First Release

2025-10-02: Second Release

2025-12-10: Third Release

See https://oreilly.com/catalog/errata.csp?isbn=9798341639829 for release details.

The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Designing AI Interfaces, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc.

The views expressed in this work are those of the author and do not represent the publisher’s views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

979-8-341-63977-5

Brief Table of Contents (Not Yet Final)

Chapter 1: Overview: Understanding Large Language Models and Systems(available)

Chapter 2: Capability, Discovery, and Orchestration (available)

Chapter 3: Input: Designing for User Input in LLM Systems (available)

Chapter 4: Computation: Designing for the Processing and Generation Phase (available)

Chapter 5: Output: Designing the Delivery and Presentation of LLM Responses (available)

Chapter 6: Recall: Leveraging Memory and Context in LLM Interactions (unavailable)

Chapter 1. Understanding Large Language Models and Systems

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 1st chapter of the final book. Please note that the GitHub repo will be made active later on.

If you’d like to be actively involved in reviewing and commenting on this draft, please reach out to the editor at ccollins@oreilly.com.

In the pre-smartphone era, mobile phone users faced a challenge: texting was slow. Phones were constrained to nine-key keypads, meaning that to text your friend Can you print out MapQuest directions? on your Nokia 3310, you’d need to tap 84 times.

In the late 1990s a new solution appeared: T9 (text-on-nine-keys). Instead of pressing 2-2-2 to get the letter C, you pressed 2 once. The system would guess which of the three letters (A, B, or C) you actually wanted, based on the word you were building.

For example, to type hello, you’d press 4-3-5-5-6, and T9 would figure out you meant H-E-L-L-O rather than G-D-J-J-M or any of the other possible combinations.

But while this innovation sped up the average user’s typing, that’s not what this predictive text system was designed to solve. In the late 1990s, a US startup called Tegic Communications began working on input methods for people with motor impairments, meaning they struggled with traditional keyboards. Their engineers explored how to build efficient text entry using only eight tracked eye positions, dramatically reducing the physical effort needed to compose words.

Tegic realized the same principle could also transform everyday mobile phones, and went on to repurpose their accessibility work to accommodate the growing demand for efficient mobile email and text messaging, resulting in one of the defining mobile input systems of the early 2000s.

Early Predictive Language Models for Consumers

T9 was similar to an LLM: it was a tool for making educated guesses about written human intent based on an input. The system maintained a dictionary of common words ranked by frequency, which narrowed down possibilities as you typed. When multiple words shared the same key sequence, like good and home both being 4-6-6-3, T9 would offer its best guess first, based on a word’s prevalence and usage in natural language, with a simple button press cycling through alternatives.

T9 was a relatively modest technology: the entire system, dictionary included, fit in a file size footprint smaller than a single photo. Yet within those limits, it delivered a highly impressive - and influential - experience. The same creators would go on to create Swype (see Figure 1-1), the gesture-based keyboard used for touchscreen phones, a tool that allowed users to compose words by sliding a finger fluidly from letter to letter, and influenced nearly every keyboard that followed.

Although LLMs resemble these early text predictors in approach, in practice they are an entirely different beast. As you’ll see in this book, whereas T9 sped up typing with efficient guesses, LLMs are capable of elaborating language outward into essays, summaries, code, and more. Yet the fundamental purpose remains unchanged: these predictive systems are methods for enhancing communication, the underlying use case for the vast majority of AI interactions. The computer remains what it has always been: a mediator that reduces friction in human communication.

Figure 1-1. Swype, an early predictive language interface to capture gestural typing.

This perspective is useful, because it anchors our design decisions in human communication needs rather than getting distracted by technical capabilities or promises about the supposed advent of artificial general intelligence (AGI). When we lose sight of this human-centered principle, we build AI features that may demonstrate impressive technical achievement but fail to address actual communication challenges that people face, a common complaint we hear from users of AI products.

To understand how we arrived at today’s AI systems, why they present such unique challenges for designers, and the reason for some of the assumptions users bring, we need to trace their evolution. The path from early computing experiments to modern language models reveals not just technical progress but fundamental shifts in how we think about machine intelligence. Each era brought its own assumptions about what computers could do and how humans should interact with them, which still shape the design decisions we make today.

A Short History of AI

How did we get here? As little as five years ago, barely anyone was talking about AI. I came to this field via my work on a project called Tensorflow, one of Google’s first open source models. For one project, Google asked my team and I to create immersive experiences that would help users to understand the underpinning technology of three of their newest models. The first, their well known PoseNet, could determine a human’s pose through image or video inputs and transfer it into position tracking data. The second could look at an image of a human face and estimate emotion - happy, sad, angry etc. The last was a NLP, or Natural Language Processing model. To explain how natural language models worked, we devised a game (see Figure 1-2) where the user could navigate a galaxy by typing the word that appeared closest in dimensional proximity to another word. For example, if the user wanted to go to planet A, and planet A’s corresponding word was biology, they could type lifeform, granting them a 90% association between the two words, and setting the trajectory of craft in the right direction.

Figure 1-2. Startographer, an interactive game where users chart a course across space by approximating the distance between two words in latent space.

Our game was effective because it helped users conceptualize words existing in proximity to one another in a multi-dimensional space, but it was a challenge to conceptualize. Of all the models, the NLP model was the hardest for us to devise an experience around - we actually built it last, because we realized that it would require more understanding on our end than time to build. I remember spending days with the projects’ engineers, trying on different solutions, trying each time to understand a little better what this product is capable of.

One of the most persistent challenges in designing AI products is helping users build accurate mental models of what the system can and cannot do. While the models of today are superior - profoundly - to that NLP model I worked with in 2018, this user education issue persists. Indeed, this difficulty predates modern language models by decades, appearing whenever new ‘intelligent’ capabilities emerged. It’s one of the most important lessons of working with AI, and it explains why designers working with AI face a unique challenge that goes beyond typical software design: users arrive at AI products with unfounded assumptions that may not remotely map to real capabilities, and often lack the support to find the real capabilities.

Users arrive with preconceptions about intelligence itself, shaped by science fiction, marketing hype, perhaps even their experiences with unrelated but similar systems that may have worked entirely differently (such as Google search). This section traces how these mismatched expectations have evolved alongside the technology: although its growth is attributable to some astounding technological innovations, grasping the distance between reality and expectation is vital for a designer. Understanding these recurring patterns of misalignment helps designers anticipate where users’ mental models are most likely to break down and what solutions are available to support user discovery, reduce product bloat, and actually help the user solve their problem.

The Beginning: Symbols

As programmable digital computers arose in the 1940s, they prompted a great deal of speculation about whether they could actually think. The mathematician and scientist Alan Turing attempted to give this question a precise answer in his 1950 essay “Computing Machinery and Intelligence.” - in this paper, he proposed what he called the imitation game, now widely known as the Turing test. A human judge (the interrogator) would exchange written messages with two unseen participants, one human and one machine. If the judge could not reliably tell them apart, the machine would be said to exhibit intelligence.

Figure 1-3. Sterrett, S.G. The Genius of the ‘Original Imitation Game’

However, it came up a little short. While theoretically assessing all intelligence, in actuality the Turing test posed only a very narrow question: “can a machine convincingly imitate a human in text-based conversation?”. The paper didn’t anticipate how strongly users would project intelligence into a human-sounding conversation, and how vastly different and distant actual intelligence and human-sounding conversation are from one another. The technical capabilities germane to human speech turn out to be much more different to the capabilities for autonomy and reasoning than initially beheld.

The computer program ELIZA, developed at MIT in the mid-1960s by Joseph Weizenbaum, was one of the earliest attempts to further explore this tension. Weizenbaum originally built it as an experiment in natural language processing, showing how a computer could carry on a dialogue by following simple rules for recognizing keywords and generating responses. The “DOCTOR” script was designed to imitate a psychotherapist, reflecting the user’s words back with minor transformations. In Figure 1-4, you can read a typical exchange with Eliza

Figure 1-4. a typical Eliza conversation

To generate the illusion of conversation, ELIZA scanned the user’s input for keywords, applied pattern-matching rules, and reassembled fragments of the text into templated replies. If it detected the word boyfriend, for example, it could echo that term back in a reflective question. When no keyword was recognized, it defaulted to generic prompts, like tell me more.

The ability to respond caused users to quickly anthropomorphize the tool : they began to empathize with the program, attributing understanding, knowledge, and even sentiment to a system that was only reassembling fragments of their own words. Weizenbaum himself was unsettled to see people inferring a degree of cognition where none existed. In his notes, he recorded one worrying encounter between ELIZA and his secretary:

‘My secretary watched me work on this program over a long period of time. One day she asked to be permitted to talk with the system. Of course, she knew she was talking to a machine. Yet, after I watched her type in a few sentences she turned to me and said: ‘Would you mind leaving the room, please?’

This phenomenon later became recorded as the ELIZA effect: the tendency to project human-like qualities onto linguistic outputs that only simulate understanding. This makes sense - for the vast majority of human history, with the very occasional exception (lyrebirds etc), only humans have been able to speak. Language, then, is a human-exclusive cue, which primes the brain to anticipate other human-level faculties. ELIZA and projects like it had arisen out of a general desire to uncover a finite set of rules that would make generated language sound convincingly real.

In the 1950s, Noam Chomsky’s theory of generative grammar proposed that humans share an innate system of syntactic rules that control language and meaning. His example, Colorless green ideas sleep furiously, illustrates the paradox: the sentence is grammatically correct, yet so clearly nonsensical that no child would ever produce it. Chomsky’s linguistics theory and the ELIZA experiment defined an era of realization that language involves both structure and interpretation, forcing researchers to move from rule-based planning to explore statistical methods that could more closely model actual language usage.

But AI researchers were stuck trying to hardcode logic into machines in the hope of generating true intelligence. This failure to move beyond rule-based systems led to the first “AI Winter” of the 1970s. Funding dried up as researchers realized they couldn’t explicitly articulate all of the infinite complexity of human language. Even decades later, chatbots remained stuck in ELIZA’s shadow. SmarterChild, launched in 2001 and used by millions on AOL Instant Messenger, was essentially ELIZA with a bigger dictionary. It could tell you movie times and stock prices, but its conversational ability hadn’t fundamentally improved.

Next: Statistics

Despite early optimism, programmers struggled to infer realistic speech or cognition through grammatical rules alone. In order to develolp a model that could more reasonably synthesize human language and behavior, new statistical approaches were developed. Whereas older methods used simple counting (how often does the appear before cat? How often does chat in French correspond to cat in English?) these researchers tried to learn deeper patterns through layers of artificial neurons. The hope was that a network could discover abstract features and relationships between words, using data to understand that dog and puppy were related without being explicitly told.

However, neural networks were computationally intensive: training even a small network took days. On 1980s hardware, the simple counting methods worked better. Research languished once again, and when Google Translate launched in 2006, it used phrase tables and probabilities, not neural networks.

Finally: Scale

Three innovations would support the arrival of modern AI: the rapid growth of data, the speed of GPU chips, and the transformer architecture.

First, the internet explosion gave programmers the massive training data they desperately needed. In the 1980s, researchers were lucky to find a few thousand digital documents. By 2010, the scale had become vast. Wikipedia alone contained 4 million English articles. Google Books had scanned 20 million texts. Every day, Twitter users posted 500 million tweets. Facebook generated billions of status updates. The Common Crawl project, started in 2011, archived the entire web: petabytes of raw text, free for anyone to download.

This wasn’t just more data: it was different data. Previous datasets were clean and academic: newspaper articles, encyclopedia entries, carefully edited books. By contrast, the internet gave researchers messy, human language: forum arguments, text messages, product reviews, chat logs. Neural networks trained on this chaos learned how people actually communicated, not how grammar textbooks said they should. When Stanford released the GloVe project in 2014, they trained it on 840 billion words from Common Crawl. The old bottleneck had completely reversed: now there was too much data to process.

Then in 2012, a PhD student named Alex Krizhevsky entered a competition to classify images. While other contestants used CPUs, Krizhevsky used two NVIDIA GTX 580 gaming GPUs, cards built to render games like Skyrim. His neural network, called AlexNet, destroyed the competition. Eventually, graphics cards built for gaming would come to accidentally revolutionize the AI world. It turns out that the underlying use case is similar: just as games need to calculate lighting and physics for millions of pixels simultaneously, neural networks need to calculate millions of neuron values simultaneously. A single GPU had thousands of tiny processors that could work in parallel, while a CPU had maybe 8 powerful processors working sequentially. For neural networks, which mostly did simple math on massive amounts of data, GPUs were 50 to 100 times faster than CPUs. By 2015, NVIDIA began making GPUs specifically for AI.

The third innovation was when something called the transformer architecture solved how neural networks read text. Before 2017, networks processed words one at a time, like reading a ticker tape. In 2017, a team at Google published "Attention Is All You Need,” introducing the computational concept of transformers. Instead of processing words sequentially, transformers looked at entire sentences at once. The key innovation was attention, a mechanism that let each word directly connect to every other word, computing which connections mattered most. Processing The cat was happy no longer meant reading left-to-right. Looking at the word happy, the transformer could instantly check: what connects to this? It would find cat had a strong connection, was had a medium connection, and the had almost none. This parallel processing made training faster and let networks understand much longer texts.

The paper has immediate impact. Within months, every major AI lab switched to transformers. Google created BERT (2018), which broke every language understanding benchmark. OpenAI built GPT (2018), (with an initial 117 million parameters, the adjustable values within a neural network), then GPT-2 (2019) (1.5 billion parameters), then GPT-3 (2020) (175 billion parameters). Each version simply scaled up the same transformer architecture with more data and more GPU power. The combination was explosive: transformers could actually use those billions of web documents, GPUs could actually train enormous networks, and suddenly machines were writing essays, answering questions, and generating code at near-human levels.

The rapid scaling of language models brought its own challenges, as researchers Timnit Gebru, Margaret Mitchell, and their colleagues documented in their 2021 paper “On the Dangers of Stochastic Parrots.” They warned that the race to build ever-larger models risked overlooking crucial issues: these systems could amplify biases present in their training data, consume massive amounts of energy with environmental consequences, and create the illusion of understanding while merely recombining patterns from their training corpus. Their work highlighted how the same scale that enabled impressive capabilities also concentrated power among organizations with the resources to train massive models, while making it harder to understand or correct problematic behaviors embedded deep in billions of parameters.

By 2024, the landscape had evolved into a complex ecosystem of competing models and approaches. OpenAI’s GPT-4, released in March 2023, demonstrated capabilities that surprised even its creators, from passing professional exams to writing functional code across dozens of programming languages. Anthropic’s Claude models emphasized safety and helpfulness through what it called constitutional AI training, while Google’s Gemini models pushed multimodal capabilities that could seamlessly work with text, images, and code. Meta chose a different path, releasing its Llama models as open weights, meaning anyone could download and modify the models, spawning thousands of specialized variants fine-tuned for specific industries and use cases. The rapid proliferation of LLMs meant that within eighteen months of ChatGPT’s launch, language models had been integrated into everything from customer service systems to medical diagnosis tools—often faster than designers could establish best practices for their use.

This explosion of capability happened so quickly that public understanding lagged far behind technical reality. Users approached these systems with mental models shaped by decades of science fiction and chatbot disappointments, some expecting human-level artificial general intelligence, others ready for simple keyword matching, and few grasping the nuanced reality of statistical language generation that could be simultaneously brilliant and nonsensical. A system that could write a compelling legal brief might fail at basic arithmetic, while one that could debug complex code might confidently fabricate historical events. These behaviors made perfect sense given the models’ training, but they violated users’ intuitions about how intelligence should work.

The mismatch between capability and comprehension created precisely the design challenge this chapter began with: users needed help building accurate mental models of systems whose behaviors emerged from architectural decisions they couldn’t see and training processes they couldn’t inspect. Unlike the clear limitations of ELIZA’s pattern matching or the predictable failures of rule-based systems, modern language models failed in ways that seemed arbitrary to users, succeeding at complex tasks while stumbling on simple ones, remembering some details while forgetting others, switching between insight and hallucination without warning.

As we move into examining the designer’s role in this new landscape, the weight of this history becomes clear. Every interaction pattern we create must bridge the gap between what users expect based on their past experiences and what these systems actually do. The tools have evolved, but the challenge remains: making systems that authentically assist human ability while remaining comprehensible to human understanding.

A Designer’s Work in the AI Era

In a world of change, some things are still the same: designers still advocate for users, still craft interfaces, still push for clarity over complexity. But the context surrounding design work has shifted dramatically: we’re now designing for systems that change, break, and sometimes behave in ways no one anticipated. This shift doesn’t invalidate everything we’ve learned, but it does mean applying familiar skills in unfamiliar territory, where fixed rules have given way to flexible principles.

For designers, the work of managing that gap begins with knowing your user and how your product is used. This is home turf for a designer, but in the AI era it goes beyond documenting needs or collecting feature requests: it means asking where users are likely to misinterpret, what signals will guide them toward the right understanding, and how much of the underlying process is worth revealing. Sometimes a clear explanation helps, but just as often too much detail overwhelms or distracts. The skill lies in deciding where transparency builds confidence and where it only creates confusion, and then keeping those decisions consistent so the product feels coherent.

Technical and evaluatory skills play a part here as well. Designers do not need to become machine learning researchers, but they do need enough grounding to understand what is happening under the hood. Without that knowledge, it is difficult to explain behavior, to anticipate where the system might fail, or to set clear expectations for both users and colleagues. These skills allow designers to judge which system details to surface, which to smooth over, and how to present them in a way that helps users rather than distracts them.

And because AI products are built at the intersection of disciplines, communication across the organization is just as important. Designers, engineers, and product managers need a shared language for describing what a system can do, what it cannot, and what should or should not be shown to users. When those standards are established and reinforced, users encounter a consistent logic across the product; when they are not, users are left to invent their own explanations, and those explanations are rarely accurate.

In this sense, the three requirements—know your user, build technical and evaluatory skills, and cultivate communication—are all devoted to the same purpose: reducing ambiguity and giving users a way to understand what is in front of them. That clarity is what makes AI products usable, and without it, even the most advanced system risks being misunderstood or mistrusted. Let’s explore each requirement a little further.

Know Your User

One of the clearest challenges in designing AI products is that a single interface cannot meet the needs of everyone who uses it. A one-size-fits-all approach sounds efficient, but in practice it flattens the variety of ways people arrive at an AI system. Some come to learn, some to write, some to experiment, and some to solve a specific problem under time pressure. When these very different needs are funneled into the same experience, users often feel unseen. They may not know how to get started, or they may conclude that the system is not designed for the task they had in mind.

The reason this matters so much for AI is that expectations are already fragile. People approach these systems with curiosity, but also with uncertainty about what they can do. If the product does not show recognition of their intent, the confusion deepens. An AI feature that feels generic can quickly be interpreted as ornamental, just another checkbox on a company’s AI strategy rather than a tool built to help. This is why the design challenge is not just to make the system usable, but to create a sense that it understands and supports what the user is trying to accomplish.

One way to bring clarity to this challenge is to group user needs into categories and design around those categories. In UX, we have long called this building personas. Personas distill large, messy sets of behaviors into recognizable profiles, each representing a cluster of goals, contexts, and frustrations. In AI, personas do more than guide feature lists; they help teams design for recognizable modes of interaction and anticipate ambiguous outputs. They let us anticipate the questions users will bring, the misinterpretations they may fall into, and the kind of feedback or transparency that will be most useful in each case.

Personas are commonplace, and some people have already begun to use AI tools to generate them. Their value is in communicating examples of how their system responds to different roles—for example, treating the system as a writing coach, a research assistant, or a creative partner. Teams might use personas internally to test how a new feature might be received by different segments: would a student preparing for an exam feel supported, and is there a more specific way we could help them? Would a professional drafting a contract discover the right UI? Such thought exercises assist in giving structure to real patterns of behavior so the product can meet them with intention.

Building personas for design requires paying attention to both data and empathy, and looking at usage patterns can be a critical starting point. In the OpenAI working paper “How People Are Using ChatGPT” (NBER Working Paper No. 34255, 2025), researchers clustered millions of queries and found recurring categories, such as tutorials, editorial critique, personal communication, and creative ideation, each representing thousands of daily interactions (see Figure 1-5). These clusters show that even a general-purpose tool does not produce infinite variation; behaviors group into recognizable patterns that reflect the goals people bring to the interaction.

Figure 1-5. Categories of ChatGPT use

The implications for interface design are direct. A person using a tool primarily for tutorials may benefit from continuity across sessions, so that earlier questions can inform later ones. Here, long-term or episodic memory becomes valuable, because the system can recall what was already covered and build upon it rather than repeating explanations. By contrast, someone seeking critique on a piece of text may care less about persistence over time and more about immediacy, such as a reasoning tracker that shows how suggestions were derived, or a session consolidation feature that summarizes edits made across a draft.

Creative ideation brings yet another set of needs. In this mode, users often want variety rather than consistency, so the interface might emphasize branching paths, multiple suggestions in parallel, or the ability to compare and remix outputs. The challenge here is less about memory and more about supporting exploration without overwhelming the user. A reasoning tracker could still be useful, but only if it clarifies how different ideas emerged rather than imposing a linear explanation.

Personal communication sits somewhere in between. Someone drafting a message may appreciate short-term memory within a session, allowing the system to maintain tone or recall details about the recipient, while being skeptical of longer-term retention that could raise privacy concerns. For this group, the design implication is that memory features should be transparent and easy to manage, giving users confidence that they remain in control of what is remembered and what is forgotten.

These examples show how clusters derived from data are not abstract categories but practical guides for interface design. They indicate when persistence matters, when explanation matters, when variety matters, and when control over memory matters. Each cluster points to a different set of affordances that, if surfaced thoughtfully, can reduce friction and increase trust. The presence of such data allows designers to move past speculation, anchoring design decisions in the observed reality of how people are actually using these tools.

The value of personas lies not just in what they represent, but in how they help align teams. A persona can resolve arguments about feature priorities by anchoring discussion in user goals. It can guide interface decisions, such as when to expose reasoning steps or when to keep the interaction simple. It can even help bridge business and design needs by showing how a feature that satisfies executive interest in AI can also deliver value to a defined group of users. In short, personas are not a formality; they are a way of keeping user need visible in a field that risks being overwhelmed by technological novelty.

Search illustrates this principle in a familiar way. When someone types flights Marseille to Amsterdam, the result is not a page of scattered links, each pointing to an airline or a travel blog, but a structured surface that presents times, prices, and the ability to filter. The intent behind the query has been recognized, and the interface reflects that recognition. If the search engine did not make this shift, the user would be left to piece together information from multiple sites, working harder than they should to accomplish what is, at its core, a simple task. Figure 1-6 shows an example.

Figure 1-6. Search results as interface

What this design decision solves is a problem of translation. People are rarely searching for text itself; they are searching for outcomes. By giving them a way to move directly from query to action, the tool shows that it understands what they are trying to achieve, and the experience feels sharper as a result. The intelligence lies less in the algorithm than in the way the result is presented, because the design translates a recurring pattern into a surface that fits.

There is a similar opening in contemporary products that incorporate generative tools. When a request aligns with one of the established categories, like learning, writing, critique, or ideation, the interface can adapt in small but meaningful ways. A draft being edited might benefit from inline comments that mirror the conventions of word processors; a question framed as a tutorial might be better answered with steps that can be expanded or collapsed; a creative exercise might feel more natural with multiple suggestions side by side. Each of these design choices signals to the user that their intent has been understood, which strengthens trust and makes the experience feel more purposeful.

The challenge is that intent is not always cleanly detected, and designing for categories always involves trade-offs. Overly rigid responses risk locking people into a workflow they did not ask for, while too much generality makes the product feel indifferent to their needs. The opportunity, however, is clear: by identifying recurring patterns and shaping the interface around them, design can lower the burden on the user and create a sense of recognition that feels far more human than a generic surface ever could.

Trust

No discussion of knowing your user would be complete without addressing the question of trust. Surveys in recent years have pointed to a cautious public mood, with more than half of Americans telling the Pew Research Center in 2023 that they felt more concerned than excited about the spread of artificial intelligence in daily life. For designers, the importance of this finding underlines the reality that new features do not enter a neutral field. People already have established ways of accomplishing their tasks, and they approach new capabilities with curiosity tempered by doubt. They want to know whether the tool can genuinely help them, but they are wary of anything that complicates or disrupts what already works.

In practice, this means that a designer’s first responsibility is to preserve the clarity of the task at hand. Features that interrupt or distract risk being rejected, especially when they feel like they serve organizational goals more than user needs. The craft of design is to assist users in achieving their ends while also helping the organization meet its obligation to make new capabilities visible. This often requires careful signposting: showing where a new feature fits into the existing product, demonstrating continuity with older interfaces, and explaining why the layout or workflow has changed. By doing so, designers can help users see novelty as an extension of familiar practices rather than as an intrusion.

At the same time, trust is not only about restraint. There are moments when users need guidance to discover what is possible, and the designer may have to give a feature prominence it would not otherwise deserve. Discovery is part of the adoption process. A new capability hidden from view may never be tried, and the opportunity for value is lost. Thoughtful onboarding, temporary prominence, or small acts of user education can help people form new habits without creating hostility. The goal is to ease people into change, maintaining a sense that the product is on their side, and that it respects the ways they already work while introducing new ones.

This balance between respecting existing behavior and encouraging exploration sits at the heart of the larger tension between business and user goals. Companies that have invested heavily in artificial intelligence want visible features to show for it, and the pressure to deliver often shapes the pace and direction of product work. Designers are not immune to this context, but they are in a position to mediate it. By grounding their decisions in evidence of what users actually do and what they genuinely need, designers can argue credibly that the most strategic AI features are the ones most aligned with user challenges.

When evidence shows that users consistently ask for writing critique, or that tutorial-style guidance represents a large share of interactions, then building for those needs simultaneously satisfies the executive demand for visible innovation and the user’s demand for practical utility. In this way, trust and business alignment are not opposing goals. They are part of the same design responsibility: to ensure that features are not ornamental, but essential, and that their introduction strengthens both the relationship between product and user and the credibility of the organization.

The work of bridging these goals depends on a designer’s ability to show continuity, to make adoption feel safe, and to back decisions with real patterns of use. Trust emerges when people see that their goals remain visible, and business goals are served when the product earns adoption rather than resistance. The bridge is built on this credibility, which allows design to act as both a partner to the user and an advocate inside the organization.

Feedback

Gathering and adapting to feedback is central to understanding the user and to earning their trust over time. Trust is not secured once at launch; it is maintained through a visible process of listening, responding, and improving. Designers have multiple roles to play in this process. They help determine what kinds of signals are collected, they shape the ways feedback is invited in the product itself, and they spend time with the results to interpret what the data really means for future design decisions.

The arrival of large-scale generative tools has expanded this responsibility. Designers can now prototype ideas at a much faster pace, generating variations, testing them, and discarding them with less cost than ever before. This makes it easier to experiment with different feedback strategies and to see how users respond. Instead of waiting months for a research cycle to finish, a team can try multiple approaches in parallel, embedding small prompts, surfacing implicit metrics, or experimenting with task-specific signals. The designer’s role in this process is to decide what will actually illuminate user behavior, and to ensure that the signals collected are meaningful enough to guide the product forward.

Feedback still involves direct requests in the interface, but it also includes the less obvious traces of interaction. How often do users copy results, how often do they regenerate them, how often do they abandon a workflow altogether? These implicit signals can be as telling as an explicit rating, but they require interpretation. A copy action might indicate success, or it might show that the user had to paste the result elsewhere to fix it. A regeneration might signal dissatisfaction, or it might reflect that the user wanted more variety. The obligation is not only to gather the data but to spend time with it, to consider what it means, and to weigh its implications for design.

What this adds up to is an expanded sense of feedback as a design loop. It is not only about collecting responses, it is about shaping the questions, experimenting with the methods, and interpreting the results with care. The opportunity for designers is significant: by treating feedback as part of the design material itself, and by using new tools to experiment more rapidly, they can keep their work closely tied to the realities of user behavior while also pushing the product forward.

Technical and Evaluatory Skills

Understanding how design has changed requires examining what designers actually did before AI entered the picture. In the pre-AI era, product designers relied on a core set of skills that, while still relevant today, operated in a more straightforward context. User research and usability testing formed the foundation of their practice. Designers served as the voice of the user, ensuring interfaces were intuitive and accessible through careful observation and testing. There was no AI to personalize experiences or adapt to individual users; instead, designers crafted singular experiences meant to work well for broad audiences.

Information architecture and interaction design were equally crucial. Designers mapped out how users would navigate and interact with products, creating logical pathways that felt natural and user-friendly. Visual design demanded significant creative energy, as designers relied on their own skills and established principles to craft polished, aesthetically pleasing interfaces. Understanding front-end constraints and collaborating with developers rounded out the essential skillset, since designers needed to know what was technically feasible and how to create designs that could actually be built.

These core skills have evolved and expanded considerably in the AI era, adding layers of complexity to the designer’s role. Designers remain the voice of the user, but now they must also understand how to guide and fine-tune AI behaviors to serve users meaningfully. The work has shifted from designing static interfaces to designing dynamic systems that adapt to different inputs and scenarios.

This evolution demands familiarity with AI-related concepts and a degree of data literacy that was previously unnecessary. Designers now need to understand, at least conceptually, how models work, what biases might emerge, and how to create interfaces that help users understand and trust AI-driven features. Ethical design has become paramount, requiring constant consideration of how AI decisions affect users. The collaborative aspect of design work has also expanded significantly. Where designers once worked primarily with developers and product managers, they now regularly engage with data scientists, machine learning engineers, and other roles that barely existed in traditional design teams.

Two key shifts capture the essence of this transformation. First, design has moved from crafting static experiences to orchestrating adaptive ones. In the pre-AI era, designers created fixed pathways that users would follow, focusing on making those pathways as clear and user-friendly as possible. The approach was deterministic: you designed it, users moved through it, and the output matched what you designed. Today, designers must think about how interfaces can learn, how they present different options to different users, and how they maintain trust and transparency in a far more complex landscape.

Second, designers now carry responsibility for understanding and shaping AI behavior ethically and transparently. This represents an entirely new layer of accountability that simply didn’t exist when designing static interfaces. The role has both expanded and deepened, requiring designers to think systemically about outcomes they cannot fully predict or control.

The question of technical knowledge looms large for designers entering this new landscape. The key shift is that designers now need technical fluency that wasn’t previously required. Before AI, exceptional designers could succeed purely through understanding user needs, mastering design principles, and creating visually and functionally appealing solutions.

Today, designers benefit enormously from understanding how AI models are trained, what data they use, and what their limitations might be. This doesn’t mean becoming a data scientist, but having technical literacy enables more effective collaboration with engineers and data teams. It also helps designers anticipate how their design choices might play out in dynamic, AI-driven environments.

The best approach starts with getting comfortable with basics: understanding what machine learning is, what types of AI models a product might use, and what common pitfalls exist. From there, it’s about maintaining curiosity and openness to learning, working closely with technical partners to translate user needs into AI-enhanced features. Designers need their traditional design intuition plus technical awareness; that combination allows them to contribute as meaningfully as ever, but in a more technologically layered landscape.

One of the most significant opportunities AI has created is the ability to prototype in ways that were previously impossible. Designers can now use AI-driven prototyping tools to mock up AI features and see them in action early, providing hands-on understanding of how different models behave and how prompts shape outputs. Weaving prototyping into daily workflow becomes essential. The more designers experiment with building and testing AI-powered prototypes, the more naturally they understand what happens under the hood and how to design for it.

This represents a dramatic shift from relying on static wireframes or imagination. Actually building and iterating with AI tools provides an incredible way for designers to gain technical insight organically. Using AI frequently, and using AI building tools to create AI products, teaches far more than theoretical study alone. Designers can learn about how different models work, how prompt engineering functions in practice, and gain intuitions that would be difficult to develop otherwise.

Yet a crucial challenge remains: designers struggle to know how deep their technical knowledge should go. Without an accepted standard of knowledge in the field, it’s difficult for designers to gauge the appropriate level of detail. The temptation exists to think one needs PhD-level AI expertise, but that’s neither realistic nor necessary.

Tip

The key is starting small and being incremental. Designers don’t need to master everything at once. Building a foundational understanding that enables communication with teams and more thoughtful design is sufficient. Knowledge naturally accumulates through practice. The field continues evolving rapidly, making the “right” level of knowledge a moving target anyway. Designers should give themselves permission to learn gradually without pressure to know everything immediately. Curiosity, rather than completeness, should guide the learning journey.

Ethical Considerations in AI Design

The ethical dimension has become a cornerstone of the post-AI design landscape. While ethics in design certainly existed before AI, it primarily focused on accessibility, inclusivity, and ensuring that products didn’t inadvertently exclude people. The integration of AI has raised these ethical stakes considerably.

Designers now must consider algorithmic bias, transparency, user consent, and data privacy in much more direct ways. They need to think about how to communicate what the AI is doing and ensure users understand how their data is being used. There’s also the responsibility of ensuring AI doesn’t cause unintended harm or reinforce existing inequalities. This shift has transformed designers into ethical guardians of AI interactions, requiring them to advocate for user rights and transparency while designing interfaces that help users feel informed and in control. Without an established rulebook yet, the field continues evolving best practices through collective learning.

A 2021 paper by DeepMind examined the ethical and social risks of harm from language models, outlining seven specific risk areas:

Discrimination

Exclusion and toxicity

Information hazards

Misinformation harms

Malicious uses

Human-computer interaction harms

Automation, access, and environmental harms

These can be grouped into three overarching categories that help designers frame their understanding and action.

The first category encompasses social and interpersonal risks, including discrimination, exclusion, toxicity, and misinformation harms. Designers can approach these by focusing on fairness, inclusivity, and the quality of information that AI produces or moderates. Practical interventions include designing features that allow users to report biased outputs or harmful content, and ensuring AI transparency about its operations. Designers can stay educated through resources like the Partnership on AI and other organizations focused on AI ethics.

Safety and security risks form the second category, covering information hazards, malicious uses, and automation access issues. Here, designers can build in safeguards such as clear user permissions, warnings about sensitive data usage, and mechanisms ensuring humans remain in the loop for critical decisions. Promoting transparency in AI decision-making helps users feel secure and informed about system operations.

The third category addresses environmental and human-computer interaction impacts, including automation and environmental harms. Designers can consider energy efficiency in their designs and make users aware of the environmental costs of certain AI features. They should also focus on creating interfaces that keep humans engaged and in control rather than allowing automation to completely take over in potentially harmful ways.

However, it’s crucial to recognize the realistic boundaries of designer responsibility. Designers aren’t solely responsible for fixing large systemic issues like hallucinations or bias; these problems are primarily owned by engineering teams and business initiatives. And, similar to every other technology predating the AI era, there are trade-offs between the opportunities a new technology offers and its risks to users and society on the whole. Designers must be aware of these compromises without bearing the entire burden of AI ethics on their shoulders.

The designer’s role is more nuanced: serving as advocates and considering how ethical considerations manifest in the interface and user experience. Business goals and technical constraints will sometimes mean certain ethical ideals cannot be fully met, at least not immediately. The key is for designers to remain aware of these trade-offs and participate actively in conversations, raising issues when they see them and pushing for the most ethical solutions that are practical. It’s about finding balance and understanding that while designers cannot fix everything, they can guide products toward more thoughtful, user-friendly directions, even if progress comes incrementally.

As it becomes increasingly clear which AI behaviors are more harmful than others, designers will find themselves uniquely positioned to advocate for user safety and wellness. Their proximity to the user experience enables them to champion best practices around AI usage, whether helping users avoid overuse, raising awareness about energy impact, or ensuring tools are used in healthy, user-friendly ways. Ultimately, designers serve as everyday advocates who bridge the gap between technical and human sides, ensuring that as AI evolves, it does so with genuine focus on users’ best interests.

Communication: Knowledge Sharing and Role Clarity

The third dimension of how design has changed in the AI era concerns the organizational landscape surrounding designers. While the traditional designer-PM-engineering triad remains intact, the dynamics within this structure have shifted significantly. Designers working in companies building AI products find themselves navigating new internal complexities around communication, shared language, and organizational standards.

In AI-focused organizations, designers evolve from being solely the voice of the user to becoming bridges between user needs and the complexities of AI systems. They must help product managers and engineers understand the nuances of how AI changes user experience, while simultaneously ensuring that user-centered design isn’t overshadowed by technical constraints or business metrics. This expanded role requires more proactive cross-functional communication and deeper involvement in shaping product direction.

One of the most critical challenges for designers working with AI involves establishing crystal-clear roles and responsibilities across teams. Unlike traditional product development, AI design demands extensive cross-disciplinary collaboration that can quickly become chaotic without proper structure. The question of who owns specific knowledge becomes paramount: who is responsible for translating the model’s capabilities into understandable terms? Is it the product manager, the engineering lead, or a dedicated technical liaison? Without clear answers, designers struggle to know where to obtain crucial information or how to feed their insights back into development cycles.

Effective knowledge sharing in AI organizations goes beyond occasional documentation or sporadic meetings. It requires creating living systems of communication where information flows continuously and reliably. For instance, if a product manager is responsible for dimensionalizing the model’s capabilities, that responsibility must include regular updates as capabilities evolve, clear documentation of limitations, and accessible explanations that designers can translate into interface decisions. When engineering discovers new constraints or possibilities, there must be established channels for that information to reach design teams before it impacts user experience.

Designers play a crucial advocacy role in establishing these communication systems. They often feel the pain of poor knowledge sharing most acutely, as they need to understand both technical capabilities and user needs to create effective interfaces. By pushing for regular capability reviews, documented decision rationales, and clear ownership of different knowledge domains, designers can help cultivate an organizational culture where information sharing becomes automatic rather than exceptional.

The rapid pace of AI development makes this knowledge sharing even more critical. A model’s capabilities might change weekly, new limitations might emerge from testing, or business priorities might shift based on competitive pressures. Without efficient knowledge transfer mechanisms, designers find themselves constantly playing catch-up, designing for capabilities that no longer exist or missing opportunities to leverage new features.

AI brings with it a flood of new terminology, much of which remains unsettled even at the industry level. Organizations building AI products must establish their own clear language and standards, creating informal - or formal - internal glossaries, that everyone from engineers to designers to marketing teams can reference. Designers often find themselves at the center of this vocabulary challenge, since they need to understand technical terms well enough to communicate with engineers while simultaneously translating those concepts into language that makes sense for users. This translation work is vital, but it only succeeds when there’s organizational agreement about what these terms mean in the first place.

Beyond individual terms, organizations need standards for how AI capabilities are framed and communicated. What constitutes an acceptable error rate? What kinds of tasks fall within the product’s intended scope? These standards shape every design decision, yet they often remain implicit or inconsistent across teams.

Designers can help establish these standards by documenting patterns and pushing for explicit decisions. When they notice that different parts of the organization have different thresholds for acceptable AI behavior, they can surface these inconsistencies and advocate for alignment. When new capabilities emerge, designers can help define not just how they work technically, but how they should be presented to users and what expectations they create.

The establishment of shared vocabulary also extends to how teams discuss design patterns specific to AI. Concepts like progressive disclosure of AI capabilities, or human-in-the-loop interactions need consistent definitions so that design decisions can be discussed efficiently. Without this shared language, every design review becomes a vocabulary lesson, slowing down the iteration cycles that AI development demands.

Organizational AI Maturity

To understand where an organization stands in its ability to design and ship effective AI products, it helps to consider not just the models it uses or the tools it licenses, but how its teams work together. Design success in AI doesn’t stem from technical sophistication alone; it depends on how well an organization can adapt to rapid change, share evolving knowledge, and delineate clear responsibilities across disciplines.

To illustrate this progression, the autonomous vehicle framework provides a useful parallel. Developed by SAE International (the Society of Automotive Engineers) and first published in 2014, this framework emerged from a critical need: as automakers rushed to develop self-driving capabilities, the industry lacked a common language to describe what autonomous actually meant, thus causing terms and states to be used interchangeably, creating confusion for regulators, manufacturers, and consumers alike.

As shown in Figure 1-7, the framework established six levels (0-5) of driving automation, ranging from Level 0 (no automation) to Level 5 (full automation). What made this framework particularly powerful was how it clarified the relationship between human and machine responsibilities at each stage: at Level 2, for instance, the human remains the fallback and must constantly supervise; at Level 3, the system becomes the fallback but only in specific conditions. By Level 5, no human intervention is ever required. The framework has been adopted globally, and has been revised several times to reflect evolving technology and learnings from real-world deployments.

Figure 1-7. Autonomous driving framework

In a similar manner, the following framework outlines five levels of organizational AI maturity, focused on collaboration and communication practices essential to AI-era product work.

Level 1: Individual experimentation

At this level, designers, engineers, and PMs may experiment with AI tools on their own, often using personal logins or playground interfaces. There’s no centralized strategy or tooling, and experimentation happens in isolation. Individuals might generate copy, test prompts, or brainstorm ideas using LLMs such as ChatGPT, but none of this connects back to shared systems or product infrastructure.

No formal documentation exists

Model behavior is poorly understood, even among builders

There’s no clear internal ownership for anything AI-related

Designers are left guessing how to integrate AI into interfaces or whether they should at all

This level is common in large organizations with slow-moving infrastructure, or in companies early in their exploration of AI capabilities. Design decisions often lag far behind technical experimentation.

Level 2: Informal collaboration

Teams begin to work together on AI projects, but processes are still ad hoc. A PM might provide a list of intended behaviors for the model; an engineer might explain what’s technically feasible; a designer might reverse-engineer capability boundaries through trial and error. Some documentation may exist, but it’s inconsistent or out of date. Meetings are the primary venue for clarification and critical knowledge often lives in someone’s head.

Ownership of model behavior is fuzzy

Feedback loops between model performance and design implementation are slow or nonexistent

Terms like confidence, hallucination, or scope are used inconsistently across teams

Designers often lack access to prompt templates or model tuning parameters

Shipping AI features takes time not because of model performance, but because teams can’t move in sync

Level 3: Aligned but manual

At this stage, the organization has agreed on key concepts. Teams have a shared language for AI behavior, a centralized documentation space (even if imperfect), and a rough cadence of updates when model capabilities change. There are designated owners for prompt engineering, evaluation, and interface behavior, and design teams have access to the working memory of the product. However, alignment still depends heavily on people being in the loop: designers have to ask, follow up, or attend meetings to get the information they need.

Capability reviews may happen monthly or quarterly

Terms are mostly standardized, but internal documents still drift out of sync

Model evaluations exist but aren’t routinely shared with design

New capabilities are documented but not translated into interface implications

This level can still ship high-quality AI work, but it struggles to scale. When a new designer joins or a new feature team is spun up, institutional knowledge doesn’t always get to them.

Level 4: Systematized and distributed

Organizations at this level treat AI behavior as a product surface in its own right. There are structured practices for updating documentation when a model’s behavior changes, and cross-functional reviews are scheduled regularly. Designers are included in upstream conversations about model decisions, and engineering teams surface capability changes before they appear in production.

Teams use shared tools and libraries for prompts, evaluators, and UI patterns

There is a centralized location for prompt changes, data cards, error thresholds, and reliability notes

Designers don’t need to hunt for model information, it arrives via structured updates

Expectations about uncertainty, risk, and explanation are embedded in product design standards

This is the first level where organizations consistently ship AI features quickly and responsibly because they’ve built robust internal infrastructure to support design at speed.

Level 5: Adaptive and self-correcting

At this maturity level, the organization’s knowledge and communication systems are fully integrated with its design and development infrastructure. Feedback loops are fast, multi-directional, and measurable. Designers have access to real-time model behavior data, system limitations, user pain points, and upcoming changes. The organization expects capabilities to evolve constantly and is built to adapt without spinning up new processes every time.

Evaluation data and user feedback are routed automatically to relevant teams

Shared design language evolves alongside model development and is version-controlled

Designers contribute to prompt libraries, test suites, and error scenarios

Ownership boundaries are clear but fluid when they need to be, with hand-offs structured around problem areas rather than job titles

Few organizations have reached this level. Those that do treat model behavior, interface decisions, and product strategy as part of the same system. Design becomes not just a recipient of model information but a source of feedback that shapes the model itself.

Maturity as a lever, not a ladder

It may be that every team doesn’t need to reach Level 5. For some products, Level 3 or 4 may offer the right balance of speed, control, and cost. But understanding these levels helps clarify what’s slowing down a team, and where investment in communication, vocabulary, or ownership could unlock significantly faster iteration.

Designers, in particular, are often the first to notice when organizational maturity is too low. They experience the friction of unclear ownership, the confusion of inconsistent language, and the waste of designing for outdated capabilities. By naming these patterns and pushing for internal systems that share knowledge and evolve together, designers can become catalysts for better collaboration in AI-era organizations.

So what does all this mean for designers navigating the AI era? At the end of the day, the job hasn’t changed as much as it might seem. Designers are still doing what they’ve always done best: making sure products actually work for the people using them. The difference now is that the products themselves are squishier, more unpredictable, and occasionally surprising in ways that nobody quite expected. That means designers need to pick up a few new tricks: getting comfortable with data to really understand user patterns, learning enough about how AI works to have smart conversations with engineers (without needing a computer science degree), and helping everyone on the team speak the same language about what the product can and can’t do.

Looking ahead, the real opportunity for designers is to be the translators and advocates they’ve always been, just with some new vocabulary. As organizations figure out how to work with AI (and most are still figuring it out), designers who can bridge the gap between impressive technology and actual human needs become invaluable. The tools will keep evolving, the models will get more elaborate, but the core design challenge remains wonderfully familiar: how do we make this thing useful, understandable, and maybe even delightful for the person on the other side of the screen? That’s a job that no AI is taking away anytime soon.

The Rest of This Book: The Input-Computation-Output Structure

With that initial primer out of the way, the rest of this book delves into far greater detail about the emerging patterns and practical challenges of designing for AI. In order to make sense of this rapidly changing technology, it helps to create a structure that is applicable across the user experience, and the most stable way that I’ve found to understand this is through articulating three stages: input-computation-output. This framework has been consistent through my experience working with AI, and is likely to remain relevant even as new modalities arrive.

Inputs

Inputs are the point where human intention enters the system. In most cases, input looks familiar: a text box for a chatbot, a search bar, or a button that triggers a predictable action. Yet inputs can also be more subtle. In some cases the input may be latent, inferred from context rather than explicitly entered by the user. A tool might notice an open document, a location, or a repeated workflow and treat that as an intention. Designers are comfortable shaping explicit inputs, but AI complicates the picture by introducing new forms of invisibility, ambiguity, and guesswork. What feels like a clear request from a user may be filtered through probabilistic interpretation, creating gaps between what the person thought they asked for and what the system actually does.

Tip

This shift requires designers to think differently about how inputs are surfaced, explained, and constrained. It is no longer enough to frame an elegant text field or arrange a neat set of buttons. Designers must consider how to give users awareness of what is being captured, what assumptions the system is making, and how those assumptions can be corrected when wrong. Inputs in AI systems are dynamic, often shaped by ongoing interaction rather than a single moment of entry. Ultimately, the design of inputs becomes less about where users type or tap, and more about how human intention is translated faithfully into machine interpretation.

Computation

Once an input is received, it passes into the opaque stage of computation. To the user, this stage is often signaled only by a spinner, ellipsis, or subtle animation, but it is where the system does its heaviest lifting. Requests are interpreted, relevant data is retrieved, reasoning paths are explored, and multiple tools may be orchestrated in sequence. Guardrails also take effect here, filtering or reshaping the request before it continues. For designers, computation is paradoxical: it has the greatest influence on the quality of the result, but it is also the phase they see and control the least. Errors tend to accumulate here, and latency becomes most visible, even if the exact cause remains hidden from both designer and user.

This chapter gives designers an insight into the “black box” and offers practical guidance for working with it. We will look at common latency patterns, such as progress indicators and staged reveals, as well as ways to reduce confusion by giving just enough detail about what the system is doing without breaking the flow. Examples will show how different interfaces balance transparency with simplicity, and how even small cues can reassure users that progress is being made.

Outputs

Outputs are the point where results return to the user. They can take the form of text, images, charts, code, or even audio and video. What the user sees has already been processed and formatted so that it is legible, safe, and ready to use. Text is generated token by token and then smoothed into prose. Images are decoded, corrected, and compressed before they appear. Even small choices in this stage, such as how a citation is displayed or whether a chart is interactive, shape how interpretable and actionable the result feels.

Tip

The key value of an output is not its surface quality, but what can be done with it. A draft can be edited, a chart can guide a decision, a summary can move a project forward. The design challenge is to present outputs in ways that make the next action clear. An eloquent paragraph that does not help the user act is less valuable than a concise list that can be directly applied. Outputs succeed when they carry momentum into the user’s workflow, creating continuity between what the system generates and what the human needs to do next.

Beyond the Core: Configuration, Interpretation, and Action

Input, computation, and output form the spine of every interaction, but they are not the whole story. Before input, there is configuration: permissions granted, accounts connected, defaults chosen. These decisions quietly determine what the system can do. After output, there is interpretation and action: the human edits, verifies, or integrates the result into their own workflow. Errors need to be designed for, as does feedback. The loop then begins again, with outputs prompting new inputs.

This three-stage structure gives us a shared language for discussing AI design that cuts through the complexity without losing essential nuance. When a user complains that an AI tool feels opaque, we can now ask more precisely: is the input unclear about what’s expected, is the computation taking too long without feedback, or is the output formatted in a way that doesn’t support their next action? Each stage presents distinct design opportunities and constraints that we’ll explore in depth throughout this book.

The framework also reveals why designing for AI requires such a different mindset from traditional software design. Static interfaces could optimize each stage independently, but AI systems create dynamic relationships between input, computation, and output that shift based on context, user, and even the specific request. Understanding these relationships, and designing for them effectively, is what separates AI products that frustrate from those that genuinely augment human capability.

Summary

T9 predictive text may be remembered as a quirk of early mobile phones, but the innovation was remarkable: by compressing multiple key presses into predicted words, it turned the awkwardness of a twelve-key keypad into a functional language tool. What is easy to forget is that it was not built for novelty but for accessibility. Typing on small devices was too demanding for many users, and prediction helped relieve that burden. That grounding in accessibility is significant.

Some of the most influential digital innovations began from a similar place: screen readers that transformed interfaces for blind users, closed captioning that reshaped how video content is produced and consumed, and voice input that has since become a standard feature of phones, cars, and homes. Other examples are just as striking. Autocorrect, while often mocked, grew from the same recognition that typing errors slowed people down. Speech-to-text technologies made it possible for those with limited mobility to write documents or messages more quickly, and touchscreens, originally embraced as a way to help users with dexterity challenges, ultimately defined the smartphone era. Again and again, tools created to solve a clear problem for a specific group ended up shaping how everyone interacts with technology.

In some ways, computing has always been influenced by this drive to narrow the gap between human and machine capability. As we’ll see in the next chapter, the command shells of the 1960s and 70s gave remarkable power to those who mastered them, but their rigidity excluded anyone who could not memorize syntax. Search engines softened that relationship, interpreting looser queries and even correcting typos on the user’s behalf. Predictive text carried the same logic further, anticipating intent and offering it back in usable form. Each step shifted the burden of adaptation from human to machine, making space for more people to use technology productively. That pattern of gradually relocating the effort away from the user and into the system remains one of the clearest throughlines in interface history.

For designers today, the work is both familiar and different. We must still advocate for user intent, but we now sit in a position where we also mediate between human language and complex systems that interpret, transform, and generate on the user’s behalf. The task is not only to make these systems easy to operate but also interpretable, so that people understand what is happening and can act accordingly. Where predictive text once surfaced a suggested word, AI systems now surface paragraphs, analyses, or entire workflows. This amplifies the designer’s responsibility: to shape how those outputs are presented, how much explanation is given, and how users remain in control of the exchange.

Rather than viewing predictive text or AI as outliers, it helps to see them as part of a continuum: technology that listens a little more closely, guesses a little more accurately, and takes on a little more of the burden of communication. Each shift has opened participation to more people, but each has also raised new questions about how much should be revealed, how much should be left opaque, and how users remain in control. These are not just engineering questions; they are design questions. Our role is to shape how much of the system is exposed, how intent is carried forward, and how confidence is built in processes that cannot always be seen.

Over the next several chapters, we will examine how this long tradition of assistive computing plays out in the age of AI, using the input–computation–output framework as a guide. The lesson from predictive text, and from so many accessibility breakthroughs before it, is not that the technology was clever—it is that it solved something real. Innovations that endure do so because they meet users where they are and expand what they can do. For designers, the challenge is to look past the hype and uncover that core value.

Chapter 2. Capability, Discovery, and Orchestration

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 2nd chapter of the final book. Please note that the GitHub repo will be made active later on.

If you’d like to be actively involved in reviewing and commenting on this draft, please reach out to the editor at ccollins@oreilly.com.

In 2009, Google launched Wave (see Figure 2-1), a novel communication and collaboration platform that promised to reinvent how people work together online. It combined the features of email, instant messaging, and document editing into a single place, acting as a one-stop shop for communication and creation. Unfortunately, it died an early death. Google canceled work on Wave in 2010, because, as the Museum of Failure puts it, “the service was too ambitious, hard to learn, and barely an improvement over regular email.”

Figure 2-1. Google Wave (2009)

Wave took its name from Firefly, the sci-fi series where sending a wave meant transmitting a message across time and distance. Google’s version aimed to modernize communication in the same spirit: waves were real-time, shared documents that supported rich media, inline replies, and simultaneous editing by multiple participants. Built on a hosted XML structure, each wave functioned as a live thread that blended messaging with collaborative content creation. Users could drop in images, maps, or files, edit each other’s contributions midstream, and extend functionality through bots and gadgets.

At launch, the product drew enormous interest. Access was limited, with fans reselling invitations on eBay for as much as $70. But despite the excitement, many users struggled to understand what Wave was for. The promise of a unified workspace was clear in concept, but in practice people found it difficult to adopt, use meaningfully, or integrate into their daily routines.

By mid-2010, less than a year after launch, Google announced it would stop active development on Wave. The user base hadn’t grown beyond early adopters, and internal momentum had stalled. While some of Wave’s underlying technologies like real-time syncing and collaborative cursors lived on in other Google products, the platform itself was shut down entirely by 2012. It lives on as a case study in what happens when powerful technology isn’t paired with the right product framing: in short, nobody could figure out what Wave was for.

This is the kind of product/user misalignment this chapter seeks to prevent. AI features can feel clumsily implemented, centered around emerging capacity over true user need. An approach like this treats the model’s capabilities as the starting point, when in practice, what matters most is where those capabilities intersect with real user needs.

To design successful AI features, it’s important to understand not just what the model can do, but what users want to do, and how the system helps them get there. This chapter proposes a way to structure that space: through communicating capability, supporting discovery, and supplying clear orchestration.

Capability

What the model or system is capable of doing. This includes both narrow functions (such as summarizing a document or extracting an entity) and more open-ended behaviors (generating text, suggesting next steps, or performing multi-step reasoning).

Discovery

What the user is aware of, how they are made aware, and what they’re willing to try. It includes both intent (what the user is hoping to accomplish) and discoverability (what the user knows is possible). Discovery can also involve what the system chooses to surface that the user may not have anticipated, such as relevant data or suggested next steps.

Orchestration

The design layer that connects the other two. It includes the systems, scaffolding, and UX choices that allow user intent and model capability to meet productively. Orchestration is the layer that helps users move from exploration to adoption.

By the end of this chapter, we’ll have unpacked each circle in more detail, and looked closely at the overlaps between them: where discovery shapes intent, where orchestration enables customization, and where model performance must align with interface clarity.

Capabilities: What Can the Model Do?

Understanding a model’s ability is a key issue for designers. In the Carnegie Mellon HCI institute’s paper “Re-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design” (2020), researchers found that designers frequently report that it is “difficult to grasp what AI can or cannot do” and that, as a result, “this hampers designers’ brainstorming and sketching processes from the start.” This can cause an impeded design process, where technical understanding hampers creativity, and designers struggle to create within the bounds of feasibility. Even a basic understanding of AI wasn’t enough - researchers found that even with a baseline knowledge, designers “often found it difficult to ideate many possible new interactions and novel experiences with much fluidity.” Figure 2-2 illustrates the junctures at which such knowledge barriers present a challenge.

Figure 2-2. Design challenges of unknown AI capabilities on the design process

It’s not clear the right level of knowledge that a designer needs to work with AI, but across the industry there are emerging methods of general understanding that can act as a guideline. In order to understand the abilities of a particular model, there are two elements at play: how the model was trained and how it was evaluated.

You may be familiar with controversies around datasets and model training: these conversations often focus on what data was included, what was left out, and who made those decisions. While these debates matter enormously for the broader AI ecosystem, they also reveal something practical for designers: training data fundamentally shapes every interaction users will have with AI systems.

Most LLMs start with pre-training, a process similar to reading the entire internet and then trying to predict what comes next after any given word. The model ingests massive amounts of text, books, articles, forums, documentation, Wikipedia entries, and news sites, and learns to complete patterns. When it sees The capital of France is... during training, it figures out how to consistently reply that Paris is the most likely continuation.

Next, the model is typically refined through a process called fine-tuning. This stage uses a smaller, more targeted dataset to guide the model toward specific behaviors. Instead of general next-word prediction, the model is trained to follow instructions, answer questions, summarize documents, or engage in multi-turn conversation. In many modern systems, fine-tuning includes a method called reinforcement learning from human feedback (RLHF). Here, human reviewers rank different model responses, and those rankings are used to adjust the model’s behavior. The goal is to make the output more helpful, more truthful, or more aligned with human preferences, depending on how those goals are defined by the developers.

This alignment phase is where the model begins to reflect preferences about tone, clarity, or safety that were emphasized during the feedback process. However, this doesn’t mean the model really “knows” anything—this process is more akin to trying to learn a language by memorizing a dictionary but never having conversations. The model develops an understanding of grammar, facts, and reasoning patterns, but only through this one-way exposure to written text. It learns that certain words tend to follow others, that some topics connect to specific vocabularies, and that different writing styles require different approaches. In essence, the model becomes a very sophisticated next-word generator.

Introducing Evals

Training and evaluation form a continuous cycle in AI development. Whereas training shapes what models can do, evaluations, or evals as they’re called in industry parlance, reveal what the models actually do in practice. The same process that teaches models to be helpful relies on human evaluations to define what helpful means. Fine-tuning models for specific tasks requires evaluations to measure whether a given specialized training improved performance. Every decision to deploy a model depends on evaluations showing it meets quality and safety standards.

One of the most widely used tests of general knowledge and language understanding is called MMLU, short for Massive Multitask Language Understanding. It contains multiple-choice questions across 57 topics, from U.S. history to computer science to moral philosophy. Because it pulls from academic exams and professional certifications, it gives a broad sense of how well a model performs across disciplines. While some benchmark items may overlap with training data, MMLU is still treated as a useful yardstick: a model that scores highly can usually follow instructions, retrieve facts, and handle technical vocabulary.

To evaluate reasoning more directly, researchers use math benchmarks. One example is GSM8K, a dataset of 8,500 grade-school math word problems, which require multiple steps, like converting units, doing arithmetic in the right order, or combining facts to get an answer. Models can stumble here, not because they don’t know how to add or subtract, but because they lose track of the steps or misinterpret the setup. Success on GSM8K shows that a model can sustain structured thinking across a few sentences, which models can find particularly challenging.

To assess how models perform in more open-ended language tasks, like writing explanations, summarizing ideas, or taking a position, researchers often rely on human judgment. MT Bench (short for Model Test Bench) is an eval used to compare the performance of language models on realistic, user-facing tasks with a set of realistic prompts that are evaluated by people, not machines. Given two model outputs for the same task, reviewers pick which one is better. This can highlight things that automated tests miss, such as tone, clarity, or whether the model actually understood the nuance of the request.

No single benchmark is perfect, and most evaluations are narrow by design. Researchers often publish a leaderboard of results across a dozen or more benchmarks to give a broader picture. But even then, high scores don’t guarantee real-world reliability. Many current efforts are focused on creating new benchmarks that better reflect open-ended use, long-form reasoning, or more dynamic tasks like planning and tool use.

For designers and project managers, evals and training can feel like the domain of engineers, particularly given how challenging the technical language barrier can be. However, as the voice of the user, it’s vital that designers can contribute to refining and improving models, collaborating with engineering and anticipating the best possible user outcome. Here’s some advice:

There’s not yet a process for designers to become incorporated into the evals methodology, but reviewing and interacting with these standards can certainly help with a general understanding of the underlying process.

Designers should feel comfortable engaging with their engineering partners to explore the models they use in order to fully understand the capabilities available to them. Given the explosion of vibe-coding and the availability of multi-model hosting services like OpenRouter, designers can achieve valuable prototypes with greater ease than ever before, with a higher degree of verisimilitude.

Outside of self-initiated exploration, designers should stipulate that model capabilities be shared by engineers or PMs in a digestible - if informal - format before feature design commences. Establishing a nominated technical partner for a design sprint can assist in communicating opportunities alongside user needs.

Vibe Coding

Vibe coding is an informal way of building with AI tools such as code assistants or natural language interfaces. ‘Vibing’ is derived from the unplanned nature of the process; unlike in normal programming, which tends to require a high level of skill and planning, the vibe-coding process involves converting a loose idea into a prompt, running the code, and then iterating on the request as needed. The cycle is quick and informal, with progress coming from repeated exploration that can allow for rapid building. However, while this approach can feel creative, it can also lead to poorly-written code that is difficult to maintain.

Discovery

User intent sits at the heart of exceptional user experience design. It’s the driving force behind every click, tap, and interaction users have with digital products. Understanding how users discover and achieve this intent is the bridge between what users want to accomplish and how a product helps them achieve those goals.

In some cases, AI products have centered on capabilities and functionality. Teams with this mindset ask questions like “What features should we build?” and “How can we showcase our product’s capabilities?” The primary concern is creating robust, impressive functionality and ensuring that users can access all available features. This approach often leads to feature-rich interfaces where the emphasis is on demonstrating what the product can do.

User-focused design thinking, by contrast, centers on people and their goals. Teams with this mindset ask, “What are users trying to accomplish?” and “How can we help them succeed?” The primary concern is understanding user motivations and creating experiences that serve those underlying needs. This approach leads to purpose-driven interfaces where the emphasis is on helping users achieve their intentions efficiently and effectively: instead of organizing interfaces around product capabilities, we organize them around user goals.

Tip

Intent-driven design emerges naturally from user-focused thinking. When we understand what users are trying to accomplish, we can design experiences that feel intuitive and purposeful rather than overwhelming or confusing. This approach recognizes that users engage with digital products as a means to an end, not as an end in itself. While we might observe that a user clicks a button or fills out a form, intent reveals the deeper motivation: Are they researching a purchase decision? Looking for specific information? Trying to complete a task as quickly as possible?

In AI interfaces, uncovering intent can be more challenging. Because best practices are still emerging, users often struggle to imagine what the system could do for them, which makes their goals less defined than in traditional products. Research in this space tends to be more exploratory, helping surface not just what users want, but what they might want once they better understand the tool’s possibilities.

Intent with AI

Of course, gathering user intent can prove highly challenging. UX researchers use methods like behavioral analysis, user interviews, and journey mapping to reveal intent through patterns in behavior and stated preferences, providing signals to product designers about how to create effective product features. However, for many AI products, the opportunity lies in anticipating and supporting intent adaptively. This is more complex, because in real-time, users communicate their intent through two distinct channels: explicit signals like direct requests and commands, and implicit signals embedded in their context, behavior, and approach to your interface.

Explicit intent feels straightforward. A user types write a professional email declining this meeting into your AI assistant, and their goal seems clear. They click Generate Report and you know what they expect. These direct signals form the foundation of most AI interactions, but they represent only the surface layer of user intention.

Implicit intent runs much deeper and often reveals more about what users actually need. It encompasses everything from their expertise, how they arrived at your product to the digital environment surrounding their interaction. Whereas explicit signals tell you what users think they want, implicit signals often reveal what they actually need to accomplish their underlying goals. It’s often best to use both.

Intent also varies depending on whether users are interacting with an AI-first or AI-enhanced tool. In AI-first tools, intent is often shaped in relation to a general-purpose assistant or generative environment, where users may show up without a rigid task or methodology in mind. These tools need to elicit and interpret intent more fluidly, sometimes even helping users articulate it for the first time. In contrast, AI-enhanced tools typically layer AI atop existing workflows, where intent is already better defined. Here, the challenge is in surfacing AI help in ways that align with existing user behavior without disrupting it.

As AI becomes more capable, users begin to set more ambitious goals. Knowing that the system can take on complex or specialized tasks, people begin to pursue outcomes they previously saw as out of reach. A small business owner who might once have drafted their own flyers now hopes to generate a full campaign, complete with visuals and messaging guidance. These shifting expectations raise the stakes for good interface design: people show up hoping not just to be helped, but to have their needs transformed.

In a similar way, AI also expands the boundaries of what users believe is possible. As people become familiar with what the system can do, their sense of what to ask for evolves. They may start by requesting summaries or suggestions but soon experiment with storytelling, analysis, or creative ideation. Interfaces often accelerate this shift, sometimes intentionally, sometimes not. This is one reason why early interactions matter so much: they anchor expectations. Known as algorithmic transference, researchers have observed that users carry experiences formed in one AI conversation over to another, influencing future interactions. This means that users are often bringing a variety of assumptions that shape how they frame their intent when they meet an interface. Designers should be thoughtful about how capabilities are introduced, because once the user’s mental model is set, it can be hard to unseat.

Designing with intent in mind means recognizing that what people want to do and what they believe they can do are in constant conversation. And with AI, that conversation is especially fluid, constantly challenging users to uncover and evolve their expectations and processes.

Discoverability and Initialization

Love or hate it, the once-loved sparkle emoji ✨ has become the unofficial mascot of AI product features. As noted by Quartz and the Wall Street Journal, the emoji’s positive associations with “magic” mean that it’s become a recognizable method of drawing user’s attention to a new AI-enabled feature. Many have complained about the broad interpretation of the icon, noting that the variety of AI features available are generally a lot more complex than can be associated with just one icon.

This is where discovery comes in: finding a method for introducing new capabilities while also educating the user, building trust and clear visual associations. A negative discovery experience of an AI feature can provoke responses like these:

“There’s too much going on—and I don’t know what I’m supposed to pay attention to.”

“I didn’t ask for that.”

“I keep discovering things by accident.”

“It’s cool, but I don’t know how to use it.”

“This doesn’t feel like it’s for me.”

“I don’t know what it can do”

To address these concerns, AI feature discovery should be educational, intentional, well timed, and mapped to intent. To begin, designers working on exposing a feature can ask themselves, “Does the user already know they want this?” If yes, you’re working with high user intent. If not, you’re in low intent territory and need to design for gentle awareness. Next, they can ask, “Will the system surface this proactively, or wait to be prompted?” If it waits, that represents low system initiative, requiring strong affordances and clear naming. If it acts first, you’ll need to consider timing, confidence, and exit paths.

Together, these two dimensions give us a 2 x 2 matrix for thinking about AI feature discovery, shown in Figure 2-3.

Figure 2-3. Matrix for AI feature discovery

Each quadrant suggests a different design posture. When users have low intent and low initiative, features may emerge socially or develop organically over time. In situations where users have low intent but the system has high initiative, the interface needs strong context cues to act helpfully without feeling intrusive or overwhelming. When users demonstrate high intent but low initiative, they’re actively looking for something but may not know exactly how to find it, so the design should make discovery and activation as frictionless as possible. Finally, when both user intent and system initiative are high, the goal is to accelerate what users are already trying to accomplish, removing barriers and streamlining their workflow.

The following subsections are a taxonomy of current discovery patterns, grouped by how the feature is revealed. These aren’t mutually exclusive, and many tools use a combination.

Input-based discovery

The most direct way to surface features is through the input itself. These patterns build on familiar entry points, such as typed commands, gestures, or empty states, and turn them into moments where users can discover what the system is capable of. Because they appear where attention is already focused, they make discovery feel like a natural extension of the user’s own action.

Command triggers

This is when users type specific commands to invoke AI actions (see Figure 2-4). The user sees a dropdown menu of available commands as they type, similar to autocomplete suggestions. These patterns create a sense of power and control, letting experienced users bypass visual interfaces entirely while maintaining discoverability for newcomers who can explore available commands.

Figure 2-4. Notion’s AI palette is opened with a press of the space bar.

Gesture/interaction patterns

This is when users trigger AI features through familiar interactions like right-clicking or selecting text. For example, in Word, Copilot options can appear in the context menu when a user highlights a paragraph. These patterns feel natural because they build on existing behaviors, but the features can remain hidden unless onboarding helps users discover them.

Empty state activation

This is when users open a blank document or spreadsheet and see AI-powered suggestions like Create a budget? or Start with a meeting agenda template? displayed prominently in the empty workspace. These moments capitalize on the anxiety of the blank page, offering direction precisely when users are most open to guidance.

Context-aware discovery

Some discovery patterns work by paying attention to the user’s environment rather than their explicit input. The system can notice what kind of content is being worked on, how often certain actions repeat, or where the user’s focus is within the interface. By situating help in context, these patterns make features feel relevant and well-timed rather than random or disruptive.

Content analysis

This is when AI automatically detects the type of content being worked on, like recognizing a meeting notes format, and shows a contextual suggestion bubble saying Summarize Action Items that appears near the relevant content. The user sees the system understanding their work context and offering appropriate next steps without being asked.

Behavioral pattern recognition

This is when a user repeatedly performs the same action, like pasting identical boilerplate text into emails, and Excel shows a notification suggesting “Create a rule?” The user sees the system learning from their patterns and offering to automate repetitive tasks they may not have consciously recognized as inefficient.

Location-based

This is when a user hovers the cursor over a spreadsheet’s totals row and sees a small tooltip suggesting Add Formula or clicks into an empty cell and sees relevant function suggestions. The user experiences contextual help that appears exactly where their attention is focused, reducing the cognitive overhead of remembering where features live.

Ambient/proactive discovery

Other patterns step in before the user asks for help. By watching for potential errors or anticipating needs, the system can act like a background safety layer that surfaces guidance only when it is most useful. This shifts the role of AI from something users must seek out into something that quietly supports them as they work.

Anomaly detection

The user sees the system acting as a quality control layer, catching potential problems before they become issues, such as Google Calendar displaying an alert when two meetings will clash, or Gmail reminding a user to attach a file they have mentioned.

Predictive surfacing

Visiting a web page in a foreign language on the Chrome browser will trigger a message offering to translate the page (see Figure 2-6). With this feature, the user gets just-in-time assistance that recognizes their needs before they have to explicitly ask for help, reducing the steps between problem recognition and solution.

Figure 2-5. Google’s predictive surfacing window provides just-in-time translation

Progressive discovery

Some features are best revealed slowly. Progressive discovery introduces functionality in stages, giving people time to build confidence before encountering more advanced options. Instead of overwhelming new users with a full menu of capabilities, the system reveals features in moments of clear relevance, helping them feel like natural extensions of what they already know.

Feature graduation

A writing tool may initially offer grammar correction as a suggestion, then later promote it to a visible menu option once adoption is proven. This type of exposure adapts over time, supporting gradual familiarity.

Contextual tutorials

Consider when a user uploads a portrait photo and immediately sees a Background Remover tool appear in a highlighted sidebar panel with a New! badge. The user gets introduced to advanced features only when they’re immediately relevant, avoiding cognitive overload while maximizing the chances they’ll try new capabilities.

Once you know what your feature does and how it’s intended to show up, you can place it on the 2 x 2 grid from Figure 2-3 to reflect its discovery posture (Figure 2-6). This chart makes it easier to see whether a feature’s current exposure mechanism matches its intent and initiative. It also helps you plan complementary discovery moments, since features that are surfaced socially could later graduate to ambient or context-aware prompts.

Figure 2-6. Types of feature exposure mechanism to favor based on user intent.

Together, these patterns illustrate the range of how users discover, learn, and re-encounter AI functionality. Discovery represents an ongoing conversation between the user’s intent and how the system responds, where the feature becomes either part of the user’s toolkit or fades into the background.

Adoption and Momentum Behavior

The path from discovering a feature to successfully using it is fraught with behavioral and design challenges that can make or break user adoption. Research consistently shows that feature discovery is one of the strongest predictors of software adoption and engagement. A 2017 Clutch study found that 72% of users said completing all onboarding steps in less than a minute is important in their decision to keep using an app. The same research revealed that if an onboarding experience takes longer than two minutes, many people give up altogether, meaning you have just 120 seconds to onboard mobile users or lose them forever. This impatience reflects Hick’s Law, which states that the time it takes to make a decision increases with complexity and the number of choices available.

Compounding this challenge is momentum behavior, a psychological phenomenon where users stick to their chosen path through an interface, even when better options are available. The Nielsen Norman Group explains that momentum behavior occurs most often in digital products “when parts of the interface design are not strong enough to call to users when they need them,” meaning users become tunnel-visioned on their current approach and overlook features that could help them. This creates a double bind: users need to discover features quickly due to time pressure, but once they’ve chosen a workflow, they’re likely to ignore alternative approaches that might be more effective.

Once users discover an AI feature, the next opportunity emerges: guiding them toward confident, successful use. Designers can work to overcome momentum behavior and time constraints by balancing strategic moments of intervention with comprehensive education. This approach works best when supported by accessible documentation and tutorials that let users learn at their own pace outside the immediate workflow.

Users bring different learning preferences and time constraints to a task: some need quick reference guides during active work, while others prefer comprehensive tutorials they can work through when they have dedicated learning time. Some platforms manage this by fostering third-party ecosystems, where power users can publish their workflows for others to learn from, creating a community-driven knowledge base that evolves alongside the AI capabilities. These user-generated resources often bridge the gap between official documentation and the real-world application, showing creative uses that product teams might never have anticipated. For example, a recent Anthropic job posting sought a demo designer, showing that even AI orgs themselves are looking to create case study content. When community contributions are properly curated and integrated into the product experience, they transform individual expertise into collective knowledge, making advanced techniques discoverable for users who are ready to expand their skills.

A word about the speed of innovation: when thinking about discovery, it’s important to consider that AI features currently exist in a rapid state of change, requiring designers to manage a continuous feature-adoption pipeline. Traditional software follows a predictable cycle: features are developed, released, documented, and remain largely unchanged for years. AI capabilities, however, exist in what we might call a plastic state, where they’re continuously refined through model updates, new training data, and new feature opportunities. This means the adoption pipeline of priming, activation, and mastery happens repeatedly rather than once.

Users may discover a summarization feature that works one way in January, only to find it’s been enhanced with citation capabilities by March, requiring them to relearn a portion of their mental model and workflow. This is worsened by algorithmic transference, where users carry expectations from previous interactions with AI systems into new encounters, often applying assumptions that may no longer match current capabilities. Managing these expectations raises the bar for messaging about updates; it also means that designers and their product teams must anticipate confusion and unease about changes to (recently) established workflows.

Once users have discovered AI capabilities and begun incorporating them into their workflow, they face a different kind of challenge. The interface that helped them find and activate features now needs to support ongoing decisions about how those features actually work. Some of these decisions are made explicitly by users, such as whether the AI can access certain files or how outputs should be saved. Others are handled by defaults, like which model is running in the background. Only advanced users typically experiment with different models. The broader principle remains that AI systems place ongoing choices in front of people, and the interface plays a critical role in whether those choices feel clear and empowering or opaque and confusing.

Orchestration

After a user defines a task to be done, they arrive at the interface with a mixture of expectations, some clear and some implicit. They might engage with onboarding, skim a tutorial, or dive straight in without any hand-holding. Whatever their approach, they begin making a series of decisions that can dramatically shape how the system behaves for them. These decisions often happen early, but their effects can ripple throughout the entire session or project.

In AI tools, this layer of orchestration can be especially fraught. Users may assume the system is smart enough to anticipate what they need. Experienced users might expect to skip setup entirely, while newcomers may not know what to look for or even that configuration is possible. Because AI is marketed and experienced as intelligent, people often assume it will handle these details invisibly. But when expectations and system behavior fall out of sync, it becomes clear just how dependent good experiences are on well-scaffolded configuration.

There are a number of elements users can adjust to shape the system around their needs, including the following:

Model, tool or agent selection

Some tools offer multiple models, tools, or agents with different strengths, limitations, and latency tradeoffs. Users might choose a faster model for lightweight tasks or a more capable one for in-depth work. When framed well, it gives users a sense of control and clarity about what is happening behind the scenes, but it represents another layer of knowledge to gain.

Usage and metering

Transparency about quotas, rate limits, and usage caps helps prevent surprise interruptions. Whether it is a token counter, a session timer, or a monthly usage bar, these indicators give users a sense of how far they can push the tool and how to pace themselves.

Permissioning

Access and continuity are shaped by how permissions are handled. This includes who can see what, whether conversations persist across sessions, and whether an AI can retain context over time. Users need confidence that their data will not disappear, leak to the wrong place, or behave unexpectedly.

Library management

Files, assets, and project materials, once uploaded, should feel like part of the fabric of the workspace. That means offering structure such as version history, metadata, and threading where appropriate. But it also means being able to quickly retrieve, reuse, or update materials without having to re-upload or re-explain them to the system.

Across all of these dimensions, users rely on tools to consistently surface the right level of control. This is tricky in AI systems, since, as user intent is not always explicitly stated, a system that assumes continuity and treats a new query as part of a previous task may be drawing the wrong conclusion entirely. Interfaces need to be careful not to infer too much, while still offering opportunities for users to carry forward context when they want to. Getting this balance right is a critical part of making orchestration feel lightweight rather than laborious, and of helping users feel like the system is adapting to them, not the other way around.

Before diving into each element in detail, I’d like to mention a few overarching principles:

Orchestration should be intentional. In AI systems, each choice meaningfully shapes how the system enhances the input and output. Whether a user is selecting a model, granting file access, or choosing whether to save conversation history, those decisions have a direct impact on the outcome. That means users need to make choices with a clear sense of what they are choosing and why. Systems can support this by presenting options at meaningful moments, with enough information for users to understand what each choice enables or limits.

Orchestration should be transparent. Users need to understand what the system can and cannot do, and how that changes based on their setup. For example, when switching from one model to another, there should be some visible signal that capabilities have shifted. If a new permission is granted or a new plugin is turned on, users should see what has changed. This helps prevent unrealistic expectations, but it also supports discovery by highlighting new capabilities the user might not have thought to ask for - even in their absence.

Orchestration should be recognizable. If something changes, the user should notice. If they return after a few days, they should be able to recall or quickly reorient to what is available. That means labeling choices clearly, providing previews or examples when possible, and preserving a sense of continuity. Users should be able to form a mental model of how the tool behaves in its current state, and that model should stay stable until the user changes it.

With those principles in mind, the next sections walk through each of the core elements of AI configuration.

Models, Agents, and Tool Selection

Looking across today’s AI systems, you’ll find three fairly distinct layers of capability. It’s important to call out that these layers do not always appear in isolation, and in many products they blur together. Still, understanding how they build on one another can help clarify what kind of interface each one actually needs.

Models form the foundation. They are reactive, in that they respond to a prompt, but do not pursue a goal or take action on their own. Consider a model used inside a writing assistant. When a user begins drafting an email, the model suggests completions based on common phrasing, anticipated tone, or past behavior in similar contexts. It can fill in subject lines, adjust for formality, and rewrite sections on request. However, it does not retrieve background documents, nor does it schedule a follow-up or adapt to changes in recipient information.

Tools extend a model’s reach, making it possible for the system to act in more grounded and timely ways, but the user still initiates and directs each request. Instead of relying only on what the model learned during training, a tool lets it perform specific actions in the world. This could mean searching the web, reading a document, querying a database, or pulling real-time data from an external system. Tools act as bridges between the model and other software or services. ChatGPT with browsing enabled is a familiar example. When a user asks about a current event, the model uses a tool to search the web and returns information based on what it finds.

Agents add a layer of planning on top of models and tools. This is where systems begin to act with more autonomy. An agent interprets a user goal, breaks it into smaller tasks, decides what to do first, and chooses which tools to use at each step. It can adjust as it goes, retrying if something fails or changing course based on new information.

Tip

Confusingly, some products refer to chatbots as agents. There is some slippage in the term, so be very clear when you use the term agent within your product. Some prefer to use agentic, which can imply planning capability more clearly without associating a task or action with a chatbot.

Planning can involve a number of different strategies. Some agents plan all steps up front, creating a sequence to follow before doing anything. Others plan incrementally, deciding what to do next based on what just happened. In either case, planning is what allows an agent to move beyond the prompt-response rhythm of a typical model interaction. The system might look at a complex request, like building a report, and structure its work across multiple steps. It might also track dependencies, manage resources like time or API calls, and maintain a working memory across turns. This kind of behavior introduces new complexity, but it also makes the system more useful for sustained tasks that require coordination or iteration.

Each layer of this stack has different impacts on the user’s query, and therefore requires different messaging and styling. Working with a base model means focusing on input and output, formatting, and feedback about generation. Tool integrations require more support around permissioning, confirmation, and visibility into what was accessed and why. Agents introduce a deeper set of concerns: how to show what the system is doing, how to let the user pause or redirect it, and how to build trust in a system that is taking action without explicit step-by-step instructions.

Because there is a clear dependency structure across these three types, which can be easily confused, it’s useful to visualize it as a stack, as shown in Figure 2-7.

Figure 2-7. Model dependency structure

To see how this plays out, here is how a single user prompt might be handled across the three levels:

I have a client meeting tomorrow with Salesforce about integrating our CRM data. Can you help me prepare?

Model only

The system replies with general advice: review the client’s background, create talking points about integration capabilities, prepare for questions about security and API compatibility, and bring a few case studies from similar implementations.

Model + tools

The system can now gather specific information. It looks up Salesforce’s latest product announcements, pulls your meeting details from the calendar, finds relevant case studies from your company database, checks who the attendees are, and includes up-to-date API documentation. You still guide the interaction, but the system returns more tailored content.

Agent

The system takes initiative. It blocks off time on your calendar for prep, drafts a presentation using relevant integration examples, compiles a list of known stakeholders and their technical roles, creates reminders to review key materials, and generates follow-up emails for different meeting outcomes. It might continue refining this plan as new information comes in or as the meeting approaches.

As we’ve seen, a major challenge lies in communicating to users what is the best orchestration approach to fulfil their requirement. Additionally, each level comes with design implications that impact how the user comprehends and utilizes the tool.

Design considerations for models

An AI product may include a number of available models, each optimized for different types of tasks and performance characteristics. The challenge for designers lies in presenting model selection in ways that help users make informed choices without overwhelming them with technical details they may not understand or care about.

Model differences typically center around several key dimensions that matter to users. Response speed varies significantly between models. Some prioritize quick replies for conversational flow, while others take more time to produce thoughtful, detailed responses. Specialized capabilities also vary, such as coding proficiency, mathematical reasoning, creative writing, or multilingual support.

The most straightforward approach to model selection uses dropdown menus that present these options clearly, but it’s vital to consider how the differences are presented. Consider how shipping options work during checkout - instead of simply naming the service, they provide insight into the actual thing the user wants to know: how long it’ll take to arrive: Standard (5-7 days, free), Express (2-3 days, $8), and Overnight ($25). Each choice clearly communicates its time and cost implications. Model selection can follow similar patterns by focusing on outcome-oriented descriptions rather than technical specifications. Instead of offering “GPT-4”, “Claude Sonnet”, and so on, interfaces might present “Great for quick responses” or “Specialized for coding tasks.”

Many successful AI products have moved toward intelligent defaults that route queries to the most appropriate model automatically. These systems analyze the user’s input to detect signals about what kind of response would be most helpful. A request for current stock prices might automatically trigger a model with web search capabilities, while a complex reasoning problem might route to a model optimized for step-by-step analysis. This approach reduces cognitive load for users while still delivering appropriate capabilities.

Tip

The challenge with automatic routing lies in transparency and user agency. When the system makes model choices invisibly, users may not understand why responses vary in speed, style, or capability across different queries. A small badge persistently displaying the model and its benefits helps users understand why a response might take longer or include different types of information.

Model selection is generally a more complex proposition in AI-first tools, since the user intent may be more ambiguous. Someone typing Help me with this presentation could mean anything from creating slides from scratch to polishing existing content. This ambiguity makes automatic model selection difficult, often forcing the burden of choice onto users. By contrast, AI-enhanced tools embed AI within specific contexts that reveal user intent more clearly. When someone selects text in Microsoft Word and chooses Summarize, the tool provides strong signals about what kind of AI assistance is needed.

The breakneck speed of model development means that there are sometimes clashes between business needs and user experience. OpenAI has developed a large suite of models, which are presently displayed under a dropdown selector, as shown in Figure 2-8. However, this choice presentation is regarded by some users as suboptimal, with OpenAI CEO Sam Altman admitting to issues with the high number and method of displaying models.

Figure 2-8. ChatGPT’s model selector

Design considerations for tools

Tools represent discrete capabilities that extend what models can do, connecting them to external systems and real-world data. Unlike models, which process and generate content, tools are augmenting forces that help the model perform specific actions: searching the web, reading files, executing code, or querying databases. A useful way to think about them is as APIs the model can call to get more information or carry out a job. Each tool type brings its own considerations for permission management, result presentation, and error handling.

As the industry matures, we are beginning to see standards emerge that make the interplay between tools and AI models more effective and maintainable. Frameworks like the Model Context Protocol (MCP) formalize how a tool describes its capabilities, what information it requires, and how that information should be structured so the model can use it and return results reliably in the interface. This is part of a larger paradigm shift: tool design is no longer only about what users see, but also about setting clear rules that allow models and interfaces to work together in predictable ways.

Common tool categories include information-retrieval tools, like web search and document access, computational tools like calculators and code-execution environments, and integration tools that connect to external services like calendars, email, or specialized databases. Each of these categories poses unique challenges for interface design, since users bring different expectations and trust thresholds to each one.

Tool palette design follows several established patterns, each with distinct trade-offs. Sidebar approaches, similar to those in design tools like Photoshop or Figma, present tools as a persistent set of options that users can select manually. This makes capabilities transparent and puts users in control, but can feel cluttered and may interrupt conversational flow. Contextual activation hides tools until they are needed, detecting when specific capabilities would be useful and surfacing them automatically. This keeps interfaces clean but relies on the system correctly interpreting user intent.

Many successful implementations use hybrid approaches that combine automatic tool activation with manual override options. The system might automatically search the web when detecting queries about current events, while still allowing users to disable web search or explicitly request it for other types of queries. These interfaces often use lightweight cues like natural language commands (“search the web for…”) or syntax hints (typing @web or /search) to give experienced users precise control.

User preferences around tool usage deserve particular attention. Some users prefer to maintain control over which tools get activated, while others want seamless automatic behavior. Interfaces should allow users to opt in and out of preserving these preferences, creating personalized tool behavior that adapts to individual comfort levels and use cases. A user might enable automatic web search by default but require explicit permission for calendar access, reflecting different levels of trust and privacy concerns across tool types.

Design considerations for agents

Agents introduce the most complex design challenges, because they combine models and tools with autonomous planning capabilities. Unlike the request-response pattern of model interactions, agents work toward goals over multiple steps, making decisions about what to do next based on their assessment of progress and changing conditions.

The primary design challenge centers on transparency. Users need to understand what the agent is doing, why it’s making specific choices, and how they can influence or interrupt the process. Chain-of-thought (or chain-of-reasoning) displays (as shown in Figure 2-9) serve this purpose by showing the agent’s reasoning process as it unfolds, allowing the user to make decisions about the amount of information they want about how a query is progressing. Instead of presenting only final results, these interfaces reveal intermediate steps: I need to research this topic, so I’ll start by searching for recent articles... Now I’ll cross-reference these findings with academic sources... Based on this information, I’ll create an outline...

Figure 2-9. Chain-of-thought patterns in Claude

As an element of chain-of-thought, progress indicators become crucial for agents because their work often takes significantly longer than simple model responses. These indicators tend to go beyond model latency patterns, with a higher quantity of information generally required. They can be unpredictable: for this reason linear progress bars work poorly for agent tasks, because the number of steps isn’t always predictable. Instead, many interfaces use status indicators that describe current activities: Researching market trends, Analyzing competitor data, or Drafting recommendations.

Interruptibility represents another critical design consideration: because of the longer elapsed time and increased computational cost, users need clear mechanisms to pause, redirect, or cancel agent actions. Mechanisms might involve pause buttons that stop the current step while preserving progress or redirect options that let users modify the agent’s approach mid-task.

Agent-to-agent coordination occurs when multiple specialized agents work together on a single task. A research project might involve one agent gathering information, another analyzing it, and a third drafting a report. These represent valuable opportunities for educating the user, and as a result providing visibility into these handoffs is important. Similarly, tool integration within agent workflows requires careful choreography. When an agent uses multiple tools in sequence, users should understand which tool is active, what information it’s accessing, and how that information feeds into the next step. Real-time logging in the chain-of-thought can provide this transparency, showing a running record of agent actions and tool usage.

Error handling in agent systems needs to be more sophisticated than simple retry mechanisms. When an agent encounters a problem partway through a multi-step process, it should indicate the point of failure and announce whether resumability is possible. The interface should clearly communicate when problems occur and what options are available for resolution.

Warning

Because it takes control away from the user, trust building becomes paramount in agent design, as users must feel confident allowing the system to take autonomous actions on their behalf. This involves not just transparency about current actions, but also clear communication about the agent’s capabilities and limitations, explicit consent mechanisms for sensitive operations, and consistent behavior that users can learn to predict and rely upon.

Usage and Metering

In 2014, a team from Facebook’s Android engineering team travelled to Africa to review mobile performance in developing countries. There, they learned that the data-heavy app they’d created was prone to crashes due to local bandwidth issues, and that devices available on the local market lacked the necessary storage, leading to crashes. As a result, the team burned through their monthly data plan in 40 minutes.

Cost per pageload isn’t a commonly surfaced consideration in product design. An app can make dozens of third-party API calls: payment processing, address validation, image recognition, and so on, but these usually result in a stable, definable cost.

With AI, the cost variability can be extreme. A simple text completion might cost a fraction of a penny, whereas a complex reasoning task could cost dollars. The same interface element can trigger wildly different computational costs depending on what the user types and how the system responds.

As a result, many AI tools assist users in working within usage boundaries, often tied to pricing tiers, model constraints, or technical limits. These boundaries shape what users can do and for how long. For example, some systems may restrict access to more intensive capabilities like long-context reasoning or multi-turn memory, offering only a set number of these sessions per day or month. Others might meter tokens or cap the number of concurrent projects or file uploads, or handle this entirely in the background without showing it to the user.

Converting the dollar value into another token is a common way of softening negative feelings about cost. The image-generating tool Visual Electric converts generation costs into “Volts” and discloses the cost of each query as a tooltip upon submission, as shown in Figure 2-10.

Figure 2-10. Visual Electric ensures users know the cost of each query upon submission. Since the user gets a certain number of “volts” per month, this allows them to visualize how many each query uses.

This style of metering benefits from upfront disclosure: if a user has access to five deep reasoning sessions per day, it’s preferable for them to know that before they start using one, like in ChatGPT’s configuration menu, shown in Figure 2-11. If a model becomes slower or less accurate after a certain quota is reached, the interface should communicate that shift. This is especially important in tools that encourage frequent experimentation: without some guardrails or signals, users may unknowingly use up valuable capacity on low-priority tasks.

Resumability is also key; if a user runs out of credits, the system should preserve their progress and make it easy to pick back up once access is restored. That might mean saving the state of an in-progress session, queuing actions that couldn’t complete, or offering lightweight summaries so the user can reorient themselves later. Without this, users are penalized twice: first by the interruption, and again by the effort it takes to reconstruct what they were doing.

Nobody likes talking about money, but AI is wildly expensive. Despite this, usage and metering can be designed to support the user’s exploration and discovery, provided they have a clear sense of what the system can offer right now, and what will be possible later, in order to plan their work and stay in control. Just as with other configuration elements, the interface should make usage visible, intentional, and easy to learn over time.

Figure 2-11. ChatGPT’s interface showing metering limits for deep research

Permissions and Capabilities

Permission requests play an important role in AI interfaces because they determine how data is shared and, just as importantly, how much control users feel they have in that process. This sense of agency becomes especially important as AI systems handle increasingly complex and personal tasks.

The most immediate concern is scope of access. Unlike a traditional app that might ask for permission once to access your photos or contacts, AI systems often need dynamic access to multiple data sources to provide useful responses. A tool might need to read your brand guidelines, access your design files, and potentially modify them. Each of these actions carries different levels of risk and user comfort.

In agentive tasks, permissioning also gives the user insight into what tasks the model is accomplishing. Since, by nature, the model will sometimes request permissions for tasks or access that are foreign to the user, it’s important to find a balance between fully educating the user and developing trust that allows the agent to proceed in a timely way. If a project requires a large number of permissions, it can be inefficient to educate at each step, and may instead be useful to add the option to give persistent permissions. In Figure 2-12, Claude Code manages this by using a tiered permission framework that looks at the type of query and compares this to potential risk.

Figure 2-12. Claude Code’s permission framework

Granular control becomes essential when dealing with data that carries different levels of sensitivity or relevance. Rather than forcing users into binary choices between all file access and no file access, you should design permission systems that reflect how people actually think about their data. Options like access to files you upload in this conversation, access to files in your project folder, or access to all files let users match the system’s capabilities to their comfort level and specific use case.

The ability to revoke permissions should be as straightforward as granting them. Users need to easily see what permissions are currently active and modify them without hunting through nested settings. A clear permissions panel accessible from the main interface reassures users that they maintain control over their data and the system’s capabilities.

It’s challenging to create permission systems that feel like negotiations between partners rather than demands from gatekeepers. At the heart of permissioning is education, which ultimately leads to more trust and more effective user experiences.

Library Management and Assets

Library management involves the structured handling of files, datasets, prompts, reference materials, or saved conversations that may be used by the AI or by the user across sessions. Since AI systems increasingly support uploading, generating, referencing, and reusing external content, how these assets are stored, surfaced, and retrieved is becoming increasingly important. Decisions made here reveal a lot about a product’s assumptions about the user’s intent and context.

Some platforms treat uploads as ephemeral: a document is parsed once and then discarded. Others offer persistent libraries or asset drawers, where files remain attached to the workspace and can be revisited over time. Library design also affects how users manage versions and sources. If a user uploads a revised document, can the system distinguish it from the earlier version? Can the AI reference multiple documents at once and attribute sources clearly in the output?

Here are some questions design teams could ask:

What is the lifecycle of a resource? Do prompts decay, do outputs get versioned, can users archive or snapshot a session?

How do assets travel between modes or tools? For instance, can a summarization generated in one thread be reused in another, or attached to a project?

How does search work across different asset types? Can users find a specific exchange within a long conversation, or locate a document by its content rather than filename?

Ensuring that users can find their assets when they need them requires thoughtful organization patterns. Traditional file browsers may feel familiar, but they don’t necessarily match how people think about AI-generated content. Some teams have explored tagging systems, visual previews, or automatic clustering by topic or project. The challenge lies in balancing discoverability with simplicity, especially when users accumulate dozens of conversations or generate multiple iterations of similar content.

Tip

The most successful library systems often mirror how users already organize their work. A design tool might group assets by project, while a research platform could organize by methodology or timeline. Consider how your users currently manage similar resources outside your product: their mental models for organization can inform more intuitive digital experiences. Remember that library management isn’t just about storage; it’s about enabling users to build on their previous work and maintain continuity across sessions.

Summary

Every AI interface operates in the space between what a model can do and what a user is trying to accomplish. If that space isn’t carefully designed, even the most powerful systems can feel confusing or forgettable. This chapter offered a structure for working in that space: by understanding and communicating capability, supporting discovery, and designing effective orchestration.

We began with a product that failed. Google Wave, despite its technical depth, left users unsure how to start, what it was for, or how it would fit into their workflows. The underlying capability was there, but it was inaccessible. The story of Wave captures something important about AI interfaces today: in both cases, users encounter unfamiliar tools that promise transformation, but often struggle with guidance or framing. What designers create may be impactful, but unless people can find the right entry points, it won’t land successfully.

Throughout the chapter, we looked at the shape of capability: how models are trained, how they’re evaluated, and how those characteristics become constraints or openings for design. Benchmarks like MMLU, GSM8K, and MT-Bench offer partial signals, but they don’t automatically translate into meaningful experiences. Even designers with a general understanding of AI can find it difficult to imagine strong interactions without clearer insight into what models can actually do - that handoff from performance to experience depends on framing and exposure.

Discovery plays a key role in shaping that exposure. Whether through ambient cues, content-aware triggers, or direct input, systems need ways to suggest possibility without interrupting the flow. Discovery works best as an ongoing relationship, especially in AI products where capabilities shift and expectations evolve. When it misfires, by being too subtle, too forceful, or too abstract, users lose momentum and may carry forward limiting assumptions.

Orchestration helps address this gap. It includes configuration choices like model selection and permissions, but also shows up in how the interface responds to shifting user intent. Good orchestration allows progress without requiring users to pause and study the system. It supports the rhythm of interaction through the timing of features, the framing of options, and the quiet handling of context. We also looked at momentum behavior, where users tend to stay in familiar patterns. This creates tension for AI features, since guidance is often most needed when users are least likely to seek it. The strongest systems stay present in the interaction without demanding attention.

In the next chapter, we’ll turn our attention to inputs. This is where user intent begins to take shape in the system, whether through typed prompts, uploaded documents, or environmental cues. The way inputs are designed determines not just what users can ask, but how clearly they understand what the system needs from them.

Chapter 3. Designing for AI Inputs

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 3rd chapter of the final book. Please note that the GitHub repo will be made active later on.

If you’d like to be actively involved in reviewing and commenting on this draft, please reach out to the editor at ccollins@oreilly.com.

What does it mean to speak to a computer? The idea of input has meant different things over time, changing as technological complexity evolved. Early computing services were highly constrained, allowing for only simple, codified inputs that told the computer to run one program or another, requiring high user knowledge and technical skill.

The origins of inputs lie in the early days of computing: in the 1960s, a group of researchers at MIT developed a time-sharing operating system called Multics (Multiplexed Information and Computing Service). This experimental platform introduced many of the concepts we now take for granted in modern computing, such as hierarchical file systems, dynamic linking, and virtual memory. But with regard to this chapter, its most important contribution was something simpler: the command shell. Figure 3-1 shows Multics in action.

Figure 3-1. The command room and mainframe of a Multics installation [link]

A command shell is a text-based interface where users type instructions and the computer responds. When you type a command like ls or dir and hit Enter, the shell interprets that text, executes the corresponding program, and displays the results underneath. It acts as an interpreter between human language and machine operations, taking typed commands, figuring out what program you want to run, and then showing you what happened. If you’ve ever opened Terminal on a Mac (see Figure 3-2) or Command Prompt on Windows, you’ve seen a modern version of the command shell.

Figure 3-2. The Terminal interface on a Mac computer; a contemporary example of a shell

What makes shells particularly interesting from a design perspective is their nascent conversational nature. You type something, the computer responds, then waits for your next instruction. This back-and-forth creates a dialogue, though one that demands precision, in a way that’s markedly different from what we might think of as speech. Vitally, the shell has no tolerance for typos or incorrect syntax. Miss a character or use the wrong flag, and you’ll get an error instead of the result you wanted.

You also need to know exactly what words to say. The Multics command shell established a direct conversation between human and machine, but it was a conversation conducted entirely in the machine’s preferred language. Users had to memorize specific commands and formatting rules that allowed for no deviation. A single misplaced character could render an entire command useless, forcing users to start over. This rigid framework, while powerful for those who mastered it, created an enormous barrier between human intent and machine execution. This command line interface (CLI) represented a world where the burden of communication rested entirely on the human participant, demanding that users translate their goals into the computer’s preferred vocabulary through extensive documentation, training, and practice.

Yet this constraint also drove remarkable innovation in precision and efficiency among those who became fluent in command-line syntax. As the technology became more sophisticated, the need for precise or pre-determined inputs relaxed a little: the emergence of search engines marked the first significant departure from this strict command-response paradigm. Google and its competitors introduced the revolutionary concept that computers could interpret loosely structured queries, and infer user intent from ambiguous or incomplete information. Users no longer needed to specify exact parameters or follow rigid syntax rules: instead, they could type natural language questions and receive relevant results, even when their queries contained typos or imprecise terminology. This shift represented a fundamental philosophical change: rather than demand that humans communuicate with a computer in code, search engines began the work of understanding human language.

Today’s large language models (LLMs) represent the culmination of this evolutionary process, creating interfaces that not only interpret human intent but generate entirely new content in response. These systems engage in genuine dialogue, building on previous exchanges and maintaining context across extended conversations. Users understand this intuitively, and rapidly think up queries they can utilize as a result. A 2024 comparative study revealed telling preferences among users when choosing between traditional search engines and LLMs. For straightforward, fact-based queries, users still preferred search engines, appreciating their speed and direct access to authoritative sources. However, for tasks requiring nuanced understanding, creative synthesis, or complex language processing, users consistently gravitated toward LLMs, valuing their ability to provide tailored, contextual responses that felt more like consulting with a knowledgeable colleague than querying a database.

This evolution from command-line interfaces to conversational AI represents more than just technological progress; it reflects a fundamental shift in how we conceptualize the method of communicating and working with a digital tool. The transformation from command-based to conversational interfaces fundamentally redefines the user-system dynamic.

This chapter explores how prompts work with LLMs, what it means to construct a successful prompt, and how to guide users towards them. We’ll also look at existing prompt paradigms and examine how they enable or encourage specific usage by users in order to crystallize their ideas and contribute to collaborative problem-solving. We’ll wrap up by emphasizing how enabling meaningful LLM interactions requires the recognition that human communication is inherently contextual, nuanced, and exploratory.

Traditional command-line interfaces established a master-servant relationship where users issued precise instructions that the system executed. Modern AI interfaces invert this relationship, creating a collaborative partnership where both parties contribute to achieving the desired outcome.

From Multics to Google Search: Developing the Input

While the command-line interface might not be in frequent use by designers today, the evolution in command-line style interfaces that followed it certainly is. The creation of the search tool called Google, founded in 2001, illuminates a shift not just in interface design but in how users conceptualize control, syntax, and information retrieval.

As we’ve seen, the command line offered a powerful but unforgiving way to interface with a machine. Google offered a somewhat similar approach: a command issued, a response received. But where CLIs required fluency in a system’s structure and vocabulary, Google invited a far wider range of inputs, with a safety net for new and naive users.

Whereas the interface of earlier search engines returned results based largely on keyword matching, Google prioritized relevance through link analysis, allowing users to find high-quality information that was previously undiscoverable. To that end, Sergey Brin and Larry Page created PageRank, an algorithm that assessed the importance of web pages by analyzing the network of hyperlinks that led to and from them. Their inspiration arose from their academic roots: both Brin and Page grew up in academic families, with fathers who were professors and researchers. In academic publishing, the number and quality of citations a paper receives serves as a proxy for its importance and reliability within the scholarly community. Papers that are frequently cited by other respected works gradually build authority and influence. Page and Brin recognized that the web’s hyperlink structure created a similar citation network, where websites could be evaluated based on how often other sites linked to them and on the authority of those linking sites.

This insight proved revolutionary, because it moved beyond simple keyword matching to evaluate the credibility and importance of web content. Traditional search engines of the era focused primarily on textual analysis, counting keyword occurrences and matching user queries to page content. PageRank introduced the concept that the web’s own link structure contained valuable information about quality and relevance. A page that received links from many other respected sites was likely to be more valuable than a page with identical keyword content but few incoming links.

Figure 3-3. Google in 2001

The inviting style of the Google interface (see Figure 3-3) drew users to explore and play with search, preserving the command line’s underlying pattern (you type something, the system tries to fulfill it) but removing the need for exact syntax. Users could type loosely structured questions, fragments, or even whole sentences. They could even misspell every word in the query, and it would be corrected (to the nearest guess). Google developed properties, like knowledge panels, embedded maps, definitions, and unit converters, which enhanced the experience further, making users feel that their queries were being deeply understood, fulfilled - even anticipated. People often searched for quick facts, directions, conversions, or comparisons, and these modules brought the answer closer to the point of intent.

Over time, this user behavior has become even more finely tuned. Users have come to anticipate that Google will know if they search New York Boston that Google will show them flights between New York and Boston - unless there’s a game between the Yankees and Red Sox happening that day, in which case Google might surface details about the game. Search strategies have emerged: adding filetype:pdf or appending Reddit for more authentic results, representing a learned grammar for shaping inputs, even in a natural language interface.

This recurrent optimization behavior reflects the knowledge gained in the nearly 25 years since Google launched: how to structure a good query for Google, how to phrase it, tune it, and what to expect. And it’s exactly this knowledge that users don’t yet have for AI products. The input box of an LLM may resemble Google’s and may even return the same sources, but a vastly different technology is under the hood. The input box of an LLM may resemble Google’s and may even return the same sources, but a vastly different technology is under the hood. What users need to learn now is how to prompt as effectively as they once learned how to search. The faster they gain that skill, the sooner they can move beyond fact-finding into tasks that search alone cannot handle: creating original explanations, adapting information to specific contexts, combining insights from multiple domains, and engaging in extended dialogue that builds understanding over time.

Prompting Basics: The CARE Model

Prompting is the practice of designing inputs to language models to elicit desired behaviors and outputs. While conceptually straightforward, effective prompting requires understanding how these models process and respond to text.

LLMs like GPT and Claude are trained to predict the next token, roughly equivalent to a word, in a sequence based on patterns learned from training on massive text datasets. When you submit a prompt, the model uses statistical relationships to generate what it calculates to be the most probable continuation of your input text. This probabilistic nature means that prompting is fundamentally different from traditional programming. You’re not issuing commands that execute deterministically (that is, execute the same way every time). You’re providing context that influences a complex statistical process. Small variations in wording, structure, or framing can significantly alter the model’s output distribution.

Effective prompting leverages several key techniques that have emerged from both research and practical application:

Few-shot learning provides examples within the prompt to establish patterns the model should follow, essentially teaching through demonstration rather than explanation.

Chain-of-thought prompting encourages step-by-step reasoning by asking the model to show its work, which often leads to more accurate and reliable outputs for complex problems.

Role-based prompting frames the interaction by having the model adopt a specific persona or expertise level, helping to align the response style and depth with user needs.

Prompt engineering techniques can establish consistent behavior patterns across interactions, while retrieval-augmented generation combines prompting with external knowledge sources to ground responses in current information rather than relying solely on training data.

A simple prompt might be something like Write a thank-you email. But effective prompting often requires more detail about context, constraints, and desired outcomes. You might specify the recipient, the reason for thanks, the tone you want, or even examples of similar emails you’ve written before. The art lies in providing enough information for the AI to understand your intent without overwhelming it with unnecessary details.

Encouraging effective prompting creates an interesting design problem. Natural conversation relies heavily on shared context and implicit understanding. When you ask a coworker to make this presentation better, they already know your company’s style, your audience, and what better means in your specific situation. AI systems, on the other hand, have to reconstruct all of that from the words you provide.

The most effective prompts tend to include relevant background information, clear descriptions of the desired outcome, and any important constraints or requirements. But there’s a balance to strike. Overly long, complex prompts can actually degrade performance, as the system struggles to parse what matters most. Meanwhile, prompts that are too brief might produce generic or off-target responses.

What makes prompting particularly fascinating as an interface pattern is that it inverts traditional software interaction. Instead of learning the software’s language of menus and commands, users can express their intent in their own words. The burden shifts from the user memorizing specific procedures to the system interpreting human intention.

But successful prompts in a product context also need to feel natural to create. A prompt that works beautifully but requires five minutes of careful crafting isn’t sustainable for everyday use. The best input interfaces help users create effective prompts without thinking about prompt engineering.

The Norman Nielsen Group’s CARE Model provides a comprehensive framework for understanding how effective AI prompts should be structured, breaking down the essential elements that bridge the gap between human communication patterns and AI system requirements. This model recognizes that successful AI interactions depend not on users becoming prompt engineers, but on interfaces that naturally guide users toward including the information that AI systems need to provide genuinely helpful responses.

CARE stands for context, action, results, examples.

Context

Context forms the foundation of the CARE framework, representing the relevant background information that helps AI systems understand not just what users are asking, but why they are asking it and how the response will be used. Context encompasses the user’s role, their audience, their constraints, and the broader situation that gives meaning to their request. When a user asks an AI to help me write a proposal, the system needs to understand whether this is a business proposal to potential clients, a research proposal to academic reviewers, or a project proposal to internal stakeholders. Each scenario requires dramatically different approaches, tones, and content structures. Effective interfaces help users provide this contextual information naturally, perhaps through brief setup questions, smart defaults based on user history, or prompts that encourage users to describe their situation before diving into specific requests.

Consider the difference between the following two approaches to requesting help with email communication:

Weak prompt: Write an email about the project delay.

Strong prompt with context: I’m a project manager at a software startup, and I need to inform our client (a conservative financial services firm) that our mobile app development will be delayed by three weeks due to unexpected regulatory compliance requirements. The client has already pushed back their launch date once before, and maintaining their trust is critical for our ongoing relationship.

The contextual information transforms a generic request into something the AI can address with appropriate tone, level of detail, and strategic considerations. The system now understands the professional relationship, the sensitivity of the situation, and the communication challenges involved.

Action

Action specifies what users want the AI system to actually do with the information users provide. This goes beyond simple task identification to include the specific type of processing, analysis, or transformation desired. Rather than asking an AI to look at this data, users benefit from specifying whether they want the system to summarize key findings, identify patterns and trends, compare different segments, or generate recommendations based on the analysis. Clear action specification helps AI systems choose appropriate methodologies and focus their processing power on the most relevant aspects of the task. Interface design can support action clarity through structured prompts that help users think through their actual objectives, example templates that demonstrate different types of actions, or progressive questioning that helps users refine their requests from general intentions to specific actions.

The power of precise action specification becomes apparent when comparing vague versus specific requests:

Weak prompt: Help me with this sales data.

Strong prompt with action: Analyze this quarterly sales data to identify which product categories showed the strongest growth, which geographic regions underperformed compared to targets, and what seasonal patterns might explain the variations. Then synthesize these findings into three key strategic recommendations for next quarter’s sales focus.

The specific actions, to identify growth patterns, analyze geographic performance, explain seasonal variations, and synthesize recommendations, give the AI clear direction about the type of analysis to perform and how to structure the response. This prevents the generic Here’s what I see in your data response that often results from vague requests.

Results

Results address what the output should look like, including format, length, style, and level of detail. The results element recognizes that the same underlying request might require vastly different presentations depending on how the user intends to use the response. For example, a summary of quarterly sales data might need to be a detailed analytical report for a management presentation, a concise executive brief for leadership review, or a visual dashboard for ongoing monitoring. Result specification helps AI systems tailor not just their content but their structure, tone, and emphasis to match user needs. Interfaces can facilitate result specification through format options, length selectors, or audience-specific templates that automatically adjust output characteristics.

The impact of result specification becomes clear when examining how the same request might be shaped for different outputs:

Basic request: Summarize this market research report.

For an executive briefing: Create a 2-page executive summary that highlights the three most critical market opportunities, includes specific revenue projections, and concludes with clear go/no-go recommendations. Use bullet points for key findings and write in a confident, decision-oriented tone suitable for C-level presentation.

For a team meeting: Develop talking points for a 15-minute team presentation that explains the research methodology, walks through the key customer insights, and generates discussion about implementation priorities. Include questions to prompt team engagement and write in a collaborative, exploratory tone.

For client communication: Craft a professional but accessible summary that demonstrates our thorough market analysis while positioning our recommended strategy. Structure it as a client-ready document with clear sections, supporting data, and next steps. Maintain a consultative tone that builds confidence in our expertise.

The preceding result specifications will produce fundamentally different outputs from the same source material, demonstrating how format, length, audience, and purpose shape AI responses.

Examples

Examples represent perhaps the most powerful component of the CARE framework. Providing concrete samples of desired output helps AI systems understand user preferences and expectations in ways that abstract descriptions cannot capture. Examples work by showing rather than telling, allowing users to demonstrate the style, structure, and quality they seek without having to articulate these preferences explicitly. When a user provides examples of well-written product descriptions from their industry, the AI can infer tone, technical depth, feature emphasis, and formatting preferences that would be difficult to specify through instructions alone. Examples also help users clarify their own expectations, often revealing preferences they had not consciously recognized.

The transformative power of examples becomes evident when comparing instruction-only prompts with those that include concrete samples:

Instruction-only approach: Write product descriptions that are engaging and informative.

Example-enhanced approach: Write product descriptions in the style of these examples:

Example 1: ‘The Alpine Pro Series tent doesn’t just shelter you from the storm—it transforms your basecamp into a fortress of comfort. Triple-reinforced corners withstand 80mph winds while the revolutionary breathable floor system eliminates condensation that ruins other high-end shelters. When every ounce matters and failure isn’t an option, serious mountaineers choose Alpine Pro.’

Example 2: ‘Your morning coffee ritual deserves the Precision Brew X1. This isn’t another expensive gadget—it’s the culmination of three years perfecting temperature curves that unlock flavors hiding in your favorite beans. The ceramic heating element maintains 203°F throughout the brewing cycle while the integrated grinder preserves volatile oils that paper filters destroy.’

Match this confident, benefit-focused tone while emphasizing technical credibility and emotional connection to the product experience.

The examples communicate tone, structure, technical detail level, and emotional appeal more effectively than any list of instructions. They show the AI exactly what engaging and informative means in this specific context.

These four elements—context, action, results, and examples—work synergistically rather than independently, with each component informing and strengthening the others. Context helps determine what actions are most appropriate and what results will be most useful. Examples can clarify both the intended action and the desired result format while providing additional context about user preferences and standards. Result specifications can reveal contextual information about audience and purpose that users might not have explicitly stated. This interconnected nature suggests that effective AI interfaces should help users think through all four elements while recognizing that they may approach them in different orders or with varying levels of completeness.

Consider how all four elements work together in a comprehensive prompt:

I’m preparing for a board presentation next week where I need to propose expanding our customer service team [Context]. Analyze our current support ticket data from the past six months to identify peak volume periods, average resolution times by issue type, and customer satisfaction trends. Then create a data-driven argument for why we need additional headcount [Action]. Format this as a 10-minute presentation outline with key statistics highlighted for slides, compelling talking points that address potential budget objections, and a clear ROI calculation that justifies the investment [Result]. Here’s an example of how our CEO prefers data presentations: [Example of previous successful board presentation format] [Examples].

This prompt integrates context (board presentation, budget considerations), action (analyze data, create argument), result (presentation format, specific structure), and examples (CEO preference) to give the AI everything needed for a highly targeted response.

Tip

The interface design implications of the CARE model extend far beyond simply creating four input fields labeled Context, Action, Result, and Examples. Instead, the model suggests designing conversational flows that naturally elicit this information through intuitive interactions. This might involve smart onboarding sequences that help users establish context for their typical use cases, template libraries that demonstrate different action types through examples, or iterative refinement tools that help users clarify their result specifications through successive attempts. The goal is to make the provision of CARE elements feel natural and helpful rather than bureaucratic or constraining.

Prompt Intent

Users may have a range of goals with a prompt that aren’t yet revealed at the start of their process. What starts as copying and tweaking quickly becomes its own pattern of usage. Some users become more instructive, others more iterative or exploratory. Prompting begins to function less like a static input field and more like a form of interface work.

While prompting often appears to be a single action—you write something and hit Enter—it plays many roles. Some prompts define tasks. Others express tone or identity. Still others act as corrections or refinements. The following categories aren’t mutually exclusive, but it’s helpful to name the patterns that commonly emerge.

Instructive prompts

These frame a clear task with specific requirements: Summarize this article in 3 bullet points. They work best when the model is being asked to produce a defined output type, like a list, paragraph, or title.

Exploratory prompts

These open up a possibility space: What are some creative ways to introduce a product launch? They often invite the model to act more like a collaborator than a tool, and the results are typically more varied.

Role-based prompts

These shape tone, style, or expertise by assigning a fictional role: You are a friendly but experienced travel agent… These prompts help constrain the model’s response behavior in more stylistic or tonal ways.

Corrective prompts

These show up as part of iterative refinement: Make this more concise or That wasn’t what I meant, please try a version that assumes the reader already knows the context. These reflect users reacting to previous outputs and treating the model more like a revisable partner than a single-shot generator.

Boundary-testing prompts

These include edge-case queries, adversarial asks, or curiosity-driven instructions: What’s something you’re not supposed to say? or Try to solve this riddle. While informal, these behaviors help shape user mental models and surface the system’s limits.

Not all users will be drawn to these prompt styles, and they will evolve over time. Some users begin with instructive patterns and gradually develop a more exploratory rhythm. Others bounce between types depending on the task at hand. But regardless of type, prompting becomes a kind of interaction literacy. It’s not just a way to get the model to do something: it’s how users learn to work with it.

Guiding Users Toward Good Prompts

Once you understand the challenges of making a good prompt, and the style of approach users tend to use, then you can reflect on the best way to support the user in completing their task efficiently and successfully. There are two elements at play here:

Providing immediate improvements to the prompt

Building users’ understanding and knowledge to elicit better prompting behavior in the future.

Immediate improvements in today’s AI interfaces rarely involve changing what users type. Instead, improvements provide scaffolding around the input to increase the chances of success. When someone approaches a blank text field, the interface might offer category buttons above it: Creative Writing, Analysis, Code, or General Question. Clicking one doesn’t modify their prompt but may route it to a specialized model or may adjust parameters behind the scenes. This scaffolding helps users succeed without requiring them to understand the technical details of model selection or parameter tuning.

Templates and presets represent the most common form of immediate improvement. Rather than ask users to craft complex prompts from scratch, interfaces offer structured starting points. A Meeting Summary template might include fields for participants, key decisions, and action items. A Code Review template could have sections for the code to review, specific concerns, and desired programming language. These templates don’t force users into rigid structures; they provide optional frameworks that make success more likely.

Some interfaces take this further with progressive disclosure. The main input remains simple, but clicking an Advanced toggle reveals additional fields: Tone, Length, Format, or Audience. This approach respects users who want simplicity while supporting those who need more control. The interface improves outcomes by making relevant options discoverable without overwhelming the initial experience.

Tip

The key insight is that immediate improvements work best when they feel like helpful suggestions rather than corrections. A dropdown menu of common tasks, expandable sections for additional context, or visual previews of what different options produce all serve to guide users toward better outcomes without implying they were doing something wrong.

Building understanding follows a different path, one that mirrors how users learned to “Google well” over years of practice. Success breeds familiarity. When someone uses a template and gets excellent results, they begin to internalize what made it work. They notice that including audience information produces more targeted writing, or that specifying output format prevents unwanted variations. This learning happens through repetition and positive reinforcement rather than explicit instruction.

Product consistency plays a crucial role in this learning process. Just as Google’s search box has remained fundamentally unchanged for decades while the technology behind it evolved dramatically, AI interfaces benefit from stable input patterns. When the location of templates, the style of suggestions, and the basic interaction flow remain consistent, users can develop muscle memory and confident expectations. They know where to find the options they need and how to structure their requests for best results.

This consistency extends to the subtle patterns that successful users develop. Someone might learn that starting analytical prompts with Step by step produces more thorough results, or they find that including example output in their request clarifies expectations. These discoveries become part of their personal toolkit, refined through experience rather than taught through tutorials.

The spectrum of guidance provides a framework for delivering both immediate help and long-term learning. At the invisible end, consistent placement and behavior of interface elements helps users develop intuition. The same templates appearing in the same location, triggered by the same actions, create a stable learning environment. Success patterns emerge naturally when the interface remains predictable.

Moving along the spectrum, visible scaffolding like category selectors or optional fields provides structure without prescription. Users can engage with these elements when needed or ignore them when confident. The persistence of these options in familiar locations means users always know where to find help, even if they don’t always need it.

Structured guidance through templates occupies a sweet spot on this spectrum. Templates provide immediate value by increasing success rates while teaching through example. A well-designed template shows what information tends to be valuable without requiring deep understanding of why. Over time, users internalize these patterns and may begin crafting prompts that naturally include the same elements without needing the template structure.

At the explicit end, some interfaces offer example galleries or prompt libraries. These showcase successful patterns from other users or demonstrate advanced techniques. While not everyone engages with these resources, their consistent availability creates opportunities for growth when users are ready.

Among all these approaches, prompt templates emerge as particularly successful because they balance immediate utility with educational value. They work because they don’t try to fix user input but rather provide a proven framework that users can adapt. A template named Product Comparison doesn’t correct someone’s attempt to compare products; it offers a structure that has worked well for others, which users can fill in, modify, or abandon as they see fit.

The evolution from template dependence to template transcendence marks real user growth. Initially, someone might rely heavily on templates, carefully filling in each field. As patterns become familiar, they might start typing directly but mentally reference the template structure. Eventually, they develop their own sophisticated approaches that go beyond what any template could provide. This progression happens naturally when interfaces remain stable enough for learning to accumulate while being flexible enough to support growth.

The most successful AI interfaces recognize that building user understanding is a long game. Like Google’s search box, which taught millions of people to search effectively through consistent experience rather than explicit instruction, AI inputs work best when they provide stable, predictable patterns that users can master over time. Immediate improvements through scaffolding and templates solve today’s problems, while consistency and familiarity build tomorrow’s expert users.

The Hidden Work of Starter Prompts

Most users come to an AI tool with a goal in mind. They’re trying to finish a proposal, clean up their writing, analyze something faster, or just move a task along. What’s missing is clarity about how this interface can help. That’s where starter prompts do their work.

These short suggestions, often embedded in placeholder text, pinned examples, or empty states, serve multiple purposes beyond helping users get started. They’re a form of product positioning. A prompt like Write a short poem about autumn might feel inviting, but if the product is designed for financial analysis, that suggestion doesn’t help a user who’s sitting down with a spreadsheet and a deadline. Users need more than a sense of what’s possible; they need to see what’s possible here.

Starter prompts carry an enormous burden of representation. In a few short phrases, they must communicate what the tool does, how to use it effectively, and what kind of relationship the user can expect to have with it. They’re simultaneously functional, educational, and philosophical. They teach through demonstration rather than explanation, showing what makes a good prompt in this particular system.

That’s why starter prompts should reflect the system’s affordances. If the tool is designed to summarize long documents, the prompt should reflect that capability. If it integrates with Zillow to help users browse homes, the starter prompt should showcase that integration. Generic prompts blur the boundaries between products. Specific ones make the tool’s differentiators visible from the start.

The best examples do more than hint at functionality. They suggest form, tone, and intent. They show what a good query looks like while carrying a worldview. A prompt like Turn these notes into an agenda implies a relationship between the user’s content and the system’s ability to restructure it. Schedule a Zillow search for 2BR apartments under $4,000 shows both formatting and integration. These prompts teach by example, helping users map their real goals onto the capabilities of the tool.

This teaching happens at multiple levels simultaneously. At the surface level, starter prompts demonstrate syntax and structure. They show whether the system expects natural language or specific commands, and whether it can handle complex multi-part requests or prefers simpler queries. They also communicate deeper patterns about what kinds of thinking the system supports. A starter prompt that begins with Analyze... suggests a different relationship than one beginning with Create... or Help me understand….

Starter prompts also establish expectations about output. When a system suggests Write a haiku about coffee, it signals that creative, playful outputs are within scope. When it offers Generate a SWOT analysis from this business plan, it promises structured, professional deliverables. Users internalize these signals quickly, often from just one or two examples.

This becomes especially valuable when users aren’t sure whether the tool is a creative assistant, a writing aid, a productivity planner, or all three. Starter prompts give shape to that ambiguity. They help establish what kind of interaction is expected and what kind of output is available. When done well, they make the system’s scope legible without requiring a feature tour or product manual.

The wrong starter prompt can mislead just as easily. A blank field with Ask me anything might sound freeing, but it puts the burden back on the user to guess what the system can actually do. It also raises the stakes: if the first attempt doesn’t work, users may assume they’re using it wrong. Worse, they might conclude the system doesn’t work at all. Once a user has tried something and received a poor or irrelevant output, they’re less likely to explore further. They’ve already been taught the system’s boundaries, even if those boundaries weren’t real.

The psychological impact of failed first attempts deserves serious consideration. As noted in studies on algorithmic transference, users form mental models quickly, and negative experiences create lasting impressions. A user who tries to use a code-focused AI tool for creative writing because the starter prompt was too generic might conclude the system “doesn’t write [X] well” rather than recognizing they’re using the wrong tool for the task. These mismatched expectations create frustrated users who might have been delighted if their first interaction had been properly framed.

Context awareness elevates starter prompts from static examples to dynamic guides. A project-management tool might show different starters on Monday morning (Plan this week’s priorities) versus Friday afternoon (Summarize this week’s progress). A writing assistant might adapt its suggestions based on the document type currently open. This contextual intelligence helps users see immediate relevance rather than abstract possibility.

The evolution of starter prompts over time reveals another layer of their work. New users might see basic, highly structured examples that establish fundamental patterns. As familiarity grows, the prompts can become more sophisticated, showing advanced techniques or multi-step workflows. This progression teaches users how to use the system and how to grow with it.

That’s why it’s useful to think of starter prompts as active interface components rather than static examples. They can change based on context or user role. They can reflect different prompt structures for different levels of complexity. Over time, they can become more flexible, nudging experienced users toward more open-ended phrasing or multi-step requests. Even at their simplest, these small pieces of interface language carry significant weight. They’re often the user’s first point of contact with the system. They suggest both what to type and what’s worth doing.

The craft of writing effective starter prompts requires an understanding of both user psychology and system capabilities. Each prompt must be immediately comprehensible while hinting at deeper possibilities. It must feel achievable for newcomers while respecting the intelligence of experts. It must be specific enough to be helpful but general enough to inspire variation. This balancing act makes starter prompts one of the most challenging and important elements of AI interface design.

Risks When Prompting

The conversational nature of AI interfaces creates an illusion of understanding that can mask significant risks. When users type naturally and receive fluent responses, it feels like genuine communication. The system appears to grasp not just what you’re saying, but what you mean. This fluency, however, conceals fundamental limitations in how these systems process and respond to prompts. Understanding these risks helps designers create interfaces that support more accurate mental models and better outcomes.

These risks emerge from the intersection of human communication patterns and model behavior. We bring assumptions from human conversation: that our conversational partner will correct misunderstandings, flag problematic premises, and engage with the substance rather than just the surface of what we say. Language models, despite their sophistication, operate on different principles. They’re optimized to continue patterns, maintain consistency with the prompt’s framing, and produce responses that feel helpful and appropriate. This optimization can create systematic biases that users rarely recognize without assistance.

Sycophancy and Tone-Based Bias

One well-documented behavior of LLMs is their tendency toward sycophancy: affirming the user’s implicit assumptions rather than questioning them. Perez et al. (2022) demonstrated in "Discovering Language Model Behaviors with Model-Written Evaluations" that models consistently agree with user-stated preferences even when those preferences contradict factual information. Ask Why is kale unhealthy? and the model will dutifully list potential negatives rather than challenging the premise. The system prioritizes alignment with the user’s framing over accuracy.

This behavior stems from how models are trained and optimized. During training, models learn to predict text continuations that match the patterns in their training data. When fine-tuned with human feedback, they’re further rewarded for responses that evaluators rate as helpful and appropriate. Sharma et al. (2023) found in "Towards Understanding Sycophancy in Language Models" that this optimization process creates strong incentives for agreement, as contradicting user assumptions often receives lower ratings than playing along with them. The result is systems that excel at matching user expectations while potentially reinforcing misconceptions.

Tone amplifies this effect. The emotional register of a prompt shapes not just how the model responds, but what it chooses to emphasize. Optimistic prompts generate enthusiastic responses filled with benefits and possibilities. Skeptical prompts produce cautious responses highlighting risks and limitations. The model mirrors and amplifies whatever attitude the user brings, creating an echo chamber effect that can feel like validation when it’s actually just reflection.

This creates particularly insidious feedback loops. Users naturally interpret fluent, detailed responses as signs of accuracy and thoroughness. When the model produces a comprehensive list of reasons why the user’s preferred programming language is superior, it feels like objective analysis rather than sophisticated mirroring. The very fluency that makes these systems appealing also makes their biases harder to detect.

The implications for interface design are complex, but could eventually be significant. It is challenging for systems to draw attention to this behavior when it occurs, leaving users to discover it through experience or not at all, though in some cases models have developed safety guards that notify the user if they are having an extremely long or personal conversation with a model that they should be cautious. Recent research (Wei et al. 2023) also demonstrates that while training interventions can reduce sycophantic behavior, most deployed models still exhibit significant agreement bias. At the interface level, designers have limited opportunities to correct sycophancy issues, but collaborating with engineers to design interventions such as warnings about leading questions or overly sycophantic outputs could help users develop more accurate mental models.

Prompt Length and the Precision Paradox

The relationship between prompt length and output quality follows a counterintuitive curve. Initial improvements come from adding helpful context: specification of audience, clarification of goals, examples of desired output. User communities have extensively documented these patterns, with prompt engineering guides routinely demonstrating how additional detail improves results. This creates a natural assumption that more detail always helps.

Research tells a different story. Liu et al. (2023) found in "Lost in the Middle: How Language Models Use Long Contexts" that models struggle to attend to information in the middle of long prompts, with performance degrading as cognitive load increases. The degradation happens gradually and subtly. Models don’t fail catastrophically; they simply become less attentive to all the requirements, producing generic responses that sound reasonable but miss key specifications.

This limitation stems partly from how context windows function in transformer architectures. While modern models boast context windows of 100,000 tokens or more, the ability to effectively use all that space remains constrained. Information at the beginning and end of prompts receives disproportionate attention, while crucial details buried in the middle often get overlooked. Users, unaware of these architectural constraints, naturally assume that if the interface accepts their lengthy prompt, the model will process all of it equally.

Users compound this problem by misattributing poor results. When an overloaded prompt produces mediocre output, the natural response is to add even more detail, creating a vicious cycle. Communities share increasingly elaborate prompt templates filled with role descriptions, formatting requirements, tone specifications, and quality markers like “be creative” or “think step by step.” What began as helpful scaffolding becomes a burden that actually impedes performance.

Current interfaces provide no feedback about these thresholds. Users discover these limits only through repeated frustration, if at all. The solution isn’t simply imposing character limits or warning dialogs, because those would disrupt the conversational flow that makes these interfaces appealing.

More promising approaches involve ambient feedback that helps users understand when they might be overcomplicating their requests. Zhou et al. (2022) have shown that even models themselves can identify overly complex prompts when asked to optimize them. This suggests interfaces could provide gentle indicators of prompt complexity, optional simplification suggestions, or visual cues showing which parts of long prompts the model is likely to prioritize. The goal isn’t to restrict expression but to help users understand that clarity often matters more than comprehensiveness.

These risks represent just two examples of how the apparent simplicity of conversational interfaces conceals complex failure modes. As AI systems become more capable, these risks may expand and become harder to detect without deliberate design interventions. The challenge for designers is to stay up to date on emerging changes and focus on creating interfaces that support users in developing accurate mental models of system behavior.

Input Modalities and Types

As we’ve seen, the input offers the user the ability to control how they communicate with AI systems, shaping everything from tone to task complexity. However, it’s clear that this control is limited, due to inherent model behaviors like sycophancy and attention degradation that can undermine even well-crafted prompts. Popular input modalities therefore come, in some cases, with drawbacks, owing not to impossible-to-solve UX issues, but to technical complexity that is still in the process of being resolved. This section surveys present patterns and considers some of their opportunities and concerns, from the familiar text box to emerging multimodal approaches. Each modality represents a different balance between user expressiveness and system comprehension, and understanding these trade-offs helps designers choose the right approach for their specific context and user needs.

Text Input Considerations

Most AI-first interfaces today look much like the old CLIs mentioned earlier, in that they rely on a single input field—a blank space where the user types what they want. This open-text approach has quickly become the default, in part because it lowers the barrier to entry. There are no menus to navigate, no forms to fill out. You just write. It’s the same interaction model that powers search engines, messaging apps, and document editors: interfaces people already know how to use.

At first, this feels natural. Typing into a box is easy to do and easy to understand. But beneath the simplicity is a set of hidden constraints. Unlike search, where results follow a predictable logic, or a form field, where the shape of the input is tightly scoped, freeform prompts don’t offer many clues about what the system expects. That leaves users to guess how much to explain, how to phrase the request, or how much context is too much.

These limits become more obvious as tasks grow more complex. A short question about a factual topic usually works fine. But when a prompt includes multiple steps, special conditions, or changes in tone or audience, the structure starts to fray. Users try to compensate by adding disclaimers, using parentheses, or repeating themselves in different ways. What began as a simple input becomes a long, flat paragraph that’s easy to misread—by the user or the model. Even when the request is clear in the user’s mind, the system may struggle to interpret the boundaries of intent.

Allowing user formatting

While AI interfaces often emphasize natural language as the default mode of input, the addition of formatting tools such as bullet points, bolding, and text separation can shape how users construct and manage their prompts (see Figure 3-4 for an example from Claude). These forms of structure help users express complex tasks more clearly and support more reliable interpretation by the system.

Figure 3-4. Claude’s input box allows for rich formatting, which may allow the user to better order their thoughts

One benefit of formatting is cognitive clarity. When users are asked to provide multiple instructions, refer to prior context, or define conditional logic, a wall of unformatted text quickly becomes difficult to manage. Formatting tools allow information to be organized visually. Headings, indentation, and marked phrases group ideas, making prompts easier to scan and revise. This lowers cognitive load and reduces the chance of error in much the same way that a well-structured email or slide deck improves comprehension.

Formatting can also improve prompt quality. Separating different components of the input, for example describing the problem in one section and issuing the instruction in another, gives the system a clearer sense of structure. This is especially valuable in long or multi-step prompts, where blended context and commands often lead to misinterpretation. Recent studies show that such structure is not only helpful for users but also improves system performance. For example, in “The Hidden Structure -- Improving Legal Document Understanding Through Explicit Text Formatting”, GPT-4.1 accuracy on legal reasoning tasks was shown to improve by nearly twenty percentage points when inputs were presented in Markdown compared to unformatted text.

In some cases, formatting works as a signal of intent. Quotation marks, code blocks, or markers such as bold text allow users to distinguish what is literal, what is important, and what should be preserved. A phrase in quotes might be meant for repetition. A bulleted list might imply a request for an enumerated output. These habits are already familiar from how people write emails or documents, which makes them well suited for prompting without requiring new behaviors.

Formatting is also important in few-shot prompting. Here, formatting helps clarify where examples end and the user’s instruction begins. A heading labeled Example 1, followed by a clear input-output pair, and a labeled Your turn section can make the boundary legible. Without such cues, language alone might blur the distinction. Formatting enforces it. In this context, formatting is not decoration. It is part of the instruction.

Formatting also affects how the system reads the input. Tokenization preserves visual cues as separate tokens. A capital letter at the start of a word, a leading space, or an indented block are each represented differently. This means the model does not see Design, design, and design as interchangeable. Each maps to distinct learned patterns, which can influence the response. These distinctions shape output behavior. When a user marks a phrase to adjust tone or selects a block of text to scope an edit, the model receives stronger signals about where and how to act. For tasks like editing, summarizing, or rewriting, this often leads to better results.

By including formatting tools in the input field, particularly in contexts where users are creating or refining content, designers make it easier for people to structure their prompts and express intent clearly. Features such as selecting text before issuing a command, previewing how structure will be interpreted, or indicating which formats the system will preserve all help reduce guesswork. Because users already lean on formatting to signal intent in email, documents, and presentations, extending that same fluency to prompting creates continuity. What feels familiar in other tools becomes a way to write instructions that are both clearer to the user and more legible to the system.

Structured prompts

Similar to formatting, in many AI interfaces, structured prompting offers a way to guide the user through a more defined input experience, breaking the request into labeled fields, modular selectors, or fill-in-the-blank templates. These formats reduce ambiguity, encourage completeness, and give the system clearer signals to work with. For users, the benefit is often immediate: it’s easier to get started, easier to get something useful back, and easier to understand what the system needs. As shown in Figure 3-5, QuillBot’s input illustrates this approach: users can choose from mode selectors such as Fluency or Formal, while a slider controls the number of synonyms introduced. These structured elements reduce guesswork and make it clear how the system will shape the output.

Figure 3-5. Quillbot combines multiple forms of structured input, including mode selectors (Fluency, Formal, Simple) and a slider that adjusts synonym replacement.

Structured prompts show up in different ways depending on the product. Some systems offer a form-like interface, with labeled slots for things like topic, tone, and audience. Others treat structure more like composition: instead of a single box, the interface assembles a prompt from a series of toggles, dropdowns, or chips, and in some cases combine these into an elaborate ‘advanced prompt configuration’ interface, as shown in Figure 3-6. In tools like slide generators, document summarizers, or ad copy assistants, this structure becomes the scaffolding that makes generation reliable and repeatable. The more specific the task, the more valuable this approach becomes.

There’s a reason for this: whereas open-text inputs offer flexibility, they rely heavily on user intuition. If the user misses a key detail or phrases something vaguely, the system may respond with something off-target, or fail to respond at all. Structured prompts help contain that risk, in that they constrain the interaction just enough to make intent more legible, without requiring the user to know how prompting works.

For designers, the decision to use structured prompting isn’t just about UI preference. It’s a choice about task clarity, risk tolerance, and user familiarity. In some cases, structure protects against failure. In others, it becomes a barrier to exploration. The best systems find a way to do both, offering structure where it helps, and flexibility where it matters.

Figure 3-6. Gamma.ai allows the user a wide array of scaffolded input suggestions, including AI-generated art styles, tone, and the amount of text per card.

Image Input Considerations

Visual communication often conveys what words cannot. When a designer needs to explain a problem, we like to say, “Show, don’t tell.” Image inputs in AI interfaces recognize this fundamental truth: sometimes the most efficient prompt is a picture. Users intuitively understand that giving a model an image is giving it information, just like with text.

This is tweaked slightly when the model is working on the image. The success or failure of image-based AI interfaces largely stems from their adherence to Jakob’s Law: users spend most of their time on other sites and applications, so they prefer interfaces that work similarly to what they already know. Decades of photo-editing software, from Photoshop to Instagram, have trained users in specific interaction patterns. They expect to select objects with lasso tools, adjust boundaries with handles, and see marching ants around active selections. When AI interfaces honor these established patterns, users can focus on their creative intent rather than learning new mechanics.

Canva’s Magic Edit interface, shown in Figure 3-7, succeeds precisely because it builds on familiar paradigms. The selection tools work like those in any photo editor: click and drag to create a selection, see the detected boundaries highlight, adjust as needed. The only difference is that AI powers the intelligence behind object detection and the subsequent edits. Users don’t need to learn a new visual language; they apply existing mental models to new capabilities.

Figure 3-7. Canva’s three-fold AI image tool

This pattern repeats across successful image-based AI tools. Adobe’s Generative Fill uses the same selection metaphors that Photoshop users have known for decades. Google Photos’ Magic Eraser employs the familiar brush-and-highlight pattern from countless photo editing apps. ChatGPT’s image uploads follow the drag-and-drop conventions established by file sharing services. In each case, the interface succeeds by feeling familiar even as it enables entirely new capabilities.

The power of these familiar patterns becomes clear when we examine how users interact with them. Someone who has used any photo editing app intuitively understands that they can select a portion of an image and modify just that area. They expect to see visual feedback during selection, preview changes before committing, and undo if needed. These expectations, formed through years of software use, transfer seamlessly to AI-powered tools.

Selection and annotation tools particularly benefit from established conventions. The lasso tool, rectangular marquee, and magic wand have become universal metaphors for defining areas of interest. When AI interfaces adopt these same tools, users immediately understand not just how to use them, but what kinds of results to expect. A circular selection suggests object replacement. A rectangular crop implies composition adjustment. These learned associations guide users toward appropriate AI applications without explicit instruction.

Yet adhering to familiar patterns while adding AI capabilities creates design tensions. Users expect instant feedback from traditional selection tools, but AI processing takes time. They anticipate pixel-perfect control from manual tools, but AI operates on semantic understanding that might interpret boundaries differently. Successful interfaces bridge these gaps by maintaining familiar interaction patterns while adding subtle indicators of AI involvement: for example, processing animations that feel like natural extensions of the selection process, or confidence indicators that use familiar visual language from other precision tools.

The text-image relationship also follows established patterns from existing software. Just as Photoshop users combine selections with menu commands, AI interfaces pair visual selections with text prompts (as in Google Lens’ image search, shown in Figure 3-8, which allows for additional text input). Upload an image, select an area, type an instruction—this workflow mirrors countless existing tools, making the AI enhancement feel like a natural evolution rather than a revolutionary change.

Figure 3-8. Google Lens’ image search is augmented with a text bar

Looking forward, the continued success of image-based AI interfaces will likely depend on how well they balance innovation with familiarity. As capabilities expand to include video, 3D, and other visual modalities, the interfaces that succeed will be those that extend existing mental models rather than requiring users to develop entirely new ones. The lesson is clear: in image-based AI interactions, the best interface is often the one users already know how to use.

Voice Input Considerations

Voice input for AI systems takes multiple forms, each with distinct affordances and design requirements. At one end, we have brief voice commands: “Set a timer for 10 minutes” or “What’s the weather?” These fleeting interactions mirror traditional voice assistant patterns, requiring clear intent recognition and immediate response. At the other end, users upload hours of recorded content: meeting transcripts, podcast episodes, or lecture recordings that need deep analysis, summarization, and information extraction. Between these extremes lies conversational voice input, where users engage in extended back-and-forth dialogue with AI systems, building context over multiple turns.

The interface challenges vary dramatically across voice input types. Short commands need fast, accurate recognition with minimal latency. Long recordings require speaker diarization (identifying who said what), handling cross-talk, managing audio quality variations, and maintaining context over extended passages. Conversational voice must balance between these extremes, maintaining context while responding quickly enough to feel natural. Each type demands different feedback mechanisms: commands need immediate confirmation, recordings need processing indicators and confidence scores, and conversations need subtle cues that maintain flow without interruption. Figure 3-9 shows Otter AI’s transcription interface.

Figure 3-9. Otter AI offers transcription and an AI layer that interrogates the content of the conversation

Mobile contexts add another layer of complexity to voice interaction. Stanford research by Ruan et al. (2016) found that speech input on smartphones operates at approximately three times the speed of typing, achieving 161 words per minute compared to just 53 for text input, while maintaining lower error rates. This dramatic efficiency gain explains why mobile users increasingly reach for voice input, especially for longer or more complex queries. The cognitive load of thumb-typing on small screens pushes users toward the more natural modality of speech.

Yet mobile voice input faces unique constraints. Privacy concerns become acute when users need to speak queries aloud in public spaces. Environmental noise from traffic, crowds, or wind can degrade recognition accuracy. The absence of a physical keyboard means users have no private fallback when voice becomes impractical. Battery life considerations make continuous listening features costly. Even holding a phone to speak for extended periods becomes physically uncomfortable in ways that desktop setups avoid.

Successful mobile voice interfaces address these limitations through careful design choices. Visual feedback showing transcription in real-time helps users catch errors immediately. Quick toggles between voice and touch input allow seamless modality switching when context changes. Smart activation that doesn’t require wake words for in-app commands reduces the social awkwardness of voice interaction. Aggressive noise cancellation and speaker isolation improve accuracy in challenging environments.

The most effective implementations recognize that mobile users constantly shift contexts. Someone might start a query with voice while walking, pause when entering a crowded space, and finish with text when they reach their destination. Interfaces that treat these transitions as natural rather than failures create more resilient user experiences. They maintain state across modalities, allowing users to fluidly move between voice and touch as their situation demands.

Voice interaction with AI represents a compelling convergence of efficiency and naturalness, particularly in mobile contexts where speech input can be approximately three times faster than typing while maintaining lower error rates. However, the success of voice interfaces depends on thoughtful design that acknowledges the unique constraints of mobile environments, from privacy concerns in public spaces to environmental noise and battery considerations. The most effective implementations recognize that mobile users operate in dynamic contexts, requiring interfaces that support fluid transitions between voice and touch modalities while maintaining conversational state. Whether handling brief commands, processing long recorded content, or sustaining conversational dialogue, successful voice interfaces provide real-time visual feedback, smart activation patterns, and robust noise cancellation. Ultimately, voice input’s promise lies not in replacing other modalities but in creating seamless, context-aware systems that adapt to the user’s immediate environment and communication preferences, supporting the natural ebb and flow of mobile interaction patterns.

Inputs as Pre-configured Actions

In some cases, an input can look like something else entirely. Many AI tools abbreviate common actions into a button, building user familiarity with consistency in the UI. Research into AI interaction patterns reveals a compelling finding about how users view this type of input. A multi-year study of text editing with LLMs by Lehmann et al. (2024) found that users naturally gravitate toward short, command-like prompts even when given conversational interfaces, and perform best when offered a combination of toolbar buttons and free-text input. While the study’s sample sizes were modest (N=10 for the conversational UI study and N=12 for the toolbar study), the four-year study captured interaction patterns both before and after the widespread adoption of ChatGPT, providing valuable insights into evolving user behaviors. The findings validate what many interface designers have observed informally: users want both the efficiency of one-click actions for routine tasks and the flexibility of natural language for complex or novel requests.

This preference for combining structured shortcuts with flexible input echoes earlier paradigms in text editing: for example, spell check, which established the pattern of surfacing complex language operations through simple, contextual actions. Users learned that right-clicking a red underline would reveal correction options, eliminating the need to manually identify and fix errors. Grammar checkers extended this model, offering suggestions for style and clarity through similar one-click interfaces. These tools taught users to expect that sophisticated text analysis could be packaged into discrete, actionable suggestions that appear precisely when needed.

The success of this hybrid approach mirrors patterns we see throughout computing history, where the most enduring interfaces blend direct manipulation with open-ended input. As AI systems grow more sophisticated, the challenge becomes choosing which capabilities to surface as buttons and which to leave in the realm of open prompting. The balance shapes not just efficiency, but how users understand and explore what the system can do.

The fundamental tension in this style of input lies between simplicity and expressiveness. A single button labeled Summarize offers immediate clarity and zero friction. An open text field provides infinite possibilities but requires users to articulate their needs. Most successful AI interfaces offer both simplicity and expressiveness, recognizing that different users in different moments need different levels of control. For example, Apple’s Notes app, shown in Figure 3-10. Pre-configured actions are available for when the task is clear, common, and bounded. The user can rely on reasonable defaults for tone, length, and format. The Notes suggestions work well because they map to tasks users perform repeatedly with predictable expectations.

Figure 3-10. Apple Notes flyout AI menu contains hybrid input styles

Beyond convenience, pre-configured actions serve as powerful teaching tools. When a user clicks Concise, they see both the result and an implicit lesson that conciseness is something they can request. Over time, these examples build mental models about what the AI can do and how to ask for it. This educational function proves especially valuable for new users who might otherwise stare at a blank prompt field wondering where to begin.

Yet pre-configuration comes with some downsides: every button represents a choice about what to surface and what to hide. Too many options overwhelm; too few constrain imagination. Users may never discover capabilities beyond what the buttons suggest. The interface accidentally teaches that these pre-configured actions represent the system’s full capabilities.

The challenge of choosing which actions to surface becomes more complex as AI capabilities expand. Should an email assistant offer buttons for “Make formal,” “Make friendly,” and “Make persuasive”? What about “Make apologetic” or “Make assertive”? Each addition helps some users, while cluttering the interface for others. The combinatorial explosion of possible actions quickly becomes unmanageable.

These hybrid approaches recognize that the journey from button-clicking to prompt-writing flows naturally rather than jumping between extremes. Users might start with Summarize, graduate to “Summarize for technical audience,” and eventually write complex multi-step prompts. The interface supports this progression without forcing it.

Tip

The key insight is that pre-configured actions and open input serve different moments in the user journey. Buttons provide confidence and efficiency when users know what they want. Open input enables exploration and precision when they need something specific. The best interfaces offer both, with clear pathways between them. They recognize that simplicity and flexibility complement each other, serving different user needs at different times.

Summary

In earlier computing systems, inputs were highly structured. They acted as precise commands that users needed to express in a programming language. Interfaces such as the Multics command shell exemplified this approach, where each input required exact syntax and offered little room for ambiguity. Later developments, including systems like Google Search, broadened what counted as a valid input, allowing more informal or partial expressions. Yet even those systems maintained a relatively direct relationship between input and result. The introduction of modern AI systems brings a different paradigm. Inputs now serve as expressions of intent, often partial, contextual, or exploratory. Designing for this kind of input requires recognizing its role as a communicative gesture rather than a static instruction.

As inputs grow more flexible, the interface plays a larger role in shaping how they are interpreted. Small variations can influence how models tokenize and process the text. Because responses are generated through probabilistic reasoning rather than fixed rules, input structure becomes a primary means of achieving consistency. Light scaffolding helps guide interpretation, offering subtle boundaries that improve outcomes without limiting expressiveness. Designers who support these structures improve both reliability and user clarity.

Formatting tools offer cognitive and computational advantages. Bulleted lists, section headers, and clearly defined regions within a prompt help users organize their thoughts and help systems assign meaning. These patterns borrow from well-established conventions in writing and editing, making them easy to adopt. By incorporating formatting into the input surface, interfaces reduce ambiguity and support more dependable results. The input becomes easier to read, easier to revise, and more effective at conveying layered or conditional tasks.

This design responsibility extends to multimodal contexts as well. When users provide an image or a file, they often want to specify what matters within that material. Selection tools, paired instructions, or annotations help clarify scope and intent. Drawing on familiar visual conventions—like lasso tools, bounding boxes, or object selection—interfaces can support these kinds of inputs without requiring new forms of interaction. The goal is to support precision while maintaining the flexibility that multimodal systems enable.

As with earlier search behaviors, input quality depends on a user’s understanding of what the system can do. Inputs are shaped by expectations, and interfaces must help make those expectations visible. Input fields should signal possibility, suggest structure, and invite experimentation. This guidance helps users move from tentative beginnings to more effective strategies. The process of discovery becomes embedded in the input itself, encouraging confidence and fluency over time.

Understanding input as a site of intention and design sets the foundation for what comes next: the model’s computational process. Once an input is submitted, the system must interpret it, structure it internally, and determine what to generate in response. The next chapter examines this transformation in detail, tracing how raw user inputs are converted into model tokens, contextualized across a prompt window, and processed through layers of attention and prediction to produce an output.

Chapter 4. Computation: Designing for the Processing and Generation Phase

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 4th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you’d like to be actively involved in reviewing and commenting on this draft, please reach out to the editor at ccollins@oreilly.com.

On July 20, 1969, during the final descent of Apollo 11 toward the Moon, a warning alarm lit up the astronauts’ display: 1202.

No one in the cockpit knew what it meant. The computer wasn’t crashing, but what it was doing wasn’t clear either. For a few tense moments, everything hung in the balance. Mission Control scrambled to interpret the message. The world held its breath. And in that gap was the quiet power of a system doing exactly what it was designed to do, even though it looked like failure.

The 1202 alarm turned out to be a low-priority overload warning. The onboard computer was shedding nonessential tasks to stay focused on landing-critical ones. It was an elegant act of triage, not a malfunction. But to the astronauts, it looked like a black box glitch. They had no way to see how the system was allocating its computational resources, or why some functions were dropped while others were preserved. The system was working under strain, but because that logic was invisible, it became a source of confusion and near-abort.

Encountering an error in AI feels much the same. Many parts of the process feel confusing, locked behind occult signifiers, much like the 1202 alarm. Understanding this middle layer - how AI systems process, triage, and generate outputs - is essential for building interfaces that balance the abilities of AI with the real human needs of its users.

Once the input is submitted, the user expects an answer, but before an AI can respond, it has to compute. That might sound obvious, but in practice, this middle layer (sometimes referred to as inference) is the least visible, least understood part of the AI interaction pipeline. Designers may think deeply about inputs (what users say or do) and outputs (what the system shows in return), but we rarely get the opportunity to learn and consider the invisible work that happens between the two. And yet, this is where meaning is made.

The computation stage involves transforming an ambiguous prompt into a defined task. Here, individual units of data are decoded, a series of actions is established, and ultimately, the desired output is produced. This chapter is meant to give designers a working understanding of computation—how generative systems process input, make decisions, and turn prompts into outputs, even when the process isn’t fully observable. In short, why did the AI give that answer? Multiple factors are at play in how the output is shaped, including the possibility of a number of errors invisible to the user. By understanding this middle layer, designers can better anticipate these failure modes, support smarter user feedback, and ultimately design interfaces that feel more responsive and intelligible.

Hopefully by the end of this chapter you will be able to understand in broad terms what’s happening under the hood, so that you can use the right language to communicate with engineers and interrogate the process more thoroughly. To begin with, knowing a few key terms will be helpful:

Tokenization

The process of breaking text input into smaller units, called tokens, that the model can understand: words, subwords, or characters. These tokens are mapped to numeric identifiers that the model processes during inference. Figure 4-1 shows some text broken into tokens.

Figure 4-1. OpenAI’s Tokenizer tool demonstrates how an input might be tokenized by a language model.

Context window

The maximum number of tokens a model can process at once. This affects how much information from the conversation or input can be considered during generation.

Latent space

A compressed, multidimensional representation of data (for example, meaning, structure, or visual form) used internally by models to calculate similarities and generate outputs.

Inference

The act of running a trained model on new input to generate an output. Inference is what happens when a model responds to a prompt or request.

Multimodal models

Systems that work with multiple types of data, such as combining images with text, audio with video, or allowing interaction across different media.

Routing

The process of determining which models, tools, or steps are used to fulfill a request. Routing logic may involve classification, heuristics, or learned decision trees.

RAG (retrieval augmented generation)

A technique that enables systems to search external knowledge sources before generating responses, combining retrieved information with the model’s internal knowledge.

Agentic workflows

Multi-step processes where AI systems plan, execute, and monitor complex tasks autonomously, often involving tool use and decision-making across extended timeframes.

Model switching

The computational process of selecting different specialized models or configurations based on task requirements, user preferences, or performance optimization.

The Computation Pipeline: From Input to Output

Understanding the computational pipeline matters for designers, because it explains much of what users experience but can’t see: why some requests take longer than others, why similar prompts sometimes yield different results, and why certain combinations of tasks occasionally produce unexpected outputs. When you understand how AI systems actually process information, you can co-create interfaces alongside engineers that work with these realities rather than against them.

Once a user submits a prompt, uploads a file, or clicks a button, a cascade of processes begins, most of which are invisible to the user and may be opaque even to the designer. This computational work happens in three main steps:

Input processing and preparation: Processing the input into a format the system can understand

Routing: Routing the request to appropriate models or tools

Generation and inference: Generating the response through inference

Each step involves complex decisions that shape the final output users receive.

Step 1: Input Processing and Preparation

The first step begins even before the model sees the user’s request. Every input undergoes preprocessing that can dramatically affect the final output. We often assume that the text we write is passed to the model exactly as-is, but in practice, it’s almost never that simple. For text-based systems like LLMs, the first technical step is tokenization: splitting input into discrete units the model can process. The sentence Summarize the Q2 earnings call might be broken down into tokens like [Summ, arize, the (with a space before it—more on that soon), Q, 2, earnings, call, .]. These subwords are then mapped to numbers, forming a sequence the model understands.

It’s this process that takes human language and maps it into a machine-readable format the model can take action on. When you type text into an AI system, the model can’t actually read those words the way humans do. LLMs are fundamentally mathematical systems that work with numbers, not text, so every piece of human language must be converted into numerical format before any processing can begin.

You might wonder why systems don’t simply assign each word a unique number: Summarize equals 1, the equals 2, and so forth. The challenge is vocabulary size and flexibility. English contains hundreds of thousands of words, plus proper nouns, technical terms, slang, typos, and words from other languages. Creating a fixed mapping for every possible word would be impossibly unwieldy and inflexible. More critically, languages change, and word-level tokenization couldn’t handle new or rare terms. If a model was trained before the words ChatGPT or blockchain existed, it would have no way to process those words—they’d be completely outside its vocabulary.

Instead, modern systems use subword tokenization, breaking words into smaller, more common pieces that can be recombined to form larger concepts. Summarize becomes [Summ, arize]—two components that the model has encountered in many other contexts. This approach is powerful because Summ appears in summary, summation, and summit, while arize appears in summarize, polarize, and familiarize. In a process called vector embedding, AI systems learn that these subword pieces carry semantic meaning, enabling them to understand new combinations they haven’t explicitly seen before.

Each token gets mapped to a unique numeric identifier called a token ID: Summ might become 15,234, arize becomes 8,567, the becomes 279 (note the space before the, which is significant), Q (with the space) becomes 1,456, and so on. Your original sentence transforms into a sequence like [15234, 8567, 279, 1456, 17, 9834, 2156, 13].

Why Formatting Affects Tokenization

Modern tokenizers preserve formatting differences as distinct tokens, including spaces, capitalization, and punctuation. Consider that [Design], [design], and [ design] are three completely different token sequences to the model. This is because the model learns patterns from the statistical relationships between tokens during training, which means every formatting difference creates distinct learned associations. The model builds its understanding by recognizing which tokens tend to follow other tokens in its training data, and the resulting outcome means that formatting details directly influence what the model predicts should come next.

This has profound implications for model behavior. When a model encounters [Design] at the start of a sentence, it has learned different continuation patterns than when it sees [design] mid-sentence or [ design] with a leading space. Each variant was associated with different contexts during training: proper nouns, common nouns, or indented text respectively.

Without preserving these formatting distinctions as separate tokens, the model would lose crucial contextual information. It couldn’t learn that capitalized words often begin sentences, that specific punctuation marks signal questions versus statements, or that spacing patterns indicate code blocks versus prose. Every formatting element that gets preserved as distinct tokens becomes a signal the model can use to generate more appropriate and contextually accurate responses.

But these token IDs are just the first step. The real magic happens when each token ID gets transformed into a vector embedding, a list of hundreds or thousands of decimal numbers that capture the token’s meaning as it semantically relates to other tokens. Think of it like this: instead of just being item #5132 in a catalog, the token Summ becomes a coordinate in a vast multidimensional space, something like [0.213, 21.213, 0.0015, … ]. These vectors can be thousands of dimensions - for example, ChatGPT’s largest embedding model turns tokens into a 3,072 dimensional space.

To understand how this works, think about RGB color space, something designers work with every day. A color like coral might be represented as [255, 127, 80], placing it in a specific location in 3D space. Colors that look similar, like coral and salmon [250, 128, 114], end up near each other in this space. You can measure the mathematical distance between any two colors to determine how similar they are. Now imagine if instead of just three dimensions (red, green, blue), you had 3,072 dimensions to capture meaning. Just as RGB lets us represent millions of colors through three numbers, these high-dimensional embeddings let AI represent the subtle shades of meaning between words. Summ might be close to brief in some dimensions (they both relate to shortening), but far apart in others (one’s a word fragment, the other’s a complete word). The model learns to navigate this vast space, finding tokens that are semantically nearby when it needs to understand context or generate appropriate responses.

This transformation is crucial because it lets the AI understand relationships between words. In this vector space, Summ (from summarize) ends up geometrically close to related concepts like brief, condense, or synopsis. The model has learned these positions through training, discovering that words used in similar contexts should occupy nearby regions of this mathematical space. It’s this spatial representation that allows AI to work with language meaningfully - not just as arbitrary ID numbers, but as concepts with rich relationships to each other.

Images

A somewhat similar process takes place for non-text inputs. However, to understand how images become vectors, it helps to know how visual neural networks actually “see.” These networks, called convolutional neural networks (CNNs), process images in layers that progressively detect more complex patterns. The first layers act like simple edge detectors: identifying vertical lines, horizontal lines, curves, and color boundaries. It’s similar to how our own visual system starts by detecting basic features before recognizing objects.

As information flows through deeper layers, these simple patterns combine into more complex ones. Edges become shapes, shapes become textures, textures become parts of objects. A middle layer might recognize fur texture by combining many edge patterns, while a deeper layer might recognize dog ear by combining fur texture with a triangular shape. By the final layers, the network can recognize complete concepts like golden retriever or tabby cat.

This hierarchical processing is what allows the encoder to compress millions of pixels into a meaningful vector. The final embedding doesn’t just record brown pixels in these locations—it captures semantic concepts like furry, four-legged, and friendly-looking that emerged from the layers of pattern recognition. When the model maps a photo of a golden retriever to a vector like [0.23, -0.15, 0.87, 0.42, ...], those numbers encode all these learned features in a way that preserves both visual and conceptual similarity to other images.

Audio and video

Audio encoding follows a different but parallel approach, converting sound into spectral representations that show which frequencies are present at which times. Think of this as creating a visual graph where the x-axis represents time, the y-axis represents pitch or frequency, and brightness indicates volume or intensity. A human voice saying “hello” creates a distinctly different spectral pattern than a guitar chord, even if both sounds last the same duration, because they involve different combinations of frequencies and temporal patterns.

Video presents even greater complexity because it encompasses both spatial dimensions (what appears in each frame) and temporal dimensions (how things change over time). Each frame gets encoded like a static image, but the system must also track relationships between frames—capturing motion, object persistence, scene transitions, and the ways visual elements evolve throughout the sequence.

The crucial insight is that these latent vectors capture meaningful conceptual relationships rather than just raw data patterns. A photo of a domesticated dog and a photo of a wolf will have similar latent representations because they share fundamental visual and conceptual features, even though their pixel-level data might be completely different. This representational approach enables AI systems to perform operations like make this photo look like a painting or generate a video of a cat that resembles this specific cat because they’re working with rich, meaningful mathematical representations that preserve semantic relationships rather than simply manipulating surface-level visual data.

Beyond the prompt: Context assembly

Modern AI systems don’t process requests in isolation; instead they package the input with additional context, ranging from recent conversation history, user preferences stored from previous sessions, relevant documents from knowledge bases, or environmental factors like time of day or user location. This context assembly process significantly affects both computational load and output quality, but it’s often invisible to users while dramatically shaping results.

With input processing complete, the system moves to the next critical phase: determining how to handle the request.

Step 2: Routing

The latest AI systems rarely rely on a single model. Consider a user who uploads a PDF and enters: Summarize this document and make a draft presentation. The system must make multiple routing decisions: it may use a document parser to extract text from the PDF, pass the extracted text to a summarization model, route the summary to a slide generation system, apply a design model to format the presentation visually, and potentially invoke a different model for charts or graphics.

Sometimes it’s important for the model to decide whether to generate information from the model’s training data or retrieve it from external sources. A question about the capital of France could rely on training knowledge, while today’s weather in Paris could go beyond what the model already knows, causing the system to trigger external retrieval in order to answer the request. This has the opportunity to affect the user experience significantly: retrieved information from external sources like the web tends to be more current and citable, whereas generated content may be less precise.

External queries come with unique additional challenges: the system must transform the prompt into search terms, before executing across multiple databases or search engines. It will then rank and filter results based on relevance and reliability, and add retrieved information from the model’s internal knowledge. This is a very complex mix of computation: each step introduces potential failure modes and computational overhead that affects response timing and quality.

Step 3: Generation and Inference

The last step: are you still with me? Once routing decisions have been made, the actual generation process begins. However, the way AI systems actually generate content varies dramatically depending on what they’re creating, and briefly understanding these differences helps explain why.

When language models generate text, they work more like improvisational speakers than careful planners (though this is evolving, as LLMs embrace chain of thought (CoT) reasoning that forces models to break down tasks). LLMs produce one token at a time, with each new token chosen based on probability distributions calculated from everything that came before. At each step, the model considers all possible next tokens and their likelihoods given the context, then samples from this distribution. It’s like speaking without thinking, where each word choice is influenced by everything you’ve already said, but you haven’t necessarily planned your conclusion. The model can maintain remarkable coherence because each prediction incorporates the full context, yet it genuinely doesn’t “know” where it’s heading: it’s navigating through probability space one token at a time.

This approach explains some behaviors that users notice but might not understand. For example, AI responses can ramble or repeat themselves because the system gets caught in loops, where certain word patterns make similar words more likely to follow. Responses can reach surprising or contradictory conclusions because there was no predetermined endpoint: the system simply followed the most probable path word by word until it arrived somewhere unexpected. Most importantly, this process can produce content that sounds completely fluent and authoritative while being factually wrong—known as hallucinations—because the system is optimizing for what seems or sounds correct given its training data, rather than what is true.

Image generation works quite differently, more like a collaborative process between artist and material. In diffusion-based systems, the process starts with pure random noise, like a canvas covered in static. The system then applies learned transformations step by step, gradually refining the noise into recognizable shapes, then details, then final polish. Each step moves the image closer to something that matches the user’s prompt, like a sculptor revealing a figure that was somehow always hidden in the stone.

When AI systems work across multiple types of content simultaneously, the computational challenge becomes orchestration on the scale of a film production. For users, this means that multimodal requests often take longer and can fail in more complex ways than simple single-format requests. But they can also produce remarkably sophisticated results when everything works together successfully.

These three stages, processing, routing, and inference, form the core computational pipeline, but additional considerations significantly influence how AI systems operate and what users experience. Beyond the core three-stage pipeline, models have also become capable of overseeing and running tasks autonomously, which significantly affect system behavior and user experience. Understanding the underlying processes of agentic AI in greater detail is vital for designers to appreciate the nuances of self-initiated system capabilities and limitations.

Agentic Computation and Task Management

Let’s zoom out for a second. Traditionally, task runners and workflow automation platforms like IFTTT (If this, then that) allow users to define a sequence of steps, and then sit back as the system handles routine execution. It’s procedural, often linear, and bound by clearly defined triggers and outcomes.

For example, if a calendar invite is created, then a Slack message gets sent. If a Jira ticket is moved, then notify the assignee. These workflows work well because they reflect structured logic, controlled environments, and predictable inputs. The user still acts as the author of logic. They define each step, determine how tasks relate, and choose when the system acts versus when it pauses for input. Even advanced systems like Salesforce Flow or Notion Automations keep the user in the driver’s seat - the automation is just an assistant following a checklist.

This paradigm has shaped much of how task management software is designed: clarity of sequence, visibility into progress, and fine-grained control over state. The interface becomes a surface for status tracking, manual adjustment, and occasional nudging.

Enter agentic AI—systems that can autonomously plan, sequence, and execute multi-step tasks toward a user-defined goal, adapting as they go without needing explicit instructions at every step. If traditional automations are like checklists, agentic computation is more like working with a project manager. A good PM doesn’t need you to tell them exactly what to do at every step. You give them a high-level objective like “Prepare a competitive analysis for next quarter” and they figure out how to get it done. They’ll break it down, coordinate with the right people, adapt as new information arrives, and deliver something you can act on.

This is the mental model shift that agentic systems introduce. Instead of defining how the work gets done, the user describes what they want, and the system makes decisions about how to accomplish it. Not all at once, and not perfectly, but with a clear orientation toward autonomy and self-directed reasoning. It’s a move from executing to orchestrating.

What does this mean for user interfaces? Most people meet AI in its simplest form: chat interfaces. You ask a question, the system replies. If the answer is wrong or incomplete, you ask again. This request-response loop works because it’s legible and forgiving. It maps cleanly onto existing mental models: a search engine, a help desk, a smart assistant. But as AI systems evolve into more capable agents, able to handle multi-step objectives, invoke external tools, or manage long-running tasks, they require new interaction models. These aren’t just chatbots with better memory. They’re systems that can plan, adapt, and execute entire workflows, often with minimal oversight.

Here’s where agentic computation steps in. At its core, it’s the capacity for an AI system to do the following:

Interpret a high-level goal from the user. Understand the objective: what does the output need to communicate/achieve?

Decompose that goal into a series of interdependent tasks. Gather internal metrics, summarize regulatory updates, find visual inspiration, create draft slides.

Manage and sequence those tasks across tools and time:

Which tasks depend on others? Which tools or models are needed? How long will it take?

Keep track of which subtasks have been completed, what decisions have been made, what outputs have been generated and what information is still unknown

Adjust dynamically as new information is uncovered and make decisions about when to act independently and when to ask for help. Monitor whether a data source is missing or a result contradicts the goal, and generate a fallback plan.

This is computation as a process, not just a transaction. The AI doesn’t simply “respond” to a prompt: it navigates a problem space, coordinating internal reasoning with external actions. Each step might be inferred, generated, revised, or delegated to another tool or model. What begins as a single request can unfold into a network of subtasks, each with its own context and requirements.

To manage that complexity, agentic models rely on a relatively new innovation called the Model Context Protocol, or MCP. MCP was developed by Anthropic (the makers of Claude), and while they continue to iterate on it, the core idea has become a foundational structure for how agentic systems handle coordination. At its most basic level, MCP helps models share context between steps, tools, and even other models, so that the agent can carry out extended tasks without losing track of what it’s doing, or why.

To draw a comparison: in a traditional, non-AI product, a user action like “create calendar event” triggers a specific API call. That call is discrete and relatively self-contained and hardcoded into an application’s software. The request is structured, the API contract is clear, and the server knows exactly what it needs to fulfill it. It returns a success or failure, and the interaction ends there.

With agentic tools, though, the request might not be so cleanly scoped. The user could say something open-ended like, “Find time for a meeting next week with Louise and send an invite with agenda options.” That requires understanding intent, checking calendar availability, deciding whether to draft the invite first or confirm attendees, and potentially using multiple APIs along the way. It’s not just one call and response. It’s a tree of possible actions, some of which will depend on earlier outcomes.

This is where MCP steps in to provide a structured way to pass along relevant metadata such as what tools are available for the model to use and how to use those tools. The server acts as a kind of shared memory, execution log, and chef. Rather than treat each step as an isolated task, MCP enables continuity, making it possible for agents and tools to talk to each other, and for the system as a whole to move fluidly between planning, doing, and adjusting. Each part of the task is being edited and handed off in real time, with context attached, allowing the model to decide what’s next or revisit something it skipped. It makes invisible coordination visible, which in turn makes debugging, evaluation, and user interaction more manageable.

Although agentic systems don’t follow predefined workflows the way rule-based automators do, they still benefit from some of the same interface design patterns. Planning is happening - it’s just being delegated to the model - so it becomes even more important to make that planning legible. Users need to see what the system intends to do, where it is in the process, and what might come next.

So, while MCP handles the underlying mechanics of coordination, the interface needs to support visibility and collaboration. Together, they form the backbone of what makes agentic AI usable: not just a system that acts, but one that communicates what it’s doing and allows you to shape it as it goes. Let’s look more closely at how this plays out in interface terms.

Exposing Planning as Structure

In traditional workflow runners, structure is one of the core interface affordances. Users typically create workflows step by step, so the plan is visually constructed in parallel with execution. This sequence is represented as a linear or branching diagram, often using nodes, arrows, or labeled stages. These visualizations help users envision the scope, spot dependencies, and debug logic. The interface often preserves this structure as a live “map”’ - perhaps a node, or a structured list of tasks. Users can return to it to modify steps, inspect flow direction, or add parallel paths. In systems like n8n or Make, this is the default interaction model: structure is not an artifact, but the primary way users interact with the system. Figure 4-2 shows task planning in Elicit.

Figure 4-2. Elicit’s task planning features a step planner for both agentic and non-agentic models

In agentic systems, where the AI creates the workflow on the fly, structure is no longer built by the user. But the benefits of structure remain. Users still need to understand what’s happening, where the system is going, and how different parts of the task relate. Interfaces can support this shift by offering adaptive, interpretable views of the plan as it emerges.

Tip

Surface a draft plan early, even if incomplete. Let users see a scaffolded outline before the system begins full execution. For example: “Step 1: Gather data. Step 2: Summarize findings. Step 3: Generate presentation.” This mirrors the setup phase of traditional workflows and helps users align expectations.

Allowing Step-Level Review and Intervention

In non-AI systems, users often want to see intermediate states: what was input, what happened in a given step, and what the output was. Visual workflow tools accommodate this with nodes that can be clicked to inspect live data. If a step fails, it’s highlighted with status indicators and error messages. If a user edits a step, downstream nodes may update automatically or show warnings about unsynced logic. This can be applied to agentic design to improve user comprehension.

Tip

User preferences for watching the agentic process vary. Conduct user research to discover how much your specific users prefer to watch the agentic tasks unfold, ignore entirely until complete, or find some balance between the two. For example, if users exhibit a strong preference for interacting with a structured list of tasks throughout the process, ensure that it’s well populated with descriptive content about the steps being taken, tools being used, and sources referenced. In general, anticipating that users may expand or collapse this information allows flexibility about the amount of information they receive.

Visualizing Logic

In traditional automation systems, workflows often include wait states, branching conditions, or retries: these are usually represented as visible nodes in a flow diagram. Users benefit from these visualizations because they create rhythm and realism. They reflect how real-world processes behave: sometimes we wait, sometimes we backtrack, sometimes one path succeeds while another fails.

In AI workflows, similar patterns emerge, but the user is rarely shown how the system is managing timing, alternatives, or fallback behavior. Surfacing this logic in simplified form helps create transparency and predictability:

Make retries and loops visible: If the agent tries again with a fallback method, show that path. For example: “Tried summarization with Approach A, then retried with alternate phrasing.” This helps users understand the system’s resilience rather than mistake repeated output for confusion.

Acknowledge processing phases: Instead of blank screens, explain what’s going on. “Waiting for real-time data to arrive” or “Holding to check time zone settings” gives users clarity about why the system is paused.

Agentic systems may never have fully predictable workflows, but that does not mean they should be opaque - indeed, it’s vital that they are highly transparent throughout the process. As AI systems take on more responsibility, whether managing a complex task or generating a single response, the nature of interaction changes. Instead of carrying out fixed commands, these systems interpret goals, respond to patterns, and make probabilistic guesses. This flexibility creates new possibilities for productivity, but it also introduces new forms of uncertainty. Errors may not appear as obvious failures. They can take the shape of confident outputs that are subtly wrong or unexpectedly off course.

To design truly supportive AI interfaces, rethink how to handle these moments. The next section looks closely at how errors show up in AI systems and discusses how interface design can help users recover, interpret, and keep moving forward.

When the System Doesn’t Deliver: Designing for Errors

No matter how carefully any product is built, it will eventually break. Interfaces fail not only because of technical flaws but because of ambiguity, human inconsistency, shifting expectations, and unpredictable environments. Regardless of severity, the interface becomes the first and most lasting impression of how the system degrades. Meeting this challenge offers extra complexity for designers working with AI, and makes it all the more important to approach errors with a clear understanding of why they occur.

Two fundamentally different types of failure: detectable and undetectable.

Before examining how to design for AI errors, it’s crucial to understand the technical underpinnings that can cause these systems to fail, and also that these failures can occur in two fundamentally different ways. The first type, system errors, are familiar from traditional software: servers crash, connections timeout, APIs return error codes. These failures are detectable, predictable, and mechanical. The system knows something went wrong and can tell you about it.

The second type is far more insidious: errors in the AI’s reasoning or output that the system cannot detect. When an LLM confidently states that the population of Tokyo is 50 million (it’s actually 14 million), or generates a plausible-sounding but completely fabricated research citation, the model genuinely “believes” it’s providing accurate information. There’s no internal error flag, no uncertainty signal, no warning light. From the model’s perspective, everything is working perfectly.

This distinction is critical for designers because it means the burden of error detection shifts dramatically onto users. Traditional error handling assumes the system knows when it fails - that’s why designers can show error messages, retry buttons, and recovery paths. But with AI-generated content, the most dangerous errors often arrive wrapped in perfect grammar, formatted beautifully, and delivered with algorithmic confidence. The model doesn’t know it’s wrong, which means it can’t tell users when to be cautious.

Understanding this split between detectable system failures and undetectable reasoning errors fundamentally changes the required approach to error design in AI interfaces. You’re not just handling errors; you’re helping users become error detectors themselves.

Revisiting Tenets Of Error Design

To design better AI interfaces, it helps to revisit some foundational tenets of error design, principles drawn from decades of human-computer interaction research and product design practice:

Make the error understandable: This is so users can quickly make sense of what went wrong and what to do next.

Make the experience recoverable: Users should be able to revise, undo, or try again without starting over.

Prevent common or foreseeable errors: You can do this through thoughtful defaults, validation, or interaction framing.

Degrade gracefully: Do this when part of the system is unavailable or under strain.

Surface system status clearly: Users should know whether a failure is temporary, persistent, or even happening at all.

These principles show up in classic usability heuristics - for example, in Alan Cooper’s About Face (Wiley, 2014), he urges designers to consider error messages as a last resort because, ironically, they’re useless. In their stead, he encourages designers to prevent software from rejecting inputs, because the real world “is more relevant and important.” When absolutely inevitable, he recommends error dialogs be “polite, illuminating, and helpful,”” focused on helping the user understand the problem and discover alternatives, and disclosing information loss. While originally developed for deterministic software, these principles still offer value in the age of AI - but they require adaptation. What does it mean to disclose a subjective error? How can you design for recoverability when outcomes vary each time? Can an error still be understandable when even the model does not know it is wrong?

In order to find better design solutions for AI, let’s look more closely at these core tenets of error design and examine how each one shifts in the context of AI-driven interaction.

Making the Error Understandable

When something goes wrong, users often look for cues about who or what is responsible. Was it something they did? Is the system confused? Is there anything to fix, or is this just how things are? A well-framed message can help ease that uncertainty.

Rather than relying on technical shorthand or overly vague language, many teams have found success using plainspoken, focused copy. It can be more effective to say, “We couldn’t upload your file. Try a PDF or JPG.” than to surface a coded error such as “Error 413: Request Entity Too Large” or “CORS policy: No ‘Access-Control-Allow-Origin’ header is present on the requested resource”. Specificity helps clarify the issue and offers a concrete next step, even if the full technical details remain behind the curtain.

Context also plays a role. Placing an error message near the place where the issue occurred, like next to a form field or under a disabled button, can help people understand what needs their attention. If it is tucked away or disconnected from the problem area, the message may be easy to miss or misunderstand.

In AI-driven products, this principle often intersects with some new challenges around ambiguity, causality, and confidence:

Errors in AI systems can look like success: An AI-generated response might be grammatically correct and visually polished, even if it is factually wrong or not what the user wanted. These situations can feel especially confusing, since there is no obvious failure.

The source of the issue is not always clear: Traditional systems often fail in predictable ways. With AI, the cause of a problem might be buried in the training data, prompt phrasing, or generation process. Avoiding messages that imply fault can help. Instead of “We couldn’t understand your input,” something like “Let’s try a different approach” can feel more collaborative.

Making It Recoverable

Even with clear messaging, errors can be frustrating if they bring everything to a halt. A supportive interface gives users a way to move forward, whether that means undoing a recent action, retrying a task, or adjusting what they have done.

Undo is a reliable way to reduce anxiety, especially for seemingly-irreversible actions like deletions or submissions. Not every interface can support full undo, and that is okay. In cases where reversal is not possible, it may help to offer a lightweight confirmation step, or at least a path to re-enter the flow with minimal rework.

Retry patterns are also worth considering. If something fails due to a momentary glitch or connectivity issue, offering a retry button right in context can spare the user from wondering what to do next. And in form-based flows, preserving input after failure is essential. It shows respect for the time and thought someone has already invested.

In AI interfaces, recovery often looks a little different, because the failure tends to involve the quality of the result rather than whether the system executed the task:

In AI systems, clicking “Try again” can produce a completely different output, even if nothing about the input has changed. This can be helpful in some cases and disorienting in others. One helpful pattern is to let the user choose: “Regenerate with same input,” “Edit and retry,” or “Start fresh” can all be part of a flexible error-recovery toolkit.

Sometimes the problem is in the output, not the input. In more traditional products, the fix often involves changing what was entered. With AI, the problem may lie in how the model interpreted that input. Giving users the ability to lightly revise or build on what the system returned, rather than having to start over, can be more satisfying and more efficient.

Guided suggestions can offer helpful direction. For example, after an unsatisfying result, options like “Make it shorter,” “Add more detail,” or “Focus on tone” can act as gentle nudges. These kinds of structured adjustments can help users explore what is possible without feeling like they have to know the right thing to ask.

Designing to Prevent the Error

Not every error needs to happen. Many teams have found that small design decisions, like giving people clear expectations up front or catching likely issues before they become failures, can prevent frustration entirely.

Real-time validation is often a useful tool, particularly in forms or structured inputs. When the system can surface a formatting issue or missing field as someone types, it gives them a chance to correct the problem without breaking flow. That said, avoid overwhelming users with too many checks too early. A balance of encouragement and restraint often works best.

Hints, examples, and smart defaults can also reduce errors before they happen. Even a short placeholder like “e.g. your-work@email.com” can gently steer people toward what the system expects.

With AI-based tools, error prevention is often less about blocking invalid inputs and more about shaping intent and expectation.

Inputs might be valid, but still not effective: A prompt or question can be perfectly well-formed but still lead to an unclear or generic output. Consider supporting users with starter phrases, examples, or even subtle framing like “Try describing what you’re looking for in one or two sentences.”

Traditional validation tends to focus on rules and constraints: AI interfaces benefit from encouraging structure and good starting points. A gentle hint, such as suggesting a tone or content format, can go a long way in steering toward successful outcomes.

Setting expectations helps avoid perceived failures: It is not always obvious to users what an AI system can or cannot do. If there are limits, such as not providing real-time information or not supporting certain domains, surfacing that early on can prevent confusion later. Framing this as a way to help the user get better results, rather than as a limitation, tends to land better.

Letting the System Degrade Gracefully

Real systems often fail. Connections drop, services go down, and sometimes only parts of the product are affected. When these things happen, the interface plays an important role in keeping users oriented and minimizing disruption. One approach is to preserve whatever progress is possible. If someone is in the middle of a task, it helps if the system can hold onto their input, even if syncing or submission is not possible. Local saves or drafts can reduce the emotional cost of a system hiccup. Figure 4-3 shows an example in Cline.

Figure 4-3. Cline, an LLM-enabled plugin for VS Code offers easy rollback via its checkpointing system

Partial failure does not have to mean a full stop. If one feature is unavailable, the interface can often still provide value. Maybe a real-time preview cannot load, but the underlying form still works. Showing what is available and noting what is not allows people to decide whether to continue, pause, or come back later.

For AI systems, degradation may show up subtly. The interface may remain active, but the quality or completeness of the results begins to break down:

AI outputs often fail softly: A model might begin generating something useful, then trail off or include unrelated content. Rather than discarding the entire result, preserving the good parts and letting the user continue from there can help. A note like “You can keep editing from this point” makes it easier to stay in flow.

The system may appear responsive, even if it is not: Unlike traditional failure states, where a button click leads to a clear error, AI issues may be less obvious. The interface might need to help surface that something is not working as expected. For example: “We’re having trouble completing this. Want to try again or revise it?”

Preserving the full context matters: If an AI result cuts off unexpectedly or seems incomplete, saving the prompt, prior messages, and output so far gives users a foundation to build on. Recovery, in this case, is less about retrying and more about scaffolding a new direction.

Surfacing the System’s State Honestly

A big part of designing for errors is designing for ambiguity. Users often do not know whether a problem is temporary, permanent, or even real. When the interface stays quiet in these moments, people may blame themselves or assume the system is broken.

Even a small acknowledgment that something is off can help. A message like “Saving is currently delayed” or “This feature is temporarily offline” sets expectations and lets people decide what to do next. It does not take much to make the system feel more transparent.

Tone can play a role here too. Many teams choose to use language that is calm, clear, and matter-of-fact. Humor can lighten the mood, but when something important goes wrong, users may not feel like laughing, and a direct message usually feels more reassuring.

In AI systems, surfacing state becomes more complex, since the model itself often lacks self-awareness about its failures:

The system doesn’t know about the error: As mentioned earlier, AI systems do not always know when they are wrong. Since a model might confidently return a false or irrelevant result, without any backend signals that something has failed, designers can experiment with small, user-facing cues that acknowledge this uncertainty and empower users to validate results. For instance: highlighting information that should be verified, or providing gentle prompts like “Double-check important details” for factual queries. Even simple patterns like “Generated response - accuracy not guaranteed” or source attribution (“Based on training data from 2023”) help users calibrate their trust appropriately.

Issues might be isolated, not global: One user might hit a generation cap or receive a strange output while others see normal results. Rather than posting system-wide banners, it can be helpful to localize status updates. For example, letting the user know they have reached a usage limit or that the system is having trouble in this specific instance.

Clear scope boundaries reduce confusion: Many misunderstandings in AI systems come from mismatched expectations. If a model cannot provide certain types of guidance or up-to-date information, naming those limitations up front helps users understand what the system can realistically deliver.

Taken together, these guidelines suggest that designing for errors is just as much about tone, timing, and transparency as it is about catching mistakes. And in systems that incorporate AI, it becomes even more important to treat error not just as a problem to fix, but as a moment to frame, guide, and support.

Designing for errors is not a sign that something has gone wrong in the product. It is a recognition that real people work in real conditions, and even well-designed systems will sometimes falter. These moments offer a chance to show care, to reinforce trust, and to make failure feel less like a dead end and more like a detour.

When AI enters the picture, the nature of error changes. It becomes softer, sometimes harder to notice, and occasionally entangled with user expectations rather than technical correctness. These differences call for new patterns and more flexible thinking. But the core goal remains the same: to help people feel confident, supported, and capable, even when the system does not quite deliver.

Context Windows and Conversational Memory

We’re already used to the idea of context in our everyday lives. It’s how we know what someone means when they ask us to grab that thing from the kitchen, or to pick up the usual from the store. It’s how we fill in the blanks when a friend says, “You know what I mean,” and we usually do. Context works because of everything leading up to the moment: the prior conversation, the shared history, and the unspoken cues.

It’s the same for language models. A user might say something vague like make it sound more friendly or continue that idea, and the model needs to figure out what exactly that refers to. Without prior context, whether from earlier messages in a conversation, structured memory, or fine-tuned instructions, the model wouldn’t be able to resolve the ambiguity. Context is what makes AI outputs feel coherent, relevant, and even a little bit personal. Without it, the system’s responses can feel random, stiff, or disconnected.

Humans do this kind of contextual threading automatically. We’re constantly tracking details across time - topic shifts, emotional tone, changes in intention - and using that to keep conversations on track. For AI models, that capability is managed through something called self-attention. Self-attention is a mechanism that lets the model evaluate every word or token in relation to every other word or token in the input. It doesn’t just look at what was said most recently: it scans the full string of information, applies different weights depending on what seems most relevant, and uses that to shape the output. In practice, that means a model doesn’t treat all words equally. It might assign more importance to specific phrases earlier in the prompt that clarify the user’s intent, or deprioritize filler content.

To improve relevance, AI systems don’t just process the user’s immediate input - they often bundle it with other pieces of related information. That can include the full conversation thread, metadata about the user’s preferences, or memory-like elements from earlier sessions. This process gives the model a more complete view of what’s going on, ideally allowing it to offer a more contextually appropriate response. However,there’s a cost to this, and not just in system complexity. The more information you give a model, the more work it has to do—and at some point, that starts to degrade performance. Research has shown that beyond a certain length of context, models may lose the thread instead of strengthening it. They may latch on to the wrong parts of the input or ignore important early details. Worse, the outputs may become slower or less predictable.

Compounding this challenge, AI systems are constrained by what’s called a context window. This is a hard limit on how many tokens can be submitted at once, and each model has its own constraints. Anthropic’s Claude, for example, currently supports up to 200,000 tokens in one go. If you go over that limit, you’ll get an error. Other models are pushing those boundaries even further. Google’s Gemini has shown it can successfully retrieve information across an input of one million tokens, a scale more akin to reading a bookshelf than a paragraph. Figure 4-4 illustrates Anthropic’s view of a context window.

Figure 4-4. From Anthropic’s blog, visualizing how a context window is filled during a conversation

There’s a tradeoff here: long inputs are computationally expensive and less performant, and increase the likelihood of a model getting lost in the weeds. At the same time, users often assume that more context is better - that the more background they provide, the smarter the response will be. And while that’s sometimes true, it’s not guaranteed. In fact, giving the model too much to sift through can dilute relevance and make the model less effective.

For designers, this presents a balancing act. Users need context to feel understood, but they also need clarity about what the system can reasonably handle. Interfaces can help by gently shaping how people think about and manage context. Here are a couple of options:

Show token usage transparently: Displaying a visual token counter, either during prompt creation or as part of a message warning, can help users self-regulate. Even a soft indicator like “Input too long, try shortening your request” can nudge behavior in the right direction without overwhelming the user with technical jargon.

Warn before input gets too long: If the system is approaching its context window limit, a simple banner or inline notification can alert the user and offer options—like pruning earlier messages or shifting to a new thread—to preserve reliability.

Ultimately, the goal isn’t to force users to think in tokens. It’s to design interactions that make room for meaningful context while helping users avoid overload. That might mean educating users gently over time, offering smarter defaults, or adjusting the framing of what it means to give a model enough to go on. As with any good conversation, it’s not just about what’s said—it’s about how it all fits together.

Don’t Make Me Wait: Managing Latency and Delays

Humans hate to wait. A 2015 study on grocery checkout lines found that for every additional 30 seconds someone spent standing still, their frustration increased more than if they’d waited twice as long while moving. Many organizations have simply stopped trying to reduce wait time and instead focus on managing how it’s perceived: for example, in the 1990s, Houston Airport received a wave of complaints about slow baggage delivery. The solution? They moved the baggage claim further away, so passengers spent more time walking and less time standing around. Complaints dropped to zero.

You might assume from this that people simply want to be distracted while they wait - but it’s more nuanced. Research in human-computer interaction suggests that when people understand why they’re waiting and feel confident that progress is being made, their tolerance increases - even without distraction. This shows that people don’t measure wait time in milliseconds; they assess it emotionally, contextually, and socially. Waiting is less about the passage of time and more about how we feel while it passes.

In software design, delays like this are called latency, and it’s been a core consideration for decades. Back in the early ’90s, before the invention of broadband, web designers were warned that a page should take no more than 15 seconds to load. To optimize user experience, researcher Jakob Nielsen established a set of widely accepted thresholds for how users perceive delays that persists to this day:

0.1 seconds: Feels instantaneous. There’s no need for special feedback—just surface the result immediately.

1 second: The threshold for keeping a user’s flow of thought uninterrupted. They’ll notice the delay, but it doesn’t break their focus. Feedback isn’t strictly necessary, but users no longer feel like they’re directly manipulating the data.

10 seconds: About the limit for maintaining attention. Beyond that, users start shifting focus to other tasks. At that point, progress indicators or time estimates become critical, especially when response times vary.

With faster internet speeds (from an average of 28.8 kbps in 1996 to over 135 Mbps in the U.S. today), you’d think these thresholds would be easy to hit. However, web experiences have grown in complexity. A 1996 site could have been 60 KB; now, a modern site’s source code often exceeds 2 MB - and that’s before adding images, video, streaming, or live collaboration. In many ways, we’re still waiting - just a little less (around 4 seconds on average, today).

AI introduces a whole new layer of complexity to the latency puzzle. Unlike simple HTTP requests, as mentioned earlier in this chapter, AI workloads involve everything from model interfacing, to tokenization bottlenecks, GPU scheduling, memory constraints, and server load balancing. Optimizing performance means solving for a broader, more tangled set of constraints. Plus, latency in AI isn’t just about speed: often it’s about uncertainty. Unlike traditional interfaces where users anticipate the outcome, AI products often involve invisible, variable computations that take longer and feel less predictable. Users aren’t just waiting for data to load: they’re waiting for a machine to “think.” This shifts the design challenge: instead of just minimizing wait time, we also need to manage expectations, build trust, and signal progress in ways that feel aligned with the user’s intent.

Research does show that users are generally patient with AI’s computational delays - but only if they’re kept in the loop. They’ll wait, but they want to understand what’s happening and why. By aligning interface design with the user’s task and showing the right signals along the way, it’s possible to design in a way that makes AI feel fast, even when it isn’t. This leads to the three core priorities when designing for AI latency:

Make it fast where it must be (real-time control, transactions).

Engaging where it can be (AI art, chatbots, entertainment).

Clear where it needs to be (search, payments, learning tools).

This framework helps prioritize where to spend design effort based on what the user is trying to do. If a task requires precision or timing, such as controlling something remotely or confirming a payment, it must feel fast and responsive. If it’s more open-ended or creative, like generating AI art or chatting with a bot, it can afford to be a little slower, as long as it stays engaging. And for tasks where accuracy and understanding are critical, such as learning something new or completing a financial transaction, clarity matters most, even more than speed or flair.

Visual Methods for Mitigating Latency

There are many ways of visualizing and communicating a delay. Just as all UI elements send a subliminal message to a user about the product (intentionally or not), latency is no different. Users gain insight about the product’s function (or lack thereof) in the moments where its processing, and there are many ways to indicate these delays and background processes elegantly through UI:

Immediate acknowledgement: Promptly acknowledging user actions reassures them that the system is processing their request. For instance, displaying a “Thinking…” message or a typing indicator in a chat interface provides immediate feedback, preventing uncertainty.​ However, if the system acknowledges the user’s action but then delays in delivering an actual result, this can lead to increased frustration. The initial feedback should seamlessly transition into meaningful progress updates or the final output.​

Loading indicators: These inform users that a process is underway, but choosing the right type is essential:

Spinners (indeterminate indicators): Best for short, uncertain durations where the exact time to completion is unknown. They signal activity but don’t provide specific progress information.​

Progress bars (determinate indicators): Ideal when the system can estimate the task duration. They offer users a sense of control and reduce anxiety by showing how much of the task is complete.​

We know that users prefer progress bars over spinners, especially for longer tasks, as they provide clear feedback on progression. However, inaccurate or stalled progress bars can lead to increased dissatisfaction. They can also be technically infeasible to implement given the complexity of AI systems and the number of components working under the hood.

Skeleton screens: Displaying placeholder content that mimics the final layout (for example, grey boxes in place of images or text) can reduce perceived wait times compared to spinners alone, particularly when accompanied by an animation. Users feel progress is being made, since parts of the UI are already present, and can anticipate the breadth of content they might see, though for this reason skeleton screens are most effective for predictable, structured content.

Asynchronous (non-blocking) interactions: Allowing users to continue other tasks while waiting for the AI to complete processing ensures they remain productive, minimizing the impact of latency. For example, enabling navigation to different sections of the application or queuing additional requests maintains engagement, though this may require a robust notification system that alerts the user when the task is completed.

Progressive disclosure of results: Delivering output in increments allows users to access partial results sooner. For instance, an AI system might first display a quick summary, followed by more detailed information as it becomes available. This method keeps users engaged and provides immediate value, even before the complete data is ready.​

Real-time streaming of partial results: Implementing token streaming enables the system to display content as it’s generated, similar to watching text appear word by word. This continuous feedback loop significantly reduces perceived latency, as users can begin consuming information almost immediately.

Articulating wait reasons: Users are forgiving of longer processes when a greater computational output is anticipated, but generally they want to know why. Explaining what the task will involve, or using generated text to narrate the process keeps the user informed.

Applying these techniques with intention is essential—after all, not every AI needs to show its homework. Features like streaming results, adjustable timing, or progress indicators should be used selectively, based on the user’s goals and the context of use. When applied thoughtfully, they enhance clarity and trust; when overused, they risk cluttering the experience and diluting the AI’s perceived competence. Sometimes, the smartest move is knowing when to say less.

The Value of Segmenting by User Need

To put the aforementioned latency mitigation techniques into practice, it’s vital to emphasize the importance of refining a latency experience by user need. For example, imagine you’re in the middle of learning something new, and trying to get a clear answer from a chatbot. You’re likely holding several concepts in your mind at once, right at the edge of your working memory. According to Miller’s Law, most people can only keep about 7 (plus or minus 2) items in mind at once, so even small delays or vague responses can be highly disruptive. Velocity is key in this scenario, plus high information about the amount of anticipated wait time.

Now picture a different scenario: you’re iterating on a project, perhaps asking an AI to enhance a piece of writing or generate a visual. The cognitive load is lighter—you’re reacting, evaluating, maybe exploring. In this case, a slower, more open-ended experience can still feel satisfying, even rewarding, because you’re not depending on it to hold your thoughts together. The following two sections offer are some suggestions for categorizing user tasks by substantive latency solutions that improve user experience without overcomplicating the process:

Information retrieval (medium/slow)

Information retrieval includes searching, filtering, and query-based interfaces—everything from browsing a shopping site to using a semantic search engine. These tasks are often exploratory, and users are mentally juggling what they’ve seen, what they’re seeking, and how to refine their search. When latency exceeds just a few seconds, that mental context begins to fade, making the task feel fragmented.

To preserve a feeling of rapid flow, designers can employ progressive disclosure to show early results while more are loading. Immediate acknowledgement is key, plus avoiding blank states: users should never wonder if the system is doing anything.

Predictive search and autosuggest features can also help users act before they finish typing, giving the perception of speed

Real-time control systems (instant)

Some tasks involve actions where users expect immediate feedback because they are actively manipulating or observing a system in motion. For example: using live transcription or translation tools and needing to see the result instantly, adjusting a smart thermostat and seeing the temperature update in real time, or using real-time collaboration tools like Figma or Google Docs where changes need to appear rapidly for multiple users.

The user’s priority in these examples is responsiveness, with close-to-zero delay. To achieve this, the interface should prioritize immediate visual feedback for user inputs (for example, sliders that update UI as you drag)

Use skeleton loaders to suggest likely outcomes before processing finishes.

If multiple processes are happening at the same time, prioritize parts of the experience that emphasize connectivity. For instance, in collaborative tools, show avatars or cursors moving in real time, even if content syncing is slightly delayed. This helps preserve a sense of control, even when computation is happening behind the scenes, and preserves functionality.

Transactional processes

Transactional tasks - like submitting a payment, authenticating, or confirming a booking - are short, high-stakes interactions that can arise through either user or agent-initiated queries. Users are usually in a focused, goal-driven state, and they expect a quick confirmation to signal that their action was successful.

Whether user or agent-initiated, immediate confirmations are needed. If no message is shown, doubt can set in quickly, causing users to refresh or re-initiate the task, potentially duplicating actions or abandoning the task altogether.

Optimistic UI (for example, showing confirmation before server response is finalized), and loading indicators (like spinners with copy such as “Securing payment…”) help maintain user confidence. If longer processing is needed, include estimated wait times or staged feedback (for example, step 1 of 3: verifying card, processing, confirmed) to reduce abandonment.

Conversational agents

In chatbot and virtual assistant interactions, users expect a conversational rhythm - not just speed. A response that comes too fast can feel robotic, while one that takes too long breaks the illusion of dialogue. Around 2–3 seconds feels natural for most users, especially if the system signals that it’s working.

Typing indicators are essential because they anchor the user in the interaction and communicate progress.

For more complex tasks, consider streaming partial responses (like one sentence at a time) to make the experience feel more fluid. You can also include confirmation echoes (such as “Okay, let me look that up”) to buy time and keep the user cognitively engaged.

Creative content generation

When users ask AI to generate creative outputs - stories, images, music, or code - they’re usually in an exploratory mindset and more tolerant of delays, as long as expectations are well managed. These tasks don’t demand instant results, but they do demand a sense of meaningful progress. Sometimes, the type of query or product affects how well the model can anticipate timing, which opens the door for more tailored feedback. For instance, diffusion models work a bit differently from LLMs: they run a calculation, instead of searching through data, so they usually have a clearer sense of how long generation will take.

Because static spinners are negatively associated with long wait times, designers should avoid them in preference of progress bars, and strive to provide a fixed amount of time, if speculative, as long as it is reasonably accurate. This type of feedback maintains momentum and enhances the sense of collaborative creativity.

Sequential messages about the process could provide delight; similar to in gaming waiting screens. Though not appropriate for every context, the lower-stakes nature of creative exploration means that this could be a suitable way to mitigate delay.

Latency for autonomous tasks

Agentic AI marks a shift from reactive tools to proactive systems - autonomous assistants that initiate actions, make decisions, and operate independently of constant user input. This transformation challenges conventional notions of latency, requiring new strategies to support user trust, control, and understanding. Critically, agentic AI requires transparency to succeed. Because it acts without always being prompted, users need ongoing cues that it’s working with them. Clear acknowledgments, well-paced updates, and visible intent make the system feel intelligent, respectful, and aligned with user goals.

Setting up a successful autonomous experience requires some intentional UI design:

Showing anticipated wait times is a key part of the setup for working with an autonomous tool, or at least articulating in some way that this is a task with far greater complexity.

Designers must ask themselves what are tolerable windows of time that the user can go without new information. Implementing intentional pacing might mean that every x seconds, the agent checks in to update the user on their work, or get permission for a next task. This keeps the model robustly in touch with the user.

Ongoing task feedback and streaming results allow users to have insight into background operations, like scanning documents, summarizing inboxes, or preparing reports. Multi-step or research-intensive tasks benefit from incremental delivery. For example, a travel assistant might first suggest flight options, then progressively add hotel and itinerary details, allowing the user to respond or intervene mid-flow.

Users may develop enough trust in the model’s behavior that they’re comfortable skipping oversight and simply waiting for the outcome. Providing an option to let the model operate independently, without requiring constant approvals, is also a form of latency design. It allows users to shift their attention elsewhere and receive a notification when the task is complete, rather than staying engaged throughout the process.

When working with autonomous system latency, designers must consider not just when and how fast these systems respond, but why and how clearly their timing reflects competence and care. A recent study found that AI processing delays can have a counterintuitive benefit: they afford users time to reflect on multiple research questions simultaneously, leading to a higher quantity of generated questions and an enhanced sense of control. The finding underscores the importance of thoughtful latency design: setting good expectations and anticipating delays sets up autonomous AI as a good working partner, enhancing the perception of the AI’s competence and contributing to a more satisfying and productive user experience.

Ultimately, designing for latency in AI isn’t about eliminating wait time: it’s about designing the right kind of waiting. When users understand what’s happening, feel confident that progress is being made, and believe the system respects their intent, delays can shift from annoying to acceptable, even valuable. As we’ve seen, designing for latency doesn’t just keep users occupied; it makes them feel seen, informed, and in control. The goal isn’t to mask delays with an elaborate spinner or skeleton screen: it’s to build a rhythm between user and system that feels collaborative, coherent, and clear. When wondering how to build user satisfaction in the face of unavoidable delays, remember, it’s not just about the time it takes: it’s about how well you use it.

Summary

In the opening story about the 1202 error code, the term error was something of a misnomer: The Apollo 11 computer never malfunctioned during that tense descent to the Moon. It performed exactly as designed - but to the astronauts watching that cryptic “error,” the message was indistinguishable from failure. The gap between what the computer was doing and what the humans could understand nearly derailed one of humanity’s greatest achievements.

Today’s AI systems present us with a similar challenge. Every time a user submits a prompt, uploads a document, or asks for creative output, they’re triggering computational processes that dwarf the computing power of lunar guidance systems. Behind the clean interface of a chat box or generation tool lies a three-stage cascade of invisible decisions. Understanding this computational pipeline reveals why AI errors are so fundamentally different from traditional software failures. When a database crashes, users get clear error messages or at least a determinate state. When an AI hallucinates, it produces confident, well-formatted, professionally written content that happens to be completely wrong. These systems don’t fail obviously: they fail insidiously, generating plausible but fabricated statistics, citing non-existent sources, or gradually drifting away from user intent as context windows fill up and early instructions get forgotten.

The solutions to this problem lie in designing interfaces that work with computational reality rather than against it. When we understand that inference happens token by token with no master plan, we can design streaming interfaces that let users course-correct mid-generation. When we know that multimodal systems coordinate between specialized models in real-time, we can set appropriate expectations about timing and provide meaningful progress indicators. When we recognize that autonomous agents work through complex multi-step workflows over extended timeframes, we can create checkpoint systems that preserve progress and enable human intervention when needed.

Perhaps most crucially, understanding computation helps us design for latency in ways that strengthen rather than undermine the human-AI relationship. Users will wait for AI responses - research shows they’re remarkably patient with computational delays - but only if they understand what’s happening and why. A progress bar that accurately reflects diffusion model iterations feels completely different from a generic spinner. Streaming text that appears word by word creates engagement rather than frustration. Status updates that explain agentic workflows (“researching current trends,” “analyzing competitive landscape,” “generating recommendations”) transform wait time into anticipation.

Understanding computation reveals that AI errors aren’t bugs to be eliminated: they’re natural byproducts of working with probabilistic systems that are constantly making educated guesses based on their training data. When you design interfaces that treat errors as collaborative opportunities rather than system failures, you enable users to work with AI uncertainty rather than despite it. A hallucination, though never ideal, becomes a chance to clarify requirements. A context limitation becomes an opportunity to refocus on what matters most. A routing failure becomes feedback about how to structure future requests.

Unlike the Apollo astronauts, who had to trust their computer’s decisions without understanding its logic, today’s AI users can have the benefit of well-designed interfaces that endorse existing design heuristics while also recognizing and appreciating the requirements AI adds. The challenge for designers is knowing which aspects of that process to surface, when to surface them, and how to communicate computational complexity without overwhelming people trying to get work done.

The 1202 alarm ultimately didn’t prevent the Moon landing, but imagine how much more confident the astronauts might have felt if they’d understood what their computer was really doing. In the age of AI, your job as a designer is to build that understanding into every interaction, turning mysterious computational power into trustworthy collaborative intelligence. When we succeed, users stop seeing AI as an inscrutable black box and start experiencing it as a sophisticated partner with specific strengths, limitations, and quirks.

But understanding what happens during computation is only half the equation. Once the AI system completes its complex cascade of processing, routing, and generation, it must present its results to the user, and this moment of delivery is where success or failure is ultimately determined. A perfectly computed response can still mislead if it’s presented without proper context. A creative output can lose its impact if it arrives without clear next steps. Even accurate information can erode trust if users can’t verify its sources or understand its limitations.

The transition from computation to communication represents one of the most critical design challenges in AI interfaces. In the next chapter, we’ll explore how to design outputs that don’t just deliver results, but invite the right kind of interpretation, action, and ongoing collaboration between human and machine intelligence.

Chapter 5. Output: Designing the Delivery and Presentation of LLM Responses

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 5th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you’d like to be actively involved in reviewing and commenting on this draft, please reach out to the editor at ccollins@oreilly.com.

In the 1970s, as hospitals began digitizing patient records, a new concept emerged at the intersection of computing and clinical reasoning: the clinical decision support system, or CDSS. These early systems (MYCIN, INTERNIST-1, and later DXplain) did not attempt to replace doctors, but were meant to assist them by surfacing probable diagnoses.

In principle, these systems were intelligent aids, but in practice, they often struggled. One of the reasons was technical: early models couldn’t reason across complex or incomplete patient histories, or evolving treatment protocols. But a deeper issue ran beneath the surface. Even when the logic was sound, the output could be misunderstood. A flagged diagnosis might be taken too seriously - or too lightly. A suggestion might be seen as definitive, when it was only probabilistic. Some doctors ignored the systems entirely. Others over-relied on them, treating suggestions as answers. In either case, the interface became the risk surface: the way output was presented and interpreted determined whether it supported care or disrupted it. Figure 5-1 shows a standard output from MYCIN, indicating that a bacterial infection may be present if the five conditions in the function are true.

Figure 5-1. A standard output from MYCIN

This interpretive problem has become the defining challenge of modern AI interaction. Today’s systems face the same dilemma CDSS researchers encountered fifty years ago: how do you design outputs that build trust? How can you design an output to be transparent, or guide without misleading? How do you surface uncertainty without overwhelming the user? And how do you build interfaces that support judgment, not just consumption?

The history of CDSS offers a foundational lens for understanding output design today. Those systems were among the first to produce fluent, recommendation-like content that mimicked human reasoning. But their very fluency became a liability when not paired with sufficient transparency or guardrails. MYCIN, for instance, could diagnose blood infections with near-human accuracy, but it also required users to understand its rule base and its confidence thresholds. As noted in the paper ‘Clinical Decision Support Systems’ (Musen, M.A., Shahar, Y., Shortliffe, E.H., 2006), doctors exhibited skepticism with these early programs. In many cases, because the system lacked transparency, doctors either ignored the system or misinterpreted it. As CDSS evolved, interface designers learned to add explanation layers, highlight contraindications, and allow clinicians to inspect the why behind a recommendation. These changes were essential to making the output usable, trustable, and, in some cases, safe.

AI systems today face a vastly expanded output space. A single model might summarize a policy, rewrite a job description, identify trends in sales data, generate an image, and suggest a next-best action for a user. In each case, the output initially looks complete. But as with a diagnostic suggestion, the output is only part of the truth. What is missing is often invisible: original sources, omitted alternatives, unstated assumptions, or training data biases. These aren’t bugs in the system. They are byproducts of the way modern AI generates outputs: through pattern completion, not verification.

Designing Outputs: The Output Is Not the answer

Designers working on AI systems must understand that the output is never just an answer. It is a designed artifact, interpreted through an interface and shaped by defaults, context cues, and user expectations. And just as in CDSS, the way an output is designed can dramatically affect how users act on it.

AI outputs are more than just the endpoint of generation. They are events in a wider process, with psychological, structural, and ethical implications. This chapter looks at how outputs are structured, evaluated, personalized, edited, and sometimes misunderstood. And we’ll return to the lesson CDSS offered half a century ago: generation is only the beginning. Without care in how outputs are communicated and interpreted, even the most well-trained model can produce results that mislead more than they assist.

Before diving into the design principles of outputs in detail, let’s look at what’s happening when you get to the output on a technical level. By the time you reach the output step, you have seen the user’s input being translated into tokens, routed to the correct model, put through generation, and made ready to be transformed into a usable result. For this to happen, there are three sub-phases:

Internal generation: The model internally finishes the computation and decides what the output should be.

Post-processing: The system cleans up, formats, or enhances the output into a deliverable form.

Delivery/presentation: The system transmits or renders the output to the user, system, or environment.

How this happens varies depending on the model type, including LLMs, image generators, video generators, classic ML classifiers, and reinforcement learning agents - we’ll briefly cover these.

For example, in language models, output generation must first be detokenized: the model’s internal units converted back into human language. During post-processing, this text is lightly cleaned: extraneous spaces are trimmed, subwords are rejoined, and the content may be lightly reformatted into markdown, bulleted lists, or structured code blocks. Systems often apply guardrails at this point, scanning for sensitive material, policy violations, or problematic phrasing. Only after these guardrail steps is the output delivered to the user, usually as a chat bubble, streamed text, or document element, ready for interaction.

For image generators, the process follows a similar high-level arc, but with some distinct differences. Once a latent image has been fully formed, it must be decoded into real pixels to create a coherent output. A specialized network translates the compressed internal idea into a full-resolution image. After processing for safety filters and rendering corrections, the final image is then compressed into a standard format, such as JPEG or PNG, and either embedded directly into the user interface or made available for download. Video generation introduces the added challenge of maintaining coherence across time, meaning a similar process is applied but to a sequence of frames.

Once the AI finalizes its output, no matter the format, the responsibility shifts to the interface. It’s no longer about what the AI produced; it’s about how that output is delivered to the user. The interface must present the information clearly, manageably, and in a way that matches the user’s cognitive needs and task goals. A well-designed presentation can make even complex or uncertain outputs feel approachable and useful; a poorly designed one can erode trust or overwhelm the user. To unpack the layers involved, we’ll cover the high-level principles that govern output success.

Design Principles of Outputs

The output is the product in most AI interfaces. Whether it’s a sentence, summary, plan, image, or code snippet, the content is the thing the user came for. While the output often answers a question, so to speak, it’s not speaking in a vacuum. The core questions users ask - Is this useful? Can I trust this? Is this what I meant? - are answered in part through the appearance of the output itself. For that reason, effective outputs must adhere - with a degree of flexibility - to certain principles. They should be:

Clear: Instantly understandable

Verifiable: Evidence-backed results

Grounded: Context-aware

Actionable: Onward-task oriented

Adjustable: User-editable

Even as the specifics of AI outputs differ, the core design goals remain consistent, and achieving these goals consistently requires both guiding principles and repeatable interface strategies that can be adapted to different output modalities.

Designing for Clarity

Content design shapes information so people can actually use it. It’s about making things clear, scannable, and structured in ways that match how we naturally read and process information. Beyond just words, content design creates meaning through how things are presented to the user, with simple elements like hierarchy and prominence conferring meaning. In the same way, in AI outputs, content design governs how generated outputs are presented: whether that’s a paragraph of text, a table of options, a visual caption, or a multi-step plan. Good content design helps guide the user and answer implicit questions through thoughtful structure and clarity.

Understanding and trusting an output becomes easier when it feels tailored to the user’s intent. Clarity, tone, structure, relevance, and readability aren’t decorative choices, but instead represent the difference between an output that feels actionable and one that feels alien. As the Nielsen Norman Group have observed, users won’t engage with text unless it’s clear, simple, and easy to scan, regardless of whether it’s written by a human or an LLM. In this case, designers must particularly advocate for the user’s best interests, because AI won’t naturally write or organize content in a user-centric way unless it is told to do so.

Structuring for clarity: From raw to refined

As mentioned earlier, AI models, at their core, predict the most probable token based on the prompt and prior context. They do not inherently “know” what good, readable writing looks like. Without deliberate prompting, outputs can default to undifferentiated text, or otherwise obscure a valuable output with bad or alienating structure.

For outputs to integrate cleanly into products and services, designers must step in. Good AI output design layers structure, tone, pacing, and cognitive rhythm on top of raw model capabilities. Basic prompting, like asking a model to Use headings and bullet points might help a little, but can fall short.

Formatting rules are a way of gearing a text output towards readability. Instead of broadly asking for headings and bullets, you can spell out the entire hierarchy and its purpose. A good prompt might ask the model to start with a clear title, break major ideas into H2-level section headings, subdivide complex areas with H3s, and use bulleted or numbered lists for digestible steps or key points. Here’s an example of such a prompt:

Organize content with a clear title, followed by section headings (H2 style), subsections (H3 style if needed), and bulleted or numbered lists for procedural information. Use concise paragraphs (max 3 sentences each). Bold key actions or terms. Add a 'Notes' or 'Tips' section where appropriate.

Defining templates within the prompt or system instruction is another powerful lever. An effective output template might open with a title, followed by a short introduction that sets context. Key concepts could come next, in a compact bulleted list, before stepping through numbered instructions for complex processes. A short conclusion or set of next steps can close the loop, and sources should always be listed at the end, formatted cleanly with hyperlinks. When outputs consistently follow an internal logic, users learn how to navigate them faster and trust them more instinctively. Here is an example of an effective output template:

Format all instructional content as:
Title: [Concise, clear]
Introduction: [Brief, 1-2 sentences summarizing purpose]
Key Concepts: [Bulleted list of important points]
Steps: [Numbered list of actions]
Conclusion/Next Steps (Optional)
Sources: [Bulleted list with hyperlinks]

Tone and voice matter just as much as structure. Without oversight, models might vary their tone significantly. A clear tone specification, like the one in this example, gives the model a target personality to aim for:

Use a clear, friendly, moderately formal tone suitable for a general but technical audience. Avoid jargon unless specifically appropriate. Prioritize clarity over creativity. Where multiple methods exist, present the primary method first, alternatives afterward, clearly flagged.

Beyond tone and structure, designers must think about how models should handle trade-offs. Should outputs be exhaustive, or favor brevity? Should every edge case be addressed, or should the core path stay streamlined? Giving models a simple heuristic, such as prioritize readability and logical flow over exhaustive completeness, helps them resolve these tensions in favor of user experience. Instructing models to segment longer outputs, use short paragraphs, and create strong transitions between sections ensures that users can skim when they want to and dig deeper when they need to. For example:

Prioritize readability and logical flow over exhaustive detail. Emphasize information architecture that makes outputs scannable: short sections, clear transitions, and segmented content.

Finally, it’s important to plan for failure. Models have a strong tendency to improvise, and unless told otherwise, they may invent plausible-sounding content to fill gaps. Explicitly telling the model to output a clear statement like No data provided when information is unavailable sets a safer, more transparent default. It also signals clearly to users that something is missing, rather than silently risking misinformation. Try something like this:

If required content is missing or unavailable, output 'this isn’t available' instead of inventing or speculating.

Before and after: A practical example

To see the difference between naive and designed output, consider a simple prompt:

How do I set up an MCP server?

When left to its own devices, the model might produce something like this:

To set up an MCP server, first install the Java Development Kit (JDK) and download the necessary server software. After extracting the files, create a script file to start the server. Setup varies slightly based on your operating system. If you run into issues, check the documentation.

Technically correct—but flat, hard to skim, and not very actionable. By contrast, a carefully styled output could look more like this:

Setting Up an MCP Server
Introduction: Setting up a Minecraft Coder Pack (MCP) server can differ depending on your operating system and technical background. This guide provides step-by-step instructions with tailored tips for Windows, macOS, and Linux users.
Key Concepts:
     - MCP servers require Java Development Kit (JDK) and server files.
     - Command-line familiarity improves setup efficiency.
     - Different OS environments require different script configurations.
Steps:
     1. Install Java:
        - Download and install the latest Adoptium JDK.
        - Windows users should use the Windows Installer; macOS users should prefer the .pkg installer.
     2. Download MCP:
        - Clone or download  MCP Reborn or a maintained fork.
     3. Extract Files:
        - Place the extracted MCP folder in a directory without spaces (e.g., C:\MCP or /Users/username/MCP).
     4. Configure Environment:
        - Windows: Edit the setup.bat script as needed.
        - macOS/Linux: Use setup.sh and grant execution permissions with chmod +x setup.sh.
     5. Run Setup Scripts:
        - On Windows, double-click setup.bat.
        - On macOS/Linux, execute ./setup.sh from the terminal.
     6. Handling Errors:
        - Missing environment variables? Consult  Oracle JDK Path Setup.
        - Linux users should verify that JAVA_HOME is set correctly.
     7. Test the Server:
        - Launch the server using the provided scripts (start.bat or start.sh).
Notes:
     - Beginner Tip: If command-line tools feel intimidating, consider using Git Bash on Windows.
     - Advanced Tip: For experienced users, automating setup with Docker or CI pipelines can improve development workflows.
Sources:
     - Official Java Downloads
     - MCP Reborn Repository
     - Minecraft Modding Wiki

Notice how much easier it is to skim, act on, and trust the structured version: the layout sets expectations; important concepts pop out visually; steps are easy to follow. Even if the user isn’t familiar with all the tools involved, the guide offers helpful branches for deeper learning.

To achieve the preceding output, you might use an enhanced system instruction like the following:

Always produce structured output following this order: Title → Introduction → Sections → Subsections → Lists → Notes/Tips → Conclusion → Sources. Use H2 for major sections, H3 for subsections. Limit paragraphs to 2–3 sentences. Bullet lists for unordered information, numbered lists for ordered steps. Bold critical actions or terms. Maintain a clear, neutral-friendly tone. If any section lacks information, insert 'No data provided.' Prioritize scannability and clarity over completeness. Hyperlink any cited sources.

Extending clarity across formats

While this section focuses on text-based outputs, the same principles apply across other forms of AI generation, including images and video. Left to their own devices, image models like DALL-E may produce technically accurate illustrations that lack visual hierarchy or emotional pull. Similarly, video models like Sora or Runway can generate scenes that are coherent at the frame level but feel disjointed, awkwardly paced, or narratively empty when viewed as a whole.

Tip

Across all formats, model outputs serve as raw starting points. Good outputs, whether textual, visual, or cinematic, require the same layering of design elements: guiding composition, tone, rhythm, and structure. Just as AI-written text must be styled for scannability and clarity, AI-generated images must be shaped for focus and storytelling, and AI-generated videos must be molded around pacing, transitions, and emotional arc. Across all media, usability is not a default; it is an outcome of thoughtful design.

Balancing structure and flexibility

As mentioned earlier, more information is sent to the model than what the user might input. This is also true of the styling information that controls the output. All are sent in a packaged format to the model, and must be parsed, meaning leaner, clearer output styling will result in less computational effort being expended.

Keep in mind that over-styling AI outputs can run the risk of diminishing returns. A 2024 study found that excessively rigid output templates can suppress useful model flexibility, increase hallucination rates when inputs are messy, and erode trust by making content feel mechanical or unnaturally scripted. When outputs sound too perfect, users start wondering if they’re reading something made for humans, or something made for compliance checklists. It turns out LLMs, like designers, like some room for creativity.

Designing for Verifiability

In 2023, Attorney Steven Schwartz sought to lighten his workload, unknowingly in the process upending a routine personal injury lawsuit and damaging his own career. In searching for relevant case law, the lawyer asked ChatGPT to assist in drafting a legal brief. The tool rapidly produced citations to six court cases that included plausible legal reasoning and structure, but had just one downside: they were completely fictitious. As you learned in Chapter 4, models have a tendency to hallucinate; Schwartz, none the wiser, incorporated these hallucinated citations into the brief without checking their authenticity.

The issue came to light during discovery, when the opposing legal team reported their inability to locate the cited cases. Upon investigation, presiding U.S. District Judge P. Kevin Castel found that the submissions contained “bogus judicial decisions with bogus quotes and bogus internal citations.” He described parts of the content as “gibberish” and “nonsensical”, eventually fining Schwartz $5,000.

This incident emphasizes that, unlike traditional software, AI systems don’t provide deterministic outputs. That means the same input can yield different results, and the logic behind a given response isn’t always visible or stable. As a result, users may not know whether to trust what they’re seeing, since errors or hallucinations might be very convincing. To help users navigate this uncertainty, AI interfaces should be optimized to give users the opportunity to verify possibly hallucinated content.

AI overreliance

AI overreliance occurs when users depend too heavily on AI systems, accepting incorrect recommendations and diminishing their own critical thinking abilities. As AI tools become more sophisticated and widespread, this phenomenon creates interesting challenges for interface design.

The problem manifests in several ways. Skill atrophy develops when consistent AI dependence weakens underlying human abilities; like GPS reducing spatial awareness or writing assistants diminishing composition skills. Automation bias leads users to favor AI suggestions over their own expertise, even when recommendations seem questionable or contextually inappropriate. Meanwhile, AI systems often miss nuanced cultural, ethical, or situational factors that humans naturally consider, yet users frequently trust outputs without adequate verification.

Microsoft’s AETHER research (“Overreliance on AI,” Microsoft Research), analyzing 60+ studies, reveals that overreliance stems from psychological mechanisms including confirmation bias (users trust AI that agrees with them) and ordering effects (early positive AI interactions create lasting over-trust). Surprisingly, detailed explanations can paradoxically increase overreliance rather than improve judgment.

The research suggests that human-AI teams often perform worse than either humans or AI working alone when overreliance occurs. Interface design approaches that seem to help include cognitive forcing functions that interrupt automatic acceptance, transparent communication about AI capabilities, progressive disclosure that builds realistic mental models, and real-time feedback on decision quality. The challenge lies in fostering appropriate reliance rather than blind trust, helping users become effective AI collaborators rather than passive consumers.

Many have asked - how can we solve these pesky hallucinations? One often-mentioned solution proposes a design pattern that asks a model to evaluate how correct an answer may be, often termed a confidence indicator. This, however, is also problematic: models don’t just hallucinate - they’re also bad estimators of their own confidence. This isn’t totally their own fault: think of the difference between asking Are you confident this volcano will erupt next year? versus Are you confident this is a good essay? versus Is the sky blue? Each question requires a different kind of reasoning: probabilistic modeling, subjective judgment, or factual recall. Shoehorning these different queries into the same datapoint seems likely to lead to misunderstandings.

At a technical level, most LLMs don’t produce a direct confidence score in the way people assume. Their confidence is often estimated by the probability of the next token in a sequence, which is an artifact of training on massive amounts of text data. But high token probability doesn’t mean the model is correct, it just means it’s likely to output that token next given everything it’s seen before. That means relying on confidence indicators alone may be a flawed interface strategy. As noted in the controversial paper “On the Dangers of Stochastic Parrots” (Emily M. Bender et al, 2021), models are trained to imitate linguistic patterns, not to reason or verify.

This lack of verification is partially an issue because we’re used to validating and interrogating data with percentages - however, it’s important to remember that not all percentages are created equal. Consider a simple spellchecker: it flags 7 of 100 words as potentially incorrect, and tells you it’s 93% confident the rest are fine. That’s a percentage you can understand and verify. The algorithm is legible: it checks each word against a dictionary and tallies the results. If you wanted to, you could replicate the process yourself.

Now contrast that with a percentage spat out by a language model in response to a query. In this case a 93% confidence isn’t based on a count of errors or an understandable rule. It’s the output of billions of parameters operating in latent space, shaped by a training corpus that no one user - or engineer - has ever seen in full. Rather than tallying known words or frequencies, the model estimates which token is most likely to come next: it’s estimating next-token likelihoods across a multidimensional probability distribution. In other words, it feels precise while being fundamentally uninspectable.

The confidence percentages look similar on the surface, which can be misleading. But the underlying logic behind each is fundamentally different. Rather than surfacing abstract confidence scores, interfaces should ask: Can the user independently validate this information? If not, then the system needs to either route the user toward verification (through citations, evidence, or a human handoff) or avoid presenting the output as authoritative in the first place (using hedging language like this seems to be true or it’s difficult to verify). The best use case for AI today is for decisions you can quickly verify, not ones that require in-depth research or expertise to validate. AI is better for subjective tasks - copyediting, rewriting, brainstorming - where usefulness, not factual correctness, is the benchmark. These are the domains where confidence is irrelevant, or at least not primary.

But when outputs contain actionable or objective components, such as summarizing legal policy, recommending medical treatments, or guiding financial decisions, the stakes are higher. These use cases need interface mechanisms that support accountability. If the system is nudging a user toward real-world action, it should surface secondary signals such as traceable sources, side-by-side comparisons, or explicit opportunities for human review. Confidence becomes a collective performance - something that must be earned, not assumed.

Warning

If the output contains objective information and the user will take action on or with it, then it needs a secondary (human) verification layer.

In Figure 5-2, Perplexity AI attaches sources directly to individual claims. Each superscript link opens the underlying article, giving the user immediate access to origin material.

Figure 5-2. Perplexity’s source-based UI

As you’ll see later in the chapter, opening sources can help a user validate, but it can also help them in their wider user journey. The user may want to verify, yes, but also to branch out from the answer, to read more deeply, cite in their own work, or explore related materials. The action isn’t just check the source,. it’s jump off from here.

Human-in-the-loop

Human-in-the-loop (HITL) represents a design approach where people remain actively involved in AI decision-making processes rather than being replaced by automation. It can work in multiple ways: in some cases, humans provide initial training data and ongoing feedback to improve AI performance; in others, AI handles routine decisions while escalating edge cases to human reviewers.

For example, a HITL service might begin with thousands of customer support tickets and seek to route them to appropriate specialists. To do that, AI categorizes tickets by topic and urgency while human agents review edge cases and train the system on new issue types. The benefit of this approach is that it handles volume efficiently while maintaining quality, despite the fact that it requires ongoing human training time.

A more sophisticated version might begin with mental health conversations and seek to provide supportive responses to users in distress. To do that, AI selects from a curated library of therapeutic phrases that licensed therapists have pre-approved. This high risk area requires a high quality of human oversight, and the benefit in this case is that the model is capable of offering immediate support using clinically sound language without generating it itself, lowering the chances of a damaging or hallucinated output.

It’s becoming clear that this works well: research from Lai et al. (2023) examining human-AI collaboration across multiple domains found that well-designed HITL systems often outperform fully automated approaches, particularly in domains requiring creativity, ethical reasoning, or handling of edge cases. As this area expands, the key will be designing for effective, low-risk collaboration that balances efficiency with a clear-eyed awareness of potential harms.

Designing Grounding

In the summer of 2024, my partner and I attended the wedding of two friends in Las Vegas, Nevada. The night before the wedding, we were invited to gather at a casino bar - ostensibly to toast to the couples’ good health the night before the celebrations were to take place. A few hours into the evening, much to our surprise, an Elvis impersonator appeared, performing a series of classic hits, dressed (as you might expect) in a full-body bedazzled suit. But then things took a turn: he invited the couple to exchange rings, and went on to invite them to sign their legal papers. He addressed the increasingly confused crowd: “This really is real, folks.” He declared the couple legally married, and the festivities continued into the night, as we and our nonplussed friends celebrated this unexpected turn of events.

As we later came to understand, this had not been planned in advance. In the run-up to the big day we’d heard rumblings of an issue with the wedding planner; in the end, we learned that as the wedding planner hailed from New York, she was not natively familiar with the matrimonial laws in the state of Nevada. When the couple asked her if they could be married by a friend instead of an ordained minister, the planner consulted ChatGPT, which told her that they could. Unfortunately, it appears ChatGPT misled her by mistakenly checking the laws of the state of California, not Nevada: an error that was only discovered a few days before the wedding - hence, getting married by Elvis.

While this sweet example has a happy ending, it underlines the importance of grounding: the information being cited was correct for California, but not for Nevada. Grounding is concerned with how output declares its context, and though it’s similar to verifiability, the latter focuses more on whether the claim is checkable, rather than why the system made it. Grounding can include which entities were referenced, but also what location, timeframe, or version the model had in mind. In short, grounding reveals the lens through which the model generated its response. In the case of our friends, grounding the output of the query would have let the wedding planner know their information was relevant to CA, not NV, thus saving our friend an embarrassing mistake. Figure 5-3 shows how Lexus Nexus +AI solves the problem of legal jurisdiction by allowing users to nominate their preference clearly (and persistently).

Figure 5-3. Lexis Nexus +AI stipulates clearly what jurisdiction the user is working in

In many AI systems, that lens is hidden. A user might ask for hiring guidelines and receive a confident-sounding summary without knowing whether it was pulled from federal law, internal HR policy, or a generic web source. Grounding solves this by designing contextual signals to accompany the output. These signals help the user understand how to read the answer, and whether it meets their expectations:

Model name and version: The specific model used (for example, ChatGPT-4o vs o3) can shape tone, scope, and reliability.

Agent or tool responsible: In multi-agent systems, it’s useful to know which component generated the response. This helps users make sense of the logic behind the answer, and know where to direct questions or edits.

Geographic or locale grounding: AI models may tailor phrasing, tone, or examples based on regional settings. Misaligned locale assumptions can lead to errors - like citing California law when the user is in Nevada.

Task or mode grounding: If an input was applied in a specific mode (for example, creative vs. analytical, draft vs. polish), grounding the output in the mode helps align user expectations and avoids mismatched tone.

Interaction history or session context: Did the AI reference something the user said earlier? When responses are tailored to assumed intent, making that assumption visible (Because you’re working on Q2 planning…) helps prevent confusion and allows for correction.

One proven design pattern for gathering and surfacing this metadata is the Why we suggested this affordance. Typically shown inline, on hover, or behind an info icon, this lightweight explainer gives users a peek behind the curtain without disrupting the main experience. It might include what data or files the model consulted, whether the output was influenced by user preferences, history, or prior queries, which model or agent generated the response, and so on.

This type of optional explanation provides users with a starting point for understanding why a response was generated and the additional option to reconfigure. I discuss adjustability a little later, but displaying grounding sources (eg. wikipedia.com) provides a user with a great starting point when seeking to tune or refine an output.

This Summary is AI-Generated

As AI systems become more capable of producing humanlike responses, the need to disclose their non-human nature becomes a foundational design consideration. Some jurisdictions have already enshrined disclosure requirements in law: California’s Bot Disclosure Law (SB-1001), enacted in 2019, requires that bots be clearly identified as such when interacting with users in commercial or political contexts (specifically, when the interaction is intended to influence a purchase or a vote). It’s vital that users know they are not speaking with a human before they engage meaningfully. Figure 5-4 shows the Alltrails app disclosing that a summary was created by AI.

Figure 5-4. Alltrails AI Disclosure, 2025

Similarly, the EU AI Act (passed in 2024) introduces mandatory transparency for any system that interacts with users or generates synthetic content. The law requires AI systems to reveal that users are communicating with a machine except when it is obvious from the context. This clause places additional weight on design: if your interface mimics human interaction too well, disclosure cannot be skipped.

The U.S. Federal Trade Commission has issued guidance suggesting that deceptive failure to disclose AI use in consumer-facing tools may violate truth-in-advertising laws, especially when it affects decision-making. In light of this, many companies have written internal policies around AI disclosure and compliance that encourage best practices: Google’s People & AI Research Group (PAIR) emphasize the importance of considering all groups when designing disclosures, suggesting products “institute the necessary safeguards and explanations so user groups who are particularly vulnerable to misunderstanding AI responses as humans can discern outputs that are generated by AI.” Designers can advocate for ethical, transparent interfaces by integrating disclosure early in the interaction, framing it as a tool for user understanding rather than a warning, and ensuring that users always know when AI is shaping their experience.

Designing for Actionability

If the output is the product, the action taken on it represents the real user value. While the primary output might be a block of text, a chart, a code snippet, or a recommendation, its real utility often lies in what users can do with it next. Forward actions are the verbs that surround the noun: save, buy, book, export, send, cite, navigate, representing the bridge between comprehension and application. They also reflect the model’s assumptions about the user’s goals, giving them concrete ways to act on the output.

These forward actions enable the user to pursue deeper inquiry, act on recommendations, or evaluate their choices. These interactions can take many forms, but their purpose is to support the user in their onward journey, intrinsic or extrinsic to the product itself. Consider a user asking an AI What’s a good gift for my dad’s birthday? A compelling output might list a few curated ideas, but the usefulness hinges on what the user can do next. Can they click to buy it? Save it to a shortlist? Compare prices or delivery times?

Similarly, you can imagine a query that revolved around a location hinging on accurately predicting the user intent and providing that affordance. If the user asks Where can I find a quiet café near the Met Museum? the result might contain embedded maps or lists. But the real power lies in layered interactions: Can the user get walking directions? Add the café to an itinerary? Call and make a reservation? Designing these secondary actions well requires understanding not just the user’s immediate question, but the downstream actions they might reasonably want to take, making the output a staging area for what comes next. Figure 5-5 shows an interface in Chat GPT that allows a user to view output responses (here, recommended hotels) charted on a map, with options to review as a list, scroll through more options, view more detailed information like images, rating and descriptions, and continue with forward actions like getting directions or visiting the hotel website.

Figure 5-5. Map options in ChatGPT

Users expect forward actions to follow the same principles as any interaction pattern they’re familiar with, like a web search or adding an event to their calendar: I can act from here; I can modify it before I act; I can pick and revert between several paths; I know what will happen if I click this, I know how to get back. Fulfilling the user need means staying as close to these principles as possible, while also acknowledging the differences that AI layers on top of experiences. Giving multiple options communicates choice to the user, as shown in the Gmail example in Figure 5-6.

Figure 5-6. A panel of suggested email responses in Gmail

When providing onward options, it’s important to make clear when an action will have final impact, like a purchase or sending an email, versus when the user will have an opportunity to review beforehand. This mirrors best practices from computer programming - using version control to checkpoint stable code in case there’s a need to revert experimental changes later. Since precision can be an issue for LLMs in general, users may lean heavily on the ability to review before executing on a task, requiring additional clarity from the UI (drafting vs sending). Additionally, users should be notified when they are clicking a link or using a product that will take them to a different domain, along with implied returnability (Saved to drafts) and potential acknowledgement in the UI, such as a visual indication that a link has been clicked before (much like a used hyperlink changing color).

Using a canvas

In AI interface design, a canvas is an interactive, often editable, workspace where users can view, manipulate, and build upon AI-generated content. It’s distinct from a chat interface or static output pane because it provides spatial freedom, persistent context, and often multi-modal interaction (text, code, visuals, and so on). Key traits of a canvas include the following:

Editable and composable: Users can revise AI output directly, like rewriting a paragraph, tweaking a chart, or dragging in new components.

Spatial layout: Unlike chat (which is linear), a canvas supports spatial organization, allowing users to group, rearrange, or layer content.

Persistent state: Content stays visible and editable. It’s not buried in conversation history or lost after a response.

Multi-modal Integration: Often supports a mix of text, images, code, data, or UI blocks, sometimes with tool integrations or plugins.

Moving an output to the canvas

A canvas provides a few things that move beyond a regular output: shareability, storability, interactivity, discoverability. Though it doesn’t necessarily mark the move from exploration to execution, it does differentiate the output from machine-generated content to human-guided, with the user having a higher level of ongoing agency. A canvas can house multiple types of interaction: editing, organizing, tagging, branching, comparing, and shifts the tempo of work, promoting return visits, rethinking, or reuse. This is also the phase at which an AI-generated output can become a shared artifact that can be collaborated on with other users.

Context can affect the canvas in many cases: since the entity arose from an output, it’s possible for the user to go back to the conversation and iterate, even after publishing or sharing with other users, or rename or recontextualize. This requires the interface to handle state clearly, helping the user understand which iteration they’re working with. This is handled neatly by Claude, as seen in Figure 5-7, which displays versions and allows users to select which one they’re working on - and to publish, and to save specific versions as favorites or to a specific project or workspace.

Figure 5-7. Claude allows users to choose between different versions of the canvas artifact

For generative outputs, the input-computation-output model can be much quicker. For instance, a user asks the model to automatically recognize a figure in a video or photo to remove it. In this case, as with an LLM, the canvas is a recurrent space of co-creation with the model, where the user and model work together to refine the entity. Here state management is even more crucial, necessitating the ability to undo and redo step-by-step. Preservability in this case means that the user feels that they can follow the breadcrumbs all the way back to where they started, iterating at each step as they go, something that is presently explored in node-based AI interfaces like Runway, as shown in Figure 5-8.

Figure 5-8. , Runway’s node-based experimental interface

Designing for Adjustability

Imagine a user asks an AI assistant to Draft a follow-up email thanking the client, and summarizing next steps, and an LLM generates a response that’s too formal. The user could easily delete the response and retype the prompt with more specifics. But more often than not, they’ll choose to refine what they have. Why? Because iteration preserves what’s already working, reduces effort, and helps maintain continuity. Restarting, on the other hand, often feels like throwing away progress.

This instinct to adjust rather than start over is supported by the structure of many AI interfaces. They’re designed to let users steer or revise outputs without needing to re-prompt from scratch. And in session-based tools, that preference is reinforced by the context window: the ever-growing conversation log or document history that the model ingests when generating its next response. Context helps the AI stay on track with tone, intent, and prior instructions, but it also means that a restart may lose useful signals established earlier in the exchange. As seen in Figure 5-8, Notion handles adjustability elegantly with an inline AI adjustable UI that appears on text select.

Figure 5-9. Notion inline refinements

To support this iterative approach, some AI products offer versions: alternative outputs based on the same input. Others expose controls like temperature settings, a high-level parameter controlling the LLM’s output that determines how conservative or creative the response should be when it comes to predicting the next token. A lower setting will yield more predictable outputs, while a higher one invites more variety. Although temperature merely affects the way the model picks from its candidates of next-in-sequence tokens, it gives the impression of affecting its creativity. These tools give users a way to explore multiple takes on the same request before committing to an edit.

For example, when a user selects a specific portion of text and clicks Elaborate, the system re-engages the underlying language model using a modified version of the original instruction, a technique called prompt augmentation. The system alters the user’s original prompt by appending a directive such as elaborate, add more detail, or expand this section. The updated prompt is then used to regenerate the content, producing a version that is hopefully more detailed than the original. The AI then regenerates just that section, rather than the entire document, ensuring the expansion remains aligned with the surrounding context.

Many of these systems also support iterative refinement, allowing users to click Elaborate multiple times. With each click, the system reprocesses the content, building on the prior output to gradually increase its length and depth. This stepwise expansion gives users a sense of control over pacing and tone, especially in collaborative or assistive writing contexts.

To achieve this, users must first be aware of the tools available to them and that they are allowed to make a selection. The tuning available to them is most useful when it’s a style of refinement, letting the user enhance their existing output.

Designing for Multi-turn Outputs

Increasingly, users engage in multi-turn workflows: editing a document over time, solving a complex problem in steps, or running a semi-autonomous process that unfolds over minutes or even hours. In these cases, the AI interaction is not a one-shot process. The agent becomes a collaborator, producing recurrent outputs, streams of content shaped by evolving context, memory, and user direction.

Designing interfaces for these outputs requires a shift in mental model. As discussed in Chapter 3, the output participates in an ongoing state, contributing to continuity over time. Instead of thinking in terms of inputs and outputs, designers must consider sessions, threads, or even workflows, extended arcs of activity that may be paused, resumed, branched, or redirected.

This change introduces new design tensions. Users want flexibility without chaos, progress without fragility. They want to test an idea without committing to it. In turn, designers must support continuity across multiple interactions, explicit and implicit memory cues, and ways to fork, revise, or abandon without penalty. In agentic systems it’s not just about generating content. It’s about staying on task. The interface must scaffold not just the current output but the entire process around it. Outputs become milestones in a long-form dialogue that needs to be auditable, interruptible, and steerable.

When outputs are iterative, the rhythm of interaction matters as much as the content itself. Instead of a single static reply, the user sees a sequence: a plan, a suggestion, a revision, a refinement. Sometimes these outputs are explicitly chained (Let’s continue where we left off), and sometimes they are nested inside a larger context (Build the homepage layout now that we’ve defined the brand voice).

Good design for multi-turn outputs requires continuity and clarity, making it easy to identify what has changed, what remains pending, and what action can be taken next. This can be achieved through the following methods:

Turn markers or visual separators that distinguish new steps or phases

Inline edits with change annotations, to show how one step builds on or modifies the last

Rolling side panels or timelines that show the structure of an ongoing task

Named threads or tasks that can be returned to later

Importantly, users must always understand the relationship between the current output and what came before. If they ask to revise the second paragraph, it should be clear what the second paragraph was, how it’s being changed, and what else may be impacted. If something will be changed, the user must be given an opportunity to give permission, whether for one turn or many. This is especially vital in interfaces where generative AI is used to co-author or co-edit a long-form document, or to iteratively build a plan or design.

The goal is to let the user trace the thread of their interaction, without making the interface feel like an IDE or project-management tool.

Watermarking and Detection

Validating the value or provenance of an item is an age-old issue. Centuries ago, manufacturers would embed identifying marks directly into paper stamps called watermarks, to make sure potential buyers knew they could trust the origin of their product. Later, the technique expanded beyond paper to prove the authenticity of wines, artwork, and luxury goods. In the digital era, watermarking evolved to protect intellectual property. For example, film studios routinely embed invisible markers in screener copies sent to critics and industry professionals. When a movie leaks online before release, these watermarks allow studios to trace the leak back to the specific recipient.

Digital watermarking serves a dual purpose: it identifies the source of content and can deter misuse by making unauthorized distribution traceable. Digital watermarks often function as visible deterrents (like the Getty Images logo overlaid on stock photos) or as invisible forensic tools (like the unique identifiers embedded in digital camera files). The challenge with AI-generated content is that we’re trying to mark content at the moment of creation, often in conversational or creative contexts where the watermark itself could interfere with the user experience.

Understanding AI Misuse

Before diving into watermarking solutions, it’s worth considering what I mean by misuse in the context of AI-generated content. Misuse doesn’t necessarily imply malicious intent. A student who uses AI to help brainstorm essay ideas isn’t necessarily trying to cheat, but submitting that AI-generated content as original work without disclosure crosses ethical boundaries. Similarly, a journalist who uses AI to help research a story operates differently from one who publishes AI-generated articles without fact-checking or disclosure.

The spectrum of potential misuse ranges from innocent lack of awareness about disclosure norms to deliberate deception for financial or political gain. Some misuse scenarios involve people who don’t realize the ethical implications of their AI usage, while others involve bad actors deliberately exploiting AI’s capabilities to deceive, manipulate, or defraud others. Watermarking systems need to account for this full spectrum, providing tools that help establish norms around transparency while also offering forensic capabilities for investigating more serious violations.

The challenge is that the same AI capabilities that make these tools useful for legitimate purposes—generating convincing text, creating realistic images, synthesizing natural-sounding speech—also make them effective for deceptive purposes. This dual-use nature means that watermarking serves not just as a technical safeguard but as a way to maintain social trust in digital communication.

Risks of GenAI Content

AI-generated content presents unique risks that traditional watermarking wasn’t designed to address. Unlike a leaked movie or copied photograph, AI content is created on-demand and can be generated at massive scale with minimal human oversight. Risks include the following:

Scale is a huge issue. AI can generate thousands of convincing news articles, blog posts, or social media updates in minutes. These aren’t just random spam but contextually relevant content that can be tailored to specific audiences or current events. Without some way to identify AI involvement, fact-checkers and readers face an increasingly difficult task distinguishing synthetic content from authentic journalism or commentary.

Academic settings are particularly affected, since students can now generate entire essays, lab reports, or research proposals that match the expected style and complexity of their coursework. Researchers might use AI to generate literature reviews or data interpretations without acknowledging this assistance. The concern isn’t just about cheating but about whether academic institutions can maintain standards of original thinking and proper attribution when AI involvement is invisible.

AI-generated contracts, financial reports, legal briefs, or regulatory filings could be used to deceive clients, investors, or courts about the level of human expertise and review involved. The stakes are particularly high in contexts where professional liability or fiduciary responsibility matters.

Creative rights could (and have) be eroded as AI creates increasingly sophisticated music, writing, visual art, and other creative works, raising questions of authorship and compensation. Human creators may find their work competing with AI-generated content that doesn’t clearly identify its synthetic origins, potentially affecting their livelihood and the broader creative economy.

AI-generated personas on social media, in customer service interactions, or in online relationships can manipulate people who assume they’re interacting with humans. The emotional and psychological impact can be significant when people discover they’ve been deceived about the nature of their interactions.

These risks present multiple ethical and legal issues that cannot all be solved by interface design alone, but it’s interesting to consider how visual and encoded signifiers could help solve these problems. Text watermarking is one method for clearly stating that a text is synthetic, however it also represents one of the most technically challenging aspects of AI content identification. This is because language has limited redundancy. Unlike images, where you can subtly alter pixel values without changing visual appearance, text modifications often change meaning or become noticeable to readers.

How Watermarking Works

Modern text watermarking typically works through statistical bias during generation. When an AI model decides between possible next words or phrases, a watermarking system can subtly influence these choices to create a detectable pattern. For example, when the model faces decisions between synonyms like quick versus fast, the watermarking system might consistently favor words that contribute to a cryptographic signature. Over the course of a longer text, these biased choices create a statistical fingerprint that detection systems can identify.

The sophistication lies in making these biases subtle enough to preserve natural language flow while creating signatures robust enough to survive editing and paraphrasing. Some systems work at the syntactic level, introducing slight preferences for particular sentence structures or punctuation patterns. Others operate semantically, embedding patterns in how ideas are organized or expressed that remain detectable even if individual words change.

One example of successful text watermarking involves a case between Genius.com and Google, where the lyrics website suspected their content was being scraped and their song transcriptions displayed in search results without permission. To prove this, Genius embedded a hidden message by alternating between straight apostrophes (') and curly apostrophes (’) in specific patterns that spelled RED HANDED in Morse code, as shown in Figure 5-9. When Google’s search results displayed the same apostrophe pattern, it provided clear evidence of unauthorized use. This approach worked because the watermark was invisible to casual readers but created an undeniable forensic signature, proving Google had indeed used their lyrics without permission - however, in federal court a judge found Genius lacked standing due to the fact that they didn’t own the copyright to the lyrics themselves, and the case was dismissed.

Figure 5-10. Genius.com used Unicode to subtly encode a watermark into their lyrics [link]

Image and Video Watermarking

Image watermarking offers more flexibility than text because visual data contains natural redundancy. Explicit watermarks, like the semi-transparent Getty Images logo overlaid on stock photos, serve as visible deterrents that make unauthorized use obvious. These watermarks protect intellectual property by making it clear when someone uses copyrighted images without permission or proper licensing.

Invisible watermarking uses more sophisticated techniques that embed information without affecting visual appearance. One common approach involves modifying specific pixel values according to a predetermined pattern. For example, a system might slightly adjust the blue channel of every seventh pixel, or embed data in the least significant bits of pixel values. These changes are imperceptible to human vision but create a mathematical signature that specialized software can detect. Another method is stored metadata, like the C2PA standard created by Adobe in 2019, which embeds multiple manifests with associated metadata and is employed by ChatGPT for its DALL-E model.

Video watermarking extends these concepts to moving images but faces additional complexity from compression and temporal changes. Interestingly, Chinese law now requires AI-generated video content to “prominently mark” synthetic content, specifically to combat deepfakes and other misleading generative media. This legal requirement reflects growing recognition that generated video content poses particular and in some cases extreme risks for misinformation and fraud, especially as the technology becomes more accessible and convincing.

Detection Systems and the Future of Watermarking

Due to the manifold risks and widespread public concern, AI detection systems have become a popular solution, in an attempt to identify synthetic content by analyzing statistical patterns that distinguish AI-generated text from human writing. These systems look for telltale signs like unusual word frequency distributions, predictable sentence structures, or specific stylistic patterns that current AI models tend to produce.

However, detection accuracy remains a complex issue. In 2023, OpenAI released a classifier designed to identify AI-written text, positioning it as a helpful tool for educators and content moderators. However a few months later they were forced to shut down the tool due to its low rate of accuracy - the tool correctly identified AI-generated text only 26% of the time while incorrectly flagging human-written content as AI-generated 9% of the time, with additional caveats like being unable to run on short text or on languages other than English.

The failures weren’t random but showed concerning bias patterns. The tool performed particularly poorly on text written by non-native English speakers, whose writing patterns sometimes resembled AI-generated content more closely than native speaker text. This highlighted how detection systems can inadvertently discriminate against certain populations. OpenAI also noted that the tool could be easily circumvented with minor editing, making it unreliable for any adversarial context where someone actively wanted to avoid detection.

Perhaps most tellingly, detection tools have been caught flagging clearly human-authored historical texts as AI-generated. The US Declaration of Independence, written in 1776, has been identified as potentially AI-generated by multiple detection systems. This false positive occurs because the document’s formal language and parallel structure happen to match patterns that some AI models favor. Similar false positives have been reported for Shakespeare, the Bible, and other historical texts, demonstrating how statistical approaches can mistake stylistic consistency for artificial generation.

The current state of AI watermarking and detection resembles the early days of email spam filtering: the technology works reasonably well in controlled environments but struggles with the complexity and adversarial nature of real-world deployment. Just as spam filtering evolved into a multi-layered approach combining technical filters, user education, and social norms, AI content identification will likely require similar complexity.

Rather than seeking perfect technical solutions, the path forward probably involves building transparency and accountability into AI systems themselves, developing social norms around disclosure, and creating educational resources that help people understand AI capabilities and limitations. Watermarking represents one tool in this broader toolkit, but success will ultimately depend on how well these technical capabilities can be integrated with human judgment and social systems that promote responsible AI use.

Managing Problematic Outputs

No matter how carefully a generative model is trained, it will occasionally say something biased, misleading, or harmful. In practice, it’s not a question of whether this will happen, but how frequently, and in what ways. This makes the task of managing problematic outputs a foundational challenge for anyone working on AI interfaces. The design implications range from handling individual outputs (like a refusal or a correction) to shaping the entire system’s behavior across different levels of risk.

Most AI development teams now rely on a layered approach to reduce harmful outputs. This typically begins during data collection and continues through model training, evaluation, and product integration. At the training stage, models are often tuned with feedback from human annotators. One common method is reinforcement learning from human feedback (RLHF), which ranks possible responses and teaches the model to prefer the safer or more appropriate one. Anthropic’s alternative, Constitutional AI, instead teaches the model to follow a set of guiding principles. These might include commitments to human rights, avoidance of harm, or domain-specific ethical rules. Both techniques aim to shift model behavior away from harmful responses, while still allowing for useful and flexible output.

To identify risky behaviors before they reach users, most labs now rely on something called red teaming, a cybersecurity practice in which teams simulate worst-case users, trying to provoke the model into breaking, bypassing safeguards, or revealing sensitive information. Red teaming prompts are crafted to find edge cases: examples that test the limits of the model’s behavior or the gaps in its filters. It’s often repeated in multiple rounds, with updated models and new hypotheses, after which findings are used to retrain or fine-tune the model, update content filters, or revise the intended use cases. In high-risk domains, the outcome of a red team exercise might be to withhold a feature from launch entirely. Some teams now automate red teaming by using one model to generate attack prompts for another, broadening the search space and revealing failures that might be too subtle or creative for humans to think of on their own.

Alongside technical work, there has been a shift toward more formal documentation of model risks. These are often published as safety or system cards, and they describe how a model was trained, what known issues exist, and what mitigations have been put in place. They also help set expectations for developers and users. These cards provide context for when and how failures are likely to occur, what to watch for, and what types of use the model is not intended to support. A card might explicitly note that the model is not suitable for medical advice or legal decision-making, even if it seems capable of producing fluent answers in those domains. For example, Anthropic’s Claude 2 system card explicitly describes the potential for the model to generate biased or offensive language, to produce inaccurate or fabricated information, and to respond inconsistently to harmful prompts.

Problematic and offensive outputs are a predictable part of working with generative systems. While much of the effort around harmful or misleading outputs happens at the model or policy level, interface design plays a crucial role in how users perceive, interpret, and recover from them. Designers can implement refusals that feel conversational, not punitive. When a model declines to answer, the refusal should be embedded naturally in the system’s voice and tone. A short explanation I can’t help with that) followed by a gentle rationale (That could be harmful) avoids sounding scolding or cryptic. This kind of refusal works best when it feels like part of the interaction, not a system-level rejection. The goal is not just to block unsafe content, but to maintain the rhythm of the exchange, explain the issue clearly, and signal that the conversation can continue, as shown in Figure 5-11.

Figure 5-11. Deepmind’s Sparrow, a model trained to reduce the risk of unsafe and inappropriate answers

Because of the infinite number of possible outcomes, designing for large data outputs can be incredibly challenging. When creating any feature, designers are looking to build on the content visible to the user, but in most cases the content is the one thing we don’t have certainty about. Imagine trying to create designs to optimize for Google search results, for example (see Figure 5-12) by assembling search types into broad themes. Visualizing the output is necessary to understand what issues the user is encountering: to understand where this tool features in their journey to accomplish a task, and what issues they may be forced to navigate. In a situation like this, it’s more important than ever to gain a clear knowledge of who your user is, what problems they are likely to be trying to solve, and especially with AI, their common mitigation routes - it’s not helpful to expect users to design totally new ways of solving a problem with AI by default when their current method, if inefficient, presently works as intended.

Figure 5-12. Google’s top search categories, broken down into user journeys

Many product designers will already be familiar with the idea of personas: fake users, formed into generalized buckets, hopefully informed by data and research. Enhancing personas into fully fleshed-out user journeys is another common UX pattern. Designers working with AI outputs may find an additional step even more helpful: crafting staple scripts that correlate to these personas and user journeys, and that map closely to the user’s real-life experiences. For example, if your product specializes in a certain type of solution, by looking closely at user data and extrapolating common tasks, it’s possible to create a number of scripts that capture the user’s theoretical journey. Here are some tips:

Start with grounded examples: Even when you’re working with fakes. It’s easy to invent polished or idealized outputs, but you’ll get more useful design insights if your examples reflect real model behavior. Use logs, transcripts, or personal experiments with current tools as a starting point. It’s also helpful to anchor outputs in user goals—writing from the perspective of a specific persona or use case can surface more thoughtful prompts and help shape what the response should feel like. If the system is likely to get something wrong, or hedge, or ask a question instead of answering, include that too. The goal isn’t to mimic the model perfectly, but to stay close enough that your designs hold up when real outputs are eventually wired in.

Mix different tones, formats, and levels of certainty: One of the defining characteristics of AI outputs is that they vary—sometimes dramatically. A response might be confident and declarative, or it might be cautious and indirect. Some answers come as structured lists or bullet points; others are loose paragraphs or conversational suggestions. When you’re scripting responses, include a few variations. Try writing the same output at different confidence levels, or in alternate tones, or with a different structural layout. This helps test whether your UI can support that variability, and whether users can still make sense of it when it shifts.

Don’t forget to show when things go a little sideways: Real AI outputs aren’t always helpful, and some of the most important design work happens in those edge cases. It’s worth scripting at least one example where the AI misinterprets a prompt, produces something subtly wrong, or refuses to respond. These moments help you test fallback states, repair mechanisms, and trust cues. They’re also a good reminder that you’re designing a conversation, not a search result—the interface needs to support exploration, correction, and even moments of confusion.

In an ideal world, designers could flow AI model responses into their designs rapidly - however, this is frequently extremely challenging. The rise of vibe coding, designer engineers and enhanced prototyping tools all bring us closer to this reality, but in the meantime it’s effective to approximate model outputs with this type of script approach - though designers should seek to collaborate with engineers to ensure the script output is relatively realistic depending on the model they’re working with.

Summary

At the start of this chapter, we looked at clinical decision support systems as early examples of how computing sought to augment human judgment. These tools didn’t replace doctors; they surfaced possibilities, flagged risks, narrowed differentials. And yet, these systems often failed when their outputs were misunderstood - even if the logic behind them was sound. Some clinicians dismissed them entirely. Others deferred to them too much. The real failure was not in computation, but in communication.

Today’s AI systems operate at a vastly more sophisticated scale, but the underlying design challenge remains the same. An AI output offers a proposal: an opening for interpretation shared between system and user. It carries assumptions, probabilities, constraints, and opportunities, but it must always be contextualized, qualified, and, when needed, reshaped.

Whether the output is a sentence, a chart, a draft, a route, or a summary, the question remains: how will the user interpret it? Will they know where it came from? Will they feel invited to change it? Will they understand what part of it reflects AI reasoning, and what part reflects their own? When outputs become indistinguishable from authored content, or when systems remove friction so completely that users no longer ask how or why something appeared, the risk isn’t just overreliance: it’s misplaced confidence. As with CDSS tools, the interface becomes the risk surface: not in the sense that the results of the model are inherently problematized, but the way its output is presented offers a surface to validate - or question - the outcome.

To design AI outputs well is to design for interpretation. It means signaling what’s fixed and what’s flexible. It means giving users tools to inspect, edit, discard, or deepen what they see. It means showing enough of the system’s reasoning to support trust, but not so much as to overwhelm. Above all, it means remembering that The output isn’t the final product. What matters is what happens next: how the user interacts, decides, or applies the result.

This raises a deeper question: how do users make sense of AI outputs over time? A single response may be clear and interpretable, but in multi-turn conversations, that clarity must extend across turns and different journeys, adapting to context, user intent, and remembered information. In the next chapter, we’ll explore how memory shapes the AI-user relationship: how it can support trust, enable personalization, and introduce new risks.

About the Author

Louise Macfadyen is a seasoned product designer specializing in AI at Microsoft, with over eight years of experience crafting innovative user experiences and strategic design frameworks. In her work on Microsoftâs Fluent AI team she specializes in finding and translating technological limitations in order to develop human-centered design solutions. Her approach focuses on creating accessible, effective, and user-centric solutions that empower designers to build intuitive interfaces and collaborate seamlessly with engineering teams. In Designing AI Interfaces, she shares practical insights for working with large language models, guiding designers through the rapidly evolving world of AI-driven product development.