← Back to portfolio

Designer-builder handbook

Slides →
Updated
October 12, 2026
Latest in AI
  • OCT 2026Anthropic files confidentially for IPO, targeting a valuation that could exceed $60B, would be the first frontier-model lab to go public. Yahoo Finance
  • OCT 2026OpenAI eyeing 2027 IPO following Anthropic's public debut, would reportedly value the company at $500B+. Bloomberg
  • JUN 2026Anthropic ships Fable 5.1 & Mythos tier, new Mythos-class models for trusted-partner Project Glasswing; access briefly suspended for export controls then restored. Anthropic
  • DEC 2025Meta acquires Manus (the autonomous general agent) for a reported $2–3B. Integrating into Meta AI; Manus continues under its own brand. Wikipedia
  • 2026Yann LeCun → AMI Labs, raised roughly $1B for a 12-person team betting the next leap comes from world models, not bigger LLMs. eWeek
Quick view
If you only read one thing, these are the moves.
  • AI is leverage, not a replacement. Every output is a draft until a human signs off.
  • Ask do we even need AI? first. When stakes are high and taste is the deliverable, the honest answer is no.
  • The workflow still has four phases. Agents fill the roles that used to be other people.
  • Agents take defined tasks, plan, execute, deliver. Observability is how you trust what they did.
  • Context is the second brain, memory is the third. Markdown docs are how one truth travels across every tool.
  • Specialists go deep, generalists connect. AI amplifies the pairing, it doesn't replace it.
  • Security starts at the prompt. Assume anything a model sees can leak.
Customize your stack
Swap based on your preferred models.
Claude · GPT-6 · Gemini 3.6 · Llama
Figma AI · v0 · Framer AI · Lovable
Claude Code · Devin · Codex · Jules
Cursor · Windsurf · Zed · VS Code
Claude Research · ChatGPT Deep Research · Perplexity Pro
Manus · Devin · ChatGPT Agent Mode
Lindy · Opal · Zapier Agents · n8n
Wispr Flow · Whisper · Willow
Dovetail · Marvin · Notion AI
Langfuse · LangSmith · Arize Phoenix · Braintrust
Saved to this browser
The arc, idea to execution
The idea-to-execution cycle Five stages arranged in a loop: Idea, Research, Design, Build, Ship. Arrows run clockwise and the loop returns to Idea so each cycle sharpens the next. A center note reads: You decide at every handoff. You decide at every handoff. 01 IDEA what if we built, 02 RESEARCH learn the problem 03 DESIGN shape the solution 04 BUILD make it real 05 SHIP release + learn each cycle sharpens the next
Click a stage for teams & typical docs.
§ 00 · Before you start

Do we even need AI?

Everything in this handbook assumes AI belongs in the project. Sometimes it doesn't. The honest answer to "should we use AI here?" is a quiet no more often than the industry pretends. Earn the yes before you commit.

Ask yourself first, six questions
  1. Is the input or output genuinely fuzzy? If a regex, a formula, or a 50-line script would do it, AI is overkill.
  2. Does it need to be auditable or reproducible? Legal, medical, financial, compliance, deterministic beats smart. Pick deterministic.
  3. Who bears the cost if it's wrong? If a user is harmed, a client is misled, or a decision can't be undone, the bar is higher, often too high for AI alone.
  4. Is this a one-off or does it scale? Agents earn their keep on repetition. A single task you'll never do again? Just do the task.
  5. Would your users rather have control or convenience? Some crafts, journaling, cooking, learning, lose their point when automated.
  6. Is the value in the output, or in the doing? If the process is the product (therapy, teaching, meditation, art), AI short-circuits the thing you're selling.
AI probably isn't the answer when…
  • The task is deterministic, a script or formula does it reliably
  • Outputs must be auditable or legally defensible
  • Errors are irreversible or unsafe (dosing, legal advice, financial trades)
  • Volume is small and quality must be high, just do it by hand
  • Privacy or regulation makes model access impossible without heroic effort
  • The craft is the point, hand-made, slow, intentional output is the value
  • You're adding AI to look modern, not because users asked for it
AI probably is the answer when…
  • Inputs are fuzzy, natural language, images, mixed data
  • Volume is high and per-item judgment is low
  • You need breadth over depth, a competent draft across many surfaces
  • The task is repeatable and your time is the bottleneck
  • You can afford to review drafts rather than need final answers
  • Multi-step coordination that would otherwise be tedious glue work
  • You're building a feature where users explicitly want magic (search, summarize, translate, generate)
The rest of the handbook is for the yes. If you landed on no, don't force it, the best AI product decision this year might be shipping without any.
01
Design frameworks

The four-phase workflow in §02 isn't invented. It stands on 20+ years of design frameworks, the methods below shaped how teams discover, define, design, and deliver. AI didn't replace them. It changed how fast each step runs and what one person can do inside them.

Double Diamond
British Design Council · 2005
The four phasesDiscover → Define → Develop → Deliver. Two diamonds of divergence and convergence.
AI-era shiftDivergence collapses from days to minutes. Convergence stays human, picking which direction to run is still the designer's call.
Design Thinking
IDEO + Stanford d.school · 2000s
The five stagesEmpathize → Define → Ideate → Prototype → Test. Human-centered, iterative, non-linear.
AI-era shiftEmpathy data gets clustered by synthesis agents. Prototypes come from natural language, not hand-drawn frames. The empathy itself still has to come from you.
Lean UX
Jeff Gothelf + Josh Seiden · 2013
The loopThink → Make → Check. Hypothesis-driven, assumption-first, measured against outcomes.
AI-era shiftBuild-measure-learn cycles tighten from weeks to days. Validation agents watch metrics continuously; the hypothesis you're testing can change weekly.
Jobs-to-be-Done
Clayton Christensen · late 1990s
The lensFocus on the job the user "hires" your product for, not the demographic or persona.
AI-era shiftJTBD interviews cluster faster. Agents scan reviews, support tickets, and Reddit threads for implicit jobs at a scale no researcher could do by hand.
Google Design Sprint
Jake Knapp · 2016
The 5-day sprintUnderstand → Diverge → Decide → Prototype → Validate. One week, one problem, one testable artifact.
AI-era shiftA 5-day sprint is now a 2-day sprint. Day 1 research becomes an afternoon. Prototype day happens in hours. The validation call at the end is still a judgment call.
Dual Track Agile
Marty Cagan + Teresa Torres · 2010s
The two tracksDiscovery and delivery run in parallel, not sequentially. Continuous validation, continuous shipping.
AI-era shiftDiscovery agents run continuously in the background. Delivery agents ship on command. The two tracks finally feel like one system.
The through-line
  • None of these frameworks is obsolete. They name the same arc, understand, decide, make, test, in slightly different vocabularies. Pick the one your team already speaks.
  • What's new is who runs each step. The frameworks assume a team of specialists trading artifacts. AI collapses that into one person coordinating a crew (see §03).
  • Diverge wider, converge sharper. AI makes divergence nearly free. The craft moved to convergence, knowing which direction to kill, which to grow.
  • Shorter cycles beat better plans. All these frameworks reward iteration. AI lets you iterate in hours instead of weeks. Use the new budget on more cycles, not longer ones.
02
The workflow

Four phases, cyclical. Human at every handoff. The arc hasn't changed in 20 years, what's changed is who runs each step. Agents cover the roles that used to be filled by other people; you cover the judgment that can't be outsourced. Before you touch Phase I, compose your crew.

0.
Compose your crew
Decide who runs what before you run anything.
For every project, write down which parts you run, which parts a human teammate runs, and which parts an agent runs. Default to agent-first for anything that scales linearly with effort (research, synthesis, draft copy, boilerplate code, release fan-out). Default to human for taste, trust, and the last 20% of craft. The crew is the first design decision, don't skip it.
You
The niche builder
Taste, judgment, final calls, the parts of the craft only you understand because you live in this niche.
Human teammates
Specialists on tap
A fractional engineer, a brand consultant, a legal review, only where expertise or accountability is non-negotiable.
Agents
Your scalable crew
Research, synthesis, design iterations, code implementation, release ops, monitoring, anything whose cost is just more cycles.
The four-phase cyclical workflow Four phases, Research, Ideate, Design, Test & Ship, each with a human checkpoint before handoff. Loop closes back to Research. I. RESEARCH deep-research agent + synthesis II. IDEATE visual design + media gen III. DESIGN design agent + coding agent IV. TEST & SHIP general agent + background agent H REVIEW H CRITIQUE H CRAFT H DECIDE iterate, each cycle sharpens the next
Four phases, cyclical. AI accelerates; the designer decides at every handoff.
03
Automation agents

An agent is an AI that takes multi-step action on your behalf, browsing, running code, reading files, calling APIs, without you at the wheel for every step. Different from a chatbot: you describe the outcome, approve a plan, then review the result. The leverage is real, and so are the failure modes.

Delegating a task to an agent You scope the task and set guardrails; the agent plans, executes with tools and memory, and delivers an artifact; you review and either accept, revise, or stop. Observability runs underneath. A dashed arrow loops back for revisions. YOU AGENT YOU DEFINE TASK scope · guardrails success criteria PLAN break into steps · pick tools EXECUTE call tools · use memory · retry DELIVER artifact + trace of what it did REVIEW accept · revise · stop OBSERVABILITY logs · traces · token & cost spend · errors revise, tighten the brief, run again
You set the task and judge the output; the agent runs the middle. Observability is how you trust what happened.
Autonomy levels, how much leash to give
L0
Copilot
Every step reviewed. AI suggests; you accept, edit, or reject each change. Use for unfamiliar work or high-stakes output. Cursor inline · GitHub Copilot · Figma AI suggestions
L1
Assistant
Approve a plan, review the result. AI breaks down the task, you sign off, it executes end-to-end, you review the finished artifact. Most research and drafting fits here. Claude · ChatGPT with tools · Perplexity Pro
L2
Autonomous
Runs on its own with a budget. Set a scope, token/time cap, and safety rails. Check back when done or when it escalates. Good for research, monitoring, data gathering, multi-step deliverables. Manus · Claude Research · Devin · ChatGPT Deep Research
L3
Background
Always-on, triggered by events. Watches an inbox, a repo, a calendar. Acts on rules. Needs the tightest guardrails, you're not in the loop per task. Lindy · Google Opal workflows · Zapier agents · custom Claude Projects
Research agent
Deep research & synthesis
Give it a question; it searches, reads, and returns a structured report with sources. Runs for 5–30 minutes unattended.
ToolsClaude Research · ChatGPT Deep Research · Perplexity Pro · Gemini Deep Research
General agent
Multi-step deliverables
Describe an outcome, slides, a website, a research report, a spreadsheet, and the agent plans, executes, and delivers a finished file. Runs asynchronously in the cloud; closes with a deliverable, not a draft.
ToolsManus · Devin · ChatGPT Agent Mode · Claude Computer Use
Browsing agent
Screen & web tasks
Clicks through websites and apps on your behalf, gathering references, filling forms, scraping competitive data. Watch the first run.
ToolsClaude in Chrome · ChatGPT Atlas · Comet · browser-use
Coding agent
Implementation & handoff
Takes a design or spec and produces working code. The natural handoff target from your design agent's packaged bundles. Runs in a terminal, a CI pipeline, or inside your editor.
ToolsClaude Code · Devin · Codex · Jules · Google Gemini CLI
AI-native editor
Where you review & refine
The IDE where you read what the coding agent wrote, tab-complete the fiddly bits, and ship. Different category from a coding agent, the editor is yours; the agent runs inside or alongside it.
ToolsCursor · Windsurf · Zed · VS Code with Claude · GitHub Copilot
Desk agent
Files, email, calendar
Handles admin overhead, organizing files, drafting replies, scheduling, triaging messages. Saves hours, needs strict permissions.
ToolsCowork · ChatGPT tasks · Gemini in Workspace
Custom agent
Packaged teammate
A prompt + your docs + specific tools, saved as a reusable agent. Share it with the team so everyone gets consistent output. No-code builders let non-developers ship agents too.
ToolsClaude Projects & Skills · Custom GPTs · Google Opal · Claude agent builder
Multi-agent
Orchestrated teams
One agent plans and delegates to sub-agents, one researches, one writes, one reviews. Powerful, harder to debug when it drifts.
ToolsClaude sub-agents · LangGraph · CrewAI · custom orchestration
Voice agent
Realtime conversational
Talks and listens in real time with sub-500ms latency. For phone support, booking, drive-thru, voice-first UX. New category; infrastructure matured fast in 2026.
ToolsLiveKit Agents · Vapi · Retell · ElevenLabs Agents · OpenAI Realtime API
The loop: how to actually work with an agent
  1. Scope tightly. "Find the five most-used onboarding patterns in fintech apps shipped in 2025" beats "research onboarding." Agents drift proportionally to vagueness.
  2. Set guardrails. Token budget, time limit, allowed tools, allowed sites. Required for L2 and above.
  3. Ask for a plan first. Before execution, have the agent show you its steps. Cheap to catch a bad plan; expensive to catch a bad result.
  4. Let it run. Resist the urge to micromanage. Interrupting mid-task often makes output worse.
  5. Review the artifact, not the trace. Look at what it made. Spot-check sources. Don't get lost reading every tool call.
  6. Iterate on the prompt, not the output. If the result is wrong, the agent will be wrong the same way next time. Fix the instructions in your skills/*.md.

When the off-the-shelf agents don't fit, you can build your own. Same five ingredients regardless of how technical you go: a model, instructions, tools, memory, and an orchestration layer that ties them together.

Anatomy of an agent, five layers
1 · Model
The reasoning engine, Claude, GPT-6, Gemini, Llama. Determines how well your agent plans, reasons, and follows instructions. Match model size to task. Small fast models for routing, large reasoning models for hard problems.
2 · Instructions
The system prompt, who the agent is, what it does, what it never does, the voice it uses. Lives in skills/*.md or equivalent. Iterate here, not in the conversation.
3 · Tools
The agent's hands, web search, code execution, file access, APIs, your own internal endpoints. Use MCP as the protocol so any tool plugs into any agent without rewriting integrations.
4 · Memory
Short-term (the conversation context window) and long-term (a vector DB, a structured store, or a retrieval index). Without memory, agents repeat themselves every session.
5 · Orchestration
The control loop, what to call, in what order, when to stop, when to retry, when to escalate to a human. This is where frameworks earn their keep.
6 · Observability
Traces of every tool call, token count, latency, cost, and failure. Agent failures appear in multi-step causal chains, not at the individual call level, you can't debug what you can't see. Non-negotiable past prototype.
Four patterns that cover most agents
  • ReAct, reason, act, observe, repeat. The agent thinks out loud, picks a tool, sees the result, thinks again. Simplest production-ready pattern. Best for: research, lookups, single-domain tasks.
  • Plan-and-execute, plan first, then run the plan. The agent breaks the task into steps before doing anything, then executes each. Best for: multi-step deliverables (decks, reports, websites).
  • Orchestrator–worker, one boss, many specialists. A planner dispatches subtasks to sub-agents that each have narrower tools. Best for: complex workflows where different steps need different skills.
  • Multi-agent debate, specialists critique each other. Researcher proposes, critic challenges, writer revises. Best for: quality-sensitive output where two perspectives catch what one misses. Token-expensive.
The agentic loop, what actually happens under the hood
Think
Reason about the goal. What's next?
Act
Pick a tool. Call it with arguments.
Observe
Read the tool's output. Did it work?
Repeat
Loop until goal reached or budget hit.
Every agent, no matter the framework or pattern, runs some version of this cycle. The sophistication is in how it decides what to think, which tool to pick, and when to stop.
Example workflow, shipping an "outfit for today" feature to the Digital Closet app. One designer-builder wearing PM, designer, researcher, developer, and marketer hats. Agents don't replace the role; they let one person run the loop the way a team of five would. Here's a week compressed to a day.
1
Research · Monday morning
"What outfit-recommendation patterns do wardrobe apps use well, and what do users complain about?" Deep-research agent reads App Store reviews, scrapes Cladwell & Whering flows, pulls user comments from Reddit's r/femalefashionadvice, returns themed report with sources.
Claude Research · Perplexity Pro
↓ H · You read the report, pick two patterns to prototype
2
Synthesis · Monday afternoon
"Cluster the 40 beta-tester surveys from last month." You dictate the brief by voice while walking the dog; synthesis agent pulls responses from Typeform, clusters by theme, writes a one-pager with verbatim quotes about what beta users actually want from "outfit for today."
Wispr Flow → Claude · Dovetail
↓ H · You validate the clusters against two beta calls on your calendar
3
Design · Tuesday
"Generate three 'outfit for today' screens, one minimal card, one weather-first, one calendar-aware." Your design agent reads the Digital Closet codebase and design tokens on connect, produces three branded prototypes with handoff bundles ready for Claude Code.
Figma AI
↓ H · You critique, pick the weather-first direction, nudge the AI on hierarchy and spacing
4
Build · Wednesday–Thursday
"Implement in the React Native codebase. Wire up the weather API. Open a PR." Coding agent reads the handoff bundle, implements against your conventions, wires OpenWeather, runs the test suite, opens a PR with a before/after video.
Claude Code · Cursor
↓ H · You review the diff, request two changes (match palette, fix tap target), merge
5
Ship · Friday morning
"Release notes, Instagram carousel, email to the 400 beta list, update the help doc." General agent handles the fan-out across tools using your Digital Closet voice profile. You review one draft of each.
Manus · Claude Projects · Canva
↓ H · You approve each deliverable, hit send on the email yourself
6
Monitor · Friday afternoon onward
"Watch 'outfit for today' adoption. Ping me if daily open-rate moves more than 10%, or if any tester posts a bug on IG or TestFlight." Background agent runs on schedule, reads Mixpanel + App Store Connect + social mentions, pings you when the threshold trips.
Lindy · Opal · Zapier Agents
↓ H · You get an alert when the data tells a story worth acting on
Four ways to build, from least to most technical
01
No-code
an afternoon
When: you need a packaged agent for a repeatable workflow, invoice triage, lead enrichment, content QA, onboarding email drafts. No engineers required. Visual canvas, drag connectors, ship.
ToolsGoogle Opal · OpenAI Agent Builder · LangSmith Fleet · Lindy · Zapier Agents · n8n · Claude Projects · Custom GPTs
02
Vendor SDK
days, not weeks
When: you've outgrown no-code, want code-level control, but don't need to mix-and-match models. Opinionated APIs, built-in tracing, tight coupling to one provider's capabilities (streaming, extended thinking, computer use).
ToolsClaude Agent SDK · OpenAI Agents SDK · Google ADK · Vercel AI SDK · LiveKit Agents (voice) · Microsoft Agent Framework
03
Open framework
weeks
When: you need multi-agent orchestration, model-agnostic flexibility, complex state, or strong observability. Frameworks give you patterns (ReAct, plan-and-execute, multi-agent) without writing them from scratch.
ToolsLangGraph (graph runtime) · CrewAI (role-based) · AutoGen / AG2 (conversational) · Pydantic AI (type-safe Python) · LlamaIndex (retrieval-first) · DSPy (programmatic prompts) · Smolagents (lightweight) · Semantic Kernel (.NET) · Mastra (TypeScript) · Agno (open framework)
04
Custom orchestration
months
When: you've hit framework limits, novel patterns, exotic infrastructure, strict latency or compliance requirements. Build the loop yourself: bare-metal model calls, your own tool router, your own memory store. Most teams never need this.
ToolsRaw model APIs · MCP for tools · mem0 / Zep / Letta for memory · Redis / pgvector · your own state machine

Agentic monitoring, also called agent observability, is the production layer that tells you when your agent failed, why, and what to do about it. LLM monitoring watches single calls. Agent monitoring traces entire sessions, tool chains, sub-agent handoffs, and multi-step causal failures. By 2028, Gartner projects 60% of software teams will use AI eval and observability platforms, up from 18% in 2025.

What to watch, the five surfaces
Traces
Every tool call, every sub-agent handoff, every retry, captured as a tree you can replay. The single most important signal. Without full-session traces, you're guessing why an agent failed.
Evaluations
Automated scoring of outputs, relevance, correctness, hallucination detection, tool-call accuracy. Run against datasets of real production traces. Evals catch regressions before shipping.
Cost & latency
Per-run token count, dollar cost, time-to-first-token, total duration. Catches the chatty agent that silently 10×'s your bill overnight.
User feedback
Thumbs-up / thumbs-down, inline comments, flagged outputs. Routes directly into eval datasets to improve the agent in the next iteration.
Drift & alerts
Signals when output quality degrades, when a prompt regresses, when a tool starts failing silently. Required for L3 background agents.
A
Open-source, self-hosted
free, you host
When: you want full control, data stays on your infrastructure, or you're cost-sensitive. The open-source core of these powers production at serious scale.
ToolsLangfuse (28K+ stars) · Arize Phoenix (OpenTelemetry-native) · Helicone (gateway-first) · OpenLLMetry / Traceloop (instrumentation standard)
B
Framework-native
zero friction
When: you're already on a framework and want the lowest-friction observability for it. One API key, done.
ToolsLangSmith (LangChain / LangGraph) · Weights & Biases Weave · Braintrust (OpenAI Agents SDK, Vercel AI SDK, Pydantic AI)
C
Agent-first platforms
best coverage
When: you're shipping production agents and need full-lifecycle coverage, simulation, pre-release evals, production tracing, in-product debugging agents. New category, rapid consolidation.
ToolsMaxim AI · AgentOps · Galileo · Latitude · Keywords AI
D
Enterprise APM
if already standard
When: your org already runs Datadog, New Relic, or Dynatrace, and you want agent traces alongside everything else. Less specialized than agent-first platforms, more consolidated.
ToolsDatadog LLM Observability · New Relic AI Monitoring · Dynatrace
Monitoring rules of thumb
  • Instrument from day one. Adding observability after the fact means you've already shipped bugs you can't diagnose.
  • Use OpenTelemetry / OpenInference for instrumentation, so you can swap platforms without re-wiring. The standard matters more than the vendor.
  • Evaluate on real traces, not synthetic prompts. Pull failing production traces into your eval dataset, that's where the regressions actually live.
  • Alert on drift, not just errors. A silently-degraded agent is more dangerous than one that crashes. Watch quality metrics, not just uptime.
  • Human review in the loop. Even the best eval models miss nuance. Thumbs-up/down from real users is your highest-signal data.
04
Context & memory

Agents are only as smart as what fits in their context window plus what they can remember across sessions. Memory is where generic AI becomes your AI.

1st
Your brain
Taste, judgment, lived context.
The reason the niche is yours. Agents can't replicate it.
2nd
Your notes
Externalized knowledge, docs, decisions, references.
Lives in: Notion, Obsidian, Apple Notes, markdown. Named by Tiago Forte, 2022.
3rd
Your agents
AI that reads your second brain and remembers across sessions.
Lives in: Claude Projects, ChatGPT Memory, mem0, Zep, Letta.
Context engineering, three moves
1 · Pack tight
Only what the task needs. Even with 1M-token windows, attention degrades past ~200K. Trim the prompt; drop irrelevant tool schemas; cache stable prefixes (up to 90% cost cut).
2 · Retrieve, don't dump
RAG beats stuffing. Chunk docs, embed them, retrieve top-k only. Compress long tool outputs, on serious workflows, context compression is its own agent step.
3 · Persist what matters
Decide what survives this session, decisions, preferences, learned patterns, and write it to long-term memory. Everything else dies with the conversation.
Where to store long-term memory
  • Managed services (zero-ops), mem0 · Zep · Letta · Pinecone. Hosted, with decay, dedup, and semantic retrieval built in.
  • Self-hosted vector stores (your infra), pgvector · Weaviate · Qdrant · Chroma · Turbopuffer. Lower cost at scale; data stays on your servers.
  • Model-native memory (provider does it), Claude Projects & Skills · ChatGPT Memory · Gemini personal context. Simplest, least portable.
05
Specialists vs. generalists

AI flattens some work and sharpens the value of the rest. Specialists go deeper, craft, systems thinking, edge cases. Generalists go broader, AI lets one person credibly span research, copy, visual, and prototype. The best teams pair both, with clear handoffs.

Specialist
Deep in a single craft
Principal designer, motion lead, design systems owner, content strategist, UX researcher. Defines quality. Sets the bar others reach for.
Best at
  • Setting standards the AI and team work against
  • Catching what AI gets wrong in their domain
  • Crafting the last 20% AI can't
  • Building reusable systems and skills
Watch-out
  • Tempted to do everything themselves; under-delegates to AI and generalists
Generalist
Broad across phases
Product designer, founding designer, PM with design chops. Ships end-to-end with AI as amplifier. Moves fast across the whole workflow.
Best at
  • Going from problem to prototype solo
  • Orchestrating AI tools across phases
  • Synthesizing between disciplines
  • Speed, coverage, first drafts
Watch-out
  • Ships AI-default work without specialist review; craft ceiling drops
Who owns what, by phase
Phase
Generalist leads
Specialist reviews
AI accelerates
I. Research
Desk research, competitive scan, brief draft
Research method, synthesis rigor, bias check
Summarizing, clustering themes, pattern-matching
II. Ideate
Divergent concepts, moodboards, naming
Direction, brand fit, narrative clarity
Volume of options, visual variations, copy drafts
III. Design
Layouts, flows, mid-fi prototypes
Craft, motion, a11y, system fit, edge cases
Wireframes, boilerplate, repetitive screen variants
IV. Test & Ship
Scripts, synthesis, specs, handoff
Methodology, severity calls, final QA
Drafting, transcribing, summarizing results
Rituals that keep the team aligned
  • Shared skill library. Specialists encode their judgment as skills/*.md prompts. Generalists inherit craft standards automatically.
  • Specialist crit, weekly. Standing 30-min where specialists review AI-assisted work in their domain. Catches drift early.
  • Generalist show-and-tell. Fast demos of what was shipped, what worked, what didn't. Specialists learn breadth; generalists absorb depth.
  • Brand-system custody. Specialists own design-system.md. Generalists propose changes via PR. The design agent reads the canonical file.
  • Co-author the prompt. When AI output misses, specialist and generalist debug the prompt together, not the output. The prompt becomes the shared artifact.
  • Named escalation paths. Before shipping, explicit specialist sign-off for: a11y, motion, content voice, brand fit. No ambiguity about who owns what.
06
Grounding documentation

AI tools are only as good as the context you give them. The same files that guide your team, written as .md, can ground every AI tool in how your brand thinks, writes, and designs. Version-control them next to the work.

brand-voice.md
Voice & tone
How your brand sounds. Pasted into Claude projects or Cursor rules so microcopy doesn't come out generic.
ContainsPrinciples · reading level · words we use · words we avoid · before/after examples
design-principles.md
Design principles
The opinions behind the work. Used to critique AI output and explain why a concept is off-brand.
Contains3–5 principles · what each looks like in practice · what each rejects
design-system.md
Design system
Tokens, components, patterns. Your design agent reads this on onboarding to apply your brand automatically.
ContainsColor tokens · typography · spacing · component usage · do's & don'ts
glossary.md
Terminology
How you name things. Prevents AI from inventing product terms or using competitor language.
ContainsProduct nouns · feature names · acronyms · preferred capitalization · what not to say
skills/*.md
Reusable prompts
Encode team-specific workflows once. "Write a release note in our voice," "critique a design against our principles."
ContainsTask description · inputs expected · output format · examples · edge cases
adr/NNN-*.md
Decision records
Why we chose X over Y. AI drafts the first pass from meeting notes; you edit for nuance and ship.
ContainsContext · options considered · decision · consequences · date
accessibility.md
A11y standards
Your WCAG baseline and house rules. Used to audit AI-generated copy and components.
ContainsTarget level · contrast rules · alt text guidance · focus states · tested assistive tech
handoff/*.md
Dev handoff
Specs, acceptance criteria, edge cases. Pairs with Figma frames for engineering pickup.
ContainsUser story · happy path · edge cases · states · analytics events · open questions
How docs plug into each tool
→
Claude & ChatGPT projects
Upload brand-voice.md, design-principles.md, and glossary.md as project knowledge. Every new chat starts grounded.
→
Design agent
Point it at your repo. It reads design-system.md, your codebase, and design files to build your system. Refine over time.
→
Cursor & Claude Code
Store repo-level rules as .cursorrules or CLAUDE.md. Engineering and design pull from the same source of truth.
→
Skills & custom prompts
Keep skills/*.md files in a shared repo. Anyone can load them into Claude or ChatGPT for consistent team workflows.
→
Version control
All of it lives in git. Change log = decision history. Review docs like code; branch them for experiments.
07
Inside AI

When you send a single message, 11 distinct steps run in sequence, and usually multiple times per turn as the agent loops through tools. Each step has its own data shape, its own latency, and its own failure mode. Understanding them explains why latency spikes, why context fills up, and why tool-heavy turns are expensive.

01
User asks
User
InputAn intent in the user's head
ProcessTypes or speaks a request
OutputRaw text, the prompt
FormatUTF-8 string, typically 1–500 words. Voice input adds an STT (speech-to-text) pass first, adding ~300–800ms latency.
AttachmentsImages, PDFs, files travel alongside as base64 or multipart uploads. These are not yet tokens, they'll be processed separately by vision or document extractors.
Failure modeAmbiguous intent, "make it better" has no anchor in the context, so the model guesses what "it" and "better" mean.
02
Serving layer
Harness
InputThe prompt via API, web, CLI, or mobile
ProcessAuthenticates and routes to the harness core
OutputA structured request object
TransportHTTPS POST with JSON body to an endpoint like /v1/messages. Streaming responses use SSE (server-sent events).
AuthAPI key, OAuth token, or session cookie validated. Rate-limit counters checked, requests above the per-minute or per-day quota return 429.
PayloadModel name, temperature, max tokens, system prompt, conversation history, tool definitions. Attachments referenced by ID or inlined as base64.
03
Orchestration
Harness
InputRequest plus prior turn history
ProcessDecides: reply directly, call a tool, or plan more
OutputA routing plan for this turn
PatternMost agents run a ReAct loop (reason → act → observe → repeat). Others use explicit planners, state machines, or DAGs.
DecisionsWhich model variant to use (cheaper for classification, bigger for reasoning), how many retries, when to spawn sub-agents, when to stop.
StateTurn counter, tool budget, timeout clock. Hard limits here prevent runaway loops, typical cap is ~25 tool calls per turn.
04
Context assembly
Harness
InputSystem prompt, conversation, memories, retrieved docs, tool catalog
ProcessStitches it all into one ordered text block
OutputThe context window, ready for the model
Order mattersTypical layout: system prompt → tools → memories → retrieved context → conversation → current turn. Later content gets stronger attention.
Retrieval (RAG)User query is embedded, a vector DB (Pinecone, pgvector, Turbopuffer) returns nearest neighbors by cosine similarity, top-k chunks are injected.
CompressionPast turns may be summarized to save tokens. Tool schemas can be auto-pruned to only what's relevant this turn.
Size limitsAs of late 2026: 1M tokens for Claude Sonnet 5.5, 400K–2M for GPT-6 and Gemini 3.6. Even with huge windows, attention quality degrades past ~200K, chunking and retrieval still matter.
05
Tokenize
Model
InputThe assembled text
ProcessSplits it into known vocabulary units using BPE or similar
OutputA list of integer token IDs
AlgorithmByte-Pair Encoding (BPE) or SentencePiece. Trained once on a massive text corpus; the vocabulary is fixed at ~50K–200K tokens.
RatioRoughly 1 token ≈ 4 characters or ¾ of a word in English. "hello" = 1 token. "antidisestablishmentarianism" = 6+ tokens. Code, JSON, and other languages tokenize less efficiently.
CostEvery token costs money on input and output. 2026 frontier pricing is roughly $3–$15/MTok input, $15–$75/MTok output, a 200K-token agent run with 10K output easily hits $1–$3. Prompt caching cuts repeat-context cost by up to 90%.
Edge caseUnicode emoji, rare characters, and non-Latin scripts can balloon into many tokens per character.
06
Embed
Latent space
InputToken IDs
ProcessLooks each up in a learned embedding table
OutputA matrix of dense vectors, one per token
DimensionsEach token becomes a vector of ~2,048–12,288 floating-point numbers. This is the model's dimensionality, higher = more expressive, more expensive.
Positional infoPosition embeddings (RoPE, ALiBi, or learned) are added so the model knows which token came first. Pure token IDs have no order.
ShapeInput is now a [sequence_length × hidden_dim] tensor. For a 1,000-token prompt at 8,192 dims, that's 8.2M numbers before any reasoning has happened.
07
Transformer layers
Latent space
InputEmbedding vectors
ProcessDozens of attention + feedforward layers, each refining representations
OutputFinal hidden states, one vector per position
Layer countFrontier models run ~40–120 layers deep. Each layer: multi-head self-attention → feedforward (MLP) → residual + layer norm.
AttentionEvery token attends to every previous token. O(n²) cost in sequence length, why long contexts get quadratically slower without tricks like flash attention, sliding windows, or state-space layers.
What emergesEarly layers track syntax and local patterns. Middle layers cluster entities and relationships. Late layers encode abstract intent, what the response should do.
KV cacheDuring generation, each past token's key/value projections are cached so they don't get recomputed. This is why output tokens stream faster after the first token is produced.
08
Output head & sampling
Model
InputFinal hidden state at the last position
ProcessProjects to vocabulary-sized logits, applies softmax, samples
OutputOne token, repeat until done
LogitsThe output head is a linear layer that maps the final hidden vector to a score for every token in the vocabulary, typically ~128K scores.
Sampling knobsTemperature (0 = deterministic, >1 = wilder), top-p / nucleus (keep the smallest set of tokens summing to p probability), top-k (keep the k most likely).
AutoregressiveThe chosen token is appended to the sequence; steps 6–8 run again for the next token. Generation is sequential, why output speed is measured in tokens/second, not tokens in parallel.
StopGeneration halts on: an end-of-text token, a stop sequence, max token limit, or the model deciding to emit a tool call.
09
Interpret
Harness
InputGenerated text from the model
ProcessParses the output against expected formats
OutputA routing decision, send, execute, or retry
Structured outputModern APIs emit tool calls as typed JSON blocks, not free text. The harness validates against the declared schema before dispatching.
ValidationSchema mismatch → retry with an error message appended. Refusals or safety-filtered content → surface to the user with an explanation.
StreamingIf the response is a plain reply, tokens stream to the UI as they arrive. Tool calls wait for the full JSON block before executing.
10
Tool execution
Harness
InputTool call with arguments
ProcessRuns it, API, file, code execution, web search, MCP server
OutputTool result, folded back into context
SandboxingCode execution runs in isolated containers with no network by default. File I/O is scoped to a working directory. Web fetches go through allow-lists.
MCPModel Context Protocol is an open standard for tool servers. Apps like Gmail, Slack, or databases expose tools that any MCP-compatible agent can call.
CostTool calls add both wall-clock latency (a web search might take 2–5s) and token cost, the result is injected back into context and re-processed.
Failure handlingTools can error, time out, or rate-limit. Errors are returned as text; the model usually retries, picks a different tool, or asks the user.
LoopTool results re-enter at step 04. The context grows each iteration (user turn + tool calls + tool results). A complex agentic turn can loop 5–25 times before reaching step 11.
11
Deliver
User
InputThe final reply text
ProcessStreams it back through the serving layer
OutputThe response shown to the user
StreamingTokens arrive at ~50–200 tokens/sec for frontier models in 2026, up from 20–100 a year ago. Distilled and smaller models hit 500+ tokens/sec. Why you see text appear word-by-word.
Post-processingMarkdown rendered, code blocks syntax-highlighted, citations linked, safety filters applied one last time.
PersistenceTurn is logged for memory, analytics, and eval. Whatever goes into long-term memory gets summarized and stored for future context assembly.
Numbers that shape the experience
First token
Time-to-first-token (TTFT) is usually 200ms–2s. Dominated by context size, a 100K-token prompt takes longer to "prefill" than a 1K one.
Throughput
50–200 tokens/sec of streaming output for 2026 frontier models. Smaller / distilled models hit 500+ tokens/sec. Reasoning models (Claude Extended Thinking, GPT-6 reasoning, Gemini 3.6 Thinking) can pause for seconds-to-minutes before any output.
Context
Modern frontier windows: 200K–2M tokens. But attention isn't uniform, models often weight the start and end of context more than the middle ("lost in the middle").
Pricing
Input tokens are 5–10× cheaper than output tokens. Tool-heavy turns that fold tool output back into context can silently balloon costs.
Caching
Prompt caching lets the server skip re-processing unchanged prefix tokens, up to 90% cost reduction for repeated system prompts or long docs.
Determinism
Even at temperature=0, outputs can vary between runs due to floating-point non-associativity on parallel GPUs. Reproducibility requires seeds and identical hardware.
Fine-tuning
Customizing a model's behavior outside this flow entirely, a separate training run. In-context learning (few-shot examples in the prompt) handles most use cases without it.
Reasoning tokens
Reasoning models (Claude Opus 5.5 Extended Thinking, GPT-6 reasoning, Gemini 3.6 Thinking) spend hidden tokens thinking before answering. These count toward cost but you never see them, monitor them explicitly.
Multimodal
Frontier models in 2026 natively handle text, images, audio, PDF, and video in one context. Images cost roughly ~1,500 tokens each; a 1-minute video can run ~15K tokens. Design for that math.
Agent runs
A long-horizon agent run (Claude Research, Manus, Deep Research) can process 1M–10M tokens across 10–100 tool calls. Observability is required to see what actually happened.
08
World models

A world model is an AI that learns how physical environments behave, gravity, momentum, spatial consistency, cause and effect, and can simulate or generate them. Not a language model, not an agent. An entirely different class of system, and an entirely different design surface. If you work on video, 3D, spatial interfaces, robotics, AR/VR, or game-like experiences, this is where the ground is shifting fastest in 2026.

Language model
Predicts the next token
Trained on text. Generates sequences by predicting what comes next. Strong at reasoning, weak at physics, time, and space.
Known by
  • Claude · GPT-6 · Gemini
  • Writes, plans, codes, reasons
  • Text in, text out, maybe with tools
World model
Predicts the next state
Trained on video, 3D, simulations. Learns environmental dynamics, how objects fall, how light moves, how scenes stay consistent as you navigate them.
Known by
  • Sora 2 · Genie 3 · Runway Gen-4.5 · Marble
  • Generates video, simulates interactive worlds
  • Prompt or control in, dynamic environment out
Generative video
Text or image → video clip
Produces short cinematic clips with increasingly realistic physics. Useful for moodboards, pitch videos, motion studies, ad concepts.
ToolsSora 2 · Runway Gen-4.5 · Google Veo 3 · Pika · Kling
Interactive worlds
Prompt → navigable 3D scene
Generates explorable environments you can move through in real time, with persistent spatial state. The frontier for spatial and game-like UX prototyping.
ToolsGenie 3 · Runway GWM-1 · World Labs Marble · Odyssey · Oasis AI
Physical AI
Simulators for robots & autonomy
Train and test agents in synthetic environments before deploying to real hardware. Relevant if you design interfaces for robotics, autonomous vehicles, or industrial systems.
ToolsNVIDIA Cosmos · Wayve GAIA-2 · Waabi · V-JEPA 2
Spatial / scene
Image → 3D world
Turn a photo or sketch into a walkable 3D scene with geometry and lighting. Early-stage but moving fast. Useful for AR, virtual staging, concept exploration.
ToolsWorld Labs Marble · Luma Dream Machine · Spline AI
What changes for designers
  • New surfaces to design for. Interactive video as UI. AI-generated environments instead of pre-rendered ones. "Play to explore" replaces "scroll to read" in some contexts.
  • Prototypes that move. You can pitch a concept as a short generated video or walkable scene, not just static comps. The proof-of-concept bar is higher.
  • Physics as a design constraint. Earlier video models morphed objects mid-scene. Sora 2 and Gen-4.5 now obey gravity, momentum, collision. You're now designing with a physics engine, not around one.
  • Consistency is the hard part. A world model that forgets the room when you turn around breaks immersion. Spatial memory and long-horizon coherence are the open problems, test for them explicitly.
  • New failure modes. Hallucination in a world model means wrong physics, missing objects, impossible geometry. Checklist a scene the way you'd QA a layout, but for space and time.
  • Most designers don't need this yet. If you ship mostly text-and-pixel UI, world models are peripheral. If you work on video, spatial, robotics, AR/VR, games, or simulation training, the next 12 months will rewrite your toolkit.
09
Applied AI

Four concrete examples of designer-builders using an agentic team to compress what used to take weeks into days. Each shows the chain of agents, the human checkpoints, and the tools behind each step. Not aspirational, these are shipping today.

Example 1, Digital Closet: shipping "outfit for today." One designer-builder wearing PM, designer, researcher, developer, and marketer hats. A week of a traditional team compressed to a few days of solo work.
1
Research · Monday AM
"What outfit-recommendation patterns work in wardrobe apps? What do users complain about?" Deep-research agent reads App Store reviews, scrapes Cladwell & Whering, pulls Reddit threads, returns themed report with sources.
Claude Research · Perplexity Pro
↓ H · Pick two patterns to prototype
2
Synthesis · Monday PM
"Cluster the 40 beta-tester surveys." You dictate the brief by voice; synthesis agent clusters by theme with verbatim quotes about what beta users actually want.
Wispr Flow → Claude · Dovetail
↓ H · Validate clusters against two beta calls
3
Design · Tuesday
"Three 'outfit for today' screens, minimal card, weather-first, calendar-aware." Your design agent reads the Digital Closet codebase + design tokens, produces branded prototypes with handoff bundles.
Figma AI
↓ H · Pick weather-first direction, nudge AI on hierarchy
4
Build · Wed–Thu
"Implement in React Native. Wire the weather API. Open a PR." Coding agent writes against your conventions, wires OpenWeather, runs tests, opens PR with before/after video.
Claude Code · Cursor
↓ H · Review diff, request two changes, merge
5
Ship + monitor · Friday
"Release notes, IG carousel, email to the 400 beta list, update help doc. Then watch adoption." General agent fans out in your voice; background agent watches Mixpanel + App Store + social on schedule.
Manus · Canva · Lindy · Opal
↓ H · Approve each; get pinged when data moves 10%
Example 2, Design researcher synthesizing 50 interviews. What used to be two weeks of transcript tagging becomes a day of validated themes.
1
Transcribe + clean
"Transcribe all 50 recordings, strip filler words, tag speakers." Transcription agent processes files overnight; cleaned transcripts land in Dovetail.
Whisper · Dovetail · Otter
↓ H · Spot-check three for accuracy
2
Cluster + theme
"Find patterns across all 50. Rank by frequency and severity." Synthesis agent clusters with verbatim quotes, produces a theme tree.
Claude · Dovetail AI · Marvin
↓ H · Validate against two real interviews; override AI's severity calls
3
Devil's advocate
"Argue the opposite of each top finding." Second agent pressure-tests to catch confirmation bias before it ships to stakeholders.
Claude (critic agent)
↓ H · Keep what holds up, drop what doesn't
4
Package + share
"Build the readout deck and a one-page exec summary. Email it to the four stakeholders." General agent produces both, drafts the email, you send.
Figma AI · Manus
↓ H · Review, approve, hit send
Example 3, Agency designer onboarding a new client brand. Day one, you have a brand brief, three competitors, and a Figma link. By end of day two, you have a design system scaffold and a critique.
1
Absorb the brand
"Read the brand brief, extract voice, values, color DNA, and typographic preferences." Agent produces a brand-voice.md you can version alongside the project.
Claude · Opal
↓ H · Validate against brand steward
2
Competitive scan
"Audit three competitors' sites. Capture patterns, color systems, typography, UI motifs." Browsing agent navigates each, screenshots, categorizes.
Claude in Chrome · Comet
↓ H · Mark which patterns to borrow or avoid
3
System scaffold
"Draft a starter design system from the voice doc + brand DNA. Tokens, type scale, component list." Your design agent generates the base; you shape it.
Figma AI
↓ H · Opinion on hierarchy, spacing, system choices
4
Critique + handoff
"Run accessibility and brand-consistency audit on the system." Second agent critiques against WCAG and the voice doc; produces a one-page punch list.
Claude · a11y skill
↓ H · Walk the client through, agree on next sprint
Example 4, Design lead rolling out a design-system update across a product team of 20. What used to be a quarter of change-management becomes two weeks of structured rollout.
1
Scope the drift
"Scan the codebase and Figma. List every component that drifted from the system in the last six months." Agent produces a prioritized drift report.
Claude Code · Figma MCP
↓ H · Pick the top 10 to fix
2
Write the migration
"For each drifted component, write the migration spec and a codemod PR." Coding agent generates PRs with before/after visuals; you approve each.
Claude Code
↓ H · Review each PR; batch-merge the safe ones
3
Communicate the change
"Draft the design-system changelog, Slack announcement, Loom script, and FAQ." General agent builds all four; you edit the Loom script and record.
Manus · Claude
↓ H · Record the Loom yourself; it carries your voice
4
Watch adoption
"Alert me when any PR lands using the deprecated components." Background agent monitors GitHub, pings Slack with educational nudges linking the migration guide.
Lindy · GitHub Actions
↓ H · Jump in only when a nudge isn't landing
What these examples share
  • Human at every handoff. Not every tool call, but every meaningful decision.
  • Specialist agents, not general ones. Research, synthesis, design, code, ship, monitor, each a different agent built for its job.
  • Compression, not replacement. The designer-builder role doesn't shrink. The time between brief and shipped shrinks.
  • Monitoring makes it durable. Shipping is step N, not step last. Background agents keep watching once you move on.
10
Security & privacy

AI security is a different animal from traditional app security. Your agents read untrusted content, call tools you authorized, and persist memory you may have forgotten about. The usual perimeter model doesn't hold, the attack surface is inside the context window. Treat this as table stakes, not optional hardening.

Threat · 01
Prompt injection
A web page, email, PDF, or tool output contains hidden instructions the agent reads and follows. The #1 agent vulnerability. Any content from outside your control is adversarial by default. Even "trusted" docs can be edited.
Threat · 02
Data leakage
Confidential work pasted into a public tier gets logged, retained, and potentially used for training. Enterprise tiers (Claude Enterprise, ChatGPT Enterprise, Gemini for Workspace) are the baseline for client, user, or employee data.
Threat · 03
Tool-access abuse
An agent with a credit card, inbox, Git write, or production access is a liability without hard limits. Scope every tool, read-only by default; writes behind explicit approval; max-spend caps; allowlisted domains.
Threat · 04
Credential handling
API keys, OAuth tokens, and secrets must never land in prompts, logs, or memory stores. Use a vault (1Password, Doppler, AWS Secrets Manager, Vercel env). Rotate anything an agent has touched.
Threat · 05
Supply-chain risk
MCP servers, agent frameworks, and third-party skills ship with permissions. Vet every MCP server before you install it. Prefer official or widely-audited servers; check what tool scopes they request.
Threat · 06
Memory poisoning & PII
Long-term memory is a privacy surface. User PII in a vector DB is PII in a vector DB. A poisoned memory entry can bias every future session. Encrypt, scope by user, enable deletion, review what's being written.
Six hardening moves, the designer-builder's security checklist
1 · Enterprise tier
Public models for public thinking; enterprise models for anything else. Enterprise tiers come with zero-retention commitments, SOC 2 Type II, HIPAA eligibility (health), GDPR & data residency, and admin controls. Switch before your first client project, not after.
2 · Least-privilege tools
Every tool starts read-only. Writes, sends, pays, and deletes require explicit per-call approval. Allowlist domains agents can browse. Cap spend. Never give an agent a tool it doesn't need for the task at hand.
3 · Human-in-the-loop on side effects
Anything that leaves the sandbox, an email sent, a payment made, a PR merged, a social post published, needs explicit human approval, not an "autonomous" setting. L3 background agents watch and report; they do not act unsupervised on anything irreversible.
4 · Treat all external content as untrusted
Web pages, emails, PDFs, support tickets, user-generated content, can and do contain prompt injection. Keep agent instructions in a system prompt the user content can't override. Review any auto-generated action that was triggered after an agent read external data.
5 · Observability as safety
You can't catch what you can't see. Full-session traces, cost alerts, and output evals are security controls, not just performance ones. Monitoring platforms (Langfuse, LangSmith, Arize Phoenix) double as the audit trail you'll need when something goes sideways.
6 · Red-team your own agents
Once a quarter, try to break your own agent. Feed it a prompt-injection payload. Ask it to exfiltrate the system prompt. See if it'll call a tool outside scope. Fix what breaks before an actual attacker finds it.
Compliance quick-reference
  • SOC 2 Type II, table stakes for any B2B AI tool. Claude, OpenAI, Google, LangSmith, Langfuse Cloud, Braintrust all carry it. Ask for the report before you integrate.
  • HIPAA, required for health data. Claude Enterprise, OpenAI Enterprise, Google Workspace with Gemini support it under BAA. Not all monitoring platforms do, check before piping health data through.
  • GDPR, required for EU user data. Covers data minimization, right to deletion, data residency. Enterprise tiers from major vendors are compliant; most consumer tiers are not.
  • Data residency, EU-only or US-only data flows. Claude, GPT, and Gemini offer region-pinned deployments; smaller tools often don't. Confirm before client work.
  • ISO 27001, broader security management standard. Langfuse Cloud carries it; LangSmith Cloud doesn't (as of 2026). Matters for enterprise procurement.
  • Model cards & system cards, Anthropic, OpenAI, and Google publish these for frontier models. Read them before integrating a new model, especially for sensitive use cases.

AI bias is a design problem, not just a model problem. Every frontier model inherits bias from its training data, the open web, licensed text, and whatever human preferences were baked in during fine-tuning. By the time an agent runs in your product, that bias has been amplified by your prompts, your tool choices, your UX, and the user's own context. You own the bias your agent exhibits in production, regardless of who trained the model.

Where bias enters, six places to look
1 · Training data
Western-English skew, over-represented voices, under-represented ones. Named demographic gaps in medical, legal, financial, and employment outputs are well-documented. The frontier labs publish bias cards, read them.
2 · Default assumptions
Models default to stereotypical personas when under-specified ("a nurse" → she, "an engineer" → he; "a wedding" → white Western). Specify explicitly or you'll get the default.
3 · Image generation
Visual bias is the loudest kind. CEOs rendered white & male, criminals rendered Black, "beautiful" rendered thin & young. Review every batch; prompt for the diversity you actually want.
4 · Your prompts & skills
The examples in your skills/*.md teach the agent what "good" looks like. If your examples skew one direction, so will your output. Audit your own skills periodically.
5 · Retrieval sources
RAG inherits bias from whatever you indexed. A doc corpus from one era, one geography, one team, that's the lens your agent speaks through. Diversify sources or document the limits.
6 · Evaluation sets
If your eval dataset only has test cases from one demographic, your agent will pass evals and still fail users unlike that demographic. Build eval sets that reflect your actual user range.
Designing against bias, what you can actually do
  • Name the demographic you're designing for, and the ones you're explicitly not excluding. Both go in the brief. Both show up in the eval set.
  • Test with diverse inputs. Not "a user", Nilufar, Keisha, Dmitri, Maya, Jim. Feed real distributional variety into every eval pass.
  • Prompt for specificity. "A nurse" generates default; "a 54-year-old Nigerian-American nurse in Lagos" generates signal. Specify in prompts; let users specify in their own flow.
  • Review image generation batch-by-batch. One output tells you nothing. Twenty outputs show you the model's prior.
  • Add bias checks to your eval set. Automated evals can test for refusal consistency, named-persona treatment, output demographic distribution. Not perfect, but catches regressions.
  • Human-in-the-loop on sensitive outputs. Health, legal, financial, employment, housing, education, any domain where bias has direct harm, needs human review before shipping, period.
  • Document the known limits. If your agent is strongest in English, weakest in Hinglish, say so. If it struggles with African names, say so. Honesty beats pretending.
  • Give users agency. Let them correct the model, pick alternatives, flag outputs. The best bias detection system is a user who speaks up, if your UX lets them.
11
Glossary

The vocabulary of AI moves fast. These are the terms used throughout the handbook, written for designer-builders, plain language, no PhD required.

Agent
An AI that takes multi-step action on your behalf, browsing, calling tools, reading files, rather than just answering a question.
Agentic loop
The think → act → observe → repeat cycle every agent runs. Frameworks and patterns are variations on this cycle.
Autonomy level
How much leash the agent runs on, L0 copilot, L1 assistant, L2 autonomous, L3 background. Match level to risk.
Context window
Everything the model can "see" in a single turn, system prompt, conversation, tool results. 2026 frontier windows: 1M–2M tokens.
Context engineering
Deciding what goes into the context window, what to pack, cache, retrieve, compress, or persist. The 2026 discipline that replaced prompt engineering.
Embedding
A text (or image, audio) chunk turned into a numeric vector, how similarity search and RAG work under the hood.
Eval
Automated scoring of AI output, relevance, correctness, hallucination, tool-call accuracy. Run against datasets of real traces.
Fine-tuning
Customizing a model through additional training, outside the normal prompt flow. Most use cases don't need it, in-context learning is enough.
Grounding
Attaching an AI's output to verifiable sources, your docs, your data, the web, so answers aren't just generated from training memory.
Hallucination
When the model generates something that sounds right but is factually wrong. Common with names, stats, product details.
Human-in-the-loop (HITL)
Explicit human approval at specific checkpoints in an agent workflow. Non-negotiable on irreversible actions.
LLM · foundation model
Large language model trained on massive text corpora, Claude, GPT, Gemini, Llama. The reasoning engine at the center of any agent.
MCP
Model Context Protocol, Anthropic's open standard for connecting tools to agents. The USB-C of agent integrations.
Memory
What the agent retains. Short-term = context window. Working = scratchpad during a task. Long-term = vector DB or structured store across sessions.
Observability
Traces, costs, latency, and evals from production agents. Required past prototype, you can't debug what you can't see.
Prompt · system prompt
Instructions you feed the model. System prompt sets persona and constraints up front; user prompts are what comes after.
Prompt injection
A security vulnerability, external content (a web page, email, PDF) contains hidden instructions that hijack the agent. The #1 agent threat.
RAG
Retrieval-Augmented Generation, chunk documents into a vector DB, retrieve the relevant chunks per query, inject them into context. Precision over volume.
ReAct
The simplest production-ready agent pattern, reason, act, observe, repeat. Named for "Reasoning + Acting."
Reasoning model
Models that think before answering with hidden "reasoning tokens", Claude Extended Thinking, GPT-6 reasoning, Gemini 3.6 Thinking. More accurate on hard tasks, slower & pricier.
Skill
A packaged set of instructions for a specific task, the agent loads it when needed. Claude Skills, OpenAI's "custom instructions."
Streaming
The model delivering tokens as they're generated, instead of all at once. Why you see text appear word-by-word. 50–200 tokens/sec on 2026 frontier.
Token
The unit a model actually sees, roughly ¾ of a word in English. Cost and context limits are measured in tokens, not words.
Tool · tool-call
A function the agent can invoke, web search, code execution, API, file read. The hands of the agent.
Trace
A full-session record of every tool call, model turn, and handoff. The replayable tree an observability platform captures.
Vector DB
A database that stores embeddings and retrieves by similarity. Powers RAG and long-term memory. Pinecone, pgvector, Weaviate, Qdrant, Turbopuffer.
World model
An AI that learns how environments behave, physics, geometry, dynamics, not just how language is sequenced. Sora, Genie, Marble, Gen-4.5.
Red-teaming
Deliberately attacking your own agent to find vulnerabilities, prompt injection payloads, scope-escape attempts, before an actual attacker does.
Get the full handbook

Enter your email to unlock all 11 chapters — frameworks, agents, memory systems, security, and more.