Bo Thompson, Founder & CEO The agent stack
Top to bottom follows the guide, one section at a time. Each layer has a code the sections use. The dashed band around the whole stack is security and governance: it applies to every layer, not just one.
Security and governance: the wrapper around the whole stack
Every layer in this guide raises the same questions: who is allowed to act, what it is allowed to touch, and who signs off. Governance is not one more layer in the pile. It is the wrapper around all of them, which is why the agent stack above draws it as a band around every layer.
Getting a first agent working is easy. Keeping it safe enough to leave running is where the expertise goes, and it is often the difference between a pilot that sticks and one that gets switched off after one bad week. None of this has to be finished before you start: begin with one painful task and build the wrapper around it as you go.
The harness is the car's brakes and seatbelts. Governance is the driver's license, the insurance and the rules of the road.
Who and what can act
Every person and every agent signs in as itself. Staff use single sign-on (SSO, one company login), so access ends the day someone leaves. Each agent gets its own credentials, never a shared login or someone's personal account, and only the permissions its task needs, known as least privilege: an agent that reads invoices should not be able to pay them.
Applies to agents, the harness and tools.
What data can go in
Sort data into a few plain tiers, such as public, internal, confidential and personal. Write down what never goes into AI tools: customer personal details (often called PII), passwords and bank details, confidential client files. Decide how long prompts, files and outputs are kept, and know where the data lives: which vendor, which region, and whether it is used to train anyone's model.
Applies to the data layer and contracts.
Human approval gates
Some actions always pause for a named person: moving money, signing or changing contracts, anything that touches safety, and anything a customer will see until it has earned trust. Put the gate in the harness's permissions, not in the instructions. An instruction can be ignored; a permission cannot.
Applies to the human, the harness and System One models.
Seeing what happened
Audit logs record who or what took each action, when, and on whose authority. Traces let you replay a run step by step. Monitoring flags unusual behavior, like an agent suddenly calling a tool far more than usual or reading records it never reads. Cost limits per agent and per run stop the loop, not just report on it afterward.
Applies to evals and observability and the harness.
Testing before changes ship
Every change to a prompt, model version, tool, threshold or contract runs the eval suite before it goes live, and the previous version stays ready to roll back to. This is the learning loop's safety catch: a system that improves itself needs a test it must pass first.
Applies to learning loops and evals and observability.
Outside inputs can't be trusted
Web pages, emails, PDFs and tool results can carry hidden instructions meant to hijack an agent (prompt injection). Treat everything a tool returns as data, never as orders. Contracts check its shape; tight permissions limit the damage if something slips through. Connect only tools and MCP servers you have vetted, from their real publisher.
Vendor and model risk
Read the terms: whether the vendor trains on your data, how long it keeps it, and where it is processed. Ask for security attestations such as SOC 2. Models change under you, so pin versions and re-run evals before moving to a new one. Limit lock-in by keeping your prompts, contracts, evals and data in your own systems, and by preferring open standards like MCP and A2A so a model or vendor can be swapped.
Applies to System One models, agents and the harness.
Ownership
Every agent has a named owner: the person who answers for what it does, reviews its logs, and decides when it changes or retires. Back that with short written guidance covering three things: the approved tools, what never goes in, and who approves new tools.
Applies to the human and every layer.
In practice these controls live in different places: logins with your identity provider, permissions in the harness, logs in observability. An orchestration and governance layer, such as SigmaEra, can bring them into one view.
Agents
An agent is a language model placed in a loop: it reads the situation, decides on an action, takes it through a tool, looks at the result, and repeats until the goal is met or it runs out of budget. A chatbot answers once. An agent keeps going.
A chatbot is a consultant you email. An agent is a contractor you give the keys to.
Goal
What "done" looks like. Vague goals are the most common cause of agents wandering.
Instructions
The system prompt, which sets role, rules and style.
Tools
What additional resources the agent has at its disposal to complete the goal.
Budget
Steps, tokens, time or money before it must stop or ask.
Tools
A tool is a function the model can ask the harness to run: search the web, query a database, send an email, write a file. The model never runs code itself. It emits a structured request with the tool's name and arguments, the harness executes it, and the result comes back as text the model reads.
What a tool needs
A name, a plain-language description (this is how the model decides when to use it), and an input schema listing each argument and its type.
MCP
The Model Context Protocol is the common plug for tools. A service exposes its tools once as an MCP server, and most major AI tools can connect to it, subject to sign-in and admin controls. HubSpot, Salesforce and GitHub all publish official MCP servers. An MCP is really a connection that allows different tools to communicate between themselves without the rigid structure of an API.
Read vs. write
Read is simply reviewing existing content with no direct action, while writing is the ability to create, edit and delete data. Reading is cheap to get wrong. Writing, sending, paying and deleting are not, so those tools usually require approval.
Skills
A packaged set of instructions (and sometimes scripts) that teaches an agent how to do one kind of job well. A tool is a verb; a skill is a playbook.
The harness
The harness is all the software wrapped around the model that turns it into an agent. Two products can use the same model and behave completely differently because their harnesses differ. Claude Code, Codex and OpenClaw are all harnesses.
The model is the engine. The harness is the rest of the car: steering, brakes, dashboard and seatbelts.
The loop
Call the model, run requested tools, feed results back, repeat.
Context management
Deciding what goes into the model's limited context window each turn, and summarizing or dropping the rest. Doing this well is called context engineering.
Permissions
Which tools run automatically, which ask a human first, which are blocked.
Sandbox
An isolated machine or container so mistakes stay contained.
Hooks
Code that runs before or after a tool call to check, log or block it.
Recovery
Retries, timeouts and checkpoints so a long job survives errors.
The data layer
Once several agents and people work on the same job, they need one place where the facts live. Without it, each agent keeps a private copy in its context window, those copies drift, and nobody can tell which is true. The data layer is the shared system of record that agents read from and write to, under rules.
Shared state
The current status of the work: tasks, owners, decisions, open questions. Sometimes called a blackboard, where every agent posts and reads.
Memory
Durable facts that outlast one session: who the customer is, what was decided last week, house preferences.
Provenance
Every fact records who wrote it, when, and from what source, so a human can audit it and an agent can weigh it.
Versioning and conflicts
When two agents write different values, the layer flags the conflict and a rule or a human reconciles it. Nothing is silently overwritten.
Access control
Which agent, tenant or person may see or change which records. Agents get the least access the task needs.
Retrieval
Search over documents and facts (often called RAG) so an agent pulls only what it needs into context.
The human sits on top of this layer, not beside it. People should read the same state the agents see, correct it, and have those corrections win.
Contracts
A contract is an explicit agreement about what crosses a boundary. Natural language is fine for people but fragile between machines, so contracts turn hand-offs into something that can be checked.
The full chain
In a real system there is a contract at every boundary, not just between the agent and its tools. The user speaks plain language; from there on, every hand-off is checked. Click a contract to see what it carries.
Read it as a chain of custody. The model is the first translator: it turns the user's words into a task contract. The agent can only reach the world through tool contracts, and tools can only reach data through data contracts. Because every record carries its source on the way back, the user's final answer can show where each fact came from.
Between agents
Input and output schemas (usually JSON Schema) so bad output is rejected before anyone acts on it. Done criteria so the receiver knows when it has finished and must report failures, not hide them. Protocols: MCP for agent-to-tool, A2A for agent-to-agent hand-off. Both are open standards under the Linux Foundation's Agentic AI Foundation.
For agents
Permissions: allowed tools, data and spend. Policies: rules that hold no matter what it reads mid-task. Escalation: when it must stop and ask. Budget: limits on steps, tokens, time or cost.
Treat a contract like an API version. Change it deliberately, and pin which version each agent was built against. Contracts matter most in parallel work: if every branch returns the same shape, merging is easy.
Orchestration and graphs
Graph engineering is planning a job as a set of related but different tasks that parallel agents run at the same time, then putting the results back together. One agent working through ten subtasks in order is slow. Ten agents each taking one, with a lead that splits the work and a merge step that assembles it, finishes in a fraction of the time. The gain is throughput.
One agent is a single contractor building a house room by room. A graph is a general contractor running the framers, electricians and plumbers at once, then doing the final walk-through.
1. Plan and split (fan-out)
The lead agent breaks the job into branches that can run independently, writes each one a brief, and gives them all the same output contract.
2. Run in parallel
Each worker agent has its own clean context focused on one branch, so it goes deeper and does not get distracted by the others' material.
3. Merge (fan-in)
The lead collects the branch results, reconciles overlaps and contradictions, and synthesizes one answer. This step is serial and grows with the number of branches.
4. Dependencies
Some branches need another's output first. That chain, the critical path, sets the floor on how fast the whole job can finish.
Lab: Prairie Ridge's due diligence on a commercial property
Prairie Ridge Property Co., an illustrative 60-person real estate firm that manages and buys commercial property, is weighing an acquisition. The lead plans for 3 minutes, five research branches run, then the lead merges. Merging costs 2 minutes plus 0.75 per branch it has to reconcile. Change the number of agents and watch the timeline.
Design rules
Split by independence
Good branches share a goal but not each other's work. Split by region, document, competitor or question, not by step in a sequence.
One output contract for every branch
Same fields, same format, so the merge is assembly instead of translation.
Shared brief, separate context
Every worker gets the same goal and rules, but only its own materials.
Write findings to the data layer
Branches post results with provenance, so the merge can see where each claim came from and spot conflicts.
Mind the serial parts
Planning and merging do not parallelize. If they are 20% of the job, no number of agents gets you past 5× faster. This limit is known as Amdahl's law.
Time drops, cost doesn't
Parallel agents finish sooner but the total agent-minutes stay the same or rise. You are buying speed.
Bound every loop
A branch that retries needs an exit, such as passing a check, a retry limit, or escalating.
You will also see "graph" used for the wiring of checks and retries in a workflow, and for knowledge graphs of linked facts in the data layer. Those are related ideas; this primer uses the parallel split-and-merge meaning.
System One models and Jev
The name borrows from psychologist Daniel Kahneman: System 1 is fast, intuitive judgment, System 2 is slow, deliberate reasoning. Large language models, especially reasoning models, behave like System 2. They write their answer word by word, which is slow and costly when all you needed was a yes or no.
A System One model does not write text at all. You hand it some state (text or JSON) plus typed questions, and it returns an answer to each with a calibrated probability. TypeSafe AI coined the category and launched Jev, the first public System One model, on September 15, 2026, backed by a $40M seed round led by DCVC. On October 9 it raised an $870M Series A at a $7.5B valuation, led by a16z. The current version, jev-1.14.0 (October 14), is still in early access.
Choice
Pick one option from a list you supply, with a probability for each. "Which trade does this tenant request need: plumbing, electrical or HVAC?"
Score
An ordered rating. "How urgent is it: can wait, this week, within 24 hours, or now?"
Noul
TypeSafe's name for its yes/no judgment type. "Is something happening right now that makes this an emergency?"
Box
Added as a preview in jev-1.14: returns a single bounding box marking where something is.
Calibration
If Jev says 0.92, it should be right about 92% of the time. That lets code set a threshold: act automatically above it, send to a big model or a human below it.
Worked example (illustrative)
Picture a 2 a.m. message to Prairie Ridge Property Co., an illustrative 60-person real estate firm that manages and buys commercial property: "water's coming through the ceiling." A System One model reads it in a fraction of a second and answers emergency? yes, 0.97 and trade? plumbing, 0.94. The on-call tech is paged before anyone opens the inbox. A vaguer message, like "the thing under the sink is acting weird again," scores lower and goes to a coordinator in the morning. Thresholds follow the stakes: the emergency question pages someone at just 0.30, because a false alarm costs a phone call and a miss costs a flooded unit. The trade question creates a work order on its own only at 0.90 or above.
Lab: set the confidence gate
Each square is a tenant maintenance request Jev has routed to a trade. Above the threshold, the system acts on its own; below it, a person reviews. These sample scores are calibrated, so a 0.80 is wrong about one time in five.
| Generative LLM | System One model (Jev) | |
|---|---|---|
| Output | Free text, parsed afterward | Typed answers with probabilities |
| How it answers | One token at a time | All options scored in parallel in one pass |
| Speed | Seconds | Roughly 70–500 ms, per TypeSafe |
| Price | Input and output tokens | $0.042 per million input tokens, no output metering (checked Oct 29, 2026) |
| Good at | Writing, reasoning, planning, code | Routing, classifying, gating, scoring, monitoring |
| Cannot | Guarantee a clean structured answer | Write anything |
In a graph, Jev is the cheap node at the junctions: it decides which branch a request goes to, whether a branch result is good enough to merge, and whether a step needs a human.
Evals and observability
Agents fail quietly. They finish, sound confident, and are wrong. Two practices catch that.
- Tracing: a record of every model call, tool call, input and output in a run, including each parallel branch, so you can replay exactly what happened.
- Evals: a fixed set of test tasks with known good answers, run every time you change a prompt, model, tool, contract or graph. This is unit testing for agents.
- Graders: code, a person, or another model that scores outputs. A separate grader that did not produce the work is more trustworthy than self-review.
Learning loops
A learning loop is how the system gets better from its own experience instead of staying exactly as good as the day it shipped. Every run produces evidence: traces, outcomes, human corrections, failures. A learning loop turns that evidence into changes and checks them before they go live.
Evals tell you how good the system is today. Learning loops are how it gets better tomorrow.
Inner loop (within a run)
The agent checks its own work and tries again: run the tests, read the error, fix, re-run. A reviewer node critiques a branch result before it is merged. Nothing permanent changes.
Outer loop (across runs)
Patterns from many runs change the system itself: a better prompt, a new skill, a corrected memory, a tuned threshold, a better split, a fine-tuned model.
What can learn, cheapest to costliest
The bars show cost and difficulty of undoing.
Where the signal comes from
- Human corrections: every edit, rejection or override a person makes in the data layer is labeled training data. Capture it on purpose.
- Outcomes: did the work order go to the right trade, did the lead convert, did the code hold up in production.
- Graders: eval scores from code, people or a separate judge model.
Risks and considerations
Ten things that go wrong with agent systems, the layer each one hits, and the usual fix. None is a reason to stay out; each is a reason to design for it from the start.
| Risk | What it looks like | Layer it hits | Mitigation |
|---|---|---|---|
| Wrong or made-up answers | A confident answer that cites a policy, figure or date that does not exist. | AG · DL · CT | Require sources through the data contract, and send low-confidence cases to a person. |
| Prompt injection | A document or email hides text telling the agent to ignore its rules or send data out. | TL · CT | Treat tool output as data, keep permissions tight, and require approval for anything sent out. |
| Data leakage | Confidential files or customer details end up in a vendor's tool, a log, or the wrong person's answer. | DL · HA | Classify data, keep a written "never goes in" list, and set access per agent. |
| Over-permissioned agents | An agent built to read invoices can also pay them or delete records. | HA · TL | Separate credentials per agent, least privilege, and approval on every write action that matters. |
| Runaway cost | A loop retries all night, or a fan-out makes far more calls than planned. | HA · GR | Spend caps per run and per agent that stop the work, plus alerts on spikes. |
| Silent regressions | A prompt tweak or model update quietly breaks something that used to work. | LL · OB | Run evals before every change ships, pin versions, and keep rollback ready. |
| Vendor or model changes | A provider changes its terms or pricing, or retires a model you depend on. | S1 · AG · HA | Pin versions and keep prompts, evals and data in your own systems so you can switch. |
| Adoption and change | Staff don't trust the system, work around it, or drift back to personal AI accounts. | HU | Start with one painful task, teach the concepts, and show the hours saved. |
| Unclear accountability | The agent gets something wrong and nobody knows who owns the fix. | HU · every layer | A named owner per agent, written guidance, and approvals logged by name. |
| No plan for when it goes wrong | An agent misbehaves in live use and nobody knows how to stop it, undo it or who to tell. | HA · OB · HU | A pause switch for every agent, rollback to the last good version, a named person to notify, and a log review before it restarts. |
One request, end to end
A tenant's 2 a.m. maintenance message, handled by a stack that uses every layer. Step through it and watch which layer is working.
A quote request that lands at 4:55 on a Friday at an illustrative 40-rep industrial supplier, handled before the weekend instead of on Monday. Step through it and watch which layer is working.
A superintendent's 6 p.m. voice note from the truck at an illustrative 120-person general contractor, turned into a recovery plan by morning. Step through it and watch which layer is working.
- S1 · JevReads a 2 a.m. tenant message, "water's coming through the ceiling": emergency 0.97, trade plumbing 0.94, urgency "now."
- OR · Lead agentEmergency clears the 0.30 paging gate and trade clears 0.90, so it takes the case instead of leaving it for the morning queue.
- GR + CT · Fan-outSplits into three parallel branches (unit and lease record, on-call vendor roster, building history), each with the same output contract.
- DL · Data layerBranches post findings with provenance. The lead merges them into one work order.
- HA · HarnessDispatching an after-hours vendor is a write action with a call-out fee, so permissions pause and page the on-call manager.
- HU · HumanApproves from a phone. The approval is written back to the data layer.
HumanData layer
- S1 · JevChecks the outgoing text to the tenant against policy before it sends.
- OB · TraceThe whole run, every branch included, is logged and becomes a candidate eval case.
- LL · Learning loopTwo weeks later a "leak" turns out to be AC condensation. The case joins the eval set, and the emergency question gets a sharper line after the change passes evals.
- S1 · JevReads the web form, "need pricing on 400 rolls of stretch wrap by month end": quote request 0.93, ready to buy yes, 0.88, buying timeline "this month" on a scale from "no date" to "this week."
- OR + CT · Lead agentQuote requests above 0.90 skip the shared inbox. It writes the task contract: the goal is a reply a rep can send, done when price, stock and lead time each have a source, 15 minutes of budget, read-only tools, and it returns a draft in the quote format.
- GR + CT · Fan-outSplits into three parallel branches (the company and its buyer, price and stock, past orders from similar accounts), each with the same output contract.
- DL · Data layerThe CRM says the buyer has 45 employees and their website says 250. The layer flags the conflict with both sources instead of keeping whichever came last.
- CT + GR · MergeOne branch returns a lead time with no source, so it fails the output contract and reruns once, within its retry limit. The lead merges the three results into one draft reply.
- OB · GraderA separate grader checks that every price in the draft cites its source before it reaches the rep.
- HA · HarnessDrafting runs on its own. Emailing a prospect is a write action, so permissions hold the draft for the account owner instead of sending it.
- HU · HumanThe rep fixes the price-break line, settles the company size, and sends. Both corrections are written back to the data layer.
HumanData layer
- OB + LL · Learning loopThe trace becomes a candidate eval case. A month later the outcome is in: did the lead convert? Reps keep rewriting the same price-break line, so the drafting prompt gains a rule once the change passes evals, and conversion shows whether it helped.
- S1 · JevReads the transcript, "inspector failed the level 3 rough-in, framers can't close walls": topic inspection 0.95, schedule at risk yes, 0.86, urgency "this week."
- OR · Lead agentSchedule risk clears its 0.80 gate, so it plans a recovery brief for the project manager instead of filing the note with the daily log.
- CT · Task contractSets the goal (options to recover the week), done when each option names the trade, the cost and the new date, 20 minutes of budget, read-only tools, and it returns three options in the recovery-plan format.
- GR + AG · Fan-outFour parallel branches: the inspection report, subcontractor availability, change-order cost and the schedule. The schedule branch needs the re-inspection date from the inspection branch first, so that chain is the critical path. All four share one output contract.
- DL + GR · MergeThe scheduling tool still shows Thursday; the city's portal says Friday. The conflict is flagged with sources and a rule makes the city's record win. The lead merges the branches into three options.
- OB · GraderA separate grader checks each date in the options against the city's record before they go out.
- HA · HarnessBooking the re-inspection and moving the framing crew are write actions that cost money, so permissions pause and send the three options to the project manager.
- HU · HumanThe project manager picks option two from a phone. The decision and the new dates are written back to the data layer.
HumanData layer
- OB + LL · Learning loopThe trace becomes a candidate eval case. Failed rough-ins keep coming back to the same building-code item, so it goes into memory, and the recurring pre-inspection check is packaged as a skill once the change passes evals.
Other terms worth knowing
Models and context
- Context window
- The model's working memory for one call, measured in tokens. Everything the agent knows right now must fit in it.
- Context engineering
- Choosing what goes into the context window each turn. Parallel branches help because each gets a clean, focused window.
- Reasoning model
- An LLM that thinks step by step before answering. Better on hard problems, slower and pricier. The System 2 counterpart to Jev.
- Model routing
- Sending each request to the cheapest model that can handle it well.
Running work in parallel
- Fan-out / fan-in
- Splitting work across parallel agents, then gathering their results back into one.
- Critical path
- The longest chain of steps that must happen in order. It sets the fastest possible finish time.
- Amdahl's law
- The speedup from parallel work is capped by the part that cannot be parallelized.
- Checkpoint
- A saved point in a long run so work can resume after a failure instead of restarting.
Safety and control
- Human in the loop
- A required human approval at defined points, usually before irreversible actions.
- Guardrails
- Checks on inputs and outputs that block unsafe or off-policy content.
- Prompt injection
- Instructions hidden in a web page, email or file that try to hijack the agent. The harness should treat tool results as data, never as orders.
- Structured outputs
- Forcing an LLM's answer to match a schema. A partial step toward what System One models do natively.
Knowledge and learning
- Knowledge graph
- Facts and entities linked by typed relationships, often with time and provenance. GraphRAG retrieves by following those links.
- Embedding
- A list of numbers representing meaning, used to find similar text for retrieval.
- Reflection
- An agent critiquing its own output before finalizing. The simplest inner learning loop.
- Reward hacking
- A system learning to raise its score without actually getting better at the goal.
No terms match. Clear the filter to see all of them.
Put the stack to work in your business
Thirty minutes. Your problem. Our honest read on what we would build.
