Keep this card open while you design or review an agent. Go through the workflow one step at a time and use it to decide who does each step: plain code, a decision model, or a generative LLM. Each item is one thing you can check.
A quick reminder of the terms. A decision model returns a typed choice from a set you define (an option, a true/false flag or a score) with a probability or confidence attached. A generative LLM returns text. Plain code returns exactly what you programmed.
1. Diagnose each step
Ask these questions about every step in the workflow, in this order.
- Does this step have to produce text that a person or another model will read? If yes, it belongs to an LLM.
- Does it have one correct answer that a formula, a rule or a lookup can compute? If yes, it belongs to plain code.
- Can you write every acceptable answer down before the step runs (yes/no, one of N tools, one of N tags, one of N legal moves)?
- Can code build that list of options and attach the facts needed to choose between them?
- Does the rest of the workflow wait for this answer?
- Does the step run many times: per request, per command, per page, per document, per frame?
- Can you tell afterwards whether the answer was right?
If the third and fourth answers are yes, the step is a decision. If the fifth and sixth are also yes, it is a strong candidate for a decision model. If the last answer is no, fix that before you change anything else.
2. Routing table
| Task in the workflow | Who does it |
|---|---|
| Math, physics, timing, unit conversion | Plain code |
| Parsing input, building the list of legal options | Plain code |
| Hard safety limits (budgets, rate limits, forbidden paths) | Plain code |
| Picking a tool, skill or command from a known catalogue | Decision model |
| Routing a request to the right handler or model | Decision model |
| Allowing or blocking a proposed agent command | Decision model |
| Pass/fail on an agent run against fixed criteria | Decision model |
| Tagging items from a fixed taxonomy at volume | Decision model |
| Deciding which tool calls in the context are still needed | Decision model |
| Choosing the next move from legal actions in a live loop | Decision model |
| Writing a reply, an email, a summary, a field value in words | LLM |
| Anything with an open-ended answer set | LLM |
A mixed step splits along the same lines. In a browser agent, for example, code turns the page into a table of elements, the decision model picks the action and its target, and a small LLM writes only the text that goes into a field.
3. Design checklist
The option set
- The model chooses only from options that code prepared. It never invents an action.
- Every option on the list is legal and safe to execute as it stands.
- Each option carries the facts needed to compare it with the others (distance, cost, risk, relevance, whatever the domain needs).
- The list includes an explicit "none of these" or "no action" option where that is a valid answer.
- The list stays short. If it grows into hundreds of items, split it into levels (provider, then resource, then command) and choose one level at a time.
Confidence and fallback
- You set a confidence threshold for acting automatically, and you wrote down why you picked that value.
- Below the threshold, the step goes to a named fallback: a human, a stricter rule, or a stronger model.
- For irreversible actions (delete, send, pay, deploy), a low-confidence answer stops the workflow and asks a person.
- Hard limits stay in code regardless of what the model returns.
Speed and cost
- Decisions that fall due at the same moment go out in one batched request.
- In a live loop, the actor that matters most right now decides most often, and background actors decide less often.
- You measured latency and cost per decision on your own traffic, not only from a vendor's benchmark.
Context
- The agent receives only what it chose (one skill, one command's documentation), not the whole catalogue.
- When you trim context, you keep the useful items verbatim instead of rewriting them into a summary that can lose file paths, commands and errors.
4. Validation checklist
- You have your own evaluation set: real cases from your workflow, each with the correct answer written down.
- The set includes edge cases and cases where the right answer is "no action".
- You compared the model's answers with human answers on the same cases.
- You ran the same cases several times and checked how much the answers vary.
- You remember that consistency is not correctness. A judge that returns the same wrong answer every time scores perfectly on consistency.
- For bulk jobs, a person reviewed a sample of the disagreements before the changes reached production.
- You log every decision with its options, its confidence and the final outcome, so you can audit it later.
- You re-run the evaluation set whenever the option list, the prompt or the model version changes.
5. When to keep the LLM or plain code
Skip the decision model when:
- the output is text a person will read,
- the answer set is open-ended or can't be listed ahead of time,
- the step is rare and doesn't block anything,
- a deterministic rule already gets it right every time,
- you have no way to check the answers yet.
The rule behind the whole card: give each step the cheapest thing that can do it correctly. Code does the exact work, a decision model picks from a known list, and an LLM writes.