Deterministic software
- Use for
- Rules, validation, database queries, API calls, and predictable application logic.
- Signal
- The same input should reliably produce the same output.
- Avoid
- Adding a model to ordinary branching logic.
Architecture knowledge map
Design the system before you add the agent
Follow the decision tree from problem definition through classification, composition, evaluation, and iteration. The goal is not to avoid agents; it is to use exactly as much agency as the problem needs.
Explore the decision tree
Begin with the six-phase overview, open one topic at a time, then select a concept when you want its full guidance.
Nothing has been removed: search spans every concept, while “All 51” preserves the complete source canvas as a reference.
Overview first · details on demand
Overview
The complete design loop: define, decompose, compose, evaluate, and improve
How to explore: choose a topic, then select a node to reveal its guidance and connections.
6 concepts · 6 connections
Press enter or space to select a node. You can then use the arrow keys to move the node around. Press delete to remove it and escape to cancel.
Press enter or space to select an edge. You can then press delete to remove it or escape to cancel.
Question Pattern Gate Process
Architecture outcomes
What each destination means
A complete product may contain several of these. Classify each responsibility separately instead of assigning one architecture to everything.
Traditional ML
- Use for
- Prediction, ranking, forecasting, scoring, and anomaly detection over structured data.
- Signal
- Success can be measured statistically against historical examples.
- Avoid
- Replacing a measurable prediction core with an opaque agent.
Focused LLM component
- Use for
- Classification, extraction, summarization, rewrite, and bounded semantic judgment.
- Signal
- Meaning matters, but input, output, and evaluation boundaries are clear.
- Avoid
- Giving the model planning, tools, or memory it does not need.
LLM workflow
- Use for
- Prompt chains, routers, parallel workers, DAGs, and evaluator-optimizer loops.
- Signal
- There are multiple steps, but their dependencies are known in advance.
- Avoid
- Letting a model rediscover routing logic already known to the application.
Agentic loop
- Use for
- Dynamic tool choice, context selection, recovery, and action based on runtime evidence.
- Signal
- The correct next step genuinely cannot be known until runtime.
- Avoid
- Unbounded permissions, retries, costs, actions, or stopping behavior.
Classification output
Turn each choice into an engineering record
Reaching an implementation node is not the end of classification. Record the boundary and reasoning so the part can be composed, evaluated, and replaced independently.
For every classified part, record:
- Chosen implementation family
- What stays deterministic
- Why AI or agency is necessary, if it is
- Inputs and outputs
- Representative evaluation examples
- Failure detection and handling
- Dependencies on other system parts
Risk and review
Ask what happens if the step is wrong, expensive, unsafe, irreversible, or difficult to verify.
Validation gate
Use deterministic checks when possible and human approval when cost, safety, privacy, legal, or irreversible consequences are high.
Worked example
Customer refund assistant
The goal is to resolve valid refund requests quickly without approving ineligible or suspicious claims. The system is a known workflow containing several implementation families, not one general-purpose agent.
Dependency shape
intent -> [order lookup | risk score] -> eligibility -> response -> approval -> action
- 01 Bounded LLM component
Understand the request
Classify intent and extract order number, product, reason, and requested action into a schema.
- 02 Deterministic tools
Retrieve the account and order
Known database and API calls should return authoritative records without model judgment.
- 03 Deterministic rules
Check refund eligibility
Dates, product categories, prior refunds, and policy limits can be evaluated explicitly.
- 04 Statistical model
Estimate fraud risk
A score over structured behavioral data can be validated against historical outcomes.
- 05 Bounded LLM component
Draft the explanation
Generate a clear response grounded in the policy decision and retrieved account facts.
- 06 Approval and action gate
Issue the refund
Validate the amount and permissions, require review above a threshold, then call an idempotent refund API.
Evaluate
Policy accuracy, false approvals, resolution rate, human-review rate, latency, and cost.
Fallback
Ask for missing information, decline safely when checks fail, or send uncertain cases to a reviewer.
When an agent helps
Only for exceptional investigations where the next information source or recovery step cannot be known in advance.
Routing and orchestration
Let models classify. Let code route.
When the available categories and their destinations are known, ask the model for a constrained label or schema. Then let deterministic application code select and start the next task.
This removes agent descriptions from the prompt, reduces token cost and ambiguity, and makes routing testable. Keep an orchestrator only for the genuinely exceptional path.
A -> B -> C
Sequential chain
Use when every step requires the previous output. Validate the contract at each boundary.
A -> [B | C | D] -> E
Parallel fan-out
Use when branches are independent until synthesis. It can reduce latency and isolate failures.
A -> B | A -> C -> D
Partial DAG
Use when some, but not all, steps depend on one another. This is common in production systems.
Observe -> Decide -> Act -> Observe
Dynamic coordination
Use only when dependencies emerge at runtime. Bound it with tools, permissions, budgets, and stop conditions.
Operational expansion
Attach these branches after classification
These concerns matter across the system, but answering them too early overloads the opening architecture decision. Attach them once each part has a defined responsibility.
01
Retrieval and context
What information is needed, where does it live, and how should it be selected?
Choose deterministic selection for known context, semantic retrieval for discovery, and dynamic selection only when runtime evidence requires it.
02
Tools and actions
What can the system do, and which actions are safe?
Define permissions, approvals, timeouts, idempotency, rollback paths, and ownership before exposing a tool to an agent.
03
Memory and state
What must persist, and what should never be remembered?
Separate temporary working state from durable memory. Define retention, correction, privacy, and evaluation behavior.
04
Retries and fallbacks
When should the system retry, clarify, fall back, or stop?
Bound attempts and cost. Prefer a safer model, explicit rules, human escalation, or a no-op over uncontrolled repetition.
05
Guardrails and safety
How are unsafe, private, prohibited, or irreversible actions handled?
Apply controls before and after model calls, and require human approval where consequences are high.
06
Observability and operations
What gets logged, inspected, and converted into an evaluation case?
Trace decisions, tool calls, latency, cost, validation failures, recovery paths, and reviewer corrections.
Measurement and evaluation
Improve against examples, not opinions
Evaluation starts before architecture and continues in production. It should test individual components and the complete system, including operational behavior and recovery.
- 01
Representative dataset
Start small but concrete. Include common cases, edge cases, known failures, risky examples, and examples from real usage when possible.
- 02
Expected outputs and rubrics
Use exact labels where possible. For fuzzy work, score correctness, completeness, grounding, format compliance, usefulness, and safety.
- 03
System metrics
Measure task success, error rate by type, regression rate, cost, latency, tool failures, human-review rate, and failure recovery.
- 04
Compare versions
For every change, ask what improved, what regressed, what became cheaper or faster, and what became riskier. Keep a regression set.
- 05
Production feedback
Collect traces, user corrections, reviewer decisions, drift signals, cost and latency trends, and failure reports after launch.
- 06
Improve the weakest step
Clarify the task first, then improve checks, data, prompts and schemas, retrieval and tools, workflows, and finally agent policy.
End-to-end guide
From an idea to a reliable AI system
- 01
Define the problem
Write down the desired outcome, inputs, triggers, expected output or action, stakeholder, value, failure impact, and how failure will be detected and fixed.
- 02
Define measurement
Create a representative dataset, expected outputs or labels, rubrics for fuzzy outputs, and success, cost, latency, and risk metrics before choosing architecture.
- 03
Test the whole level for determinism
Ask whether explicit code can solve the complete system. If yes, build deterministic software. If no, decompose it and repeat the question at smaller levels.
- 04
Decompose into concrete parts
Split the system into loops, subsystems, tasks, or subtasks. Give each an input, output, caller, context requirement, failure mode, and verification method.
- 05
Select one part
Work on one responsibility at a time. This prevents the architecture of the most complex part from being applied unnecessarily to every other part.
- 06
Classify the selected part
Choose among deterministic logic, structured statistical prediction, bounded semantic work, a known LLM workflow, or a dynamic agentic loop. Add review where risk demands it.
- 07
Record the implementation decision
Document what implementation family was chosen, what remains deterministic, why AI or agency is needed, interfaces, evaluation cases, failure handling, and dependencies.
- 08
Compose the agent system
Model dependencies as a sequential chain, parallel fan-out and fan-in, partial DAG, dynamic coordination, fallbacks, and human gates. Define every interface.
- 09
Classify any remaining parts
Return to the next subtask until the complete dependency graph has an intentional implementation choice for every responsibility.
- 10
Evaluate the complete system
Run the composed system against the evaluation set. Measure success, errors, cost, latency, safety, failure detection, and human-review burden end to end.
- 11
Improve and repeat
Find the weakest step, make the lowest-complexity improvement, update the evaluation set, and rerun the loop with evidence.
Risk changes the design
If a wrong result is expensive, unsafe, private, legally significant, or irreversible, add deterministic validation and human approval regardless of which architecture the tree recommends. Autonomy should shrink as consequences grow.