This guide is for CTOs, AI leads and engineering managers whose agent works in a demo and breaks on the fifth step in production. It is the design companion to our generative AI development service and to the AI automation and agent development service, where these patterns are built and tested.
In this guide
- Why do agents fail on multi-step tasks when single calls work?
- Which patterns prevent which failures?
- Where should agent memory live?
- How to build tool use and memory in the right order
- When should an agent have no memory at all?
- Where AI fits
- What delivery record exists, and what does not?
- What are the alternatives to a tool-using agent?
- What are the limits of this guide?
- Frequently asked questions
- Next step
Why do agents fail on multi-step tasks when single calls work?
A single model call fails once; a multi-step run compounds small failure rates across every step, tool call and retry. Most breakage in practice is not the model "being wrong" but the plumbing around it: a tool with an ambiguous description, a timeout that triggers a duplicate write, or a context window that has filled with stale tool output. A voice agent adds a hard clock to every step.
Three failure families cover most incidents. Tool failures: the agent picks the wrong tool, invents a parameter or misreads a result. State failures: a restart loses progress, or a retry repeats a side effect such as a charge or a second message. Memory failures: the agent forgets an earlier decision, or recalls an old fact with full confidence after it has changed.
Long context does not solve the memory family on its own. Research on long inputs found that models use information at the start and end of a context far better than information in the middle, and Anthropic's engineering team describes the same "context rot" as the token count grows. More context is not more memory; it is more noise unless something curates it.
Which patterns prevent which failures?
Each pattern below prevents one failure and brings in another; read the third column before adopting it.
-
Strict tool contract (typed schema, clear name, validated arguments)
- Prevents
- Invented parameters, wrong tool choice
- Introduces
- Rejected calls the agent must recover from
- Use it when
- Always; it is the base layer
-
Few, consolidated tools with focused results
- Prevents
- Wrong tool among near-duplicates, context flooded by raw output
- Introduces
- Bigger tools that are harder to test in isolation
- Use it when
- The tool list has grown past what one prompt explains clearly
-
Actionable tool errors
- Prevents
- Blind retries of the same bad call
- Introduces
- Error text the agent may over-trust
- Use it when
- Tools fail for reasons the agent can fix, such as a bad filter
-
Idempotency key on every write
- Prevents
- Duplicate charges, records or messages on retry
- Introduces
- A key store with its own expiry rules
- Use it when
- Any tool that changes a system outside the agent
-
Step and retry budget
- Prevents
- Loops that burn time and spend
- Introduces
- Runs stopped before they finish
- Use it when
- Every production agent
-
Checkpointed run state outside the model
- Prevents
- Lost progress on restart or approval pause
- Introduces
- A state schema to version and migrate
- Use it when
- Runs that last longer than one request or wait for a person
-
Working-memory notes with compaction
- Prevents
- Context rot on long runs
- Introduces
- Summaries that drop a detail that later matters
- Use it when
- Runs with many steps or long tool outputs
-
Long-term memory store with a forgetting rule
- Prevents
- Re-asking known facts across sessions
- Introduces
- Stale or wrong facts recalled with confidence
- Use it when
- Users return and preferences or facts truly persist
Tool-use patterns in practice
Anthropic's guidance on writing tools for agents makes the same point from the tool side: more tools do not always lead to better outcomes, tool names should be namespaced so overlapping tools are distinguishable, and results should return only high-signal information. One order-search tool with a filter beats three near-identical lookups.
Tool permissions belong in the contract too. The OWASP entry on excessive agency names three root causes, excessive functionality, excessive permissions and excessive autonomy, and recommends that authorisation be enforced in the downstream system rather than left to the model's judgement. A read tool should connect with a read-only identity, whatever the prompt says.
Idempotency is the pattern teams skip most often. Payment APIs such as Stripe accept an idempotency key so a retried request after a connection error does not create a second object; an agent's write tools need the same property, with the key derived from the run and the step, not generated fresh on each attempt.
Where should agent memory live?
Agent memory is three different things with different owners and lifetimes. Mixing them in one vector store is a common memory design error.
| Criterion | Run state (checkpoint) | Working memory (notes) | Long-term memory (store) |
|---|---|---|---|
| What it holds | Current step, tool calls made, pending approval, idempotency keys | Decisions and findings of this run, compacted | User preferences, confirmed facts, lessons across runs |
| Lifetime | One run or thread | One run, summarised as it grows | Across sessions, until a forgetting rule removes it |
| Owner | The orchestration code | The agent, inside rules the code sets | A named data owner, like any business record |
| Main failure | Unversioned state breaks on a code change | A summary drops a detail | A stale fact is recalled as current |
| Forgetting rule | Delete or archive when the run closes | Rewritten at each compaction | Expiry date, source link and a correction path |
LangGraph's persistence model shows the split clearly: checkpointers save a thread's state for fault tolerance and resumption, while a separate store holds application data across threads. Whichever framework or plain code a project uses, keep the boundary: run state is infrastructure, long-term memory is data with an owner.
How to build tool use and memory in the right order
-
Write the tool contracts first
Name, description, typed arguments, permission scope and the error each tool returns. Test each tool without the model.
-
Make every write idempotent
Derive the key from the run and step identifiers so a resumed or retried step cannot repeat a side effect.
-
Add a step budget and a retry budget
Stop the run and hand it to a person when either is exhausted; our guide to AI agent observability and cost control covers the ceilings.
-
Checkpoint run state outside the process
Key it by a run identifier so a restart or an approval pause resumes instead of replaying; our guide to human-in-the-loop controls covers where those pauses belong.
-
Add working-memory notes only when runs get long
Compact on a schedule and keep decisions, open questions and identifiers; drop raw tool output.
-
Add long-term memory last, and only with an owner
Every stored fact gets a source, a date and an expiry; retrieval quality matters as much here as in any retrieval-augmented system.
-
Test the failure modes, not only the happy path
Kill the process mid-run, time out a tool, feed a stale memory and check the agent recovers; an evaluation harness is where these cases live.
When should an agent have no memory at all?
Many business agents should keep only run state. If each request is independent, such as classifying an email or drafting a reply from a ticket, long-term memory adds a data store, a privacy question and a stale-fact risk without improving the answer. Retrieval from a governed knowledge base, refreshed by its owner, is usually safer than letting the agent write its own memories.
Anything an agent stores about a person is personal data with retention and deletion duties; decide those before the first write.
Where AI fits
AI does the reasoning between tool calls, drafts working-memory summaries and can grade test runs. Engineers still write the tool contracts, set permissions and budgets, design the state schema, choose what may be remembered and review every failure-mode test. Netbase works with commercial and open-weight models from OpenAI, Anthropic (Claude), Google (Gemini) and Meta (Llama), among others, chosen per project. Each item below states how mature it is at Netbase.
- In delivered work
WhatsApp AI chatbot with CRM integration
A chatbot for a client that is not named, using OpenAI language models with Dialogflow or Amazon Lex for intents and synchronised with the client's CRM.
- In delivered work
RAG knowledge assistant
A retrieval-augmented knowledge assistant for a client that is not named, the governed alternative to agent-written memory.
- Available capability
Multi-step agents with tool contracts and tiered memory
The pattern catalogue and build order on this page; not yet tied to a published case.
What delivery record exists, and what does not?
- What exists. Netbase has delivered retrieval-augmented knowledge assistants, document AI and MLOps pipelines for clients that are not named, and built a WhatsApp AI chatbot that combined OpenAI language models with an intent engine and a CRM sync, also for a client that is not named. The named delivered AI feature is 4over4's recommendation engine, which suggests products from browsing history and previous purchases. Most Netbase projects are delivered on fixed-price contracts.
- What does not. No published Netbase case reports a multi-step agent's completion rate, retry rate, memory accuracy or cost per run, and none is offered here. The catalogue, matrix and build order are method guidance drawn from the cited sources and engineering practice.
What are the alternatives to a tool-using agent?
- A fixed workflow with one model call per step. Choose it when the steps are known in advance; code owns the order and the model fills one field at a time. Our comparison of AI agents and workflow automation sets out the criteria.
- A single call with structured output. Choose it when the job is one transformation, such as extracting fields, and needs no tools.
- A framework's built-in checkpointer and store. Choose it when the team already runs that framework and accepts its state format; keep the run-state and long-term-memory boundary anyway.
- Retrieval over a governed knowledge base instead of agent memory. Choose it when the facts belong to the business, not to one user; the enterprise knowledge and RAG service covers that path.
What are the limits of this guide?
- It covers single agents calling tools; coordination between several agents adds handoff and delegation patterns of its own.
- The research findings were measured on public models and tasks; your own failure-mode tests decide which patterns your agent needs.
Plan the next step with a Netbase consultant
Frequently asked questions
A context window is what the model sees in one call; memory is what the system decides to keep, summarise or retrieve between calls. A larger window holds more text but does not decide what matters, and models use the middle of a long context less well than its start and end.
Give every write tool an idempotency key derived from the run and step identifiers, so the receiving system recognises a retry and returns the first result instead of acting twice. Pair it with a retry budget so a failing step stops and goes to a person.
Only when users return, the facts truly persist and a named owner can correct or delete them. For most business tasks, retrieval from a governed knowledge base is safer than letting the agent write its own memories.
The pattern matters more than the framework. Netbase works with the major commercial and open-source AI tools and models, chosen per project, and keeps the same split between run state, working memory and long-term memory whichever orchestration layer a project uses.
Next step
Send us the agent's task, its tool list and a trace of a run that failed, and we will book a solution review to map which patterns are missing and in what order to add them.
Related services and solutions
Generative AI development with evaluation and human review Netbase builds generative AI features for product and operations teams: drafting, summarising, rewriting and content generation inside the software your people and customers already use. Every feature ships with an evaluation set, a human review step where output matters and limits on cost. Generative AI is a capability we are growing, and we label its maturity honestly.
Learn More
AI automation and agents that keep people in charge Netbase provides AI automation and agent development for operations teams that want repetitive, multi-step work done by software while people keep approval over decisions that matter. We combine rule-based automation with AI where it adds value, build in human approval, and measure return against a baseline.
Learn More
Enterprise AI knowledge assistants: RAG engineering with access control Netbase engineers retrieval-augmented generation (RAG) over company knowledge: the ingestion, search, permission checks, cited answers and evaluation that let an internal assistant or a website RAG chatbot answer from your own documents. It is a growth capability with access control designed in from the first prototype, and Netbase has delivered a RAG knowledge assistant for an unnamed client.
Learn More
AI knowledge assistant: answers from company knowledge, with sources and access control An AI knowledge assistant is an internal tool that answers employees' questions from your company's own documents and systems, cites the source behind every answer and shows each person only what they may see. Netbase offers it as a growth capability, designed with retrieval, evaluation and human review from the first pilot, and has delivered one for an unnamed client.
Learn More
Work with Netbase remotely
Every Netbase engagement is delivered remotely, so clients in any country and any industry can work with us; Netbase has worked with clients in 80 countries. Contracts and invoices are issued by Netbase JSC, and payment is made by bank transfer to Netbase JSC's company account or PayPal Business. Talk to the CEO, Huy Nguyen (David), on WhatsApp or by email.
Discuss a project
Netbase JSC helps organizations design, build, modernize, and operate digital products and AI-enabled business systems.project@netbasejsc.com
+84 937 869 689
91 Nguyen Chi Thanh, Dong Da, Hanoi, Vietnam
Get in touch
Tell us what you want to build, modernize, or operate.