How to Give an AI Agent Persistent Memory
Short answer: to give an AI agent persistent memory, you add a store outside the session and two verbs: write facts as they happen, read them back when relevant. Everything on the market is a variation of that sentence. What you are actually choosing is where the store lives, and there are three real options: in the client app, in your own framework, or in the connection layer between agent and tools.
If you are still diagnosing the problem rather than fixing it, start with [why your AI agent forgets everything](/blog/why-your-ai-agent-forgets) and come back for the architecture decision.
The three places memory can live
In the client app. Assistants increasingly ship built-in memory: the app extracts facts from your chats and injects them into future sessions. You do nothing, which is the appeal and the limitation. You barely control what is kept, and the memory is welded to that one app. Switch from the desktop app to the IDE and your agent is a stranger again.
In your framework. If you build agents in code, you wire in a memory service or roll your own store. Mem0, Zep, and Letta are the established options here, each with real APIs for storing, consolidating, and retrieving memories. Maximum control, real engineering: you own the integration, the schema decisions, and the retrieval quality. And it only helps agents you build; your off-the-shelf tools never see it.
In the connection layer. The newer pattern: memory lives in the [MCP gateway](/blog/what-is-an-mcp-gateway) that already sits between your agents and your tools, exposed as MCP tools any client can call. Because it is protocol-level, the same memory follows you across Claude Desktop, the IDE, and any other MCP client, and because the gateway sees every tool call, it can capture IDs and results as a side effect of routing them.
| Client app | Framework | Gateway | |
|---|---|---|---|
| Setup effort | None | Engineering project | Connect once |
| Works across clients | No | Only your own agents | Yes, anything speaking MCP |
| Control over contents | Low | Total | High (explicit tools, inspectable) |
| Best for | Personal preferences | Your own agent product | Multi-tool, multi-client work |
There is no universal winner. Solo personal use: the app memory is fine. Shipping an agent product: framework. Working across several tools and clients: the connection layer is the only one that follows you.
What remembering actually looks like
Architecture is abstract, so here is the same three-day workflow with memory in place, using the MCP-tool vocabulary (remember, recall, checkpoint).
Monday. You and the agent decide the Q3 report excludes the pilot customers, and the agent creates the report doc. Two facts worth keeping just happened, so they get stored: remember("Q3 report excludes pilot customers, per decision 2026-07-27") and remember("Q3 report doc ID: doc_88213"). Ten seconds, a few hundred tokens.
Wednesday. New session, any client. You say "update the Q3 report with June numbers." The agent's first move is recall("Q3 report"), which returns the exclusion rule and the doc ID. It opens the right document and applies the right rule without re-deriving either. No re-reading the folder, no "which customers were excluded again?"
Thursday, mid-task. The update is half done when the session has to end. checkpoint("June numbers in, still missing: churn adjustment, exec summary") saves the state of the plan itself. Friday's session resumes from that line instead of reconstructing progress from the document diff.
The economics are visible in the example: each stored fact cost a small write once, and each recall replaced a re-derivation that would have cost fetches, reads, and reasoning turns. Multiply by every session and every project, and you see why we keep saying [the token bill is largely a forgetting bill](/blog/agents-keep-forgetting-token-bill-pays-for-it). We will not invent a percentage for your setup; run one week with memory and compare your own session starts.
Practical rules that make memory work
- Store decisions with their why. "Excludes pilots" ages badly; "excludes pilots because their usage skews activation" still makes sense in October.
- Store IDs aggressively. Doc IDs, database names, ticket numbers. Cheap to keep, expensive to re-find.
- Recall at session start, checkpoint at session end. Make it the opening and closing move until it is habit; a good gateway nudges the agent to do this itself.
- Scope per project or client. Memory that mixes engagements is a liability, not an asset. This matters double for [consultants and agencies](/blog/mcp-gateway-for-consultants).
- Prune. Memory is curation. A store full of stale facts recalls stale facts.
Where we fit, plainly: [Tulimoa's gateway](/gateway) implements exactly this pattern (remember, recall, checkpoint, set_goal as MCP tools, free on every plan, EU-hosted), in early beta and not self-serve yet. The pattern, though, is bigger than any vendor: pick the layer that matches your work, and stop paying your agent to rediscover Monday.
Frequently asked questions
Can I add persistent memory without writing code?
Yes, two ways: use a client whose built-in memory is good enough for your case, or route your tools through a gateway that exposes memory tools over MCP. The framework route is the only one that requires engineering.
How is persistent memory different from a bigger context window?
A window is working memory that dies with the session and gets re-billed every turn; persistent memory survives sessions and is retrieved selectively. The full comparison is in [a bigger context window still isn't memory](/blog/context-window-vs-memory).
What should an agent NOT store in memory?
Secrets (keys, passwords) belong in a credential vault, not in memory. Raw personal data deserves restraint and a retention plan. And bulk knowledge (docs, manuals) belongs in retrieval over a corpus, not in the agent's diary.