Agent Memory vs RAG: Related, Not the Same

Short answer: RAG retrieves from a corpus you gave the system in advance. Agent memory accumulates from what the agent itself experiences: the decisions it made, the IDs it created, the preferences you stated. RAG is a library card. Memory is a diary. Both involve retrieval, which is exactly why agent memory vs RAG gets muddled, and why teams keep building the wrong one first.

The confusion is understandable. Under the hood, both often use embeddings, both do similarity search, and vendors describe both with the word "retrieval". But the data, the write path, and the job are different, and mixing them up produces agents that can quote your handbook while forgetting your name.

What RAG is for

Retrieval augmented generation answers one question: how does the model get the right knowledge at the right moment? You prepare a corpus (docs, policies, past tickets), chunk and embed it, and at question time the system pulls the most relevant pieces into context. The corpus is authored by humans, prepared in advance, and read-only from the system's perspective. If the source documents change, you re-index.

RAG is the right tool when the knowledge is big, mostly stable, and external to the agent: a product manual, a legal archive, a knowledge base. We compared it against the tool-calling side of the world in [MCP vs RAG](/blog/mcp-vs-rag).

What agent memory is for

Memory answers a different question: how does the agent keep what it learned while working? Nobody writes a corpus entry saying "the customer prefers short emails" or "we decided against feature flags in March" or "the staging database ID is db_7741". Those facts are produced during sessions, by the agent's own experience, and they are worthless in a document archive but priceless at 9 a.m. tomorrow.

So memory has a write path RAG does not: the agent stores facts as they happen (a remember call, in MCP terms), and recalls them later by topic. It is personal to the agent-and-user pair, grows continuously, and contains judgments and state, not published knowledge. The deeper anatomy is in [our guide to memory layers](/blog/memory-layer-for-ai-agents).

The side-by-side

The last row is the practical one. When RAG fails, you tune chunking and ranking. When memory fails, it is usually because the system never wrote anything down, and no retrieval tuning fixes an empty diary.

Why the confusion costs real money

The common mistake has a predictable shape: a team notices the agent keeps forgetting things, diagnoses it as a knowledge problem, and builds RAG over their docs. The agent can now cite the handbook beautifully. It still greets every session with amnesia about yesterday's work, because yesterday's work was never in any document to index.

The reverse mistake exists too: stuffing policy manuals into agent memory fact by fact, hand-feeding a diary what belongs in a library. Now the memory store is bloated with static knowledge, recall gets noisier, and the actual per-session facts drown.

The rule that prevents both: if a human authored it as a document, it belongs in RAG. If it happened during a session, it belongs in memory.

They compose, and the plumbing can be shared

A capable agent uses both, often through the same protocol. A RAG pipeline wraps neatly into an MCP tool (search_docs), and memory arrives as MCP tools too (remember, recall). From the agent's point of view, both are just tools in the list; from yours, they are fed by completely different pipelines.

This is also why we think the connection layer is a natural home for memory: a gateway sees every tool call the agent makes, which is exactly the stream of experience memory is made of. That is the design behind the memory tools in [Tulimoa's gateway](/gateway) (our product, stated openly), where remember and recall sit next to whatever RAG tools you connect, and the two stop being confused because each has its own place.

Frequently asked questions

Is agent memory just RAG over conversation logs?

No. Indexing raw transcripts retrieves noise: dead ends, retries, verbose tool output. Memory works because facts are selected at write time (this decision, this ID, this preference), which is curation, not archiving. A searchable log is a fallback, not a memory layer.

Do I need RAG if my agent has memory?

For document knowledge, yes. Memory will not absorb a 400-page manual, and should not. Use RAG for the library, memory for the diary, and let the agent call both.

Can the same vector database serve both?

Technically often yes, and architecturally they should still be separate stores with separate write paths and scopes. Mixing shared corpus knowledge with per-user session facts in one index creates both relevance problems and privacy problems.