AI Agent Memory, Explained: Why Agents Forget and What a Memory Layer Does
AI agents forget by design. Everything an agent knows during a task lives in its context window, a limited working memory that is wiped when the session ends. Start a new session and the agent remembers nothing: not your preferences, not yesterday's results, not the ID of the record it created an hour ago. A memory layer fixes this by storing the important parts outside the session and bringing them back when needed.
Why agents forget in the first place
A language model does not have memory the way a database does. It has a context window: the text it can see right now. Think of a whiteboard in a meeting room. During the meeting you can write on it and read from it, but the cleaning crew wipes it every night. Tomorrow's meeting starts with a blank board, even if half of yesterday's notes are still relevant.
The whiteboard is also small. Fill it with tool descriptions, instructions, and old conversation, and there is less room for actual thinking. Agents with too much on the board get slower, more expensive, and more confused.
What forgetting costs you
Forgetting is not just annoying. It shows up on your bill. An agent that cannot remember re-fetches the same data, re-reads the same tool lists, and asks you the same questions, and every repetition costs tokens. We looked at this pattern in detail in [your agents keep forgetting, and your token bill pays for it](/blog/agents-keep-forgetting-token-bill-pays-for-it), with practical fixes in [how to reduce MCP token usage](/blog/reduce-mcp-token-usage).
The three kinds of agent memory
Working memory is the context window itself. Fast, but small and temporary.
Retrieval memory stores documents or facts in a database and searches them when a question comes up. Good for knowledge, but passive: the agent has to know to ask.
A memory layer sits in the traffic between the agent and its tools. Because it sees every call, it can capture the things worth keeping automatically: IDs that were just created, decisions that were made, goals that are still open. Next session, it hands them back before the agent even asks.
What a good memory layer does
Four things, concretely. It remembers durable facts across sessions. It recalls them at the start of a task instead of waiting to be asked. It keeps track of open loops, so a plan survives an interruption. And it stays selective, because storing everything would just rebuild the overflowing whiteboard somewhere else.
The natural place for all this is the gateway an agent already talks through, which is why we built our memory layer into one. How that works end to end is described in [what is an MCP gateway](/blog/what-is-an-mcp-gateway).
The short version
Agents forget because their only memory is a whiteboard that gets wiped. A memory layer gives them a notebook: smaller than the whiteboard, but permanent, and open to the right page when work starts.