← Blog

Your agent's memory shouldn't be a text file

· Vectros Team
  • engineering
  • dogfooding
  • agent memory

A while back we put our whole engineering knowledge base on Vectros and started running our own team on it. It did what it promised. A thousand inert markdown files became something you could actually ask. "Why is it shaped this way?" returned the decision, with its reasons, cited. That part worked, and it still stands.

Then we ran agents against it every day, and the same gap kept showing up. The knowledge base is the team's shared, reviewed knowledge. But an agent also builds up its own notes as it goes: the half-formed lesson, the state of the project it is in the middle of, the "the owner said to always do X." That half lived where it always does, in a text file on one machine. Not isolated, not audited, not searchable by meaning, impossible to hand up to the shared knowledge base, and gone the day the machine is.

So the first thing we got wrong about agent memory was thinking it was one thing. It is two.

Two tiers, not one

  • Shared: the team's reviewed knowledge. Global, read by everyone. This is the knowledge base from the first post.
  • Working: each agent's own private notes. Seen only by that agent.

Both deserve the same care. We had given it to the shared half and left the private half in a file. And that file has a failure mode you hit even with no knowledge base anywhere in the picture.

The file only grows

Every session adds to it and nothing ages out. Before long it is a long, flat wall of notes: some current, some stale, some quietly contradicting a line three screens up. Whatever loads it now has a bad choice to make. Load the whole file every turn, which is slow, costs tokens, and buries today's useful note under months of dead ones. Or skip it, and the memory may as well not exist.

This is the quiet reason so much agent memory goes unused. Not that it is missing, but that nobody can find the one line that matters inside it. A store that only grows stops being memory and becomes clutter, and the more an agent piles up, the less any single lookup is worth.

People fight this, and so did we. You trim the file by hand, cutting what looks stale. You run a consolidate or summarize command to squeeze the pile back down. Both help. Neither fixes it. Hand-trimming is work that never ends and does not scale. Summarizing is lossy by nature: it compresses distinct notes into a digest that guesses what mattered and drops the exact line you will want next week, and you are still left with one blob, only smaller. The tools are not the problem. The format is. A flat file gives you two moves, shrink it or hand-edit it, and neither of them is search.

The fix is search by meaning, plus structure

It takes two things, and it needs both.

The first is that the store is searchable by meaning, not just by keyword. A flat file you can only grep: you have to remember the words a note was written in, and if you reach for the wrong ones, the note may as well not be there. The memory tier is searched the same way the shared knowledge base is, by meaning, so the agent gets the note that fits its situation even when it shares no words with it, and results come back ranked by how relevant they are. That alone is the difference between a store you can query and a file you can only scroll.

The second is structure. We made working memory a typed store: each memory is a row, not a line in a document, and a role fences every agent to its own rows, kept separate by ownership rather than by a WHERE clause you write by hand and hope you got right every time. Being a typed row is what lets a memory carry a priority, so a small pinned set always loads and everything else is ranked, not just returned. And when a note goes out of date, the note that replaces it takes over, with the link between them kept, so the current answer wins and the old one stops competing for the agent's attention. Nothing is deleted, so the history of how a rule changed survives, but what surfaces is the version that is true now.

Together, search and structure make the move a flat file cannot. The thing that grows and the thing the agent reads stop being the same thing. The store keeps growing. What loads does not, and it is not a lossy summary of the pile, it is the few rows that actually fit. This does not remove judgment. You still decide what to pin and what replaces what. The difference is that those choices now shape what gets recalled, instead of just making a file shorter.

A store is not recall

Structure fixes what the memory is. It does not fix when the agent reaches for it, and that turned out to be the harder half. A knowledge base only pays off if the agent recalls it at the moment it acts. Telling the agent to do that, in the prompt, does not hold. A prompt is a suggestion the model follows when it remembers to, and skips the moment it is sure it already knows, or the context runs long, or the task gets interesting. So the answer sits right there, and an agent that was told to check still walks into a mistake we had already written down.

The fix was to stop asking and put recall in front of the agent instead of leaving it to the agent's discipline. That is a big enough story to be its own post, and it is the next one: we built it, turned it on, and it caught us within a day.

Why it is worth it

Put the tier and automatic recall together and two things change, the two that actually cost you. Efficiency: the agent stops re-deriving what it already worked out, so fewer turns spent re-searching for a fact it found last week, and fewer times you have to redirect it onto something it should have known. Quality: the important thing stops getting missed, because the trap that would have caused the bug, or the decision the team already made, is in front of the agent before it commits, not found afterward. Less wasted motion, fewer avoidable mistakes. That is the whole return.

Stand up your own

The agentic-SDLC blueprint ships both tiers. Stand up the knowledge base, then enroll an agent with its own private memory:

npm i -g @vectros-ai/cli
vectros login
vectros bootstrap --blueprint agentic-sdlc --no-seed
vectros join agentic-sdlc --role member

vectros access explain shows you exactly what that agent can and cannot reach. That gives you the tier itself, a typed, ranked, searchable-by-meaning place for an agent's memory to live. See what the blueprint provisions, or read the full walkthrough in the docs.

What automatically puts the right memory in front of the agent, without anyone asking, is a loop we built on top of that tier. That part is not something you install yet. It is what we run on our own work, and it is the next post.

We built this tier to hold our own agent's memory, not a demo's. Moving ours onto it, off the file for good, is the rest of this story.