The agent memory loop doesn't need you in the room

Nobody watched this happen.

An agent was mid-task, adding a new way to look something up. It's the kind of change that ships routinely, dozens of times a week, anywhere. Building it meant encoding a value into a key that a lower layer would later parse back apart. Recall surfaced a rule we'd paid to learn once already: a value doesn't get to go straight into a key just because it looks safe, because whatever parses that key apart later has no idea where any of its pieces came from. The agent checked whether that applied here. It did. It changed course.

No review flagged this. No review even saw it. The change was already fixed by the time anyone would have looked.

That's the payoff of the four months I spent building an agent's memory, compressed into one moment nobody had to supervise. Four posts got the pieces there: a memory tier that recalls on its own, a curation discipline for what belongs in it, and a queue for what doesn't belong yet. This post is the architecture in one place, what changed once we stopped being the ones watching it run, and the source, out today alongside everything above.

The recall-and-capture loop

Every session draws on two kinds of memory: the team's shared, reviewed knowledge, and the agent's own private notes. Both are typed records in the same backend, searchable by meaning, ownership-fenced so a private note stays private. Neither is a file that only grows.

Recall from that memory happens in three places, and none of them wait to be asked.

Before each step, a fast search runs over the agent's own memory and the shared knowledge base, and the top few results land in context before the model reads the turn. While the agent works, a background pass watches what's actually happening mid-task and runs a small model to judge whether what it found actually answers the need. A search score can't tell you it found nothing, so something has to. Only what passes gets surfaced. And when the agent decides it needs more, it's a capable model with real tools, so it searches directly. Most memory systems make this the default: agent-initiated search as the main mechanism. We use it least, because the two cheaper passes above already catch most of what matters.

Capture works the same way, on its own clock. Once a session has enough new transcript, a cheap model reads it and proposes candidate memories. It cannot write anywhere. That's not a detail, it's the load-bearing decision in the whole design, and I made it because a cheap model with write access once recorded a confident, wrong fact as if it were true.

So every proposal lands in a queue instead: typed, exactly-queryable, and structurally unsearchable, so an unreviewed claim can never surface as if it were settled. The capable agent reviews that queue in the same session and commits what holds up: store it, point it at existing documentation, or discard it with a reason. Nothing sits forever. An unsettled claim gets handed to whichever session is active next, and if it goes unsettled long enough that it's clearly not getting a real look, it's auto-dismissed, reversibly, never silently.

Every piece here is something we hit wrong first. The three-pass recall exists because a naive version couldn't say "I found nothing." The propose/commit split exists because of the wrong fact above. The queue exists because a proposal that isn't a memory yet still has to live somewhere. Each one was a fix for a real failure, and together they're the architecture we actually run.

What changed when we stopped watching

Every post before this one described something we ran and checked on. That's no longer quite true.

Autonomous operation is the default now: a session can run for a long stretch on real work with nobody at the keyboard between prompts. That changes what "recall matters most at the exact moment you're about to act" means, because there's no longer a natural pause where a person might notice and step in. So the real test is whether the architecture still works when nobody's watching, not just whether it helps when someone is.

It does, and not by accident. None of the three recall passes were ever built to depend on a human being present, so an unattended run gets the identical loop a supervised one does. Disposition holds up the same way: the capable agent still commits, but during a long unattended run that agent is the same one already doing the work, triaging its own queue in the same session, on its own schedule. The loop doesn't degrade when nobody's watching. It was built, piece by piece, to assume nobody would be.

The key-design catch at the top of this post is what that looks like in practice, not a special case. The value in question didn't arrive by an obvious path, and enumerating every path a value can take before it reaches a key is genuinely hard, for anyone, in any test suite. A test catches what you thought to write a test for. Recall catches what you already paid to learn once, wherever it shows up again, in a shape nobody had to predict in advance. Different net, different fish. This is exactly the fish it's for.

What's still tuning, not proof

The loop runs in our own production every day, catching things like the one above. The pilot numbers from three posts ago still stand: the right result at the top for five of eight realistic situations, top-five for seven of eight. Two dials are still loose, and neither is the mechanism itself. I set the thresholds to sensible defaults and have not gone back to hand-tune them. The cost math (a few percent of the main model's spend, at our volume, on a subscription rather than billed per call) will land differently on a heavier workload than ours.

The source, now out

Every post before this one ended the same way: this is what we run, not yet something you install. That changes here. The full loop (both recall passes, the capture worker, the queue, the disposition CLI) is a Claude Code hooks package, and it's the exact code we've been dogfooding this whole series, not a rewrite for public consumption.

Running it needs a Vectros account, and access is still invite-only. Request early access if you don't have a key yet. At the volume one person or a small team actually generates, the free plan covers it. Once you're in:

npm i -g @vectros-ai/cli
vectros login
vectros bootstrap --blueprint agentic-sdlc --no-seed --yes

npm install -g @vectros-ai/claude-code-agent-memory
claude-code-agent-memory init

That provisions the whole agentic-SDLC knowledge base, deploys the hooks, and wires them into Claude Code. The loop itself only reads and writes two of those schemas, candidate and memory; the rest is harmless if you never touch it, and the blueprint is a plain, forkable definition if you'd rather trim it down to just those two. Everything above (the three recall passes, the propose/commit split, the queue) starts running. The README has the full quickstart, the CLI reference, and the full list of what's still pre-1.0.

Want a different backend under it? That's a real option. The loop talks to a fairly plain REST contract (create a few record types, look them up, patch them), and the API base is already an environment-variable override, not something buried in the code. Fork the repo and point it at whatever you're already running.

Four posts to build it, watching it every day, catching our own mistakes in it before anyone else could. This one is where we stop being the only ones who get to.