Recall ranking demystified: similarity, importance, and recency in one formula
How Agent Brain ranks candidate memories by blending vector similarity, salience, and how recently a memory was touched.
By Agent Software
Similarity search is not the same thing as relevance
Most memory systems for AI agents stop at vector similarity: embed the query, find the nearest neighbors, return them. That works until a session runs long enough that old, resolved context starts competing with current context for the same slot in the answer. A decision made three weeks ago and a decision made three minutes ago can both be "similar" to a new query — but they are not equally useful.
Three signals, one score
Agent Brain's recall ranking combines three signals into a single score for every candidate memory:
- Similarity — how close the candidate is to the query in embedding space
- Importance — a salience score assigned when the memory was written, based on rule-based heuristics with an LLM fallback for ambiguous cases
- Recency — how recently the memory was written, accessed, or reinforced
Each signal is weighted, and the combined score decides what actually gets returned to the agent, not raw distance alone.
Why importance is scored at write time
Waiting until recall time to decide what matters means re-evaluating every candidate on every query, which is slow and inconsistent. Scoring importance when a memory is written — using heuristics first, and only falling back to an LLM judgment call for genuinely ambiguous cases — keeps the system fast and keeps importance judgments stable over time.
Why recency has to be in the formula at all
Without a recency term, a memory system will keep surfacing outdated context indefinitely, as long as it stays similar enough to new queries. Recency doesn't discard old memories — it just makes sure a fresher, equally-relevant memory doesn't lose to an older one purely because it was written first and has accumulated more "hits" over time.
What this looks like in practice
An agent revisiting a design decision weeks later gets the current answer, not the first answer that happens to sit closest in embedding space. That's the practical difference between a system that stores text and a system that actually manages memory over the life of a project.
