Retrieve useful memories within a finite context budget

A system can store a useful record and still fail to recall it at the right moment. Memory retrieval therefore deserves its own evaluation. The retrieval query should reflect the current decision: the affected service, unresolved fact, user context, or artifact being produced. A generic search using the entire latest message can be dominated by incidental words and miss the record that changes the action.

Filter records by tenant, access, type, validity, and task scope before ranking. A highly similar record belonging to another organization must never become a candidate merely because it would improve the answer. After those hard filters, rank eligible memories for usefulness. Keeping eligibility separate from score prevents a large similarity value from compensating for a failed boundary condition.

Relevance estimates whether a memory concerns the current task. Recency estimates temporal proximity. Importance records an application-specific judgment about consequences or future usefulness. Frequency counts repeated encounters, which may reflect duplication rather than truth. None of these alone establishes correctness. A new, frequently repeated rumor can be less trustworthy than an older authoritative configuration record.

A toy score can combine normalized relevance, importance, and a decay term, but its weights are design choices that require evaluation. For example, let score equal 0.6 times relevance plus 0.3 times importance plus 0.1 times recency. A record with values 0.8, 0.5, and 0.2 scores 0.65. Stating the equation makes it possible to inspect which signal caused a surprising recall instead of treating retrieval as an unexplained intuition.

MemoryBank explores memory persistence and updating inspired by a forgetting mechanism. For this course, use exponential half-life decay as a transparent independent teaching policy: after one half-life, the recency term is one-half; after two, one-quarter. The decay changes retrieval priority, not whether the historical event happened. A verified specification may deserve a different freshness rule from a casual conversational detail.

Suppose two records have equal relevance and importance, but ages of one and four days with a two-day half-life. Their recency terms are about 0.707 and 0.25. The younger record ranks higher under the chosen equation. If the older record is authoritative and the newer one speculative, the architecture needs an authority or verification distinction rather than pretending age alone expresses trust.

MemGPT frames external memory management through a hierarchy that moves information into and out of a limited context. The practical lesson is to manage the active working set explicitly. Retrieval candidates can be more numerous than the records inserted into the prompt. Select a compact set that covers the required facts and leaves room for the task, tool definitions, and output.

If the available memory budget is 300 tokens and candidates cost 180, 170, and 90, blindly taking the top two exceeds the budget. A greedy fit policy could take the first and third, but it may miss complementary coverage. A more advanced selector can optimize coverage under cost. Always measure with the deployed tokenizer when token limits matter; word counts are only a reproducible approximation for the standard-library simulation.

A summary reduces context cost but may remove exceptions, timestamps, or attribution. Keep source IDs and make it possible to retrieve the original when a decision depends on a compressed detail. If a summary says client prefers short reports, the source may reveal that this applied only to mobile notifications. Retrieve the original before extending that preference to a detailed technical review.

The code computes recency-weighted scores for three fixed memories, then selects records within a word-count budget. Its relevance values are supplied fixtures, not embedding-model outputs. Inspect why the top item fits and why another is skipped. The next lesson adds correction and retention semantics, because a beautifully ranked obsolete memory can be more damaging than a mediocre ranking of current, properly scoped records.

Synthetic relevance and importance scores combine with a two-day recency half-life. Greedy selection uses a word budget, not real model tokens. The example isolates ranking and packing; it assumes all records already passed scope and retention filters.