What an agent memory database actually stores
Agent memory is not a bigger prompt and it is not only a vector collection. The useful record is durable operational state: what the user said, what the agent did, what project it belonged to, which entity it concerned, when it happened, whether it is still current, and whether another memory already says the same thing.
That record has to answer several retrieval shapes at once. A user may paraphrase an old preference, quote an exact error fragment, ask about a project from last month, or expect the agent to forget superseded state. A database that only does nearest neighbors can recall the vibe and miss the constraint.
One RowId space for the signals memory needs
MongrelDB exposes six public secondary index kinds that resolve through the same RowId space: Bitmap for equality, PGM learned range for numeric and time bounds, FM-index for substring containment, ANN for dense candidates, Sparse for learned lexical retrieval, and MinHash for set-similarity candidates. The engine documentation describes the same set for AI-native access patterns.
| Memory question | MongrelDB signal | Why it matters |
|---|---|---|
| What did the user mean? | HNSW dense ANN | Catches paraphrases and semantic neighbors. |
| What exact term appeared? | Sparse and FM-index | Preserves rare terms, quoted fragments, and literal error text. |
| Which project, user, state, or type? | Roaring bitmap | Turns access and metadata boundaries into cheap candidate intersections. |
| How recent or important is it? | PGM learned range | Constrains retrieval by timestamp, score, confidence, or reinforcement. |
| Have we stored this already? | MinHash plus exact Jaccard verification | Finds near-duplicates before the memory store grows into repeated sludge. |
| Can the answer be explained? | Scored components and RRF fusion | Hybrid results carry component rank, raw score, fused score, and final rank. |
The public SQL surface includes ann_search_scored, sparse_search_scored, minhash_search_scored, set_similarity_scored, and hybrid_search_scored. The hybrid form wraps the core SearchRequest: named ANN, sparse, and MinHash retrievers are unioned and fused with deterministic reciprocal-rank fusion, with component scores in the result.
-- Documented scored-search building blocks:
SELECT id, body, ann_cosine_distance
FROM ann_search_scored('agent_memories', 'embedding', '[0.10,0.45,0.78]', 20, 'id,body');
SELECT id, sparse_score
FROM sparse_search_scored('agent_memories', 'sparse_terms', '[[17,2.0],[42,1.0]]', 20, 'id');
-- hybrid_search_scored(table, request_json, projection) wraps SearchRequest:
-- hard filters first, named retriever union, deterministic RRF, component scores.Own the memory boundary
Hosted agent-memory services can be convenient, but they put the memory boundary outside the product. MongrelDB keeps that boundary in the application or on infrastructure you operate. Use the embedded engine when one process owns the data directory, native bindings when a language client should run in process, or mongreldb-server when several processes need one warm local database.
The Hermes memory plugin shows the production shape: MongrelDB-backed memory with native Rust FFI or HTTP daemon mode, dense ANN by default through all-MiniLM-L6-v2 at 384 dimensions, sparse model-free retrieval as an option, and encrypted data directories by default. That plugin uses the same engine documented here rather than a private fork.
Memory writes and retrieval need one transaction story
Agent memory goes stale in two ways: the agent forgets, or the index remembers something the source row no longer says. MongrelDB stores operational rows, embeddings, and secondary indexes under one WAL and MVCC model. Generated embedding writes validate provider output before the WAL append, and replication carries the materialized vector and provenance so followers do not call the provider again.
That matters when a memory is deleted, corrected, or reclassified. Search candidates come from the same committed state as the row the agent reads back. ANN is still approximate, and MongrelDB's own production guidance says to validate recall against your model and corpus rather than treating one recall number as universal.
Private memory should not become plaintext spillage
Long-term agent memory concentrates exactly the data a privacy review worries about: preferences, relationships, private projects, debugging context, and decisions. MongrelDB supports page-level AES-256-GCM encryption for sorted-run pages, WAL frames, result cache, index checkpoints, and encrypted per-page statistics. Username and password credentials are a separate logical access layer that can stack with encryption.
The Hermes plugin enables encryption at rest by default, generating a random passphrase in a mode 0600 key file when none is supplied. Plaintext is an explicit opt-out, not the default path.
Where MongrelDB fits, and where it does not
Choose MongrelDB when
- the agent or product should own memory locally or inside its own service boundary;
- semantic recall must compose with exact text, metadata, recency, and dedup signals;
- memory writes need transactions, recovery, and optional encryption rather than a sidecar index;
- you want SQL and native APIs over the same store instead of a memory framework plus several databases.
Choose something else when
- you want a managed vendor API and do not want to operate storage at all;
- the memory store must scale as a shared multi-region service for many independent writers;
- your application only needs a small prompt cache and durability is not important;
- your organization has standardized on a hosted memory framework and accepts its storage dependencies.
Sources
Agent memory FAQ
What is an agent memory database?
It is the durable store an agent uses to remember facts, decisions, preferences, and work state across sessions. The useful baseline is hybrid retrieval: semantic recall plus exact text, filters, recency, deduplication, and transactions.
Is MongrelDB a hosted agent memory service?
No. It is an open source embedded database engine written in Rust. Run it in process, through native bindings, or as a local daemon when multiple processes need one warm store.
Can MongrelDB combine vector search with exact filters?
Yes. Hard filters can intersect with ANN, sparse, substring, range, and MinHash candidate sets in one RowId space. Scored hybrid search fuses named retrievers with RRF and returns component scores.
Does agent memory need encryption at rest?
If memories contain private user or business context, encryption at rest is a reasonable default. MongrelDB supports AES-256-GCM page-level encryption, and the Hermes plugin creates encrypted data directories by default.
When is a hosted memory framework better?
Pick a hosted framework when operations and vendor-managed APIs matter more than local ownership, or when your memory model fits the framework's storage stack. Pick MongrelDB when the memory boundary belongs inside your product.