The five-stack tax

The tax is not the hardware, although running five systems to answer one query is its own kind of extravagance; the tax is the semantics. When a user deletes a document, your application issues five deletes across five systems with five different durability guarantees, and the embedding service usually finds out last, which is how a RAG pipeline ends up confidently retrieving content the customer was promised was gone. When a write fails halfway, you now own a reconciliation problem that no single system’s tooling can see, because each store is internally consistent and the inconsistency lives in the gaps. When you want a new filter, say “only documents this tenant is allowed to see,” you get to implement it once per system and then again in the fusion layer, and each implementation has subtly different idea of what “allowed” means. None of this is exotic; it is the ordinary Tuesday of any team running retrieval over multiple stores, and it is why so many AI features quietly ship with correctness bugs that only surface when someone audits them.

One row, six indexes

The alternative shape is the one MongrelDB implements: the operational data and the model-derived representations live in the same transactional row, and each signal gets a specialized index that resolves through one shared row-id space, so a single commit lands the document, its embedding, its sparse terms, its set fingerprint, and its metadata together, or none of them land at all. The index menu covers the signals that actually show up in AI workloads, and each one exists because a real retrieval pattern needs it:

This is the part that changes the daily work rather than the benchmark chart: agent memory retrieval by meaning, entities, recency, tenant, and memory type is one table with the right indexes, not a pipeline; ingestion-time near-duplicate detection is a MinHash lookup with exact Jaccard verification on the candidates, not a batch job over an export; prompt caching and training-data dedup stop being separate infrastructure because the fingerprints sit next to the rows they describe. The delete propagates in the same transaction as the delete, which sounds trivial until you have spent a quarter chasing the version where it does not.

Fusion is a function, not a service

The query that used to be a microservice is a SearchRequest: hard filters apply first, named retrievers run against their own indexes, the survivors union, and the ranks fuse with deterministic reciprocal rank fusion, so the result carries a fused score plus a component breakdown showing each retriever’s rank, raw score, and contribution to the final ordering. In SQL that whole request is one call, with the dense and sparse retrievers side by side and the tenant filter applied as a hard condition before any scoring happens:

SELECT id, title, search_rank, fused_score, components
FROM hybrid_search_scored(
  'docs',
  '{"must":[{"bitmap_eq":{"column":"tenant","value":"acme"}}],
    "retrievers":[
      {"name":"dense","weight":1.0,
       "ann":{"column":"embedding","query":[0.12,-0.44,0.83],"k":50}},
      {"name":"sparse","weight":1.0,
       "sparse":{"column":"sparse_terms","query":[[1042,2.1],[877,1.4]],"k":50}}
    ],
    "rrf_constant":60,
    "limit":10}',
  'id,title'
)
ORDER BY search_rank;

Two details in there matter more than they look. The hard filter is not a post-filter bolted onto the fused list, because the engine applies it during candidate generation and adaptively over-fetches so a selective policy still fills the window honestly, and it will return fewer than k rows rather than leak a row the tenant cannot see. And when approximate candidates need float-precision refinement, an optional post-fusion exact-vector stage reranks a bounded candidate window from the stored full-precision vectors, keeping the original fused score, the exact score, and the final rank visible side by side, so you can defend the ordering in a code review instead of waving at it. The fusion step itself is deterministic, which is a property you only learn to demand after debugging a ranker whose output depends on which worker answered first.

The guardrails are the feature

Retrieval that can run away from its budget is a production incident with a queue, so every scored surface in the engine is bounded by default and the bounds are part of the API rather than an operational wish: final hits cap at 10,000, candidates per retriever at 100,000, retrievers per request at 32, hard conditions at 256, sparse terms or set members at 65,536, and hybrid unions stop at 250,000 candidates while materializing only the ranked result window. Every scored endpoint accepts deadline_ms and max_work, runs off the async workers with cooperative cancellation, and shares a bounded concurrency semaphore, and the server-level ceilings are yours to tighten with MONGRELDB_AI_MAX_FUSED_CANDIDATES and MONGRELDB_AI_MAX_CONCURRENT when you would rather shed load than serve stale rankings late. This is the difference between a database and a demo, because the demo answers the query and the database answers it while thirty other tenants are asking questions of their own.

When you still want the zoo

Honesty requires the other side, and there is one: if your corpus is web-scale, if you employ people whose full-time job is tuning ranking functions, or if you already operate Elasticsearch well and your AI feature is a tenant of it rather than its reason to exist, the dedicated stack earns its keep, and a single embedded engine is not going to replace a search platform that a team has spent years shaping. The case for one engine is the common case rather than the extreme one: your data already fits the operational database’s world, your AI features need the same five signals fused per request, your correctness requirements are “the delete actually deletes” and “tenant A never sees tenant B,” and your team would rather ship the feature than staff the pipeline. We ran the five-stack version of this in the 2010s with MySQL, Sphinx, and a memcached layer held together by cron, and the modern equivalent is Postgres plus pgvector plus OpenSearch plus a fusion service held together by hope; the tools got better and the seams did not, which is the entire argument for putting the seams inside one transaction and being done with it.