Embedded vector database in Rust: HNSW, SQL, and hybrid search

What makes a vector database embedded?

The engine opens local storage from the application process and vector queries avoid a network round trip. That removes a service, credentials, health checks, and synchronization between operational rows and embeddings. It also means the application owns memory, CPU contention, upgrades, exclusive file access, and backup coordination.

A useful embedded engine should persist source data and embeddings atomically. Otherwise “local” still leaves two stores whose versions can drift.

Selection criteria beyond ANN speed

MongrelDB's embedded vector path

MongrelDB is written in Rust and stores operational rows, embeddings, and secondary indexes under one WAL and MVCC model. ANN supports HNSW with dense or binary-sign vectors, plus DiskANN and IVF dense paths. Product quantization is available as a flat PQ backend; its current hnsw selector is compatibility metadata, not an HNSW graph.

Every index family returns RowIds. A query can intersect vector candidates with Bitmap equality, learned numeric ranges, or FM substring constraints before row decoding. Named ANN, sparse, and MinHash retrievers can also fuse through reciprocal-rank fusion, followed by optional exact-vector reranking over a bounded candidate window.

Condition::Ann {
    column_id: 6,
    query: vec![0.10, 0.45, 0.78, 0.23],
    k: 10,
}

The query vector must match the embedding dimension. ANN is approximate; tests and production monitoring should compare a sample against brute-force results rather than treating one recall number as universal.

Hybrid retrieval without a second search service

Dense similarity catches semantic neighbors. Sparse vectors preserve rare terms and learned lexical signals. Exact substring constraints catch literal fragments. Equality and range indexes enforce metadata boundaries. Combining those signals is often more useful than making the ANN graph marginally faster.

MongrelDB exposes scored SQL table functions including ann_search_scored, sparse_search_scored, and hybrid_search_scored. Remote ranked queries use bounded scored functions rather than unrestricted Boolean ANN predicates, keeping deadlines and work ceilings explicit.

Read benchmark numbers correctly

The published MongrelDB measurements are local release-build engineering results, not cross-machine guarantees. On the documented Intel Core Ultra 9 386H system, a single accepted put without fsync measured 4.4828 microseconds and a durable commit measured 4.6721 milliseconds. Those figures describe the surrounding operational write path, not ANN throughput.

When to embed—and when not to

Embed when

• data should remain local or work offline;
• one application or daemon owns the storage root;
• operational writes and retrieval must commit together;
• shipping one product is simpler than operating another service.

Use a vector service when

• the index must scale independently across many machines;
• many remote applications need concurrent service ownership;
• managed failover and global replication are hard requirements;
• the vector workload alone justifies a dedicated operations team.