Embedded vector database in Rust: HNSW, SQL, and hybrid search

What makes a vector database embedded?

The engine opens local storage from the application process and vector queries avoid a network round trip. That removes a service, credentials, health checks, and synchronization between operational rows and embeddings. It also means the application owns memory, CPU contention, upgrades, exclusive file access, and backup coordination.

A useful embedded engine should persist source data and embeddings atomically. Otherwise local still leaves two stores whose versions can drift.

In-process vector database: ownership and tradeoffs

An in-process vector database is loaded as a library by the application. Calls do not cross a network boundary, and the application starts, upgrades, backs up, and closes the engine. That is useful for desktop, edge, offline, test, and single-owner server workloads.

The process also inherits index memory, build CPU, file locks, and crash behavior. Native libraries require platform packaging; synchronous calls can block an event loop; independent processes usually cannot open one exclusive storage root safely. In process removes service operations, not database operations.

Selection criteria beyond ANN speed

MongrelDB's embedded vector path

MongrelDB is written in Rust and stores operational rows, embeddings, and secondary indexes under one WAL and MVCC model. ANN supports HNSW with dense or binary-sign vectors, plus DiskANN and IVF dense paths. Product quantization is available as a flat PQ backend; its current hnsw selector is compatibility metadata, not an HNSW graph.

Every index family returns RowIds. A query can intersect vector candidates with Bitmap equality, learned numeric ranges, or FM substring constraints before row decoding. Named ANN, sparse, and MinHash retrievers can also fuse through reciprocal-rank fusion, followed by optional exact-vector reranking over a bounded candidate window.

Condition::Ann {
    column_id: 6,
    query: vec![0.10, 0.45, 0.78, 0.23],
    k: 10,
}

The query vector must match the embedding dimension. ANN is approximate; tests and production monitoring should compare a sample against brute-force results rather than treating one recall number as universal.

Hybrid retrieval without a second search service

Dense similarity catches semantic neighbors. Sparse vectors preserve rare terms and learned lexical signals. Exact substring constraints catch literal fragments. Equality and range indexes enforce metadata boundaries. Combining those signals is often more useful than making the ANN graph marginally faster.

MongrelDB exposes scored SQL table functions including ann_search_scored, sparse_search_scored, and hybrid_search_scored. Remote ranked queries use bounded scored functions rather than unrestricted Boolean ANN predicates, keeping deadlines and work ceilings explicit.

Read benchmark numbers correctly

The published MongrelDB measurements are local release-build engineering results, not cross-machine guarantees. On the documented Intel Core Ultra 9 386H system, a single accepted put without fsync measured 4.4828 microseconds and a durable commit measured 4.6721 milliseconds. Those figures describe the surrounding operational write path, not ANN throughput.

Compare the embedded options

Vector-only tools can be the right answer when the workload is only a collection of embeddings. The decision changes when the same store has to own operational rows, metadata filters, lexical search, transactions, and encryption. These comparisons keep that boundary explicit:

Vector database without a separate server

Removing a separate vector server can eliminate credentials, ports, health checks, deployment manifests, and replication between source rows and embeddings. It works best when one product owns one local corpus and the vector workload does not need independent scaling.

A library is not automatically the right answer for shared remote access. If several processes need one warm index, run one authenticated owner, such as mongreldb-server, instead of allowing each process to open the files. If many machines need managed availability, use a service designed for that boundary.

When to embed, and when not to

Embed when

• data should remain local or work offline;
• one application or daemon owns the storage root;
• operational writes and retrieval must commit together;
• shipping one product is simpler than operating another service.

Use a vector service when

• the index must scale independently across many machines;
• many remote applications need concurrent service ownership;
• managed failover and global replication are hard requirements;
• the vector workload alone justifies a dedicated operations team.

Embedded vector database FAQ

What is an embedded vector database?

It is a database library or local engine that stores embeddings beside application data instead of requiring a separate vector service. The main advantage is ownership: one deployment, one transaction story, and no network hop for retrieval.

Is an embedded vector database only for small projects?

No. It fits desktop, edge, local-first, agent-memory, test, and single-node systems. The limit is independent scale-out and many remote writers, not a fixed row count.

Why do filters matter as much as ANN speed?

Because most useful retrieval includes metadata and exact-text constraints. Filtered candidate sets prevent nearest-neighbor search from returning rows the user or tenant should never see.

Can MongrelDB run hybrid vector and lexical search?

Yes. It combines dense ANN, sparse retrieval, FM-index substring search, bitmap equality, learned ranges, and MinHash in one engine, with scored hybrid fusion through RRF.

When should I use Chroma, LanceDB, or sqlite-vec instead?

Use them when their narrower storage model already matches the job: embedding collections, columnar vector workloads, or SQLite extension simplicity. Use MongrelDB when vectors have to live with SQL, transactions, hybrid retrieval, and encrypted storage.