Hybrid indexing
HOT, Bitmap, PGM, FM, HNSW, PMA, Sparse/SPLADE, and MinHash indexes all resolve through the same RowId space, making mixed predicates practical inside one engine.
Open source embedded database
MongrelDB
A local database for writes, scans, search, vectors, ranges, and encrypted storage.
MongrelDB is an open source embedded database for applications that need more than key-value lookup, but do not want to operate a separate service for every access pattern. Store data locally, query it with SQL or native APIs, and combine equality, range, text, vector, and sparse retrieval in one engine.
8index families built around a shared RowId model
8.0µssingle-row durable write in the development profile
O(1)metadata-backed count path for known cardinalities
Localembedded file storage with optional encryption
Overview
One embedded engine. Fewer moving parts.
Embedded databases are strongest when they stay close to the application. MongrelDB keeps that deployment model while adding the access paths modern software often has to bolt on later.
Write-friendly columnar storage
Fast scans should not require giving up direct writes. MongrelDB uses a WAL and memtable in front of immutable PAX-columnar .sr runs.
Practical encrypted queries
ENCRYPTED_INDEXABLE columns use equality and range tokens to narrow candidate rows before plaintext is required.
Application-ready interfaces
Use SQL through DataFusion, Arrow IPC for batches, and NAPI bindings for Node. Keep local data local until your application needs to move it.
Indexes
Different access paths. One row identity.
Each index returns RowIds, so the engine can combine filters without forcing every workload through a single data structure.
Primary key
HOT trie point lookup
Height-optimized trie access keeps primary-key lookup compact and direct, then returns RowIds that can be combined with other predicates.
- Purpose-built for primary-key lookup and exact targeting.
- Feeds the same RowId set used by the rest of the engine.
- Fits OLTP-style read and update paths.
Equality
Roaring bitmap equality
Low-cardinality values compress into Roaring bitmaps, turning equality filters into set operations instead of table walks.
- Useful for tenant, status, type, flag, and other low-cardinality columns.
- Turns equality filters into set operations.
- Combines cleanly with vector, substring, range, and sparse constraints.
Range
PGM range index
A shrinking-cone, epsilon-bounded learned model predicts where range keys live, then resolves qualifying rows into RowIds.
- Designed for date, numeric, and ordered key ranges.
- Small model footprint with bounded lookup correction.
- Combines with dense-vector, sparse, equality, and substring filters.
Substring
FM-index containment
BWT plus wavelet-tree substring search lets containment queries use an index instead of falling back to a full scan.
- Handles substring containment without adding a separate search service.
- Converts text hits into RowIds for hybrid set math.
- Complements SPLADE-style sparse retrieval and HNSW vector search.
Dense vectors
HNSW approximate nearest neighbor
Semantic similarity search lives beside text, equality, and range filters instead of requiring a separate vector database.
- ANN candidate sets resolve to RowIds like every other access path.
- Supports dense-vector retrieval workflows.
- Intersect ANN with substring, bitmap, range, and sparse filters.
Mutable sorted data
PMA cache-oblivious runs
Packed memory arrays support mutable sorted structures without tuning for a specific cache size.
- Designed for cache-oblivious ordered maintenance.
- Helps connect write-friendly ingestion with sorted-run reads.
- Another path into the shared RowId set.
Sparse text
SPLADE-style sparse top-k
Learned sparse retrieval brings inverted-token scoring into the same engine as dense vectors, substring search, and ordinary filters.
- Inverted token lists score top-k by sparse dot product.
- Works with dense-vector and substring retrieval instead of replacing them.
- Designed for retrieval-heavy embedded applications.
Set similarity
MinHash LSH
MinHash supports approximate set similarity for deduplication and join-style retrieval, then returns RowIds like the other access paths.
- Useful for near-duplicate detection and set joins.
- Works as an LSH filter before exact checks.
- Keeps set-similarity candidates inside the same RowId model.
Query model
Combine constraints before rows are decoded.
A vector match, substring filter, tenant predicate, and time range can all be represented as candidate RowId sets. MongrelDB intersects those sets first, then decodes the rows that remain.
HNSWvector ANN
FMsubstring
Bitmapequality
PGMrange
Shared RowIdcandidate intersection
ann_search(embedding, q, 50)
fm_contains(body, "needle")
tenant = "acme"
created_at BETWEEN a AND b
Storage
01
02
03
04
05
Durable writes. Efficient scans.
The write path is log-structured. The read path resolves predicates to shared RowIds, decodes only the columns needed for the result, and keeps frequently requested answers close by.
WAL group commit
Durability starts with an append-only write-ahead log that can batch fsync work.
Bε-tree memtable
A composite-key MVCC skip-list memtable absorbs updates before data is flushed into immutable runs.
PAX columnar .sr runs
Sorted-run pages support scans, compression, and projection pushdown in the embedded storage layer.
Hybrid pushdown
Equality, range, substring, vector, and sparse matches can be intersected before row decoding.
Arrow + DataFusion
SQL, Arrow IPC, and Node-native bindings provide familiar ways to work with local data.
Columnar scans without giving up single-row writes.
MongrelDB keeps durable writes in front of compressed runs, so applications can support updates and analytical reads in the same local store.
Fast reads without another service.
Memory-mapped runs, adaptive encodings, projection pushdown, page pruning, and a warm result cache keep read paths inside the embedded engine.
Encryption
Encrypted storage with practical query support.
MongrelDB is designed to protect stored pages, authenticate metadata, and still let trusted predicates narrow work without decrypting the entire dataset first.
Authenticated pages. Searchable tokens. Explicit tradeoffs.
Page-level AES-256-GCM protects sorted-run payloads, WAL segments, result cache entries, and index checkpoints. Run metadata is authenticated so tampering can be detected on open.
Search while encrypted
Encrypted-indexable columns expose query tokens so equality and range predicates can resolve candidate rows without requiring a full decrypt-first scan.
Tamper-evident runs
Run metadata is authenticated, while encrypted page payloads are individually protected with AES-GCM tags.
Key hierarchy
- Passphrase plus salt derives a table-level KEK through Argon2id and HKDF.
- Per-run DEKs protect page payloads.
- Separate domains protect WAL, result cache, index checkpoints, metadata MACs, and per-column tokens.
Compression choices
Delta, Dictionary, Zstd, and passthrough encodings can be selected per column, while memory-mapped runs and a metadata count path keep the embedded profile lean.
Developer API
Embedded deployment. Modern interfaces.
Use SQL when it is enough, native calls when query shape matters, and Arrow when columnar data needs to move efficiently.
Use it when a local database is the simpler architecture.
MongrelDB fits local-first apps, edge jobs, desktop tools, test harnesses, agent workflows, and Node services that benefit from embedded storage without a database server to operate.
Local storeDataFusion SQLNAPI addonAsync APIBigInt RowIdsArrow IPC
// Open one local database. Combine multiple access paths.
const db = openMongrel("./app-data");
const hits = await db.hybridQuery({
ann: { column: "embedding", k: 50, vector: query },
contains: { column: "body", text: "needle" },
eq: { column: "tenant", value: "acme" },
range: { column: "created_at", between: [from, to] },
return: ["rowid", "title", "score"]
});
console.log(hits.length);
Build with MongrelDB.
Explore the code, run the benchmarks, and try it against a workload that needs local writes, hybrid search, and analytical scans in one embedded database.