What one fsync buys you

A transaction in MongrelDB commits when its records are appended to the WAL and that WAL segment is fsync’d to disk, full stop. The data pages, index pages, and column metadata are updated in memory first and flushed to disk afterwards by a background writer, but they do not block the commit. If the process crashes right after the fsync, recovery replays the WAL from the last checkpoint and reconstructs the same pages; if the process crashes before the fsync, the transaction never happened. There is no intermediate state where the database is confused, because the WAL is the only source of truth that matters at crash time.

The latency win is straightforward. A single fsync on a sequential WAL append is one disk seek on a rotating drive or one flush on an SSD, and on modern NVMe it is typically sub-millisecond. Writing and fsyncing the actual data pages would mean one or more random writes per modified page, each with its own rotational latency or flash write-amplification cost, and those costs multiply quickly on a transaction that touches many rows or many indexes. By separating “must survive crash” from “nice to have on disk,” the commit path stays flat even when the transaction is large.

Throughput benefits too, because group commit can share one fsync across multiple transactions. When several transactions are ready to commit at the same time, MongrelDB batches their WAL records into the same segment and pays one fsync for the batch instead of one per transaction. The published cargo bench -p mongreldb-core --bench write_path numbers show group commit at roughly 686 µs for 1,000 rows, which is the fsync cost amortized across many writers; the single-row no-fsync put is around 618 ns, and the durable commit is around 6.79 µs, because that measurement is the fsync-bound path. The difference between a single-row commit and a group commit is how many transactions ride inside that single disk barrier.

What the rest of the storage layer does asynchronously

Data pages are written out by a background thread that watches dirty-page counts and checkpoint intervals, and the rule it follows is simple: keep the WAL from growing forever, but never let a page reach disk before the WAL record that describes its change. That ordering is what makes recovery possible; if a data page were written before its WAL record, a crash between the two would leave an updated page with no log to explain it. The background writer respects the log-sequence-number ordering, so pages are flushed only after their covering WAL records are durable.

Checkpoints are the mechanism that bounds recovery time. A checkpoint marks a point in the WAL before which all dirty pages are known to be on disk, which means recovery only has to replay from the last checkpoint rather than from the beginning of time. MongrelDB checkpoints on a configurable interval and on dirty-page thresholds, and the checkpoint itself is a lightweight metadata write; the heavy work of flushing pages happens continuously in the background, so checkpoint stalls are small.

The tradeoffs that come with this design

Recovery time is the obvious one. Because data pages are not necessarily on disk at commit time, a crash means replaying every WAL record since the last checkpoint, and the length of that replay is the product of your write rate and your checkpoint interval. On a write-heavy workload with a long checkpoint interval, recovery can take seconds or minutes, which is fine for most applications but unacceptable for a system that needs sub-second restart. The fix is shorter checkpoints, but shorter checkpoints mean more foreground disruption and more write amplification; that is the standard WAL checkpoint tradeoff and MongrelDB does not pretend to have escaped it.

The second tradeoff is that the single fsync is still the latency floor for synchronous commits, and on a filesystem or storage stack with slow fsync behavior, that floor is visible. Some filesystems stall fsyncs while other writes drain, some virtualized disks lie about fsync completion, and some network-attached storage introduces multi-millisecond latency on every barrier. MongrelDB cannot make a bad disk fast, but it can make sure you only pay that cost once per transaction rather than once per page.

The third tradeoff is that group commit adds jitter. A transaction that is ready to commit just after a group has been dispatched waits for the next fsync window, and that wait is bounded by the group-commit timeout. For latency-sensitive single-row writes, the default timeout is short enough that the jitter is small; for throughput-bound workloads, the batching is the whole point. The settings are exposed if you need to tune them, but the default is biased toward the common case where one slow fsync is worse than a few microseconds of scheduling delay.

Why this is the modern equivalent of an old pattern

In the simpler embedded engines of that era, the way to get durability was often to open the data file with O_SYNC or O_DIRECT and pay the synchronous write cost on every page, because the code path was simpler and the gap between WAL and direct-write was smaller on the hardware available then. Modern equivalent is WAL fast-commit: the same crash-recovery guarantee, but the synchronous work is concentrated in one sequential append instead of scattered across random writes, and the storage layer can schedule the rest of the work when the disk is least busy. The guarantee did not change; the shape of the work did.

What an FM-index is, briefly

A Burrows-Wheeler transform turns text into something that compresses as well as the same text runs through gzip, because the BWT groups repeated substrings the same way. The FM-index adds enough suffix-sampling metadata on top of the BWT that you can test whether a pattern is a substring by walking the BWT’s first-column matrix one character at a time, without scanning the original text at all. The asymptotic shape is O(|pattern|) per query against a corpus that is highly compressed on disk.

Inverted indexes, by contrast, store the original tokens uncompressed, hash each token to a postings list, and answer a query by intersecting those postings lists in constant time per term. They are the right choice when your data is largely read-only, your queries are repeated, and you can afford to rebuild the index out of band after writes. They are not the right choice for a transactional engine that wants the index to participate in the same commit as the rows.

What the FM-index bought us at the storage layer

Two things specifically.

The first is write-path simplicity. An inverted index wants to be immutable between commits, which is why transactional inverted indexes need separate WAL coordination, deferred rebuilds, or row-level deltas stitched onto an immutable postings structure. An FM-index in MongrelDB is materialized as a column-level rewrite, batched with the rows it covers, fsync’d through the same WAL, and rolled back by not rewriting it; the index has no separate consistency model because there is no separate index state to coordinate.

The second is the readability of the data on disk. The same BWT that makes the FM-index queryable is the same representation that compresses text well, and MongrelDB’s storage layer already pays for compressed column layouts in the rest of the engine; the search side gets that compression benefit alongside the rest of the engine’s columnar storage without a separate index cost.

That is the trade we wanted: one source of truth, one consistency model, one wire format. The FM-index was the data structure that gave us all three; the inverted index would have given us one and a half and required a separate subsystem for the half.

What we had to give up to get there

Build cost. Inverted indexes get built incrementally as documents are inserted, and the index is correct after every INSERT. The FM-index is meaningful only over a corpus; rebuilding it across a single row is wasteful. MongrelDB batches FM-index rewrites across table-rewrite checkpoints, with the same IndexBuildPolicy::Deferred default the SQL indexes use; this means write amplification on full-text-heavy workloads is higher than on inverted-index engines, and first-query-after-batch latency can spike when a checkpoint has not been built yet. The benchmark numbers are honest about this; the architecture is honest about why.

Pure-pattern ranking. The FM-index gives you containment (does the pattern appear?), not relevance (how much does it matter?). BM25 ranking is available via the mongreldb_fts_rank(text, query) UDF, which computes simplified BM25 from per-row term frequency and approximated corpus statistics at query time; accurate BM25-with-IDF lives in the separate fts_docs virtual table, which is the inverted-index path. That is a hybrid shape, not pure FM-index purity, and we note it because a search-engine purist would.

Where it shows up in the wire format

WHERE body LIKE '%phrase%' in raw SQL, ->whereFts(...) in the Kit query builder, and where('fm', ...) in the PHP fluent builder all dispatch to the same FM-index path through /kit/query. There is no separate search endpoint to point your app at, no separate consistency model to reason about, and no eventual-index lag to defend against in application code. A search query joined against the same WHERE clauses you would write for a regular typed column looks like:

SELECT id, title
FROM articles
WHERE body LIKE '%rust bwt fts%'
  AND published_at > '2026-01-01'
ORDER BY mongreldb_fts_rank(body, 'rust bwt fts') DESC
LIMIT 20;

Same transactional commit at the end of the statement, same WAL coverage for the index update on any INSERT to articles, and the same single-engine consistency model for both the row and the search result.

When this is the wrong tradeoff

If you are running a pure search-engine workload with no transactional semantics on the indexed data, with hot-write patterns the FM-index chokes on, and with strict sub-millisecond p99 on very short queries, an inverted-index engine tuned for that workload is still faster than what an FM-index on a transactional engine gives you; we did not pretend otherwise when we built it. What we wanted was transactional full-text search on the same engine as the rest of your data, on the same wire format, and the FM-index was the right way to spend the storage budget to get it.

What “in-process” actually buys you

Two things specifically, and they are the same two things every in-process database call gets when the network is gone.

The first is a round trip. A single HTTP request from any Tier 2 client to a mongreldb-server over loopback measures p50 ~0.22 ms / p99 ~0.62 ms at ~4,000 ops/s for PUT /tables/{name}/put and POST /kit/txn commit on the published BENCHMARKS.md; the equivalent call into the engine through the in-process Tier 1 binding (Rust, PyO3 for Python, NAPI for TypeScript / Node) drops the per-call cost into single-digit microseconds, with the same 6.79 µs commit (group-committed, fsync’d) bound from the write-path numbers in the same file. The network is the dominant cost in the Tier 2 path; axum handler dispatch, JSON encode/decode, and the TCP round trip add up, while the engine work underneath is sub-10 µs.

The second is transactional consistency. The HNSW graph and the row it covers get the same atomic-write semantics; either both are durable, or both are not. There is no “the database committed but the vector index lagged” state for the engine to defend against because the engine is one engine. CDC pipelines, outbox consumers, and dual-write coordinators exist in application code only because two systems need to look like one; they stop existing when the two systems collapse back into one.

The actual numbers

The HNSW graph is a column-level index in the same engine files as the rows it indexes. A vector insert inside a transaction rides the same single-digit-microsecond put path the published cargo bench -p mongreldb-core --bench write_path numbers report (put no-fsync at 618 ns, commit fsync at 6.79 µs, group commit at 686 µs for 1,000 rows). A k-nearest-neighbour query in the same transaction rides the same sub-10 µs filter path the cold-SQL-filter number (8.7 µs at N=1M) tracks. The HTTP loopback path costs the same 0.22 ms p50 / 0.62 ms p99 as every other Tier 2 op, because the dominant cost in that path is the network, not the index; HNSW-specific benchmark numbers will land in BENCHMARKS.md when the HNSW build policy is committed, and we will not invent them in the meantime.

What this looks like from a where('ann', ...) call

The fluent query builder in mongreldb-php pushes where('ann', ...) to the same HNSW index through the same /kit/query path as a regular typed column. A query for ten nearest neighbours of a 384-dimensional embedding against a documents table, filtered by tags = 'engineering', looks like:

$rows = $db->table('documents')
    ->where('tags', 'engineering')
    ->whereAnn('embedding', $vector, k: 10)
    ->get();

That call, against a mongreldb-server running on the same host, is one HTTP round trip; against an in-process Tier 1 binding it is one function call into mongreldb-core. Both paths return the same rows with the same transaction isolation; the only difference is the round trip.

When this is the wrong shape

If your workload has a single vector index that already exceeds what fits in the memory of one server, and you genuinely need sharded vector search across dozens of machines, an HNSW graph that lives inside one engine process is the wrong shape; you want a vector service that is built to shard, and we will not pretend otherwise. One engine process means one HNSW graph in memory, and that is the limit on a single-machine deployment. What you get by staying inside one process is that the row that gets updated with a new title is the same transaction that updates the row’s vector, the same WHERE clause that filters by tags = 'engineering' also filters by approximate-nearest-neighbour distance, and the same /healthz endpoint that tells you the database is up also tells you the vector index is up; for most applications that is a much better set of defaults than another microservice to operate, and when a workload outgrows it the same client code paths through the same wire format keep working against a vector service with no application-side rewrite.