<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Group-Commit on MongrelDB</title><link>https://www.mongreldb.com/articles/tags/group-commit/</link><description>Recent content in Group-Commit on MongrelDB</description><image><title>MongrelDB</title><url>https://www.mongreldb.com/assets/og-mongreldb.png</url><link>https://www.mongreldb.com/assets/og-mongreldb.png</link></image><generator>Hugo</generator><language>en-US</language><lastBuildDate>Sun, 02 Aug 2026 12:00:00 -0500</lastBuildDate><atom:link href="https://www.mongreldb.com/articles/tags/group-commit/index.xml" rel="self" type="application/rss+xml"/><item><title>WAL Group Commit: Accepted Writes and Durable Commits</title><link>https://www.mongreldb.com/articles/2026/07/wal-fast-commit-one-fsync-one-transaction/</link><pubDate>Fri, 17 Jul 2026 09:30:00 -0500</pubDate><guid>https://www.mongreldb.com/articles/2026/07/wal-fast-commit-one-fsync-one-transaction/</guid><description>MongrelDB separates accepted writes from fsync-backed durable commits, batches WAL work with group commit, and flushes log-structured columnar runs later.</description><content:encoded><![CDATA[<p>A database benchmark can make a write look impossibly fast by timing the moment an in-memory structure accepts it, or impossibly slow by forcing a storage barrier after every row, and both numbers can be correct while answering different questions; MongrelDB exposes that distinction directly, because <code>put</code> stages work on the write path while <code>commit</code> is the point where the WAL must cross the durability boundary.</p>
<p>The current release-build measurements on the documented Linux benchmark host put an accepted single-row write without fsync at 4.4828 microseconds and a durable commit at 4.6721 milliseconds. The three orders of magnitude between them are not a mystery in the storage engine. They are the storage device, filesystem, kernel, and controller agreeing that the log reached stable media.</p>
<h2 id="what-the-wal-guarantees">What the WAL guarantees</h2>
<p>A transaction stages row versions against the current database state, then commit serializes the authoritative change into the append-only write-ahead log. A commit acknowledged as durable has passed the WAL fsync. Recovery can replay committed log records that have not yet reached immutable sorted runs, while incomplete or uncommitted work does not become a visible transaction merely because some bytes were written.</p>
<p>The WAL is not the final table format. MongrelDB is log-structured: committed changes feed a Bε-tree memtable keyed by RowId and epoch, then flush into immutable <code>.sr</code> sorted runs using PAX-style columnar pages. Readers merge the mutable layers and runs under MVCC, so a snapshot sees the row version appropriate to its epoch while compaction and later writes continue.</p>
<p>That architecture keeps synchronous commit work narrow. It also makes maintenance part of the design rather than an optional cleanup script, because many small runs eventually cost more to merge at read time than one compacted run.</p>
<h2 id="group-commit-changes-throughput-not-physics">Group commit changes throughput, not physics</h2>
<p>When multiple committers reach the durability boundary together, the engine can append their WAL records and share a storage flush instead of issuing one fsync per transaction. The device still has to make the bytes durable, but more useful work rides inside one barrier.</p>
<p>That is what group commit is for. It does not make fsync take microseconds on hardware where fsync takes milliseconds, and it does not turn an accepted pre-fsync write into a durable one. It amortizes the barrier across concurrent work.</p>
<p>The published one-million-row and write-path measurements include:</p>
<table>
	<thead>
			<tr>
					<th>Operation</th>
					<th style="text-align: right">Time</th>
					<th>Meaning</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>Put without fsync</td>
					<td style="text-align: right">4.4828 microseconds</td>
					<td>Accepted into the write path, not yet durable</td>
			</tr>
			<tr>
					<td>Commit with fsync</td>
					<td style="text-align: right">4.6721 milliseconds</td>
					<td>Durability barrier included</td>
			</tr>
			<tr>
					<td>1,000 puts plus commit</td>
					<td style="text-align: right">7.7071 milliseconds</td>
					<td>Batch work plus one commit, 129.75 K rows/s on that host</td>
			</tr>
			<tr>
					<td>Durable update on a flushed 100-row table</td>
					<td style="text-align: right">4.2844 milliseconds</td>
					<td>Read-modify-write plus durable commit</td>
			</tr>
	</tbody>
</table>
<p>These are engineering measurements from one Intel Core Ultra 9 386H machine with local NVMe storage, not a promise that another filesystem, virtual disk, laptop power mode, or cloud volume will flush at the same latency.</p>
<h2 id="flush-compaction-and-garbage-collection">Flush, compaction, and garbage collection</h2>
<p>A durable WAL can recover data, but keeping every historical WAL segment forever would make startup and disk usage increasingly unpleasant. MongrelDB flushes mutable state to immutable sorted runs, compacts multiple runs into a new clean generation, persists the manifest swap, then lets garbage collection reclaim obsolete runs and stale WAL segments when pinned snapshots no longer need them.</p>
<p>The daemon checks for tables with enough accumulated runs and compacts them in the background. Short-lived embedded or CLI processes can close with an explicit flush and schedule periodic compaction. A compaction remains crash-safe by retaining the old run set until the new run is fsync&rsquo;d and the manifest points at it.</p>
<p>This is the log-structured trade: writes stay append-friendly and scans stay column-friendly, while the engine spends background IO merging generations. If the process opens, writes a handful of rows, and exits thousands of times without flush or maintenance, the run and WAL shape will eventually tell you that the deployment ignored half the storage model.</p>
<h2 id="accepted-is-not-durable">Accepted is not durable</h2>
<p>Applications should name their guarantees honestly. A telemetry buffer may accept a write and tolerate losing the final few milliseconds after power failure. A payment ledger may require each response to mean the WAL is durable. A bulk importer may stage a thousand rows and commit once because one barrier for the batch is both faster and easier to reason about than a thousand independently durable rows.</p>
<p>The API distinction lets the application choose that boundary instead of hiding it behind one overloaded word called <code>save</code>. The benchmark page uses the same language: accepted writes acknowledge before fsync, durable commits include fsync.</p>
<h2 id="what-to-measure-on-your-hardware">What to measure on your hardware</h2>
<p>Start with the deployment filesystem and storage class, because an NVMe workstation, an encrypted laptop volume, a container overlay, and a network-attached cloud disk can have radically different barrier behavior. Measure p50 and p99 durable commit latency under the expected number of concurrent writers, then add readers, compaction, backup, and the largest analytical query the application will run.</p>
<p>MongrelDB&rsquo;s mixed-load qualification makes the point. Four writers and four readers on a one-million-row workload recorded a 33.012 millisecond commit p50 and 176.885 millisecond p99 on the published host, far above the single-committer 4.6721 millisecond result because concurrency, snapshot retention, and shared resources changed the workload. Neither result invalidates the other; they answer different operational questions.</p>
<p>The runnable commands and machine details live in <a href="https://github.com/visorcraft/MongrelDB/blob/master/BENCHMARKS.md"><code>BENCHMARKS.md</code></a>. If a durability number appears without its fsync policy, transaction size, concurrency, filesystem, and storage device, it is not a database result yet, it is a caption waiting for the rest of the experiment.</p>
]]></content:encoded></item></channel></rss>