Ask a Postgres operator what their database is doing right now and they have a dozen places to look, from pg_stat_activity to the slow query log to whatever exporter is scraping the daemon that week, and ask the same question of an embedded engine and the honest answer has historically been a shrug, because the database is a library inside your process and there is no process boundary to hang a metrics endpoint on. That shrug is expensive in exactly the situations embedded engines get chosen for, since a desktop app, an edge gateway, or a CLI tool that feels slow has no DBA to blame and no dashboard to consult, just a process whose RSS keeps climbing while somebody guesses whether the problem is compaction, cache pressure, or a batch that never flushed. We got tired of guessing on our own engine, so Kit exposes the counters the storage engine already keeps for itself, per table, directly on the database handle, and this post walks through which numbers exist, what each one is actually telling you, and where the honest limits of the approach are.

Three numbers that tell you where the write backlog lives

Every row you commit travels the same pipeline inside the engine: it lands in the memtable, drains into the mutable-run tier when Kit flushes the table, and spills from there into a durable, checkpointed sorted run once the tier crosses its watermark, with the WAL carrying the durability guarantee the whole way down. Each stage of that pipeline has a counter, and the counters matter because a backlog at each stage means a different problem with a different fix, so lumping them together into one “write latency is bad” signal would throw away the part that actually tells you what to do.

tableMemtableLen is the count of row versions currently sitting in the memtable, which after a commit are durable through the WAL but not yet reorganized into a sorted run, and the reason you care is reopen cost: on disk, committed-but-unflushed writes exist only as WAL records, so a short-lived process that never flushes is stuck replaying the whole batch from the WAL on its next invocation to rebuild the in-memory indexes. A spike right after a bulk commit is completely normal, because Kit flushes after large batch commits on its own, but a floor that never falls back toward zero across the life of the process means flushes are not keeping up with your write rate, and that is the moment to look at setTableMutableRunSpillBytes rather than at your disk.

tableMutableRunLen is the middle tier, the versions that have drained out of the memtable but have not yet crossed the spill watermark that turns them into a real .sr run on disk, and tableRunCount is the number of sorted runs the table currently has, with a compaction target of one. That last number is the compaction debt made visible, since every additional run is one more sorted segment the read path has to merge across, so a run count that climbs and never comes back down is telling you compaction is losing the race against ingest, which is a different conversation than “the database is slow” and a much more useful one.

The cache counters, and why hitRate is the one to watch

On the read side, tablePageCacheStats hands back cumulative hits, misses, and a computed hitRate, where a hit means the lookup found a page visible to your snapshot in cache and a miss means the page was absent or too new for the snapshot and the engine went to disk instead. The hit rate is the single most honest answer to the question “does my working set fit in memory,” because it is measured at the point where the engine either did or did not avoid a disk read, not inferred from how much RAM the process happens to be holding. One caveat keeps that answer honest: a miss also covers the case where the page was cached but too new for your read snapshot, so under a heavy concurrent writer the hit rate can dip with capacity to spare, and you should read it against your write load rather than in isolation. A hit rate pinned near one with a growing dataset is a good day, a hit rate that slides while both your query pattern and your write rate stay flat is the cache telling you the working set outgrew it, and the Rust and Python surfaces also expose tryLockMisses, which counts lookups skipped because a cache shard’s lock was contended. That last one is a signal aimed at us rather than at you, since there is no user-facing shard-count knob today, so a sustained nonzero count is worth a bug report instead of a tuning session.

Two smaller counters round out the picture: tablePageCacheLen and tableDecodedCacheLen report the entries currently held in the page cache and the decoded-page cache, and one detail matters for how you read them, which is that both caches are shared across the tables of a database, so every table handle reports the same cache-wide occupancy rather than a per-table slice. That makes them health gauges for the database as a whole, and here is the full surface wired into a health snapshot in TypeScript, which is the shape we use in our own tooling:

import { KitDatabase } from "@visorcraft/mongreldb-kit";

const db = KitDatabase.openSync("./data", schema);

function tableHealth(table: string) {
  const stats = db.tablePageCacheStats(table);
  return {
    runs: db.tableRunCount(table),            // compaction target: 1
    memtable: db.tableMemtableLen(table),     // WAL-durable, not yet in a run
    mutableRun: db.tableMutableRunLen(table), // waiting on the spill watermark
    pageCacheEntries: db.tablePageCacheLen(table),   // cache-wide, shared across tables
    decodedEntries: db.tableDecodedCacheLen(table),  // same: shared decoded cache
    hitRate: stats.hitRate,                   // hits / (hits + misses), 0 when unused
  };
}

The Python surface is the same calls with snake_case names, and the page-cache stats arrive as a plain dict with hits, misses, try_lock_misses, and hit_rate keys, so a monitoring hook is a three-line function there too:

health = {
    "runs": db.table_run_count("widgets"),
    "memtable": db.table_memtable_len("widgets"),
    "hit_rate": db.table_page_cache_stats("widgets")["hit_rate"],
}

Counters only matter if they can change a decision

The reason these live next to the tuning setters instead of in a separate diagnostics crate is that a measurement you cannot act on is just anxiety with a number attached, so each counter has a knob that answers it: a sliding hit rate argues for growing the cache budgets, a memtable floor that never clears argues for lowering the mutable-run watermark with setTableMutableRunSpillBytes so staged rows reach a durable sorted run sooner, and a run count that climbs argues for looking at compaction settings or the index build policy rather than for buying faster disks. Back in the MySQL days we answered the same class of question with SHOW STATUS and a lot of squinting at Threads_running, and the modern equivalent is not a fancier dashboard, it is putting the counter on the handle next to the knob so the feedback loop fits in one file.

What these numbers are not

Honesty about limits is cheaper than discovering them in production, so here they are: the counters are process-local and cumulative since the engine opened, they carry no history and no aggregation across processes, and if you want fleet-wide dashboards you still export them yourself to Prometheus or a log line on whatever cadence suits you. What you get in exchange for doing that plumbing is that the numbers come from the engine that is actually doing the work, measured where the decisions are made, which is a better deal than SQLite’s near-silence and a different shape than Postgres’s daemon-hung statistics tables, and for an embedded library that is the right shape, because the engine already knew all of this and the whole feature is deciding not to keep it a secret.