NoVacDB

PostgreSQL-compatible database in Go · built by Vikrant Kumar

A PostgreSQL-compatible database built from scratch in Go that updates rows in place and keeps old versions in an undo log, so tables don't bloat and never need VACUUM.

GoPostgreSQL wire protocolB+TreesWrite-ahead logMVCCLinux

Source code

The problem

PostgreSQL never changes a row in place: every UPDATE writes a new copy and leaves the old one behind. Tables and indexes bloat until VACUUM cleans them up, and on busy tables VACUUM itself becomes the problem. I wanted a database that speaks PostgreSQL's protocol, so psql, pgAdmin and existing drivers connect unchanged, but stores data the way InnoDB does: rows updated in place, old versions kept in an undo log only as long as a reader needs them. And it had to be written from first principles, with no third-party libraries.

Decisions and why

In-place updates with an undo log instead of PostgreSQL's append-only rows

Every row carries an 18-byte header: the transaction that last wrote it and a pointer to its previous version in the undo log. An UPDATE saves the old version first, then changes the row where it is, so a table that is updated a million times stays the same size. Readers that need an older version walk the undo chain instead of the table holding every copy.

Speak the PostgreSQL wire protocol, not a new one

Writing my own protocol would have meant writing my own clients too. Implementing PostgreSQL's protocol 3.0 (startup, simple and extended queries, cancel requests) means psql, pgAdmin and every PostgreSQL driver work on day one, and NoVacDB can be compared with PostgreSQL using the same tools.

Write-ahead log with checkpoints and full page images

A commit returns only after its log records are fsynced; data pages are written later by checkpoints. Recovery replays the log from the last checkpoint, and a full image of each page after a checkpoint protects against half-written pages. The log replays rolled-back transactions too, because later records can depend on their page layout.

Stable row IDs with forwarding stubs

When an update makes a row too big for its page, it moves and leaves an 18-byte stub at home. Indexes, undo records and future row locks keep pointing at one ID for the row's whole life, so moving a row never rewrites an index.

Readers never wait for writers

Each statement reads through a snapshot of committed transactions, with READ COMMITTED and REPEATABLE READ like PostgreSQL. A writer holds a writer slot, not the read lock, so queries keep running while a transaction writes, and rollback undoes changes in place instead of blocking everyone.

The hardest part

Proving that concurrency and crash recovery are actually correct. I built a model checker that runs one writer committing and rolling back random transactions while REPEATABLE READ readers compare every read, by table scan and by index, against the exact state their snapshot should see, under Go's race detector. It found two real bugs that ordinary tests had missed: a transaction ID that was briefly invisible as 'active' between being allocated and logged, so a snapshot could see uncommitted rows; and a commit that freed its own undo a moment before snapshots stopped needing it. Crash tests cut power at random points with torn writes and check that every committed transaction survives and nothing else does. Every step is also checked by planting 25-40 deliberate bugs and confirming the tests catch them.