PyDB: a database from scratch
A small database I wrote to learn how they work. It survives being killed mid-write.
Open source · my own project
The problem
I wanted to understand what a database does between a SQL statement and the disk, so instead of using one I built one.
What I did
- Stored data in fixed 8 KB pages, with B-Tree indexes for fast lookups.
- Saved every insert to a write-ahead log before counting it as done.
- Made checkpoints write a new file and swap it in, so the data file is never half-written.
The result
- Tests kill the writer mid-workload and mid-checkpoint, and no confirmed insert is lost.
- The first version failed that test and lost every committed write. The fix is on the blog.
- A TypeScript B+Tree built on the same ideas runs live in the Lab.
How it's built
Insert → log start → change pages in memory → log commit. On restart, redo every committed insert since the last checkpoint and drop the rest.
Built with Python, B-Tree, WAL, pytest, CI.