Architecture
From the server to SQLite to reactive Svelte state
In this post
We wanted Milieu to feel fast and dependable wherever you use it. You should be able to organise your day on a flight, look up a contact on the underground, or keep working through an internet outage. Even with a good connection, there’s no reason for every interaction to wait for a server.
So Milieu keeps a copy of your data in your browser, right beside the app, and most of what you do reads and writes that copy. What you change is saved in your browser first and synchronised with the server in the background.
But when you open Milieu in a new browser, it has none of your data yet, and you shouldn’t have to watch a loading screen while your account downloads.
Instead, we start you on the server. You can get going straight away while a background worker downloads your account’s SQLite database. Once that database has caught up with everything you’ve seen and changed, your tab switches to it. You won’t notice the switch. From then on, your browser keeps synchronising in the background, and carries on working if your connection drops.
The server always has the final say. Your browser keeps what the server has confirmed, plus its best guess at what your pending changes will do. Each time it syncs, it takes the server’s latest and works those guesses out again on top.
SQLite runs on both sides: inside the server process, and in your browser through WebAssembly, in a worker. There, a sync engine does everything this post describes. Three things tie the two sides together: the same SQL queries, your changes saved as operations that can be sent and replayed, and cursors that track how far your browser has caught up. We’ll come to each.
If you know Replicache or Zero, the guessing will look familiar: pending changes are replayed on top of the server’s state rather than merged with CRDTs. As with PowerSync, your browser runs real SQLite. What’s different is where you start. Your tab doesn’t wait for a local copy before you can work; it begins as an ordinary server-rendered app and moves onto your browser mid-session, without your screen ever jumping back in time.
Starting on the server
Your first page load is plain server rendering. Svelte’s asynchronous rendering waits for the page’s queries, and SvelteKit sends their results along with the HTML. Your browser picks the page up from that data, so it’s ready to use without downloading your account or asking for those results again.
Until your browser can answer for itself, reads go through SvelteKit remote queries and writes through remote commands. And the server sends hints whenever something changes, so edits made elsewhere show up on your screen.
Meanwhile, your browser fetches its own copy of your account.
Building your account file
A background worker on the server builds a SQLite file with the rows you’re allowed to see and the columns we sync, indexes already built. The file also records its cursor: a bookmark for exactly which server state it came from. Rows and cursor are read from a single snapshot, so they describe the same moment.
We compress and cache these files. Your browser can start from an older one and pull whatever’s changed since its cursor, so signing in doesn’t need a freshly built file.
One database for all your tabs
In your browser, the database and its outbox, the queue of changes waiting to be sent, live in the Origin Private File System, or OPFS. SQLite reaches them through synchronous access handles, which are fast but exclusive, and only available in dedicated workers. That means only one tab’s worker can hold the files at a time.
A Web Lock picks that tab, and that tab starts the database worker. A SharedWorker connects every other tab to it. However many tabs you open, they share one database and one sync stream.
That arrangement has to survive whatever your browser does with tabs. You might close the tab holding the files, or the browser might freeze it or tuck it into the back-forward cache. Another tab can then take the lock, and its new worker waits for the old one to let go of the files before opening them. If an existing database won’t open, we treat it as a storage problem rather than starting afresh, because that file may hold work you haven’t sent yet.
The same queries on the server and your browser
We write Milieu’s queries once, in Kysely, against relations such as visible_todos. On the server, those relations limit a query to the rows you’re allowed to see. In your browser, they’re just the local tables, which only hold your data anyway.
That way, one query definition serves the first render, remote reads and local reads. The query never needs to know whose data it’s reading, and components only ever see rows.
The two databases aren’t identical. Your browser holds a chosen slice of the server’s data, and the server’s handlers also check permissions and do things a browser can’t. Sharing queries and domain logic cuts the duplication inside those lines.
Keeping track of what your browser has seen
Every row your browser can receive belongs to an owner account. Your address book is yours; a shared project’s to-dos, events and discussions belong to whoever owns the project. Each owner has a counter. Whenever one of their rows changes, a database trigger bumps the counter and stamps the row with the new value. On each synced row, sync_owner says whose it is and owner_seq holds that stamp.
Your browser keeps one cursor per owner it can see: one for you, and one for each person who shares something with you. A pull asks for the visible rows stamped beyond those cursors. The server reads those rows and the new counters from one snapshot, and that matters: if your cursor ever moved past a change the response left out, your browser would miss that change for good.
Counters put changes in order without trusting anyone’s clock, and having one per owner means unrelated accounts never queue behind a single global sequence.
Handing over without going backwards
A downloaded file isn’t safe to read from just yet.
Say the file contains a to-do titled “Book train”. While it downloads, you rename it “Book train to Edinburgh” through a remote command. If your tab switched to the file now, the old title would come back.
So each tab keeps a mark: the furthest server state it has seen. Every remote query result, and every response to a direct write (a change your tab sends straight to the server before the handover), comes with the relevant owners’ counters, read after the operation, and the tab keeps the highest it has seen for each owner. Your browser’s cursor moves forward as its pulls complete. The tab hands over only when, for every owner in its mark:
browser_cursor[owner] >= tab_mark[owner]
- This tab The server renders your Today list mark 412
- Local database A worker downloads the account file, built a little earlier cursor 400
- This tab You rename “Book train” to “Book train to Edinburgh” mark 413
- Local database The worker pulls what changed since 400 cursor 413
- Handover The cursor has reached the mark: every query moves to the local database
- This tab Reads and writes are local; sync carries on in the background
It also waits for its own direct writes to finish, and the engine keeps other tabs’ direct writes in order, so local changes can’t overtake them.
Once it has, every shared query in your tab switches to your browser at once. Components never choose between server and local data for themselves, so you can’t see a screen that’s half one and half the other.
Coming back to a browser you’ve used before adds another case: its outbox may hold work the server has never seen. Those changes mustn’t vanish just because the server answers first. So while there’s anything in your outbox, the tab holds the server’s answers back for a moment and waits for your local database, which already includes them. And if the server can’t be reached, your existing local database answers straight away, without catching up first.
If your browser doesn’t offer persistent storage, as in some private windows, your tab simply stays on the server. Everything keeps working while you’re online; you can’t save changes offline.
From database rows to what’s on your screen
After the handover, query results flow straight into deeply reactive Svelte state.
Within a tab, each row is kept once, by its kind and key, so two live queries that both show the same to-do share one reactive object. An update changes that object’s fields in place. A component showing the title updates when the title changes; one showing other fields doesn’t.
Those objects carry on through the handover itself, so nothing on screen needs a separate version for local data.
Re-running only the queries that changed
Reactivity is only cheap if the database work behind it is. Re-running every query on the page after every change would get slower and slower as your account grows.
Instead, the engine records which tables and rows each write changed, and each query says which rows feed its results, including through other tables. For small changes, the engine compares those rows before and after the write to decide whether a query’s result could have changed. To see the earlier state, it briefly undoes the write: inside a savepoint, it applies the write’s changeset, SQLite’s record of exactly what the write changed, in reverse, takes a look, then rolls back.
Queries whose rows the write didn’t touch, before or after, are skipped. Grouped counts, like the numbers beside your lists, adjust by the difference instead of counting again. Larger changes fall back to broader invalidation. And when results reach your tab, the worker sends only the rows that changed, plus any change to their order.
Saving your changes in your browser
Say you tick off a to-do. That becomes an operation, “complete this to-do”, which has a decider: a function that reads the rows it needs and works out what to write. It’s a pure function of those rows, the operation’s arguments and an explicit context.
Time and randomness come in through that context. If a decider read the clock or picked a random value itself, replaying the same pending change later could give a different result for reasons that have nothing to do with the server.
The engine writes the predicted rows and the operation’s outbox entry in one SQLite transaction. SQLite’s session extension records the row changes as a changeset, with enough detail to reverse it. Your screen updates as soon as the transaction commits.
Those two records do different jobs. The operation is what your browser asks the server to do, and it’s what gets sent. The changeset is what the guess changed locally; it never leaves your browser, and it’s how the guess gets undone when the server’s answer arrives.
New things get UUIDs in the browser that creates them, so an offline change can refer to something without waiting for the server to name it.
Sending your changes to the server
Networks are unreliable: a request can succeed on the server while its response gets lost on the way back. When your browser pushes your changes, they run through the same handlers as every other write. A push can carry several batches of operations, each with its own identifier, and the server checks whether it already has an outcome for a batch in the same transaction that runs it.
So resending the same identifier returns the recorded outcome instead of running your changes twice. The server keeps each outcome until your browser confirms it has the answer, so even a browser that’s been offline for months gets that answer rather than a second run.
Pulling changes
A pull brings back the resulting rows and removals, so your browser never has to rebuild the server’s state by replaying its history.
Some tables travel with a parent instead of carrying their own stamps. Take a person’s contact details: they belong to the person, so changing a detail stamps the person, and your browser replaces the whole collection when the person arrives. The schema declares these relationships in one place, and the same declarations drive visibility, change tracking and replacement in the browser, so a new relationship can’t be added to one and forgotten in another.
Some changes cross owners. Moving a to-do between a personal and a shared project changes its owner, and taking away someone’s access writes a removal for their account.
A removal means your browser can no longer see a row, not that the row was deleted. A to-do in the trash stays in your browser so you can restore it; something you lose access to goes, with everything attached to it, on the next sync.
Replaying your pending changes
Say you complete a to-do on your phone while your laptop renames it. When your phone next pulls, it takes out your completion, applies the renamed row, then decides the completion again. The to-do ends up both renamed and done.
More generally, rows arriving from the server have to be reconciled with the changes you’ve made that it hasn’t confirmed yet. The engine does this in one transaction:
- Undo each pending guess, newest first.
- Apply the server’s changed rows and removals.
- Drop pending changes the pull already includes.
- Run the remaining deciders again, in their original order, against the updated rows.
- Save the new cursor.
Your screen only sees the committed result, never the moment in between with your changes taken out.
Replaying doesn’t guarantee a change succeeds. Permissions might have changed, or the thing you changed might be gone. Each operation’s decider works out what replaying means for it, and the server still has the final say on whether to accept it.
A successful push isn’t enough to drop a guess, either. Your browser keeps it until a pull includes the server’s matching state; otherwise an accepted edit could flicker away between the server’s reply and the next pull.
Requests can also overlap: a pull that started before a push finished may bring back an older snapshot, even if it arrives later. The engine tracks these overlaps and keeps the guess until a recent enough pull replaces it.
If the server refuses a change, or it still fails after five attempts, the engine removes the guess and keeps the change in Settings → Unsaved. From there you can send it again with the same identifier, or discard it.
Testing that everything agrees, and fixing it when it doesn’t
We test the browser’s engine against the server’s real implementation. Property tests generate sequences of operations and compare the browser’s guesses with the state after the server has run them and synced. They also compare the synced browser database with a freshly built account file, ignoring the stamps the server assigns.
Other tests check what each account can see, and that pulling changes bit by bit ends up with the same rows as a fresh file. These are the seams between operations, triggers, visibility rules and rows that travel with a parent, where every part can be right and the whole can still miss a change.
While you use Milieu, fingerprints catch what the tests don’t. When nothing’s waiting to be sent and the server’s state can be compared, the engine splits each table into ranges of about a thousand rows. Each range gets a row count and a 32-bit sum of row hashes over the synced columns, worked out the same way on both sides. They’re cheap checksums, not cryptographic proof, but they catch drift.
A range that differs is fetched and replaced without downloading your whole account, once we’ve checked neither side has moved on in a way that would make those rows stale. If the difference is too big to fix range by range, the engine downloads a fresh file, keeping your outbox and the old file until the new one is ready. Every repair is reported to us as a bug to investigate.
Updating your database without losing your work
When we change the database’s shape, migrations use the same trick as replaying: your pending changes are kept apart from confirmed data. Before migrating your local file, the engine takes out guesses made against the old shape. It applies and checks the migrations in a transaction, then decides your pending changes again against the migrated data. If a migration fails, the original file is still there, and the engine can fall back to a fresh download without losing your outbox.
Pull responses include a schema version, and the engine won’t apply rows in a shape it doesn’t understand. Whether a pending operation can still be sent is checked separately from the shape of rows, so your pending work can still go while the app and database versions catch up with each other.
What runs in your browser
Projects, to-dos, notes, contacts and calendars all run in your browser. Some things the server works out, like an item’s History, are kept once you’ve opened them rather than included in your account file.
All the complexity lives in the engine: deciding when your browser can take over, holding on to your pending changes, keeping requests in order, reconciling with the server and fixing drift. Pages just use shared queries and reactive rows, and never need to know which side answered. That’s what makes a brand new browser useful from the first page load, and lets one you’ve used before keep working with no connection at all.