Someone else's keystroke

You type in a cell. Two hundred milliseconds later it is on your colleague's screen, in a grid they never refreshed. That is a small thing to describe and a surprisingly large number of decisions to get right.

Two streams, not one

A change is published to a workspace stream and, when it belongs to one, to a view stream. The split exists because those two audiences want different things. Someone with the Orders view open cares about every row in Orders. Someone in Customers does not, and sending it to them anyway means every open tab in the workspace wakes up for every keystroke anywhere in it.

Which view an event belongs to is derived rather than declared: an explicit viewUuid in the event's payload if there is one, otherwise the entity id when the entity is itself a view. That leaves an obvious question — what happens to an event that names no view at all, like renaming the workspace?

It is dropped from the view streams. Not fanned out to all of them. That is the tempting bug, because "no view" and "every view" are one if apart and the fan-out version looks like it works: everyone gets everything, everything appears, nothing seems wrong until a busy workspace starts pushing every event to every tab.

Isolation is the part that matters

Everything else on this page is a performance or comfort question. One property is not: a connection must never receive an event for a workspace it did not subscribe to.

A bug there is not a slow grid. It is one customer watching another customer's activity, in real time. Which is why the tests below spend more effort on what does not arrive than on what does — and why writing them turned up a way that property can fail today, described at the end of this post.

The broadcaster itself holds the line: an event goes only to connections registered under its key. What the tests could not vouch for is whether the key a connection is registered under names exactly one tenant.

Staying alive is its own problem

An idle TCP connection is something proxies, load balancers and corporate firewalls like to close. So every thirty seconds the server sends an SSE comment — a payload-free line that clients ignore entirely and intermediaries count as traffic.

The same pass does the housekeeping. A connection that has been closed is dropped from the set. A connection that throws when written to is dropped and closed, so one dead client does not stall the loop for everybody behind it. And when a workspace's last connection goes, its entry is removed rather than left behind as an empty set — a per-workspace leak that would be invisible until the memory was gone.

The client half

Two details on the browser side are worth knowing, both of which took a bug to find.

The stream is opened with fetch, not EventSource. EventSource cannot send an Authorization header, and the stream needs one. The cost of that choice is that the stream bypasses the interceptor that normally notices an expired token and refreshes it — so the stream has to do that itself, which is precisely what it failed to do for anyone who left a tab open overnight.

And your own changes are applied optimistically, then confirmed. Events carry the id of the user who caused them, and your client discards its own — otherwise every edit would arrive back at you and fight the local state you already applied.

Who is allowed to connect

Isolation is enforced by the broadcaster, but only over connections that exist. What decides whether one may exist is a separate check, and it refuses in three distinct ways.

An unauthenticated request is refused, which is unremarkable. A selection token is refused too, and that one is worth naming: it is the token you hold after logging in but before choosing an organisation. It carries a valid subject, so a check that asked only "is this person authenticated" would let it open a stream. And a non-member of the organisation is refused even though both of the above passed.

For workspaces there is one more step, and it is a deliberate asymmetry. An organisation's owners and admins may open a stream on any workspace in that organisation without being explicitly added to it; everyone else needs a workspace membership. The bypass is scoped to the organisation that owns the workspace rather than the one named in the URL — so being an owner somewhere does not make you an owner everywhere.

That is now tested, including the case that would matter most if it broke.

What running them found

Rewriting the three tests was not the end of it. The first honest run failed — and not because the tests were wrong. The API's cell edits, row inserts, batch edits, column fills and M2M updates were never broadcasting at all: attribution read the caller's identity inside a post-response lambda, which lands on the IO thread, where lazy authentication throws — into a catch that logged a warning nobody read. Your own edits looked live because your browser applies them optimistically; everyone else's tab stayed still until refresh. The identity is now resolved reactively at the start of the chain and carried in, and the second honest run is what proved the fix.

The three tests that couldn't fail

Now the honest part, and the reason this post exists at all.

Until today the broadcasters had no tests. That alone is unremarkable. What is worth writing down is the coverage that appeared to exist. Three end-to-end tests, all passing, none of which could fail:

• One asserted that an error banner had a count of zero. The banner is absent when the connection is healthy — and equally absent when the app never attempted to connect, or when the element was renamed. It passed with real-time dead.
• One inserted a row through the API, waited, and if the row had not appeared reloaded the page before asserting it was there. Which proved the row was in the database, something the API call had already established.
• One asserted count() was >= 0. A count is always at least zero. That assertion could not fail for any input, any state, any bug.

Green suite, three tests named after real-time, zero of them testing it.

There are now 55 unit tests across the module — 26 over the two broadcasters covering delivery, cross-workspace and cross-view isolation, the no-view-means-no-broadcast rule, pruning of closed and failing connections and the connection-count bookkeeping; 15 over the gate above, and 7 over the keys the resources route on. And because a passing test proves nothing about whether it can fail, all four behaviours were checked by breaking the thing they watch: keying every connection off the wrong workspace fails five, making the broadcast deliver nothing fails nine, allowing selection tokens through fails one, and treating an ordinary member as an admin fails two. The three end-to-end tests have been rewritten to assert something — the stream opening, and the row arriving without a reload.

That is the lesson worth more than the feature: a check that cannot fail is worse than no check, because it occupies the space where a check would go. Nobody writes another test for something that already has three.

What it doesn't do (yet)

• Workspace slugs stopped being tenant identifiers, and the stream had to learn that. A migration made slugs unique per organisation — precisely so two organisations could each have an acme-store — while the workspace stream still addressed events by slug alone, so for a while two such organisations received each other's workspace-level events (view names, member changes, key lifecycle; never row data, which travels view-scoped under a globally unique uuid). Found by writing the tests above, closed in two moves: first a guard that refused to deliver into a bucket spanning organisations — dropped events being the safe failure, cross-tenant delivery the unsafe one — and then the real fix, every workspace-stream event stamped with its organisation and delivery filtered on it, so colliding slugs now each receive exactly their own events. The guard stays as the fallback for an event without the stamp, which after the transition should be none.
• Replay covers short gaps, honestly. Every delivered event now carries a monotonic id, and a reconnecting client presents the last one it saw; the server replays what it missed — tenant-filtered, like live delivery — and says replay.complete so the stale banner comes down by itself. The honesty is in the failure: a window the buffer cannot vouch for (evicted, or a server restart) is answered with replay.gap and the banner stays, because replaying a plausible-but-partial history would be worse than asking for a refresh. The buffer holds the last 512 events per stream; a laptop asleep all night still gets the banner.
• No presence. You cannot see who else is in the view, where their cursor is, or what they are editing. You see the result of their edit, not them.
• Last write wins. Two people editing the same cell within a second of each other is resolved by arrival order, with no merge and no warning.
• New rows wait for you. Rows added by others appear as a count you click, not as rows that shove themselves into the middle of what you are reading. Deliberate, but it does mean the grid is not always literally live.