Four things that passed and shouldn't have
We publish downtime honestly because that’s the brand. This one is about a quieter kind of honesty: four separate things in the same sprint looked verified and weren’t, and all four are worth naming specifically, because “add more tests” is not the lesson any of them actually teaches.
1. A screenshot check that passed on a black frame
The project has a screenshot harness — stage a scene, capture it, exit — inherited from an earlier project specifically so visual regressions could be caught automatically. Its pass condition, as written, was: does a file exist, and is it bigger than zero bytes.
A capture run against a server that wasn’t even running produced a pure black image from a client that never connected, spawned no avatar, and rendered nothing but a directional light pointed at nothing. That run exited successfully. A file existed. It had bytes in it. The check had no opinion about what was actually in the frame, so it had nothing to say about a frame containing nothing.
2. The fix for #1, which only tested the case it had just fixed
The obvious fix is a background check: measure the color of an empty scene, and fail if too much of the frame matches it. That shipped, and it correctly rejected the specific black-ish gray an empty scene actually renders as.
It did not reject literal black. The measured empty-scene color and pure
(0, 0, 0) are different enough that a truly dead frame — the exact failure
case that motivated the fix in the first place — scored as 100% “content”
and passed clean. The fix closed the hole it was built to close and left an
adjacent, more literal version of the same hole wide open, because the only
failure case anyone had actually reproduced by the time the fix was written
was the gray one, not the black one.
The eventual real fix doesn’t enumerate failure colors at all. It asserts what should be present — desaturated terrain pixels above a brightness floor, a specific color relationship for the thing that’s supposed to be there — so an empty frame fails by construction, not by matching a list. Both the gray case and the black case are now pinned by a test that reproduces each one for real and confirms both score zero.
3. A test that “proved” a rule-breaking configuration works
Separately, a test existed asserting that cargo could be lifted three cells up through stacked Lifter blocks and continue into a belt. It passed. It built its evidence for that by constructing the block layout directly in code and never going through block placement at all — which mattered, because the actual placement rule in the game made stacking one block directly on top of another physically impossible. The test proved a configuration the game itself would refuse to let a player build. Once placement was fixed to allow stacking (a separate, later change), it exposed an unrelated grammar violation that had been sitting underneath the whole time — see the M0.3 post for that half.
4. A decision record asserting something nobody had run
The design record that specified this sprint’s terrain claimed a specific elevated plateau existed “to give the Lifter a reason to exist” — that without climbing it, the only route from the resource to the delivery point would be blocked. That sentence was false the day it was written, and stayed false through two closed issues, because nothing in the terrain actually prevented cargo from routing around the plateau instead of over it — and no block in the game can lower cargo back down once it’s been lifted, which the record itself didn’t notice meant a lifted route could never reach ground level again anyway.
Nothing caught this because nothing tried. The existing test that seemed to confirm the Lifter mattered only proved cargo could go up — which it can — never that going up led anywhere useful. A decision record is not exempt from the same rule that applies to code: a claim nobody has actually run is a guess wearing a citation.
The actual lesson
Not one of these four is “write more tests.” Two of them were tests, and passed. The pattern underneath all four is the same: each check was built to catch the specific failure someone had already seen, and each one was silent about the nearest neighboring failure nobody had reproduced yet. The fix, in every case that actually held up, was switching from “does this not match the bad thing I already saw” to “does this match the good thing that’s supposed to be true” — a positive assertion instead of a blacklist. That’s a more expensive thing to write and a much harder thing to fool by accident, which is presumably why it’s not the default.