Three weeks ago I wrote about six months of watching an AI-generated system fail without producing an error. Clean exit codes. Plausible numbers. Wrong answers.
I said the hard part wasn’t the failures. The hard part was getting the fixes back into the data the system had already damaged.
I want to show you what happened next, because it turned out I was still being too generous — with the system, and with myself.
Seventy-two notes that said nothing
On August 2nd I went to write a connection between two notes and found that neither of them had a body. Frontmatter, a mangled heading, and then white space where an argument was supposed to be.
I counted. Seventy-two.
They passed every check the vault runs. They had descriptions. They had types and topics. They had links pointing in and links pointing out. They satisfied every criterion my verification step lists except one — body supports the claim — which is the only criterion no script evaluates.
The distribution is the diagnostic: twenty-seven from a single extraction batch, thirty-two with no source recorded at all. That’s a pipeline failure, not scattered sloppiness.
Then I measured again, and both of my headline numbers were wrong.
Seventy-two empty bodies were sixty-five; seven were navigation hubs, which legitimately have no body. Forty-four corrupt headings were twenty-seven, because my regex for the corruption signature # U<lowercase> matched any word starting with a capital U. Seventeen of them were ordinary English. Undercapitalized traders. Union collective bargaining. Unsupervised anomaly detection.
The repair caught it only because I had written a per-file guard: rewrite the heading only when its first word matches the slug’s first word. Without that guard the bulk fix would’ve turned seventeen correct titles into # Ndercapitalized traders.
I wrote a sentence into that note at the time. A plausible number produced by a proxy that does not measure what its name claims. Eighth-plus instance.
I didn’t think it applied to what I was about to do.
The number went to zero
That note closed on August 16th. The clearing measurement, recorded in its own frontmatter: nodes with under forty characters of body, seventy-two to zero.
Body includes the link list.
The original count measured prose — the body with headings and link lists stripped out. The closing count measured body, and lowered the threshold while it was at it. Nothing in that note claims the prose came back. It says the opposite, in a section titled what was NOT done: re-extraction unavailable, no provenance left to re-read; deletion unavailable, because all twenty-five of the sourceless notes have inbound links and removing them would quarantine the notes that cite them. Neither recoverable nor removable. Sixty-five nodes that assert nothing, and a documented decision to accept them.
The prose didn’t return. The routes to return it stayed closed.
What moved was the ruler.
Five days later
On August 21st I ran a different measure — prose only, links and headings stripped — across 3,162 files.
Eleven notes came back with zero.
All eleven carried status: active, which in this vault means verified. All eleven were fully recoverable from git, at 1,236 to 2,530 characters apiece.
That last number is the one that matters. These weren’t notes that were never written. These were notes that had been written, and then emptied, and then marked as checked.
The anchor case is precise. One note held 3,348 bytes on April 15th. A commit on April 18th emptied it — a commit about abolition and Black Marxism, unrelated to anything the note was about. It then survived eight more commits, including one whose message reads: batch verify all 529 remaining draft nodes: promoted 528, verified all.
It was marked verified with nothing to check.
And here’s the property that kept it hidden: the defect is inversely correlated with detectability. The more links a destroyed note carries, the longer its body measures, and the healthier it scores. A 552-character body of pure links reads as adequately developed. The nodes most thoroughly connected to everything else were the ones best insulated from being caught.
I know what was in some of them, because git does.
A note on COMPAS producing twice the false-positive rate for Black defendants. A note on statistical patterns revealing discriminatory effects. A note on right-to-work law and organizational capacity.
The system marked those verified while they contained nothing. Not wrong. Nothing. A green check over a claim about algorithmic discrimination that had been hollowed out three months earlier by a commit that was thinking about something else.
What is actually better
I want to be accurate about the progress, because there’s some.
The first case in the original catalog took 101 days to surface, and the repair that ran the next day reported FAIL 82 → 0 while every body stayed empty. For three months after that, automated passes kept building carefully reasoned connections onto twenty-seven files with nothing in them.
This time: found on the 21st, restored on the 21st. Commit e81e54d4, eleven files, 141 lines spliced back out of git history. The detection carried its own recovery check, so the finding arrived with the repair route attached instead of a verdict of unfixable.
That’s a real improvement and I’m not going to undersell it. Detection got honest. It got same-day.
Now the part that hasn’t moved.
The guard I built after losing those first twenty-seven bodies — the whole point of which was to stop a write from destroying content — is gated behind a hardcoded path filter for */notes/*. My vault stores notes in nodes/. Nothing in the generator references the guard at all.
My own words, in my own repository, three weeks ago: a guard that reaches neither side, built by someone who had just written this entry.
177 commits since I published that post. The guard still can’t fire on the vault whose damage caused me to build it.
What I actually learned
The generator works. It produces a functioning system, reliably, and I’ve kept using it for six months because that’s true.
The failure is never the crash. It’s the check that reports green because someone — usually me, usually while trying to be rigorous — changed what the check was counting, and the new count was easier to satisfy, and nobody had to decide to lie.
I did that in my own vault. Twice. In the same file where I’d already named the pattern.
That’s not a story about a note-taking plugin. It’s a story about how easy it is to be the thing you can see clearly from the outside, and how little the seeing protects you.
The eleven are restored. The sixty-five are still there, asserting nothing, cited by notes that may be the only surviving record the claim was ever made.
I know exactly where they are now.