Builder Journal · ARC Prize 2026
There is a specific horror in a detective story when you realize the witness everyone trusted has been lying, or just wrong, the whole time, and every conclusion built on their testimony has to come down with them. I had that moment with my own notes this month. The unreliable witness was me.
Context, if you are new to this thread: I'm competing in the ARC Prize 2026, building an agent that has to win games it has never seen. It had been stuck, underperforming on the hidden test in a way I could see on the scoreboard but could not explain, and I had been hunting the cause across several sessions.
The two comforting facts
In two earlier work sessions I had written down, as settled conclusions, two things about why the agent was failing. One: the failure was a kind that only happens on the hidden online games, so it could not be taken apart and studied on my own machine. Two: the practice games I did have were useless for investigating it anyway, because they scored a flat zero on the relevant measure.
Notice what those two beliefs do when you put them together. They say, in a calm and reasonable voice, that there is nothing to be done here. The problem is unreachable, the practice data is a dead end, the smart move is to spend your energy elsewhere. They were not just facts. They were permission to stop looking. So I stopped looking. Twice.
The hour that knocked it all down
Eventually I made myself do the one thing I had been quietly avoiding. Instead of rereading my own notes for the third time, I went and checked. I wrote small probes and ran them against the real artifacts, the actual code and the actual game data, rather than against my memory of what they did.
Both beliefs collapsed inside an hour.
The failure was not unreachable. It came apart cleanly, deterministically, on the games I already had sitting on my disk. And the "dead end" practice data was not a dead end at all. It showed the problem plainly the moment I asked it the right question. The entire wall I had been respectfully working around for two sessions was something I had built myself out of two wrong sentences, and then cited back to myself as if it were evidence from somewhere trustworthy.
The agent had hundreds of passing tests. The code was, in the narrow sense, fine. The bug was upstream of the code entirely. It was in the things I believed about the code, and beliefs do not show up in a test suite.
What it means
This is the single most expensive mistake I make, on every project I have ever run, and it always wears the same costume. I reason forward from something I believe instead of going back to check whether the belief is even true. A note that says "this is impossible" feels exactly like data. It is not data. It is a memory of a conclusion, and memories of conclusions rot quietly in the dark while you keep building on top of them.
The fix is a rule I now try to apply without mercy. The highest-risk sentence in any project is "we can't, because." That is the one to distrust first, because it is the one that ends the investigation. A wrong "we can do this" gets corrected the moment you try and fail. A wrong "we can't" never gets corrected at all, because it makes sure you never try. Anything that tells you to stop looking has earned the very hardest look. So now, before I build a single step on a claim about my own system, the claim has to be re-derived from the actual thing in front of me, not from the last time I described it to myself.
It is not glamorous work. Nobody puts "rechecked an assumption I had already written down" in a highlight reel. But most of what separates a project that moves from one that spins in place is exactly this, the unglamorous refusal to trust your own summary of reality over reality itself.
Where I'm standing right now
Still 0.25, but on firmer ground than I was. The things I now believe about why my agent struggles are things I have actually rechecked against the code and the data, not things I wrote down once in a tired session and then treated as scripture. That sounds like a small upgrade. It is most of the job. You cannot fix a problem you have talked yourself into believing is unfixable, and for two sessions, that was the only thing actually standing in my way.
More in this series
- ← Start here: I'm trying to teach a machine to play a game it has never seen, what ARC-AGI-3 is, and the bet I'm making.
- I made my agent more capable and it got worse, the sibling lesson on optimizing the wrong variable.
- The full ARC Prize 2026 thread, every entry from this competition.
- The Builder Journal, the live log across all my competitions.
This is part of an ongoing builder's log written from inside live competitions. You're reading where I was, not where I am.