Exit Code Zero: The Hidden Failure Mode Every AI Pipeline Inherits

Jeetendra Gangele is Cofounder and CTO of BluePill AI, leading engineering for AI consumer twins and synthetic consumer research.

I built an autonomous coding factory. You describe a change in plain English, and it clones the repository, makes the edit, runs the project’s own test suite, stands up a live preview environment and opens a pull request for a human to review. It worked. It opened three pull requests against our real codebases.

Each one carried this sentence in its description: “reviewed by a second independent pass before this pull request was opened.”

None of them had been.

Three Nested Failures, None Of Which Crashed

I found this while fixing something unrelated, which is the only reason I found it.

The pull request was created before the reviewer ran. About 120 lines earlier in the same script. By the time the second pass started, the change was already on GitHub waiting for a human.

Nothing read the reviewer’s verdict. The prompt ended with a clear instruction: end with APPROVE or CHANGES_REQUESTED. Those two words appeared exactly once in the entire pipeline—on that line, in that prompt. Nothing downstream ever searched for them. The only thing checked after the review finished was whether the process had exited cleanly.

And the reviewer was reading a blank page. The review input was assembled with “git diff HEAD”—the standard way to ask “what has changed?” But the commit had already happened earlier in the run, and “git diff HEAD” after a commit is empty by definition. The reviewer received a well-formed prompt and no code. It produced a confident verdict about nothing.

The Part That Generalizes Past My Stack

The middle failure is the one I’d want every engineering leader to check this week.

An LLM command-line tool exits zero whether it approved your change or refused it. It exits non-zero when it fails—bad credentials, a timeout, a malformed request. Not when its judgment is negative. The judgment is prose in a file. The exit code is plumbing.