What green means when agents ship to main
When AI coding agents deliver code to a main branch, a passing pipeline is only as good as its definition of passing. Nicholas Hopper has found pipelines that reported success without testing anything, and wrote down what he no longer accepts as green.
A coding agent treats a green pipeline as permission to continue. That makes the definition of green the most important rule in a repository that agents work in. Nicholas Hopper has collected, across his own projects and his work at APX Stream, the ways a pipeline can say yes without meaning it.
A pipeline can pass without running the tests
On Vibecode City, the deploy gate ran type checks, a build and smoke tests. It never ran the unit test suite. Nine unit tests broke unnoticed while continuous integration stayed green. Hopper repaired the nine tests and added the suite to the gate.
Nothing in the pipeline had failed. The gate did what it was configured to do. The gap was between what the gate checked and what everyone assumed it checked.
One failure can hide sixteen others
The same project’s smoke tests ran as a single chain of commands joined with &&. A chain like that stops at the first failure. One broken smoke test stopped sixteen later ones from running, and the chain was the only thing that exercised them.
Hopper replaced the chain with a runner that executes every smoke test and reports all the results. The first full run showed 20 of 21 passing.
Checks can pass while the product is broken
On a client project, Hopper made a performance change that passed the type checker, the linter and the production export. The built bundle was broken in the browser. He found the regression only by loading the bundle in a headless browser.
On a personal project that renders documents to images, Hopper made this a standing rule for agents: after changing any source file, rebuild and read the rendered image. The image shows alignment, overflow and font problems that the source does not.
A deploy can do nothing and report nothing
One of Hopper’s personal servers deploys from its main branch. Commits authored by a cloud agent session landed on main and never deployed. There was no failure and no deployment. The hosting platform’s Git integration builds only commits whose author belongs to the project, and the agent was not a member. The deploy now runs from continuous integration, for every commit on main.
What Nicholas Hopper does not accept as green
At APX Stream, Hopper wrote the rules into the skill that agents follow when they deliver work:
- A gate that skipped tests it was expected to run is not green.
- A test of mocked wiring is not proof that a real integration works.
- A gate that leaves generated output modified is not green.
- A gate’s exit code may not be hidden behind
grep,tailortee. - A user-interface test that injects its own fixtures is not proof that the real backend path exists.
The script chooses the checks, and only the script may reduce them
Running every check on every change is slow. Hopper’s delivery skill classifies the diff with a script to choose which checks run. A path the script does not recognise forces the full set. The agent may add checks by judgment and may never remove one. The script has its own self-test, and if that fails, everything runs.
Landing parallel work without losing any
When several agents work at once, each in its own git worktree, the delivery step has to bring every commit to main. Hopper’s procedure takes an inventory of every worktree before it does anything, merges every commit with no filter for importance, bans reset, stash, clean, rebase and force push, and ends by proving that each inventoried commit is an ancestor of the result. If a person’s uncommitted work is in the way, the agent stops and asks.
The pattern
Each of these failures was silent. The pipeline reported success, or reported nothing. Hopper’s response in each case was the same: write down what success requires, and make the check for it something an agent cannot talk its way past.