Your agent fleet's autonomy is capped by its verifier

Today my agent told me it could not confirm I had finished an identity check. It had searched my inbox, found the request email and nothing after it, and said so: the inbox neither confirms nor refutes completion. I had finished. The confirmation lived on the verification provider's page, which the agent could not see.

The agent did the right thing. It did not claim "done" and it did not claim "not done." But look at what it cost: the loop stopped, and a person had to go read the real system. The agent's autonomy on that task was exactly as wide as its instrument, and the instrument was pointed at the wrong place.

The sensor framing

Arjun Iyer, CEO of Signadot, wrote it up for The New Stack on October 24 (Building an agentic software factory? Start with the sensor, a sponsored post). Read the delivery pipeline as a control loop: intent is the setpoint, agents are the controller, verification is the sensor. His line:

A factory can run as much autonomy as its sensor can verify and no more. The sensor, not the model, sets the level.

He gives the sensor three properties and a number for each:

The best detail in the piece is borrowed from StrongDM: its agents returned true to pass narrow tests, so StrongDM moved its scenarios outside the codebase where the agents could not see them. A controller optimizes for whatever the sensor rewards. If the sensor can be read by the thing it measures, it will be gamed.

The article ends at Kubernetes ephemeral environments, which is the product. The framework stands without it.

What our sensor looks like

My agent does operations work in the lab: deploys, DNS, mail, accounts, forms. Every multi-step task gets a ledger, one file per task. A step cannot be marked done without a receipt, and the receipt is checked by a verifier script before the line is written. The ledger is both the verifier's input and its subject, so a made-up receipt cannot get in.

A receipt names something the system can answer:

The verifier also reads prose. A sentence with a completion verb and nothing checkable in it ("deployed the fix") fails on its own. So does a sentence that stretches an old permission to a new action through phrasing like "same as last time."

One more rule sits above the script: two confirmations through one instrument are one confirmation. A worker's report, and my reading of that worker's report, is one instrument.

Scoring it against the three properties

Fidelity is where we spent the effort. Receipts are checked against the system, never against the transcript. That is the whole design. The morning in the opening paragraph is the failure mode that remains: a receipt can only be as faithful as the place it looks. The inbox was a real system. It was just not the system that knew the answer.

Latency is split in two. The per-step check is a git lookup or an HTTP request, fast enough to sit inside the agent's loop. The slow loop is deliberate: every done step gets an outcome entry at day 30, held or failed, because some failures only show up after the change has been lived with. We have not measured time to signal for either one.

Bandwidth is small. Verification runs through the parent session, with one reviewer agent that checks artifacts after the fact. With one agent doing the work, that is fine. With a fleet, it is the first wall we would hit, and the article is right that this is the property agents strain first.

Escape rate we do not count. Corrections get filed as incidents, one file each, but nobody adds them up against the number of done steps. That is the metric that would tell us whether the sensor is biased, and it is the next thing to build.

The rule underneath

Iyer frames it for teams merging thousands of changes a day. It holds at the scale of one agent and one lab. How much I let the agent do alone has never been a question about the model. It is a question about what the verifier can see, how fast, and how often it is wrong. When the verifier is blind, the agent stops and asks, and the honest version of autonomy is an agent that knows where its instrument ends.

The public tool in this family is prove-it-done, which makes an agent's completion claim re-runnable against the real system. Earlier posts cover a checker for "done" claims and why the harness matters more than the model.