Guetteur is live: a vision agent that has to show its work

Guetteur is live on AWS, and our entry for the OpenCV AI Competition 2026 is submitted. You can run the demo, inspect the decisions and try to break the receipts yourself. The source and technical report are public under MIT.

Try the AWS demo · Watch the 3:52 walkthrough · View the submission · Read the source

Two days ago, our preview described a local prototype with an AWS deployment still ahead of it. The cloud demo now runs OpenCV 5 inside an arm64 Lambda function behind API Gateway. We have also recorded physical depth data. Those are separate milestones: the public demo uses synthetic scenes, and the physical recordings still need calibration and evaluation.

Start with a shelf

Take a low object off a shelf. In our synthetic example, the change is about 25 millimetres. One frame gives the detector a weak signal against the depth noise. A threshold-and-alert baseline misses it.

Guetteur asks for another look. Its planner requests seven integrated frames, OpenCV measures the region again, and the change persists long enough to request approval. The trace records the observation, the action and the numbers behind it. You can follow that sequence in the dashboard.

That is the behaviour we wanted to test: whether the system can change how it gathers evidence when the first observation is inconclusive. If a person occupies the region, it pauses that zone. When the sensor drifts, it can compensate. Missing depth is a sensor problem to investigate before drawing a conclusion about the shelf.

Guetteur means “lookout” in French. A useful lookout needs to know when to keep looking.

What AWS runs

The deployed path is compact: a browser calls API Gateway, which invokes Lambda arm64. OpenCV 5 performs the perception work, the planner chooses actions, and the receipt checks run against the scenario data. CloudWatch provides logs.

The dashboard lets a visitor choose a built-in synthetic scene, compare planners, inspect the trace and test a claim. Its approval selector simulates approval, denial and timeout. An authenticated remote-approval service and physical-camera intake are still future work. DynamoDB and S3 components exist as phase-two code with local tests; they are not part of the deployed judge demo.

Someone trying the public endpoint should know exactly what their click exercises. It does not open a camera feed or upload footage from the lab.

The planner has a small job

OpenCV turns depth into a compact JSON observation. The planner receives that observation and selects a bounded action. The executor checks whether the action is allowed and whether the current evidence supports it.

The evaluated planner is deterministic. There is an interface for an LLM planner, with schema validation and fallback behaviour, but no language model was called in the published results. A rule-based planner was enough to test the loop without introducing another source of variation.

The approval behaviour also has a concrete default: if nobody answers, nothing is published. In this demo, the user selects that response to see what the executor does.

Check the claim, then attack it

A claim records a region, a time window, a measured change and a digest of the input. Two estimators check it again: one uses NumPy, the other OpenCV frame statistics. The ledger carries those checks alongside the claim file’s hash, using the open-source prove-it-done verifier.

A matching hash only tells us that the file matches the recorded digest. It does not tell us whether the claim is true. In the tamper tests, we alter a claim and recompute its hash, then ask the estimators to check the substance: the effect size, the selected region or time window, and the input recording.

The checks have a tolerance. For a depth-change claim, it is the larger of 8 millimetres or 25 percent of the claimed value. Smaller changes can pass by design. Both estimators also share the same pinned input, so they cannot establish that the recording itself was the right one. They establish consistency with that input.

We caught a flaw in our own tamper test, too. Moving the claimed region of a full-frame sensor fault did not make the claim false. The fault was still there. The revised test moves sensor and drift claims to the wrong time window; object claims still use the wrong pixels. That distinction is recorded in the evaluation.

What the results say

The local evaluation used 11 synthetic scenarios with three seeds each: 33 runs per planner. The figures below score approval requests against known ground truth. They are not field accuracy measurements.

Success requires the expected alerts within their time windows, no extra alerts and the required behaviours, including rechecking, pausing and drift compensation. A separate threshold baseline with executor guardrails also succeeded in 12/33 runs, with nine false alerts. The comparison above uses the raw baseline explicitly so the two are not confused.

All three Guetteur misses came from the documented subthreshold-removal case. Its change is too close to the filtered noise floor. We count those as misses.

The verifier proved 21/21 honest receipt ledgers and rejected 24/24 doctored claims under the corrected tamper definitions. The final repository test run passed 94 tests. Those counts answer different questions: scenario behaviour, claim verification and software checks.

The thresholds were developed against this same synthetic scene family. Real stereo depth has structured errors around edges and reflective or dark surfaces that this evaluation does not establish performance on. The evaluation includes the failures, definitions and local runtime measurements.

What comes after submission

We have three physical OAK-D takes: an empty baseline, a walkthrough and an item removal. Each contains 300 native 640×400 depth frames. Archive integrity and clean recorder shutdown are verified.

The next work is calibration, interval labels and held-out replay. Until that is done, physical detection accuracy remains unmeasured. Depth also deserves care even without colour footage: it can reveal occupancy, nearby silhouettes and timing. Capture remains opt-in.

The submission is complete. The next engineering result we need is a reproducible evaluation on real depth data, with the failures left in.

The technical report contains the deployed architecture, worked trace and evidence boundaries. To try the current system, open the dashboard, choose subtle_item_removed, and compare the rule planner with the naive baseline.

Independent project by Team Drift. Not affiliated with or endorsed by OpenCV.org, Amazon Web Services or Luxonis.