Skip to content

Remediation: fail-closed --expect, case-safe witness, accurate policy budget, working example bundle - #105

Merged
noctem-o merged 6 commits into
mainfrom
remediation/audit-fixes
Jun 9, 2026
Merged

noctem-o merged 6 commits into
mainfrom
remediation/audit-fixes

Conversation

@noctem-o

@noctem-o noctem-o commented Jun 9, 2026

Copy link
Copy Markdown
Owner

Remediation pass for the code-level findings from the read-only audit. Each fix has tests or an empirical check; cargo test is green, cargo fmt --check is clean, and the ordeal golden root is unchanged.

The README readability/scope rewrite (including the "what this is / what it's not" rewording) is not here — it stays in #104 per the agreed split.

Fixes

1. verify: fail closed on --expect without --recompute-witness-root (fix(verify))
The plain bundle check (verify --witness <dir>) returned after the self-consistency check and never consulted --expect, so a bundle that is internally consistent passed with any expected root — defeating the one mode meant to defend against a fabricated bundle. It now rejects that combination and points the caller at --recompute-witness-root. New end-to-end tests cover both the rejection and the correct anchored path (tests/verify_expect_guard.rs).

2. Witness: normalize tool-name case (fix(witness))
The agent trace stored tool names in their raw case while the transcript lowercased them at the dispatch gate, so a mixed-case tool name (e.g. Trade.Buy) made witness recompute fail closed on an otherwise valid run. Names are now normalized consistently in the witness view and the validation comparison. This is a no-op for the all-lowercase fixtures, so the golden root is unchanged; a regression test asserts a mixed-case trace recomputes and matches the lowercase form.

3. Policy: history_max_calls now allows exactly N (fix(policy)) — ⚠️ behavior change
Block previously fired only when the count exceeded max, so history_max_calls = N actually allowed N+1 calls — an off-by-one in the permissive direction for a control described as a capability firewall. It now blocks once the count reaches max, so N permits N calls and blocks the (N+1)th, matching the field name. Unit + integration tests and the policy.toml comment are updated. Reviewers should consciously accept this semantics change.

4. repro.sh: use the real CLI (fix(repro))
The script invoked the binary with no subcommand and an unsupported --output flag, so it could not run. Rewritten around run --seed --runs --out-dir.

5. Regenerate the replay/run_0000 example bundle (fix(bundle))
The shipped manifest stored repo-root-relative artifact paths (replay/run_0000/meta.json), which the bundle-relative resolver doubled, so the example failed both verify --witness and run --replay. It turned out to be doubly stale — its transcript still used the pre-rename gauntlet.lookup tool name. Regenerated with current code (non-git dir, so no commit SHA leaks into provenance); it now passes self-consistency, --recompute-witness-root, and run --replay.

6. Landing page: repair rendering + soften overclaims (docs(site))
docs/Index.html didn't render as designed: CSS variable references used en-dashes (var(–ink)), font names used smart quotes, and two stray markdown code fences sat in the body. Those are fixed. I also softened the technical claims the stubbed implementation doesn't support — the "agent cannot reach outside its envelope" line (the policy gate is cooperative, not mandatory) and "reproducibility is not a claim, it is a property" — and corrected the GitHub URL casing.

Left for you to decide: the Regulatory Alignment table (EU AI Act, FCA/SEC, FDA SaMD, IEC 61508, IETF/SCITT) is unchanged. Those claims still run ahead of what a stubbed-agent reference implementation demonstrates; whether/how to qualify them is a positioning call, not a code fix.

Note (out of scope)

cargo clippy -- -D warnings fails on Windows due to a pre-existing unused import in src/nix_provenance.rs (the only use is behind #[cfg(unix)]). CI runs on Linux/macOS where it's used, so CI is unaffected; flagging it but not fixing it here.

🤖 Generated with Claude Code

noctem-o and others added 6 commits June 9, 2026 22:44
…witness-root

The plain bundle check returned before consulting --expect, so an anchor that was never compared still exited 0. Reject the combination and point the caller at --recompute-witness-root, with end-to-end tests for both the rejection and the correct anchored path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tion

Agent-trace tool names kept their raw case while the transcript lowercased at the dispatch gate, so a mixed-case tool name made witness recompute fail closed on an otherwise valid run. Normalize consistently; this is a no-op for the all-lowercase fixtures, so the ordeal golden root is unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Block now fires once the matching-call count reaches max rather than exceeding it, so history_max_calls = N permits N calls and blocks the (N+1)th, matching the field name. Unit and integration tests and the policy.toml comment are updated to match.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
repro.sh invoked the binary with no subcommand and an unsupported --output flag, so it could not run. Rewrite it around run --seed --runs --out-dir and print the resulting witness root.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The shipped manifest stored repo-root-relative artifact paths, which the bundle-relative resolver doubled, so the example failed both verify --witness and run --replay. Regenerated with current code; it now passes self-consistency, --recompute-witness-root, and replay.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fix broken CSS variable references (en-dash instead of --), smart-quote font names, and stray markdown code fences so the page renders; soften technical claims the stubbed implementation does not support (cooperative policy gate, replay scope); correct the GitHub URL casing. The regulatory-alignment table is left unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@noctem-o
noctem-o merged commit 490eda5 into main Jun 9, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant