Back to DevLog

Fail-closed enforcement for read-only Manager and Reviewer verdict parsing

2 min read

I closed two agent-hardening gaps in memstack-skill-loader: making the Manager structurally read-only and enforcing a strict machine contract on Reviewer verdicts.

Manager is structurally read-only

I added a PreToolUse deny hook that blocks file-writing tools (Write, Edit, MultiEdit, NotebookEdit) and write-idiom Bash commands (redirection with > or >>, tee, sed -i, dd, truncate, python open(). Ambiguous Bash fails closed. The hook is injected into the Manager subprocess via inline --settings at spawn.

I added an orchestrator backstop that hashes the working tree before the Manager turn and fails the turn loudly on any change, discarding its output. The backstop uses git status --porcelain --untracked-files=all, git diff, and git diff --cached. I excluded the runner state dir (~/.memstack/agent-runner) via per-path filter so the Manager's own logs cannot false-positive the backstop. I needed --untracked-files=all because the default porcelain collapses an untracked dir to dir/ and slipped the exclusion.

I verified four ways: Write denied, Bash redirect denied, clean read-only turn passes, backstop fires on mid-turn touch.

Reviewer verdict is a machine contract

The Reviewer must now end its response with exactly one fenced JSON block containing verdict, security, reasons, blocking_issues, and notes. I added _parse_reviewer_verdict to schema-validate and fail closed. Zero blocks, multiple blocks, malformed blocks, unknown verdict values, missing keys, or consistency violations (REJECT with empty blocking_issues, APPROVE or APPROVE_WITH_NOTES with non-empty blocking_issues) all resolve to REJECT with specifics.

security=FAIL or missing security forces REJECT regardless of verdict. APPROVE_WITH_NOTES is the only caveat path. Notes are recorded to reviewer_notes.jsonl run artifact.

_review_with_contract allows exactly one corrective re-prompt on a contract violation, quoting the violation. A second violation is final REJECT. I log the raw block and parsed decision side by side per gate for audit.

This replaced the old substring match ("APPROVED" in reviewer_output.upper()) that let hedged prose like "I would have APPROVED..." parse as approval.

I verified six ways and updated advisor-test fixtures to the new JSON verdict contract. Six advisor tests regressed because their canned fixtures used the old prose format. I rewrote the two fixture constants as valid JSON verdict blocks.

Both fixes merged fast-forward-only to master. Full suite green (146 passed).

Share this post