My agent guardrail denied a Python docstring as an attempted transaction

dev.to

Yesterday I published a postmortem about my own guardrail blocking me from writing a Markdown file. The two-axis fix I proposed there — classify the argument, but also ask whether the tool can act on it — was the right shape and the wrong root cause.

The actual root cause was two matcher defects. Both are worth stealing.

Defect 1: bare-word alternation

The commerce-path matcher listed its tokens as a bare alternation. Any payload that merely contained one of those words anywhere — in prose, in a comment, inside a longer identifier — matched, regardless of context.

Confirmed live: a Python docstring in an unrelated repository was rejected as an attempted transaction.

That is worse than it sounds, because the hook is registered with a match-everything pattern in user settings. It was not misfiring on one project. It was misfiring on every project on the machine.

The fix: require a real path or fragment separator, plus a trailing word boundary, so a token that continues into a longer identifier is not treated as a path.

Defect 2: a conjunction that satisfies itself

This one is my favourite, because I stared straight at it and did not see it.

The rule was a conjunction — deny when the payload contains both a mutation action and a financial object:

if (MUTATION_ACTION.test(text) && FINANCIAL_OBJECT.test(text)) deny()
Enter fullscreen mode Exit fullscreen mode

That reads as sensible narrowing. Two independent signals, both required.

Except several tokens appear in both lists. So one word satisfies both halves by itself:

token: "checkout"

  is it a mutation action?   yes  - it is in the action list
  is it a financial object?  yes  - it is in the object list

  -> both clauses true, from one word, in any context
  -> DENY

me: it was a docstring
Enter fullscreen mode Exit fullscreen mode

The conjunction was not narrowing anything. For every overlapping token it was a single-word denylist wearing an AND's clothes — and because it looked like a conjunction, it read as conservative in review. That is the dangerous kind of bug: not one that hides, one that reassures.

The fix: the action and the object must occupy distinct, non-overlapping spans in the payload. Two signals means two places in the text, not one word counted twice.

The part worth arguing about

The reason to fix this in a day is not developer annoyance. It is that a fail-closed control with a high false-positive rate does not stay enabled.

That is the whole failure mode. Nobody sits down and edits a ruleset to be subtly worse. They add a bypass, or set the override env var, or comment out the hook "just for this branch" — and then coverage is zero, and the incident that eventually happens looks identical to a control that was never installed. The guard does not fail loudly. It fails by uninstallation, months earlier, in a commit nobody reviewed carefully.

So precision is not cosmetic for a security control. Precision is what buys you the right to keep the control switched on. Every false positive is a withdrawal from the account you need at the moment it fires correctly.

How to prove a precision fix did not cost coverage

When you loosen a matcher, the obvious question is whether you also loosened something you needed. Asserting "no regressions" is not an answer to that question. So the change shipped with a decision diff over a real payload corpus: replay every payload through the old and the new matcher, and count which direction each decision moved.

Corpus A Corpus B
payloads 234,768 414,735
decisions loosened (deny to allow) 3,229 4,499
decisions tightened (allow to deny) 0 0

tightened: 0 is the claim that matters. Coverage did not move in the dangerous direction across 649,503 payloads.

And the write-up labels these as raw counts — explicitly not "zero regressions" — because a corpus is only what it happens to contain, and a count over it is not a proof about payloads it never held. That distinction is the difference between a measurement and a marketing line, and it is the first thing I would look for in anyone else's "we improved our guardrails" post.

Two questions for your own guard

If you maintain any pattern-based control — a WAF rule, a secret scanner, a lint gate, an agent guardrail:

  1. Does any single token satisfy more than one clause of your conjunctions? Grep your lists against each other. Overlap silently turns an AND into an OR, and the code still reads as strict.
  2. Can you produce a loosened/tightened table for your last tuning change? If not, you do not know what that change did. You know what you intended it to do.

ThumbGate is MIT and local-first. It runs in the PreToolUse hook and needs no server on the local enforcement path:

npx thumbgate init
Enter fullscreen mode Exit fullscreen mode

Repo: github.com/IgorGanapolsky/ThumbGate

The census harness is in there too, if the shape of the diff rig is more useful to you than the tool itself.

Source: dev.to

arrow_back Back to News