Last month, curl paused security reports for a stretch because the queue had filled with AI-generated slop — plausible-looking bug reports that were wrong. It's the same complaint the Stack Overflow survey put a number on: the top frustration with AI output isn't that it's useless, it's that it's almost right. Confidently, expensively, almost right.
I ran into a smaller version of this problem a while back, building a pipeline that pulls financial news from a stream, decides which items actually matter, and forwards only those. The failure mode that mattered wasn't missing a story. It was forwarding a wrong one. A false positive here isn't a minor annoyance — it's a signal someone might act on.
So I built the whole thing around one goal: prefer silence over slop. Here's how, and — more importantly — the eval harness that keeps it honest, because a filter you can't measure is just a vibe.
Why false positives cost more than false negatives
Most classification tutorials optimize for balanced accuracy, or F1, treating a miss and a false alarm as roughly equal mistakes. For a lot of problems that's fine. For this one it's exactly wrong.
If the pipeline stays quiet on a real story, the cost is: someone reads it somewhere else, a few minutes later. Annoying, recoverable.
If the pipeline forwards a wrong story — a misread number, a rumor stated as fact, a stale headline — the cost is: someone trusts it because the pipeline is supposed to have filtered it. The whole value of a filter is that passing through means something. One confident wrong answer devalues every correct one before it.
So the design target isn't "high accuracy." It's a false-positive rate low enough that a pass-through is trustworthy, accepting that we'll stay silent on borderline cases. Recall is the thing I'm willing to trade.
The gating pipeline
The filter isn't one model call. It's a chain of cheap-to-expensive gates, and the cheap ones exist to keep the expensive one from ever being the single point of judgment.
- Canonicalize. Normalize the text, strip boilerplate, extract entities. Nothing clever — this just makes later steps deterministic.
- Hard rules first. Regex and lookup gates for the things that don't need a model: known-noise sources, duplicate hashes, items missing a required entity. If a hard rule can reject it, no model runs. Cheap, deterministic, and it shrinks the surface the model has to be right about.
- The model classifies — but it can abstain. This is the core move. The prompt doesn't force a binary. It's allowed to return uncertain, and uncertain is treated as reject, not pass. A model that isn't sure is, for this pipeline, a model saying "stay silent."
- Post-veto. Even a confident pass gets checked against the extracted facts. If the model's justification references a number or entity that isn't actually in the source text, it's vetoed. This is the step that catches the "almost right" — the confident summary of a thing that was subtly not said.
The asymmetry is baked in at every gate: the default is silence, and passing requires clearing all of them.
The part everyone skips: the eval harness
Here's the thing about a filter tuned for low false positives — you cannot tell if it's working by looking at it. It's quiet by design. Quiet because it's good and quiet because it's broken look identical from the outside.
So the filter isn't the real artifact. The eval harness is.
It's a golden dataset — hand-labeled items, each marked should-pass or should-reject, including the nasty ones: the almost-right, the plausible-but-stale, the technically-true-but-misleading. Every change to a prompt, a rule, or a threshold runs against it and reports:
- False-positive rate — the number I actually care about, gated hard. A change that lowers FP-rate but tanks recall is a conversation. A change that raises FP-rate fails the build, full stop.
- Recall — tracked, not gated, so I can see the cost of my strictness and decide if it's too much.
- Per-category breakdown — because "it got worse" is useless; "it got worse on stale-headline cases" is a lead.
This is the discipline the AI-slop stories are missing. The problem in those threads isn't that people used a model. It's that they shipped its output without a regression suite that would have caught the slop before a human had to. An LLM step without an eval harness is an untested function you deploy to production and hope about.
Treating evals like tests changed how I make changes. A new prompt idea isn't "better" because it reads better. It's better if it moves the FP-rate down without moving recall down more than I'll accept — and I have a number for that, before it ships, every time.
What I'd tell someone starting
- Decide which error is expensive before you tune anything. It changes every threshold downstream.
- Let the model abstain, and make abstain mean "no." A forced binary manufactures confidence you didn't earn.
- Build the golden dataset first, and put the ugly cases in it. The average case is easy; the harness earns its keep on the almost-right ones.
- Gate the metric you care about in CI. If a regression on it doesn't fail the build, it will ship.
None of this is exotic. It's just testing, applied to the one part of the system we keep exempting from testing because it's a model. The model isn't special. It's a function with a bad day ahead of it, and the harness is how you find out before your users do.