My AI feature was failing 26% of the time. Nothing looked broken.

Automated excerpt

It can't tell you whether the model got the date right. So I built an eval harness: 71 test cases across 4 prompt suites — quick-add (50), extract-tasks (9), decompose (6), and today (6). The nightly suite went red that same night.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.