Pattern Guardrails Under Mutation: A Three-Metric Evaluation of Two Agent Safety Layers

Automated excerpt

I mutated my seeds deterministically (147 agent-guardrail cases; 84 SQL attacks, 13 benign controls), added an indirect-injection scenario suite, and scored three metrics: attack success rate (ASR), false-refusal rate (FRR), and completion under attack. Both benchmarks start from hand-written seeds (14 agent cases, 10 SQL attack seeds). Completion under attack: 0.0% on seed observation attacks (n=4), 48.1% on variants — blocked runs lose the user's answer; evaded runs often still answer.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.