From 51% to 90.5%: fine-tuning a local triage model on 10,003 support tickets (open source and measured)

Automated excerpt

Hierarchical intent accuracy: 51. 0% -> 90. 5% (after fine-tuning) Flat 77-way classification is hard for a small model. At the 0. 84 confidence threshold (zero-shot): 75. 2% accuracy with 60. 5% of tickets auto-handled. The threshold gates only on cluster and intent confidence.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.