Build your own decision model

Automated excerpt

"System one" decision models are models that infer and respond with calibrated probabilities or every allowed answer. Consider your everyday language model, to get typed output from it (JSON), you may use Structured Output to constrain the output to guaranteed valid JSON. By masking other items in the vocabulary, the model can only emit those tokens. This means that the model is not calibrated. bin count confidence accuracy (0. 90, 1. 00] 809 0. 9855 0. 7009We can notice that the model tends to be extremely overconfident in the 0. 9 - 1.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.