Embedded AI evaluators won’t be an easy solution, Hugging Face’s policy chief says

Automated excerpt

AI labs have staked their reputations on model benchmarking — the supposedly neutral and accurate system of evaluations that are designed to measure whether one AI product is better than another. However, the methods behind these evaluations are complicated and still maturing, said Irene Solaiman, Hugging Face’s chief policy officer, during a discussion at TechCrunch Disrupt last week. Evaluations are now moving from merely a test of the model itself to a test of the entire system around the model — including the harness and agent.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.