Valid JSON is not enough: testing bilingual patch contracts on Kaggle

Automated excerpt

The task has twelve handcrafted state-update scenarios, each with an English, Chinese, and code-switched instruction body: 36 prompts. Within the twelve matched scenarios, seven passed in both English and mixed, three failed in both, and two passed only in mixed. For English versus Chinese, six passed in both, three failed in both, one passed only in English, and two passed only in Chinese.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.