AI article
Do LLMs Invent Japanese Law Articles? A Bilingual Benchmark
Community description: Checking every Japanese statute article an LLM cites against the official e-Gov registry, in Japanese and in English.
Dev.to | Oct 6, 2026 | Raihan
Automated excerpt
Treating deleted placeholders as non-existent gives 66/1,379 (llm-jp, Japanese), 12/597 (Qwen2. 5-7B, Japanese) and 2/753 (Swallow-8B, Japanese); the English counts do not change. LLMs cite Japanese law articles in their answers. Most Japanese LLM evals score answers against gold labels.
Selected automatically from source text; not independently written or fact-checked. Read the original for full context.