The Implications of Linguistic Illegibility for LLM Security

Automated excerpt

Abstract:LLMs are trained to generate natural language. However, various strands of evidence indicate that an LLM's externalized linguistic outputs and mechanistically-extracted linguistic features can be an unreliable lens for understanding internal model computation. If linguistic illegibility is always possible, then security mechanisms that rely on a model's linguistic self-reporting (e. g. , chain-of-thought monitoring, constitutional self-critique, activation probing for linguistically-defined feature vectors) can never be completely sound; the model sandbox will always need isolation techniques whose guarantees do not depend on reading a model's linguistic state at all.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.