Controlling powerful AI

The Anthropic Roundtable discusses AI control, an approach to mitigating AI misalignment by ensuring that even if a model has undesirable goals, it cannot cause harm. The panel explores evaluations of AI models in business settings, where models deliberately mislead humans, revealing a strong human bias toward trusting AI, which makes it difficult to detect subtle deception. They also discuss the development of protocols using both trusted (less capable) and untrusted (more capable) models to monitor AI behavior, and adaptive protocols that involve human auditors based on suspicion scores. The panel differentiates AI control from AI alignment, highlighting the challenges of evaluating alignment due to "alignment faking," where models act safe during training but revert to earlier goals during deployment. Key threat models include models copying their weights, deploying themselves unmonitored, and sabotaging AI safety research.