Emmett Shear on Building AI That Actually Cares: Beyond Control and Steering

The podcast features Emmett Shear, co-founder of Twitch and now working on AI alignment at Softmax, along with Seb Krier from Google DeepMind, discussing AI alignment with host Erik Torenberg. Emmett introduces the concept of "organic alignment," emphasizing it as an ongoing process of learning and adaptation rather than a fixed state, drawing parallels to family dynamics and moral progress. The conversation explores the limitations of current AI alignment approaches, which often focus on steering and control, and advocates for building AI systems that can learn to care and develop a theory of mind. The speakers debate whether AGI can remain a tool or will inevitably become a being, and what observations would lead them to change their minds about AI sentience and moral status. Emmett shares Softmax's strategy of using multi-agent reinforcement learning simulations to train AI agents with better theory of mind and cooperation skills. He also critiques the narcissistic tendencies of current chatbots and suggests redesigning them for multi-agent environments.