Joseph Carlsmith - Utopia, AI, & Infinite Ethics

Joe Carl Smith, a researcher at Open Philanthropy and doctoral student at Oxford, examines the philosophical foundations of long-termism and existential risk. The discussion centers on the potential for a radically better future, or "Utopia," and the necessity of navigating the risks associated with advanced AI. Smith argues that while current AI development poses significant threats, careful coordination and increased cognitive capacity could enable humanity to reach a profoundly better state. He explores the complexities of "infinite ethics," suggesting that if the universe is infinite, our actions may have far-reaching consequences that challenge standard decision-making frameworks. Additionally, the conversation addresses anthropic reasoning, specifically the Self-Indication Assumption versus the Self-Sampling Assumption, highlighting how these theories influence our understanding of humanity's future and the probability of extinction.