What a GPT-7 Intelligence Explosion Looks Like | Carl Shulman

AI researcher Carl Shulman explores the transition toward an intelligence explosion, where AI systems begin to automate their own development. A critical "second saving throw" for humanity involves using early, potentially misaligned AI to solve remaining alignment problems, such as developing neural lie detectors, while maintaining hard power constraints to prevent server takeovers. This process leverages the unique advantage of empirical verification, where humans can confirm successful outcomes—like an AI rooting an air-gapped computer—even without understanding the specific exploit used. The feedback loop accelerates as AI productivity surpasses human researchers, not by achieving flawless human-level capability, but through the sheer scale of tens of millions of GPUs performing specialized tasks. Key drivers of this acceleration include the generation of synthetic training data and self-play curricula, which allow models to transcend the limitations of human datasets and rapidly master complex programming and reasoning skills.