Ep 54: Princeton Researcher Arvind Narayanan on the Limitations of Agent Evals, AI’s Societal Impact & Important Lessons from History | Unsupervised Learning | Podwise
This interview podcast features Jacob interviewing Arvind Narayanan, a Princeton computer science professor, about the current state and future of AI, particularly focusing on reasoning models and agentic AI. The conversation explores the limitations of current AI benchmarks, the challenges of evaluating agentic AI's real-world performance (including the high cost of errors in autonomous systems), and the need for more human-centered design in AI development. Narayanan highlights the importance of construct validity in benchmarks and suggests that uplift studies, which compare productivity with and without AI tools, offer a more practical evaluation method. He also discusses the implications of AI for education and the need for policy interventions to promote responsible AI adoption, emphasizing the importance of teaching critical thinking skills alongside AI usage.
