Ep 69: Co-Founder of Databricks & LMArena on Current Eval Limitations, Why China is Winning Open Source and Future of AI Infrastructure

In this interview, Ion Stoica, co-founder of Databricks, AnyScale, and LLMarena, discusses his new company LLMarena, which focuses on AI model evaluations using human preferences and dynamic benchmarks to combat contamination and subjectivity. Stoica explains the company's origins from a Berkeley project, the challenges of evaluating conversational AI, and the importance of human-in-the-loop validation for reliable AI applications. The conversation explores the future of open-source models, the competitive landscape between the U.S. and China, and the need for collaboration and shared infrastructure to foster innovation. Stoica also touches on the potential overbuilding of AI infrastructure, the opportunities and challenges in AI infrastructure, and his evolving views on topics like quantization and the progress of open-source models.