Jennifer Zhao, co-founder and CEO of Preference Model, discusses coming out of stealth with a $16M seed raise led by Andreessen Horowitz and open sourcing Karrot. She explains why training models on RL environments to perform machine learning engineering and alignment is the highest-leverage path to safe AI development. Check out our sponsor Lovable: https://lovable.dev. Turn ideas into software people love.

MTS is a live news and interview show covering technology, business, politics, and culture as it happens.

Follow MTS:

X | ⁠https://twitter.com/MTSlive⁠

YouTube | ⁠https://www.youtube.com/@mtsituation⁠

Spotify | ⁠https://open.spotify.com/show/4HUNNmV1pjV7CW6Lphm1rM⁠

Apple | ⁠https://podcasts.apple.com/us/podcast/mts/id1891088763⁠

Substack | ⁠https://mtslive.substack.com/⁠

Interested in sponsoring the show? ⁠⁠⁠⁠⁠⁠sponsors@mts.now⁠⁠

--

Brought to you by:

Mongo DB - Anyone can prototype an agent. Learn how to get them into production on September 30th in New York - register at https://www.mongodb.com/events/mongodb-local/nyc

Lovable - Turn ideas into software people love with Lovable. Start today at https://lovable.dev/

Larridin - Finally prove what your AI investment is worth. Head to https://larridin.com/ today to start measuring your AI’s bottom line.

Neon (by Databricks) - Get up to $100K in credits for your startup at ⁠https://neon.com/mts⁠

ElevenLabs - AI that communicates at human level across every channel and modality. https://elevenlabs.io/mts

Arena - Measuring AI performance in the real world. ⁠https://arena.ai/⁠

Kong - The AI connectivity platform. Connect APIs, LLMs, agents, and systems with serious security and governance. https://konghq.com

Today’s stream is sponsored by VCX, the public ticker for private tech. Head over to http://getvcx.com to learn more

Blitzy - Autonomous software development for enterprise codebases. Ship 5x faster. https://blitzy.com

AdQuick - Make your brand a billboard. Out-of-home advertising as easy to scale as digital. https://adquick.com

Chargebee - Billing and monetization infrastructure for the AI era. https://www.chargebee.com/

--

Timestamps:

(00:00) Introduction

(01:32) Origin Story of Preference Model

(02:42) AI Performing Machine Learning Research

(03:59) Considerations for AI Alignment

(04:55) Preventing Reward Hacking in Models

(06:23) Why Environments Need Depth

(07:45) Bottlenecks in Recursive Self-Improvement

(10:33) Predictions for AI Research Progress

--

Note: This podcast is not investment, legal, or tax advice, and is intended for informational and entertainment purposes only. Hosts and guests may hold positions in the companies and securities discussed; do your own research before acting on anything you hear.

Machine Learning Engineering Is the Ultimate RL Task | Jennifer Zhao

MTS

Machine Learning Engineering Is the Ultimate RL Task | Jennifer Zhao