MIPRO and DSPy with Krista Opsahl-Ong! - Weaviate Podcast

Hey everyone! Thank you so much for watching the 103rd Weaviate Podcast with Krista Opsahl-Ong from Stanford University. Krista is the lead author of MIPRO, short for Multi-prompt Instruction Proposal Optimizer, and one of the leading developers and scientists behind DSPy!

This was such a fun discussion beginning with the motivation of Automated Prompt Engineering, Multi-Layer Language Programs (also commonly referred to as Compound AI Systems), and their intersection. We then dove into the details of how MIPRO achieves this and miscellaneous topics in AI from Structured Outputs to Agents, DSPy for Code Generation, and more!

I really hope you enjoy the podcast! As always, more than happy to answer any questions or discuss any ideas about the content in the podcast!

Michael Ryan (MIPRO Co-Author): https://x.com/michaelryan207

Thank you to the DSPy team and DSPy community members for your support! Special thanks to Omar Khattab, Chris Potts, Matei Zaharia, Heather Miller, Arnav Singhvi, Herumb Shandilya, Sri Vardhamanan, Cyrus Nouroozi, Amir Mehr, Kyle Caverly, Keshav Santhanam, Thomas Ahle, Michael Ryan, Josh Purtell, Karel D'Oosterlinck, Eric Zhang, Shangyin Tan, Manish Shetty, Peter Zhong, Jasper Xian, Saron Samuel, Alberto Mancarella, Faraz Khoubsirat, Saiful Haq, Ashutosh Sharma, Rick Battle, Dhar Rawal, Insop Song, Tom Dorr, Igor Kotenkov, Corey Zumar, Lisa Li, David Hall, Ashwin Paranjape, Chris Manning, Avi Sil, and Chuyi Zhang.

Helpful Links:

MIPRO - https://arxiv.org/abs/2406.11695

MIPRO Animations from Michael Ryan - https://x.com/michaelryan207/status/1...

DSPy for Code Generation -    • NeurIPS Hacker Cup AI: DSPy for code gener...  

Compound AI Systems - https://bair.berkeley.edu/blog/2024/0...

DSPy - https://github.com/stanfordnlp/dspy/t...

DSPy research paper - https://arxiv.org/abs/2310.03714

Large Language Models as Optimizers (OPRO) - https://arxiv.org/abs/2309.03409

The ImageNet Moment of DSPy from Professor Bo Wang’s Lab - https://x.com/lateinteraction/status/...

Unreasonable Effectiveness of Eccentric Prompting - https://arxiv.org/abs/2402.10949

BetterTogether - https://arxiv.org/abs/2311.01326

Thoughts from Weaviate Erika on the Automated Design of Agents - https://x.com/ecardenas300/status/182...

Weaviate Recipes DSPy implementation of the Automated Design of Agents - https://github.com/weaviate/recipes/b...

Shreya Shankar - Who Validates the Validators? - https://arxiv.org/pdf/2404.12272

Eugene Yan - Evaluating the Effectiveness of LLM-Evaluators (aka LLM-as-Judge) - https://eugeneyan.com/writing/llm-eva...

Joao Moura (Crew AI) - Using agents to build an agent company -    • Using agents to build an agent company: Jo...  

Chapters
0:00 Welcome Krista!
0:36 What lead you to work on DSPy?
2:32 What is Automated Prompt Engineering?
4:46 Multi-Stage Language Programs
7:10 Optimizing Multi-Stage Language Programs
9:30 The Proposal Problem
12:20 More examples of Multi-Stage Language Programs
14:25 Thoughts on Structured Outputs
18:35 Thoughts on Agents
20:20 MIPRO’s Credit Assignment
27:25 Many-Shot In-Context Learning
30:04 Automated Design of Agents
31:44 Making DSPy Easier to Use
34:24 Integrating Cheaper LMs
36:28 Deeper into Learning to Propose
44:12 DSPy for Code Generation
46:18 How much of your code is written by AI?
48:28 Multi-Agents and DSPy
51:18 AI User Interfaces
53:50 Generative Feedback Loops
56:30 What directions for the future excite you the most?