Free Lesson
De-Risking LLM Model Switches w Evals & Prompt Optimization
Part of The AI Evaluation Handbook
45 min
Oct 3, 2025 12:00 PM
Virtual (Zoom)
In this video
What you'll learn
Establishing Reliable Evals
How consistent evaluation frameworks reduce risk in high-stakes LLM migrations.
Automated Prompt Optimization
Using evals as a foundation to systematically improve prompts and model behavior.
Scaling RAG Across Models
Lessons from migrating Coveo’s flagship RAG product to new LLMs multiple times per year.
Why this topic matters
I'll be talking about LLM evals best practices in use at Coveo, and how we use them as the foundation for automated prompt optimization.
We used the above many times to migrate Coveo's flagship RAG product (Coveo Relevance Generative Answering) to new LLM models, an unavoidable and high-risk project that we have to go through a few times a year.
You'll learn from
Amir Feizpour
Founder @ Aggregate Intellect
Hugo Mailhot
CoreNLP Lead @ Coveo