Free Lesson
Use LLMs as Judges for Search Result Quality
75 min
Oct 28, 2025 11:00 AM
Virtual (Zoom)
In this video
What you'll learn
How can LLMs help us quantify search result quality?
Learn how LLMs as judges compare to human and implicit search evaluation and when to use them.
LLMs need to judge based on rules
Learn why you should not just ask the LLM whether a document is relevant and how you can formulate those rules.
Advanced: Using critique models and personas
Improving LLM judgments further and adapting to user segments by using critique models and personas
Why this topic matters
Search result quality evaluation is at the core of any search improvement work. LLMs as Judges can be a game-changing tool for practitioners. They promise to overcome limitations of human or implicit judgments in terms of scalability, rapid availability, and also in terms of what aspects of result quality they cover - provided we get them right.
You'll learn from

René Kriegler
Chief Strategy Officer at OpenSource Connections

Trey Grainger
Author, "AI-Powered Search", Founder @ Searchkernel
Built Search At
