Staging environment
Free Lesson

Use LLMs as Judges for Search Result Quality

75 min
Oct 28, 2025 11:00 AM
Virtual (Zoom)

In this video

What you'll learn

How can LLMs help us quantify search result quality?

Learn how LLMs as judges compare to human and implicit search evaluation and when to use them.

LLMs need to judge based on rules

Learn why you should not just ask the LLM whether a document is relevant and how you can formulate those rules.

Advanced: Using critique models and personas

Improving LLM judgments further and adapting to user segments by using critique models and personas

Why this topic matters

Search result quality evaluation is at the core of any search improvement work. LLMs as Judges can be a game-changing tool for practitioners. They promise to overcome limitations of human or implicit judgments in terms of scalability, rapid availability, and also in terms of what aspects of result quality they cover - provided we get them right.

You'll learn from

René Kriegler

René Kriegler

Chief Strategy Officer at OpenSource Connections

Trey Grainger

Trey Grainger

Author, "AI-Powered Search", Founder @ Searchkernel

Built Search At

OpenSource Connections
Searchkernel
CareerBuilder
Lucidworks
Presearch
See all products from AI-Powered Search