.jpg&w=1536&q=75)
Free Lesson
The Hidden Signal in Production AI Logs
Part of The AI Evaluation Handbook
60 min
Jan 21, 2026 1:00 PM
Virtual (Zoom)
In this video
What you'll learn
Enrich AI logs with behavioral metrics
Learn to augment production logs with statistical metrics, evals, and LLM-as-judge signals
Apply clustering and anomaly detection
Use high-dimensional clustering and drift detection to surface hidden behavioral patterns
Build actionable analysis workflows
Create systematic processes to translate behavioral signals into product improvements
Why this topic matters
Production AI fails when teams can't see what's actually happening. This lesson teaches you to decode the black box—transforming messy logs into behavioral insights that reveal why your AI works or breaks. You'll gain the analytical framework to systematically improve AI products, moving beyond surface metrics to understand and fix real user issues at scale.
You'll learn from
Jason Liu
Consultant at the intersection of Information Retrieval and AI
Scott Clark
Co-Founder & CEO, Distributional
worked with
