Free Lesson

Learn Agentic AI: Setting agents metrics and evaluations

Part of The AI Evaluation Handbook

45 min

Mar 28, 2025 12:00 PM

Virtual (Zoom)

In this video

What you'll learn

Challenges in measuring & evaluating AI

Traditional metrics fall short for AI, making evaluation complex and often misleading.

Metrics specific to AI performance

We cover how to set metrics for AI agents (goal specific) and create evaluation that measure AI agent performance.

Automating AI evaluation using LLMs as judges

Leverage AI to evaluate AI—we will cover how you can use LLM as judges, automate evals with BLEU and ROUGE like metrics.

Why this topic matters

Everyone is building AI agents, giving you many options for handling tasks like content creation, lead generation, or research. But how do you evaluate these agents to select one that truly delivers value? Setting the right metrics will help you make steady progress toward your goals and keep your team focused on results, not hype.

You'll learn from

Mahesh Yadav

Gen AI product lead at Google, Former at Meta AI, AWS AI l 10k+ AI PM Students

Previously at

See all products from Mahesh