


Free Lesson
Agentic Experiment Design & Analysis
Part of The Agent-Powered Super IC
60 min
Jun 5, 2026 3:00 PM
What you'll learn
Frame experiments using the Agentic Loop
Apply the 4 stages (Frame, Power, Validate, Decide) to design tests that hold up to scrutiny.
Catch the validity gaps that ship bad decisions
Spot sample ratio mismatch, novelty effects, and underpowered tests before the readout meeting.
Use AI agents to design and analyze A/B tests
Apply a design helper prompt and watch Claude Code produce a full experiment readout from raw data.
Why this topic matters
Most experiment readouts ship bad decisions: p-values without validity checks, no segment analysis, vague recommendations. AI makes it easier to crunch numbers, which makes it easier to crunch the wrong ones. Learn the Agentic Experiment Loop to design, validate, and analyze tests, then see how AI agents accelerate every stage without replacing judgment.
You'll learn from

Shane Butler
Principal Data Scientist at Ontra

Sravya Madipalli
Senior DS Leader (Ex-Microsoft)

Hai Guan
Head of Data at Ontra, Ex-LinkedIn