Reliable RAG Agents: Intent-Driven Failure Detection

Jason has built search and recommendation systems for the past 6 years. He has consulted and advised a dozens startups in the last year to improve their RAG systems. He is the creator of the Instructor Python library.

Ben Hylak is co-founder and CTO of Raindrop, building Sentry for AI

agents, transforming how engineering teams monitor their AI agents in production. Raindrop’s “semantic monitoring” pipeline allows engineering teams to instantly serve massive streams of user interactions to find and monitor issues - like tool call failures, user frustration, and custom issues. Raindrop processes tens of millions of events a day to help AI engineering teams at cutting edge companies like Clay.com, Tolans, Oleve, Websim and more find issues with their agents. Previously Ben was a Human Interface Designer at Apple working on the Vision Pro and did engineering at SpaceX and Google.