How Evals Made GitHub Copilot Happen

John Berryman is the founder of Arcturus Labs. His journey through AI and search technologies includes contributing to GitHub Copilot's early development, where he worked on the team that brought AI-assisted coding from concept to reality. Throughout his career, John has helped build search and recommendation systems that millions use daily – from GitHub's code search infrastructure to Eventbrite's discovery platform and the US Patent Office's next-generation search system. This blend of experience in both foundational search technologies and cutting-edge AI applications gives him unique insight into building practical, powerful LLM applications.

John shares his expertise through two books: Relevant Search, which reveals the art and science of building search applications, and Prompt Engine, which guides developers through the emerging practice of language model applicatio

Shawn Simister is an experienced software engineer specializing in AI-powered developer tools and machine learning systems. With extensive experience at GitHub and Google, he has contributed to innovative new projects including GitHub Copilot and TensorFlow. His work spans from designing the initial prototype that launched GitHub Copilot Chat to optimizing core TensorFlow operators and building comprehensive evaluation frameworks for AI agents.

As an educator, Shawn helped design Google's Machine Learning Crash Course and prototyped the first version of the Google Codelabs publishing framework that scaled from a dozen to over 1,300 interactive tutorials. His passion lies in creating tools that augment human intelligence rather than replace it, with particular expertise in prototyping, API d

Currently working on GroundRules.ai, Shawn continues to push the boundaries of AI-assisted software development while sharing his insights on building wor

Hamel is a machine learning engineer with over 20 years of experience. He has worked with innovative companies such as Airbnb and GitHub, which included early LLM research used by OpenAI, for code understanding. He has also led and contributed to numerous popular open-source machine-learning tools. Hamel is currently an independent consultant helping companies build AI products.