A10i-labs

Fine-tuning Small Language Models: dataset to deployment

Hamid Bagheri

AI Eng Leader, PhD CS | GenAI/LLMs | 20+ yrs software, data science, AI/ML

Write a dataset specification for a domain task, prompt templates, and instruction style.

Define the task, dataset spec, and prepare training data

1B to 7B model in Colab, locally, or a single GPU environment

Run a parameter-efficient fine-tuning

Quick offline eval, regression checks, and a simple acceptance rubric (quality, safety, latency, cost).

Evaluate and decide if the model is shippable

Stable outputs, input/output contract, and basic observability.

Deploy as base model and adapter with a production contract

Learn proven approaches for quickly improving AI applications.  Build AI that works better than the competition, regardless of the use-case.

ML Engineer with 20 years of experience

Airbnb

GitHub

DataRobot

AlixPartners

Hamel Husain

ML Engineer with 20 years of experience.

ML Systems & Applied AI Evals Researcher

Google

UC Berkeley

Stanford University

Shreya Shankar

ML Systems Researcher Making AI Evaluation Work in Practice

Cohort 5

Cohort 6

🚨 September 2026 cohort's material is completely refreshed. 🚨All students get:<ul><li>♾️ Unlimited access to future cohorts &amp; office hours: never worry about timing or missing new material.</li><li>🗄️ Lifetime access to all materials!</li><li>🤖 6 months of unlimited access to our new AI Eval Assistant (more info below).</li><li>🧑‍🏫 10+ hours of office hours to maximize the value of live interaction.</li><li>🏫 A Discord community with continuous access to instructors to get unstuck (even after the course!).</li></ul>---Do you catch yourself asking any of the following questions while building AI applications?1. How do I test applications when the outputs require subjective judgements?2. If I change the prompt, how do I know I'm not breaking something else?3. Where should I focus my engineering efforts? Do I need to test everything?4. What if I have no data or customers, where do I start?5. What metrics should I track? What tools should I use?6. Can I automate testing and evaluation? If so, how do I trust it?If so, this course is for you.All sessions are live and recorded.

Stop guessing if your AI works. Build the feedback loops that make it better.

How To Collect Data For Evals

Get immediate clarity and direction with Error Analysis

Implement Effective Evaluations

Master Architecture-Specific Eval Strategies

Run Evals In Production

Ensure That Evals Lead To High ROI

Engineers and PMs who ship prompt changes and hope nothing breaks. (You'll learn to measure impact before and after every change.)

Teams still spot-checking AI outputs by hand instead of measuring systematically. (You'll learn how build automated evals you can trust.)

Leaders who don't know where their AI is failing or where to invest resources. You'll learn how to systematically find &amp; prioritize issues.

"Despite being a student in Harvard’s Master’s in Data Science program, I've struggled to understand the concepts that make LLMs so impressive. This class gave me the answers." - Tony "Ishan takes a black box, rips it apart to understandable compartments, and clearly communicates what is going on. Highly recommend." - Joe, CFO ______________________________________________________________________ The current cohort completed recording on January 17th, 2025 but you can still enroll to participate asynchronously: •&nbsp;Get lifetime access to all recorded sessions, including interactive Q&amp;A segments•&nbsp;Connect with peers in our dedicated private Discord community•&nbsp;Master the material through comprehensive quizzes and training resources•&nbsp;Receive direct instructor support via email or Discord whenever you need help•&nbsp;The option to attend a future live cohort for free______________________________________________________________________ Why This Course Stands Out: You already know how vital AI is to your career—it's in every conversation, project, and strategy session. But diving into AI feels daunting with limited time for self-study or complex programming. That’s where this course comes in. Skip the endless prep work and gain an intuitive, hands-on understanding of AI and transformers in just a few hours—without needing an advanced degree. No other course offers such an interactive, fast, and in-depth technical understanding of AI using an actual GPT-2 model, all using an easy-to-learn spreadsheet interface. ______________________________________________________________________ Perfect for Busy STEM Professionals:Whether you're a developer, technical manager, or executive, this course demystifies AI and teaches you exactly how Large Language Models (LLMs) like GPT-2 work, with no prerequisites beyond a STEM background. It’s ideal for: • Software/Web Developers seeking an easy introduction to machine learning. • Science &amp; Tech Professionals navigating AI transformations in their fields. • Technical Sales/Marketing crafting messaging for AI products. • Product/Project Managers overseeing LLM-powered solutions. • Executives/Leaders guiding AI strategy in their organizations. ______________________________________________________________________ What You’ll Learn:Using a unique spreadsheet implementation of GPT-2 (the predecessor to ChatGPT), we’ll break down each model component with crystal-clear explanations. In just a few sessions, you’ll be able to: • Understand how LLMs work—as quickly as possible. • Explain every key component—tokenization, embeddings, attention, and more. • Gain AI literacy—so you can confidently lead and manage AI-driven teams. • Build a foundation for further AI projects—and even more advanced learning. ______________________________________________________________________ Why Spreadsheets?You don’t need to be a coder or a calculus expert to understand AI—just the right approach. This course uses an innovative, no-code, spreadsheet-based method to teach LLMs. It's fast, interactive, and designed for busy professionals like you. By breaking down complex AI principles into easy-to-follow spreadsheet exercises, you’ll gain a lasting, foundational understanding of how large language models work. This is knowledge you can carry across into any model, any programming language, and any project. ______________________________________________________________________ Hands-On Learning, No Programming Required: By the end of the course, you’ll have a working, detailed understanding of how transformers power LLMs like ChatGPT, Claude, and Llama. Plus, you'll have a spreadsheet tool you can explore and tweak to deepen your understanding further. ______________________________________________________________________ See it for Yourself: Still unsure? Watch the FREE lightning lesson:"Inside the Brain of an LLM: How it Works in 30 Minutes" to see how accessible AI learning can be: https://maven.com/p/eeea6f/inside-the-brain-of-an-llm-how-it-works-in-30-minutes ______________________________________________________________________ Join the Community of Learners: • “I could’ve read 20 papers on this and not gotten nearly the understanding I got from this hands-on course.”— Class Participant from Adobe • “Highly recommend this course for anyone wanting to build a foundation in LLMs. Ishan’s interactive approach makes complex concepts incredibly clear.”— Matt, Business Intelligence Analyst ______________________________________________________________________ Practical Details: • Course Access: All sessions are recorded and available to stream after the course concludes, so you’ll never miss a thing.• Reimbursement: Your company may cover this course if you're building or transitioning into AI at work. ______________________________________________________________________ New for this cohort! We've dropped the requirement for Excel. We have a browser based alternative. Some exercises may use Google Sheets and/or Python.

Skip the mountains of prep work and learn how AI works in just a few hours

Course syllabus

If your events are recurring and at the same time, it might be easiest to use a single line item to communicate your course schedule to students

Tuesdays & Thursdays

Feel free to type out dates as your title as a way to communicate information about specific live sessions or other events.

May 7, 2022

Schedule items can also be used to convey commitments outside of specific time slots (like weekly projects or daily office hours).

Weekly projects

Course schedule

Executives, engineering managers, and product managers leading and coordinating technical AI teams leveraging LLMs

Developers without formal training in machine learning that want to quickly understand the LLM technology in their projects

A gentle introduction to LLMs for professionals pivoting their career to AI from traditional software engineering or other STEM fields

Anyone who can read a spreadsheet that is motivated and deeply interested in how the next wave of computing works

Who is this course for

What you’ll get out of this course

This course breaks down complex AI concepts into easy-to-follow steps using Excel, making it accessible for anyone with a STEM background.

Understand how LLMs work as quickly as possible

Precisely understand in detail the architecture and function of the GPT2 model, including tokenization, embeddings, attention, multi-layer perceptron, and layernorm.

Recognize and explain every component of an LLM

You'll gain a solid understanding of how LLMs function, enabling you to confidently engage with engineers and technical stakeholders.

AI Literacy for managing technical teams

Build a foundation that equips you with the essential knowledge to explore advanced AI topics, undertake AI projects, or pursue further coursework.

Prepare yourself for further exploration, projects, or coursework in artificial intelligence

Master AI Concepts with a Real LLM Implemented Entirely in a Spreadsheet. No Complex Math. Just Clarity. 

Jan 8—17, 2025 (On-demand)

Ishan Anand

Unlock a personalized learning experience.  Dates and times will be customized to the participant.

1-on-1 Premium Version 

Transform from consumer of analytics to independent operator - asking sharp questions, validating answers, shipping decisions.

Principal Data Scientist @ Ontra 

Stripe

Nextdoor

PwC India

Ontra

AppFolio

Shane Butler

Principal Data Scientist @ Ontra | Ex-Stripe, Nextdoor, PwC, Appfolio

Data Science Leader |Ex-Microsoft

Microsoft

eBay

Grammarly

Superhuman

Sravya Madipalli

Data Science Leader@Superhuman (Prev. Grammarly)| Ex-Microsoft, eBay, Nextdoor

Head of Data @ Ontra AI | Ex-LinkedIn

LinkedIn

Pinterest

Hai Guan

Head of Data @ Ontra | Ex-Nextdoor, LinkedIn, Pinterest, Meta

Cohort 1

Cohort 2

Cohort 1 is in Session, but it's not too late to enroll!! Kick off was April 20. Final registration closes April 27. Everything has been recorded so you don't miss a thing! As AI reduces the value of deep domain silos, the people who thrive will be the ones who can work across disciplines and ship. Analytical independence - the ability to validate your own ideas and make data-informed decisions - remains the missing skill in most builders' toolkits.AI can conduct the analysis, but it can't tell you what to ask or whether the answer makes sense. That requires analytical judgment.This course teaches you to think like a senior Product Data Scientist. You'll learn the frameworks used at FAANG companies to frame problems, choose the right metrics, diagnose root causes, execute the analysis, and turn insights into decisions.-&gt; <a target="_blank" rel="noopener noreferrer nofollow" href="https://join.slack.com/t/dataneighbor/shared_invite/zt-3ox4drc3r-3g0ptSthytkgv0sD6ovvqg">Join AI Builders Slack Community!!</a> &lt;-What makes this different: No SQL. No stats lectures. Five weeks of hands-on real scenarios. You'll build end-to-end, metric specs, deep dives, experiment plans, stakeholder readouts.The result: You become analytically self-sufficient. The last dependency removed from your toolkit. Make data-informed decisions in hours, not weeks.

Ask Better Questions, Make Faster Decisions - with AI as Your Analyst

Frame Problems Like a Senior Data Scientist

Design Metrics That Actually Drive Decisions

Diagnose Root Causes Without Guessing

Design Experiments That Ship

Turn Analysis Into Action With AI as Your Copilot

Synthesize Insights and Prioritize for Impact

Product Managers, engineers, and operators who need analytical independence and frameworks to measure success and drive decisions.

Designers and researchers stepping into strategy roles who need to quantify impact, run experiments, and influence with data-driven insights

Data professionals who want to frame better questions, delegate technical execution to AI, and drive impact with agentic analytics tools.

Acquire and develop a critical skill for product managers who are leading and contributing to AI products.

GenAI Product Builder @ Google, ex-Meta

Marily Nika, Ph.D AI/ML

Product Manager AI&ML @ Google

Diego Granados

Senior Data, ML and GenAI Scientist

George Zoto

Senior NLP/AI Scientist

Cohort 4

“Does it work?”... “Is it good enough?”... “Can we ship it?”...How do you answer these questions for AI products? You’re responsible for “running evals” but what does that mean?How do you choose the right metrics, interpret fuzzy results, and make a confident decision?This course gives you a framework to do just that.<ul><li>Map user value to evaluation (eval) objectives so your metrics aren’t abstract. Define success then translate it into measurable criteria.</li><li>Choose metrics you can actually maintain: capability, safety, UX friction, latency, cost and “does this reduce support tickets or increase activation.”</li><li>Set ship/no-ship thresholds you can defend to leadership.</li><li>Build lightweight workflows that work in real teams: human review where it matters, automation where it lasts, documentation that drives decisions.</li><li>Consider domain constraints (e.g., healthcare safety) and know what to avoid: silent failures, misleading proxy metrics and tests that don’t reflect production.</li><li>Tie everything to ROI: impact vs unit cost, eval coverage vs reliability, and the minimum viable monitoring you need post-launch.</li></ul>Experience AI evals through a case-based approach with a real AI product that we evaluate together.

Eliminate uncertainty in shipping AI features

Make confident ship or hold decisions for AI features

Translate user value into clear evaluation goals

Choose the right metrics for capability, safety, UX, and cost

Set defensible thresholds leadership will trust

Build lightweight evaluation workflows teams can maintain

Navigate domain constraints and avoid common failures

PMs leading AI features, growth, or platform initiatives

PMs who partner with ML teams and want to set evaluation standards

PMs who need to make clear “ship or hold” calls without doing the engineering

Fine-tuning Small Language Models: dataset to deployment

What you'll learn

Why this topic matters

You'll learn from

Hamid Bagheri