Staging environment
Hero image

LLM Ops - Large Language Models in Production

Cohort 1 has been open-sourced @ bit.ly/llmops1 LLM Ops has expanded into The AI Engineering Bootcamp @ bit.ly/AIEbootcamp

This course is no longer available.

Explore other courses

Course overview

Become the AI Engineer that every company needs

Future-proof your career with AI engineering. Design, assemble, and operate production LLM applications with industry-standard tools.


Data science and ML skills that were relevant just a few years ago are being outpaced by AI. It is now possible to build what used to take months in a single day!


This course is designed to help you think at the right level of abstraction about AI, and to write code that will help you prototype and ship production LLM applications at the leading edge.

Who is this course for

01

Advanced developers with an understanding of fundamental ML concepts and the product development process

02

Experienced data scientists or machine learning engineers who want to build production LLM applications

03

Software engineers who want to get up to speed on Generative AI and explore new career opportunities

What you’ll get out of this course

Engineer LLM apps from scratch with Python and using LLM Ops frameworks including LangChain and LlamaIndex


Build, deploy, evaluate, and improve production Retrieval Augmented Generation (RAG) Question Answering systems


Master key LLM product development techniques like prompt engineering for retrieval, caching, and versioning


Learn to leverage Agents across LLM Ops frameworks using Reasoning-Action (ReAct) logic

Work with other talented builders to bring your LLM project or startup idea to life

What’s included

Live sessions

Learn directly from Dr. Greg Loughnane & Chris "The LLM Wizard 🪄" Alexiuk in a real-time, interactive format.

Lifetime access

Go back to course content and recordings whenever you need to.

Community of peers

Stay accountable and share insights with like-minded professionals.

Certificate of completion

Share your new skills with your employer or on LinkedIn.

Maven Guarantee

This course is backed by the Maven Guarantee. Students are eligible for a full refund up until the halfway point of the course.

Course syllabus

15 lessons • 7 projects

Week 1

Feb 13—Feb 18

    🧑‍💻 Building Large Language Model Applications in Pure Python

    No module content yet

    Session 1: Intro to LLM Ops

    • 📓

      LLM Ops vs. MLOps and ML Engineering

    • 📓

      Prompt Iteration and Optimization

    • 🧑‍💻

      Beyond ChatGPT: Your First LLM Application

      Submit by Jan 18

    Session 2: Retrieval Augmented Generation (RAG) 101

    • 📓

      Retrieval Augmented Generation (RAG) Overview and Components

    • ✍️

      Your First Retrieval Augmented Generation QA Application

      Submit by Jan 18

Week 2

Feb 19—Feb 25

    Session 3: Intro to LangChain

    • 📓

      The LangChain Framework

    • 📓

      The role of caching

    • 🧑‍💻

      LangChain RAG QA Application with Prompt and Embedding Caches

      Submit by Jan 18

    Session 4: Evaluation of RAG Applications

    • 📓

      RAG ASsessment with RAGAS

    • 📓

      Reranking and Advanced Retrieval Methods

    • 🧑‍💻

      LangChain RAG QA app improvement with advanced retrieval methods

      Submit by Jan 18

Week 3

Feb 26—Mar 3

    Session 5: Deploying Open-Source RAG Applications to Production

    • 📓

      LLM and Embedding Benchmarks

    • 📓

      Hugging Face Leaderboards

    • 🧑‍💻

      Open-Source LangChain RAG QA application deployed to production endpoint

      Submit by Jan 18

    Session 6: Intro to LlamaIndex

    • 📓

      LlamaIndex Framework

    • 📓

      Versioning Indexes

    • ✍️

      LlamaIndex RAG QA application for querying SQL and PDF data simultaneously

      Submit by Jan 18

Week 4

Mar 4—Mar 7

    Session 7: Agents

    • 📓

      Reasoning-Action (ReAct) Framework

    • 📓

      Types of Agents; Agents in LangChain vs. LlamaIndex

    • 📓

      LangSmith and the evolving landscape of LLM Ops

    • ✍️

      Build and deploy a user-friendly RAG application using your choice of framework

      Submit by Jan 18

    Session 8: Demo Day and Graduation!

    • 🧑‍💻

      Final Demo Day Presentation and GitHub Repo!

What students are saying

LLM Application Learning Outcomes

Scores represent students self-reported abilities before and after taking the course (n=27)!

Scores represent students self-reported abilities before and after taking the course (n=27)!

Prerequisite

Be able to leverage a standard Interactive Development Environment (IDE) for building LLM software

Tools include VS Code, UNIX terminal, Jupyter Notebooks, and Conda package management.


🔗 Resource Links: Video, Code

Be able to programmatically access Open AI API as a developer

🔗 gpt-3.5-turbo, gpt-4 links: Video, Code

🔗 gpt-4-turbo, DALL·E 3 links: Video, Code

What people are saying

        Cannot recommend the instructors ENOUGH when it comes to building, shipping, and sharing LLM applications. They inspire a hands-on mentality and are thoughtful and responsive to feedback and requests. They are amazing people to have in your network and were instrumental in helping me switch careers from structural engineering to Generative AI!"
Pano Evangeliou, M.Sc.

Pano Evangeliou, M.Sc.

Senior AI Engineer
        My journey with LLM Ops has been transformative! The hands-on ethos, engaging community sessions, and guidance from instructors demystified LLMs for me, making app dev accessible and enjoyable. This course fueled my confidence, empowered me to bring a passion project to life, and grew my network of LLM practitioners. An inspirational adventure!
Yi Lin

Yi Lin

Associate Manager, Digital, Analytics, and Technology
        More than a course this is an experience. Under the expert guidance of the instructors I found myself going deep into ML, understanding theory, code, and application like never before. Beyond knowledge, it's about community. If you are planning to attend a live course on AI/ML, I recommend AI Makerspace as the best place to get that education.
Juan Olano

Juan Olano

Founder, Senior AI Engineer
        Dedicated to the craft. What a team of instructors! What a community that is being built from the ground, on the cutting-edge, with enthusiasm. They have attracted a special group of practitioners and are selective in the process. At the intersection of data science, machine learning engineering, and MLOps comes the current future of LLMOps.
Dr. Todd Deshane

Dr. Todd Deshane

Software Engineer

Meet your instructors

Dr. Greg Loughnane

Dr. Greg Loughnane

Founder & CEO @ AI Makerspace

I've worked as an AI product manager, university professor, data science consultant, AI startup advisor, and ML researcher; TEDx & keynote speaker, lecturing since 2013.


From 2021-2023 I worked at FourthBrain (Backed by Andrew Ng's AI Fund) to build industry-leading online bootcamps in ML Engineering and ML Operations (MLOps):


🔗 Resources links: Deeplearning.ai demos, AI Makerspace demos, LinkedIn, Twitter, YouTube, Blog.

Chris "The LLM Wizard 🪄" Alexiuk

Chris "The LLM Wizard 🪄" Alexiuk

Co-Founder & CTO @ AI Makerspace

I'm currently working as the Founding Machine Learning Engineer at Ox - but in my off time you can find me creating content for Machine Learning: either for the AI Makerspace, FourthBrain, or my YouTube Channel!


My motto is "Build, build, build!", and I'm excited to get building with all of you!


🔗 Resources links: YouTube, LinkedIn, Twitter

Course schedule

10 hours/week

  • Class!

    Tuesdays & Thursdays, 4:00-6:00pm PT

    • Nov 28th, 30th
    • Dec 5th, Dec 7th
    • Dec 12th, Dec 14th
    • Dec 19th, Dec 21st


    * All sessions will be recorded for your viewing pleasure!

  • Programming Assignments

    2-4 hours per week

    Each class period, we will get hands-on with Python coding homework!

  • LLM Ops Hack Week

    Mon, Dec 18 | Wed Dec 20th | 3-6 PM PT

    • Monday: Team formation, project ideation, storytelling workshop, system diagram presentation
    • Wednesday: Build 🏗️, ship 🚢, and share 🚀 a RAG system with at least the following components: caching, evaluation, visibility 

  • LLM Ops Demo Day

    Thursday, December 21 at 4:00 PM PT

    • Open to the pubic - Friends, family, coworkers and bosses welcome!

  • Office Hours

    Tuesdays and Fridays

    • Greg Loughnane: Mondays @ 8 AM PT
    • Chris Alexiuk: Fridays @ 3 PM PT

Build. Ship. Share.

Build. Ship. Share.

Get hands-on with code, every class

We're here to teach concepts plus code. Never one or the other.

Pair programming made fun and easy

Grow your network. Build together. Feel the difference that expert facilitation makes.

Meet your accountability buddies

Join a community of doers, aimed in the same career direction as you are.

Frequently Asked Questions

Powered by Maven