The Essentials of Machine Learning System Design
A comprehensive step-by-step guide designed to help you work on your ML system, from preliminary steps to deployment and maintenance.
This course is no longer available.
Explore other courses
Previously at
Course overview
Learn how to build and maintain robust, durable ML systems that bring value
ML System Design is a new area in machine learning that deserves to become a separate discipline. While there are plenty of books and courses that cover specific aspects of machine learning, there is scarce literature on the overall landscape of ML system design. Even among highly experienced ML practitioners, there’s a lack of a holistic perspective. Join other specialists seeking to level out these knowledge gaps, and learn directly from two experts in ML and data science with over 20 years of combined experience.
This course introduces machine learning system design as a unified pool of knowledge. We’ve developed a comprehensive framework covering all fundamental aspects of ML system design, and we’ll provide step-by-step guidelines and insights helpful to both novices and experts.
Course highlights:
— 60+ lessons on ML system design, including interactive sessions and practical advice.
— Two use cases with real-life scenarios.
— Stories of wins and failures from our personal experiences.
— Live Q&A sessions to help you synthesize and apply the course material.
You’ll develop:
— A comprehensive knowledge of designing, training, deploying, and maintaining ML systems.
— The ability to confidently implement what you have learned in a real-world environment.
— Hands-on experience that can be shared with colleagues.
This course is for:
01
Mid-career engineers: to hone their skills in building and maintaining solid ML systems and make sure they don’t miss anything critical.
02
Engineering managers and senior engineers: to fill the gaps in their knowledge and view ML system design from a broader perspective.
03
Those starting their journey in machine learning: to have structured guidelines at hand before kicking off their first ML project.
What you’ll get out of this course
A better understanding of your system’s problem space and solution space
You will increase overall awareness of the problem your system needs to solve and define the required steps before system development has started.
Deeper knowledge of the early-stage work of developing an ML system
You will learn more about the importance of picking the right metrics and loss functions, assembling a healthy data pipeline, combining various validation techniques, and preparing the earliest viable version of your future model.
Skills to shape your system into a solid, accurate, and reliable solution
You will strengthen your skills in conducting error analysis, training your pipelines, engineering and evaluating feature sets for your model, and handling testing to evaluate the performance of your system.
Guidance for securing smooth integration and sustainable growth
You will discover the key practices of integrating your solution into the existing ecosystem, the nuances of model monitoring, the challenges of deployment optimization, and the importance of proper maintenance to make your system reliable, manageable, and future-proof.
What’s included
Live sessions
Learn directly from Valerii Babushkin & Arseny Kravchenko in a real-time, interactive format.
Lifetime access
Go back to course content and recordings whenever you need to.
Community of peers
Stay accountable and share insights with like-minded professionals.
Certificate of completion
Share your new skills with your employer or on LinkedIn.
Maven Guarantee
This course is backed by the Maven Guarantee. Students are eligible for a full refund up until the halfway point of the course.
Course syllabus
Week 1
- 📄
Problem space vs solution space
- 📄
Finding the problem
- 📄
Approximating a solution through an ML System
- 📄
Risks, limitations, possible consequences
- 📄
Costs of a mistake
- 📄
Problems as a source of inspiration
- 📄
Build or buy, open source-based or proprietary tech
- 📄
Problem decomposition
- 📄
Choosing the right degree of innovation
- 📄
Goals and antigoals
- 📄
Design document structure
- 📄
A design document is a living thing
Is there a problem?
Preliminary research
Design document
Week 2
- 📄
Losses
- 📄
Metrics
- 📄
Data sources
- 📄
Cooking datasets
- 📄
Data and metadata
- 📄
How much data is enough?
- 📄
Cold start problem
- 📄
Properties of a healthy data pipeline
Loss functions and metrics
Datasets
Week 3
- 📄
Baseline: what are you?
- 📄
Constant baselines
- 📄
Model baselines and feature baselines
- 📄
Variety of deep learning baselines
- 📄
Baseline comparison
Baseline solution
Week 4
- 📄
Learning curve analysis
- 📄
Overfitting and underfitting
- 📄
Residual analysis
- 📄
Finding commonalities in residuals
Error analysis, part 1
Error analysis, part 2
Week 5
- 📄
Understanding training pipelines
- 📄
Tools and platforms for training pipelines
- 📄
Scalability of training pipelines
- 📄
Configurability of training pipelines
- 📄
Testing training pipelines
- 📄
The essence of feature engineering
- 📄
Feature generation 101
- 📄
Feature importance analysis
- 📄
Feature selection
- 📄
Feature store
Training pipelines
Features and feature engineering
Week 6
- 📄
Measuring results
- 📄
A/B testing
- 📄
Reporting results
- 📄
API design
- 📄
Release cycle
- 📄
Operating the system
- 📄
Overrides and fallbacks
Measuring and reporting results
Integration
Week 7
- 📄
Why monitoring is important
- 📄
Data quality and integrity
- 📄
How to monitor and react to arising issues
- 📄
Data drift
- 📄
Concept drift
Monitoring and reliability, part 1
Monitoring and reliability, part 2
Week 8
- 📄
Serving and inference: challenges, tradeoffs, and patterns
- 📄
Tools and frameworks
- 📄
Serverless inference
- 📄
Optimizing inference pipelines
- 📄
Accountability
- 📄
Bus factor
- 📄
Complexity and documentation
Serving and inference optimization
Ownership and maintenance
Week 9
- 📄
Learn How to Create a Design Document
- 📄
Write your design doc
Creating Your Own Design Document
Writing a design document
Week 10
- 📄
Polishing your design document
- 📄
Present your design doc
Publishing Your Own Design Document
Presenting a design document
What students are saying
What people are saying
Reader review
Reader review
Reader review
Reader review
Meet your instructors
Valerii Babushkin
Valerii Babushkin
Senior Principal at BP, Kaggle Grandmaster
Valerii is an accomplished data science leader with extensive experience in the tech industry. He currently serves as Head of Data, Analytics, and AI at BP, where he is responsible for leading the company's data-driven initiatives. Prior to joining BP, Valerii held key roles at leading tech companies, such as Facebook, Blockchain.com, Alibaba, and X5 Retail Group.
Arseny Kravchenko
Arseny Kravchenko
Staff Machine Learning Engineer, Kaggle Master
Arseny is a seasoned ML engineer with a proven track record of building and optimizing reliable ML systems for startups, including real-time video processing, manufacturing optimization, and financial transactions analysis.
Course schedule
3-6 hours per week
Oct 5 — Dec 8
Every Saturday and Sunday, 4 p.m. BST
20 modules stretched over 10 weeks
66 lessons overall
Live Q&A sessions to wrap up each module
Questions trigger fruitful discussions, so speak up!
Learning is better with cohorts
Active hands-on learning
This course builds on live workshops and hands-on projects
Interactive and project-based
You’ll be interacting with other learners through breakout rooms and project teams
Learn with a cohort of peers
Join a community of like-minded people who want to learn and grow alongside you