12-02, 08:30–09:00 (UTC), Talk Track I
We like talking about production – one famous, but probably wrong statement about it is “87% of data science projects never make it to production”.
While giving a talk to a group of up-and-coming data scientists, a question that surprised me came up:
When you say “production”, what exactly do you mean?
Buzzwords are great, but all the cool kids know what production is, right? Wrong.
In this talk, we’ll define what production actually means. I’ll present a first-principles, step-by-step approach to thinking about deploying a model to production. We’ll talk about challenges you might face in each step, and provide further reading if you want to dive deeper into each one.
This talk will cover the following topics:
- Defining ML in production from first principles
- What types of production that aren’t deployment exist
- The difference between model deployment and pipeline deployment
- Explaining the case we’ll focus on - Deploying a single model to production, which receives data, makes a prediction and returns that prediction to an accessible location.
- “Easy” deployment solutions (Streamlit, Gradio) and their limitations
- Breakdown of stages:
1. Creating code that takes a trained model, receives data, predicts and returns the prediction
2. Wrap that in an API, which can receive the data via some request (HTTP), and returns the prediction in some standard format (JSON)
1. Caveats for authentication
3. Put that API in an environment that enables is to be portable and run across hardware types (e.g. Docker)
4. Provide infrastructure, via the cloud, and run the container to listen for requests
1. Caveats for GPUs
- Further reading
Previous knowledge expected
Always learning and a builder at heart. Dean has worked on quantum optics and communication, computer vision, software development, and design – taking a multi-disciplinary approach and applying it to build products for data scientists and machine learning engineers.
Dean is currently the CEO & Co-Founder of DagsHub, a platform for data scientists and machine learning engineers, combining popular open-source tools and formats, to version their data, models, experiments, and code.
Dean is also the host of the MLOps Podcast, where he speaks with industry experts about getting ML models to production.