Deploying Models: APIs, Docker, Tracking & CI
From a notebook to a service: saving models, batch vs real-time, FastAPI, input validation, Docker, experiment tracking, CI and monitoring.
From notebook to product
A model in a notebook helps nobody. Deployment means making its predictions available where decisions are made, reliably, and in a way you can update, monitor and roll back. Most ML projects that fail do so here, not in modelling.
Two common shapes. Batch prediction: a scheduled job (nightly, hourly) loads new rows, predicts, and writes results to a table that other systems read; e.g. tomorrow's no-show risks for the call list. It is simpler, cheaper and easier to debug. Real-time (online) prediction: a service answers each request within milliseconds, needed when the input only exists at that moment, e.g. a card payment at checkout. Choose batch unless the use case truly needs real-time.
Either way you need the same pieces: a saved, versioned model; the exact same preprocessing as in training (ship the whole scikit-learn Pipeline, not just the model); input validation; logging; and a way to monitor and roll back.
Saving models and serving them with an API
Save the fitted pipeline with joblib.dump(pipe, "model.joblib") and record which code version, data and parameters produced it. (Only load model files you trust: loading a pickle can run arbitrary code.)
An API (application programming interface) lets other software ask for predictions over HTTP, usually by sending JSON to an endpoint such as POST /predict. FastAPI is the most popular Python framework for this: you describe the expected input with a Pydantic model, and FastAPI validates every request (types, required fields, ranges) and returns a clear 422 error automatically when it is wrong.
Return more than a number: the probability, the decision, and the model version, and log each request (without personal data) so you can investigate later. Add a GET /health endpoint that monitoring systems can call.
Try it: call a prediction API
A simulated POST /predict endpoint with the validation rules a real service would have. Edit the JSON or inject a mistake, send it, and read the status code and response.
Interactive: this part runs in your browser.
Code example: the logic behind the endpoint
A worked example: read it step by step (every part is explained), then run it and experiment.
Interactive: this part runs in your browser.
Docker, CI and experiment tracking
Docker packages your code, its dependencies and the operating-system pieces it needs into an image that runs identically on your laptop, a colleague's machine and a cloud server. A short Dockerfile describes how to build it; docker build creates the image and docker run starts a container from it. This removes "it works on my machine" problems.
Continuous integration (CI), e.g. GitHub Actions, runs your tests, linting and a small training smoke test on every pull request, so broken code can't be merged. Continuous delivery then builds and deploys the image automatically after merging.
Experiment tracking (MLflow, Weights & Biases) records every training run's parameters, metrics, data version and the resulting model file, so you can answer "which run produced the model in production, and how was it trained?". A model registry stores approved versions with stages such as "staging" and "production", which makes rolling back a one-line change.
Monitoring after launch
Models degrade silently. Monitor four layers: service health (latency, errors, uptime); data quality (missing values, out-of-range inputs, schema changes); drift in input features and in the prediction distribution (e.g. PSI, which needs no labels); and outcome metrics once labels arrive (precision, recall, calibration, per group). Every alert needs an owner and a runbook, and you need a tested way to fall back to the previous model or a simple rule. The game milestone after this topic puts all of this into practice.
Code: a minimal production setup
The four files most small ML services start with.
from fastapi import FastAPI
from pydantic import BaseModel, Field
import joblib, pandas as pd, logging
MODEL_VERSION = "noshow-gb-2025-11-01"
pipe = joblib.load("model.joblib") # the whole sklearn Pipeline
app = FastAPI()
log = logging.getLogger("predict")
class Appointment(BaseModel): # FastAPI validates every request against this
lead_days: int = Field(ge=0, le=365)
age: int = Field(ge=0, le=120)
distance_km: float = Field(ge=0, le=500)
sms_sent: int = Field(ge=0, le=1)
prior_noshows: int = Field(ge=0, le=100)
@app.get("/health")
def health():
return {"status": "ok", "model_version": MODEL_VERSION}
@app.post("/predict")
def predict(a: Appointment):
p = float(pipe.predict_proba(pd.DataFrame([a.model_dump()]))[0, 1])
log.info("prediction p=%.3f version=%s", p, MODEL_VERSION) # no personal data in logs
return {"probability": round(p, 3), "decision": "call" if p >= 0.3 else "sms", "model_version": MODEL_VERSION}
# run locally: uvicorn app:app --reload then open http://127.0.0.1:8000/docsFROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY app.py model.joblib ./
EXPOSE 8000
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]
# docker build -t noshow-api .
# docker run -p 8000:8000 noshow-apiname: ci
on: [pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: "3.11" }
- run: pip install -r requirements.txt pytest ruff
- run: ruff check .
- run: pytest -qimport mlflow, mlflow.sklearn
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.metrics import average_precision_score
params = {"learning_rate": 0.05, "max_leaf_nodes": 31}
with mlflow.start_run(run_name="gb-baseline"):
mlflow.log_params(params)
model = HistGradientBoostingClassifier(**params).fit(X_train, y_train)
ap = average_precision_score(y_val, model.predict_proba(X_val)[:, 1])
mlflow.log_metric("val_pr_auc", ap)
mlflow.sklearn.log_model(model, "model") # stored with the run; register it if approvedQuiz
1. You need tomorrow's no-show risk for every booked appointment, once a day. The simplest suitable design isโฆ
2. Why save the whole scikit-learn Pipeline rather than only the model?
3. A request has age: "forty". A well-built API returnsโฆ
4. What problem does Docker mainly solve?
5. Which signal can you monitor before any labels arrive?
๐ Further reading
Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.
- FastAPI tutorial โ Official documentation
- Docker: Get started โ Official documentation
- MLflow documentation โ experiment tracking and a model registry
- Made With ML โ Free MLOps course