Skip to main content

Baselines

A baseline summarizes a model version's reference rows: the data it was trained or validated on. Drift compares the version's production predictions with its own baseline.

Building a baseline​

  1. On the model's Monitoring tab, select Build Baseline.
  2. Choose the Version the rows describe (the active version is selected).
  3. Upload a CSV, or pick one from a shared volume. It can be any size. It needs a column for every input feature the model declares. Optionally add:
    • actual and prediction to record the version's performance on its reference rows (accuracy, or mean absolute error for a regressor). For a multilabel, time series or other model, actual alone: each row's reviewed verdict on the version's output (true/false or a 0 to 1 score);
    • confidence (the probability of the predicted class) as well, so the platform can estimate how often the version is right on live traffic before actuals arrive. A regressor has no confidence calibration: its performance is its error, measured from actuals.
  4. Optionally turn on Analyze drift daily once the baseline is built. It turns on the model's daily drift schedule; you can change it any time in the drift settings.
  5. Select Build Baseline. Its progress shows live; you can close the dialog and it keeps running.

When the baseline is built it becomes the version's active baseline, and the version's first drift analysis starts, over the last 7 days of production predictions. If the version has no production predictions in that time yet, the job says so, and the first analysis runs once it has some (with Run Analysis, or on the drift schedule).

Models published from AutoML get a baseline automatically, built from the reference rows their training job saved, and their drift is analyzed daily from then on: the first day the model has production predictions, it gets a result without anyone starting one. A baseline needs at least the model's minimum baseline rows (30 unless its drift settings say otherwise) for drift to be measured against it. A model trained on a small set may get a smaller baseline from its few held-back rows: the model's Monitoring tab then says how many rows the baseline has and how many drift needs, drift is not analyzed or scheduled, and Run Analysis is not offered until you build a baseline from more rows.

Models served outside the platform​

A model registered without versions (served elsewhere, with its predictions sent to the platform) has one baseline for the whole model. Build Baseline then has no version to choose, and drift compares every prediction sent for the model with that baseline.

Managing baselines​

A version keeps one active baseline. When you build a new one, the one it replaces is kept, inactive. The Baselines tab of the drift dashboard lists every baseline with its version, number of features and rows, its performance on its reference rows (by its task) and the file it was built from. Activate makes an earlier baseline its version's active one again; drift compares with it from the next analysis on.

A version without a baseline cannot be analyzed. The Monitoring tab says so and offers Build Baseline.