Reading Results
Drift status
| Status | Meaning |
|---|---|
| OK (No Drift) | The production data matches the baseline. |
| Warning | Some drift. Investigate the features driving it. |
| Alert (Drift Detected) | Significant drift. Consider retraining. |
| Could not analyze | No algorithm could score the window; each row says why. |
Drift score
The drift score is the mean PSI (Population Stability Index) across the model's features: a plain number, not a percentage. The drift status comes from it: by default a score of 0.1 or more is a warning and 0.2 or more an alert (the PSI thresholds can be changed through the REST API or Python SDK). When PSI scores no feature, there is no drift score and the status is the worst any other algorithm gave. When a window holds few predictions, neighboring value ranges are compared together, so chance differences between two small samples do not read as drift; the same window always gives the same score.
Overview
The Overview tab shows the drift score, the analysis window, the number of predictions analyzed, how many features drifted out of those analyzed, and the most drifted features. Algorithms that were skipped are listed with why. For a result that found drift, Notified shows how it was announced: the owner in Strongly, email (to whom) and Slack, or why a channel was not sent.
Feature Drift
The Feature Drift tab lists one row per feature, most drifted first, with every algorithm's result on it: its value, threshold and status. Algorithms about the dataset as a whole are rows of their own, named like [multivariate] domain_classifier or [performance] cbpe. Search by feature name, or sort by status, score or name. A row where nothing could be scored says why.
Performance
When predictions in the window have actuals, Performance Tracking on the Monitoring tab and Prediction Drift on the drift dashboard's Overview show the version's performance on them (accuracy, mean absolute error or review score, by its task) against its performance on its reference rows, the change, and how much better or worse that is than the baseline. Until actuals arrive it says so. With a baseline built from rows that include actual, prediction and confidence, CBPE estimates the accuracy drop of a classifier or reviewed model before actuals arrive; a regressor skips CBPE, saying why.
Prediction drift raises the analysis's status: under Prediction drift on the drift dashboard's Algorithms tab, Warning when performance is worse by (%) (default 5) and Alert when performance is worse by (%) (default 10) set how much worse than the baseline, relatively, its performance may get. Lower accuracy or review score is worse, as is a higher mean absolute error. A warning or alert makes the whole analysis one, so it is announced like feature drift.
History
The History tab lists every analysis with its status, drift score, sample size and duration.
Predictions
The dashboard's Predictions tab lists the model's recorded predictions, as the model's Monitoring tab does (see Monitoring), with their actuals.
Baselines
The Baselines tab lists every baseline of the model's versions (see Managing baselines).