Skip to main content

Drift Detection

Drift detection for registry (traditional ML) models.

Access it as client.drift_detection on a Strongly client, or the same path on AsyncStrongly with await. All methods exist on both with identical signatures.

Methods​

All methods​

active_baseline​

active_baseline(model_id: str) -> ReferenceBaseline | None

Return the active baseline of the model's active version.

(what drift compares that version with), or None when it has none.

analyze​

analyze(*, model_id: str, window_days: int | None = None, window_start: str | None = None, window_end: str | None = None) -> DriftAnalysisJob

Start a drift analysis.

The active version's production predictions in the window (default the last 7 days) against its active baseline.

A version without a baseline is refused. The run's progress, and why it failed when it produced no result, is the model's latest_analysis.

analyze_status​

analyze_status(job_id: str, *, model_id: str) -> dict[str, Any]

One drift analysis run by the job id analyze returned.

Its status (pending, deploying, running, completed, failed), window, and error when it failed.

latest_result​

latest_result(model_id: str) -> DriftResult | None

Return a model's most recent drift result, or None when it has never been analyzed.

Its rows are result_features.

list_baselines​

list_baselines(model_id: str, *, search: str | None = None, sort: str | None = None, limit: int | None = None) -> SyncPaginator[ReferenceBaseline]

Return a model's baselines (newest first), each with its inputs' names.

Build one with build_baseline.

Parameters

  • sort (str | None, optional): -createdAt (default), modelVersion, sampleSize, baselinePerformance.value.

list_predictions​

list_predictions(model_id: str, *, source: str | None = None, start_date: str | None = None, end_date: str | None = None, search: str | None = None, sort: str | None = None, limit: int | None = None) -> SyncPaginator[PredictionRecord]

Return a registry model's recorded predictions.

(newest first), each with its actual when one was recorded.

Parameters

  • source (str | None, optional): Only production or test (the model page's Test tab) predictions.
  • start_date, end_date (str | None, optional): ISO 8601 bounds on the prediction time.
  • search (str | None, optional): Matches the prediction id, entity id or error, or the prediction exactly.
  • sort (str | None, optional): -timestamp (default), latencyMs, modelVersion, success or confidence (- for descending).
  • limit (int | None, optional): Maximum number of predictions (default: all).

log_prediction​

log_prediction(*, model_id: str, features: Mapping[str, Any], prediction: Any, probabilities: Sequence[float] | None = None, entity_id: str | None = None, model_version: int | None = None, latency_ms: float | None = None) -> str

Record a prediction a registry model made outside the platform.

(a copy served elsewhere), so drift includes it. Predictions the platform serves are recorded automatically.

Parameters

  • model_id (str): The registry model.
  • features (Mapping[str, Any]): The input values, by feature name.
  • prediction (Any): The model's output.
  • probabilities (Sequence[float] | None, optional): Per-class probabilities; required for a classification model (the highest is its confidence).
  • entity_id (str | None, optional): Your id for what was predicted; actuals can join on it.
  • model_version (int | None, optional): The model version that made the prediction.
  • latency_ms (float | None, optional): How long the prediction took.

Returns

  • str: The prediction id (actuals can join on it).

models​

models(*, status: str | None = None, search: str | None = None, sort: str | None = None, limit: int | None = None) -> SyncPaginator[DriftModel]

Return the registry models you can see with their drift state.

The latest result (DriftModel.latest_drift) and the latest analysis run (DriftModel.latest_analysis, with why it failed).

Parameters

  • status (str | None, optional): Only models whose latest result is ok, warning, alert or error, or no-data for models never analyzed.
  • search (str | None, optional): Matches name, description or framework.
  • sort (str | None, optional): -latestDrift.calculatedAt (default), name, latestDrift.driftScore, latestDrift.sampleSize, updatedAt.
  • limit (int | None, optional): Maximum number of models (default: all).

result_features​

result_features(result_id: str, *, model_id: str, rows: str | None = None, search: str | None = None, sort: str | None = None, limit: int | None = None) -> SyncPaginator[DriftResultRow]

Return a drift result's rows, most drifted first.

One per feature (every per-feature algorithm's metric on it) and one per algorithm about the whole dataset.

Parameters

  • result_id (str): The result's id (DriftResult.id, or a model's latest_drift.result_id).
  • model_id (str): The model the result belongs to.
  • rows (str | None, optional): Only features, or only dataset rows.
  • search (str | None, optional): Matches the feature name.
  • sort (str | None, optional): -severity (default), score, featureName, order.
  • limit (int | None, optional): Maximum number of rows (default: all).

results​

results(model_id: str, *, status: str | None = None, search: str | None = None, sort: str | None = None, limit: int | None = None) -> SyncPaginator[DriftResult]

Return a model's drift results (newest first).

Parameters

  • status (str | None, optional): Only ok, warning, alert or error results.
  • sort (str | None, optional): -calculatedAt (default), driftScore, sampleSize.
  • limit (int | None, optional): Maximum number of results (default: all).

settings​

settings(model_id: str) -> DriftSettings | None

Return a model's drift settings, or None when they were never saved.

update_settings​

update_settings(model_id: str, *, enabled: bool | None = None, algorithms: Mapping[str, Any] | None = None, performance_drop_warning: float | None = None, performance_drop_alert: float | None = None, min_baseline_sample_size: int | None = None, min_window_sample_size: int | None = None, notifications: Mapping[str, Any] | None = None, schedule: Mapping[str, Any] | None = None) -> DriftSettings | None

Save a model's drift settings and return them.

Only the settings given change; the first save stores defaults for the rest.

Parameters

  • enabled (bool | None, optional): Drift settings on; required for the schedule to run.
  • algorithms (Mapping[str, Any] | None, optional): {"enabled": [names], "overrides": {name: {"warningThreshold", "alertThreshold", "params"}}}.
  • performance_drop_warning, performance_drop_alert (float | None, optional): How much worse than the baseline the model's performance may get (relative: 0.05 = 5% worse; lower accuracy or review score, higher mean absolute error) before prediction drift is a warning (default 0.05) or an alert (default 0.10); the warning must be smaller. A warning or alert makes the analysis one too.
  • min_baseline_sample_size (int | None, optional): Fewest reference rows a baseline needs (default 30).
  • min_window_sample_size (int | None, optional): Fewest predictions with recorded inputs an analysis window needs to be scored (at least 1; default 30).
  • notifications (Mapping[str, Any] | None, optional): {"email": bool, "recipients": [addresses], "slack": incoming webhook URL}. When an analysis finds drift (warning or alert) the model's owner is notified in Strongly; with email the recipients are emailed (the owner when there are none); with slack that webhook gets a message. An address that is not one, or a webhook pointing at localhost, an internal host or a private address, is refused. Each result records how it was announced (DriftResult.notification).
  • schedule (Mapping[str, Any] | None, optional): {"enabled": bool, "frequency": "hourly" | "daily" | "weekly", "timeWindowDays": int}`; it runs while `enabled (above) is true too.