Skip to main content

Drift Detection

Drift detection for registry (traditional ML) models.

Access it as client.drift_detection on a Strongly client, or the same path on AsyncStrongly with await. All methods exist on both with identical signatures.

Methods​

All methods​

active_baseline​

active_baseline(model_id: str) -> ReferenceBaseline | None

Return the active baseline of the model's active version.

(what drift compares that version with), or None when it has none.

analyze​

analyze(*, model_id: str, window_days: int | None = None, window_start: str | None = None, window_end: str | None = None) -> DriftAnalysisRun

Start a drift analysis.

The active version's production predictions in the window (default the last 7 days) against its active baseline.

A version without a baseline is refused. The run's progress, and why it failed when it produced no result, is the model's latest_analysis.

analyze_status​

analyze_status(job_id: str, *, model_id: str) -> DriftAnalysisRun

One drift analysis run by the job id analyze returned.

Its status (pending, running, completed, failed), what started it, its window, its result in short when it completed, and its error when it failed.

create_algorithm​

create_algorithm(*, name: str, code: str, supports: Sequence[str], description: str | None = None) -> DriftAlgorithm

Create a custom drift algorithm for your organization and return it.

Parameters

  • name (str): Its name, unique in your organization (a model's drift settings enable it by name).
  • code (str): Python defining detect_drift(reference_data, current_data); numpy, math, random, json and scipy.stats can be imported.
  • supports (Sequence[str]): numerical and/or categorical.
  • description (str | None, optional): What it detects.

Raises

  • ConflictError: Another algorithm of your organization has this name.
  • ValidationError: The name or code is empty, or supports names another kind.

delete_algorithm​

delete_algorithm(algorithm_id: str) -> None

Delete a custom drift algorithm, with its tests.

Raises

  • UnprocessableEntityError: A model's drift settings run it (resource-in-use, naming the models).

latest_result​

latest_result(model_id: str) -> DriftResult | None

Return a model's most recent drift result, or None when it has never been analyzed.

Its rows are result_features.

list_algorithms​

list_algorithms(*, search: str | None = None, limit: int | None = None) -> SyncPaginator[DriftAlgorithm]

Your organization's custom drift algorithms, newest first.

Parameters

  • search (str | None, optional): Case-insensitive match on the name or description.
  • limit (int | None, optional): The most algorithms to return; every match when omitted.

list_analyses​

list_analyses(model_id: str, *, search: str | None = None, sort: str | None = None, limit: int | None = None) -> SyncPaginator[DriftAnalysisRun]

List a model's drift analysis runs (its history), newest first.

Parameters

  • model_id (str): The registry model's ID.
  • search (str | None, optional): Text to find in a run's status, trigger or error.
  • sort (str | None, optional): -createdAt (default) or durationMs.
  • limit (int | None, optional): The most runs to return; all of them when omitted.

list_baselines​

list_baselines(model_id: str, *, search: str | None = None, sort: str | None = None, limit: int | None = None) -> SyncPaginator[ReferenceBaseline]

Return a model's baselines (newest first), each with its inputs' names.

Build one with build_baseline.

Parameters

  • sort (str | None, optional): -createdAt (default), modelVersion, sampleSize, baselinePerformance.value.

list_predictions​

list_predictions(model_id: str, *, source: str | None = None, start_date: str | None = None, end_date: str | None = None, search: str | None = None, sort: str | None = None, limit: int | None = None) -> SyncPaginator[PredictionRecord]

Return a registry model's recorded predictions.

(newest first), each with its actual when one was recorded.

Parameters

  • source (str | None, optional): Only production or test (the model page's Test tab) predictions.
  • start_date, end_date (str | None, optional): ISO 8601 bounds on the prediction time.
  • search (str | None, optional): Matches the prediction id, entity id or error, or the prediction exactly.
  • sort (str | None, optional): -timestamp (default), latencyMs, modelVersion, success or confidence (- for descending).
  • limit (int | None, optional): Maximum number of predictions (default: all).

log_prediction​

log_prediction(*, model_id: str, features: Mapping[str, Any], prediction: Any, probabilities: Sequence[float] | None = None, entity_id: str | None = None, model_version: int | None = None, latency_ms: float | None = None) -> str

Record a prediction a registry model made outside the platform.

(a copy served elsewhere), so drift includes it. Predictions the platform serves are recorded automatically.

Parameters

  • model_id (str): The registry model.
  • features (Mapping[str, Any]): The input values, by feature name.
  • prediction (Any): The model's output.
  • probabilities (Sequence[float] | None, optional): Per-class probabilities; required for a classification model (the highest is its confidence).
  • entity_id (str | None, optional): Your id for what was predicted; actuals can join on it.
  • model_version (int | None, optional): The model version that made the prediction.
  • latency_ms (float | None, optional): How long the prediction took.

Returns

  • str: The prediction id (actuals can join on it).

models​

models(*, status: str | None = None, search: str | None = None, sort: str | None = None, limit: int | None = None) -> SyncPaginator[DriftModel]

Return the registry models you can see with their drift state.

The latest result (DriftModel.latest_drift) and the latest analysis run (DriftModel.latest_analysis, with why it failed).

Parameters

  • status (str | None, optional): Only models whose latest result is ok, warning, alert or error, or no-data for models never analyzed.
  • search (str | None, optional): Matches name, description or framework.
  • sort (str | None, optional): -latestDrift.calculatedAt (default), name, latestDrift.driftScore, latestDrift.sampleSize, updatedAt.
  • limit (int | None, optional): Maximum number of models (default: all).

result_features​

result_features(result_id: str, *, model_id: str, rows: str | None = None, search: str | None = None, sort: str | None = None, limit: int | None = None) -> SyncPaginator[DriftResultRow]

Return a drift result's rows, most drifted first.

One per feature (every per-feature algorithm's metric on it) and one per algorithm about the whole dataset.

Parameters

  • result_id (str): The result's id (DriftResult.id, or a model's latest_drift.result_id).
  • model_id (str): The model the result belongs to.
  • rows (str | None, optional): Only features, or only dataset rows.
  • search (str | None, optional): Matches the feature name.
  • sort (str | None, optional): -severity (default), score, featureName, order.
  • limit (int | None, optional): Maximum number of rows (default: all).

results​

results(model_id: str, *, status: str | None = None, search: str | None = None, sort: str | None = None, limit: int | None = None) -> SyncPaginator[DriftResult]

Return a model's drift results (newest first).

Parameters

  • status (str | None, optional): Only ok, warning, alert or error results.
  • sort (str | None, optional): -calculatedAt (default), driftScore, sampleSize.
  • limit (int | None, optional): Maximum number of results (default: all).

retrieve_algorithm​

retrieve_algorithm(algorithm_id: str) -> DriftAlgorithm

Return a custom drift algorithm.

retrieve_algorithm_test​

retrieve_algorithm_test(test_id: str, *, algorithm_id: str) -> DriftAlgorithmTest

Return a test of a custom drift algorithm, with its status and outcomes.

settings​

settings(model_id: str) -> DriftSettings | None

Return a model's drift settings, or None when they were never saved.

test_algorithm​

test_algorithm(algorithm_id: str) -> DriftAlgorithmTest

Test a custom drift algorithm; returns the test, pending.

A job runs its code in the drift sandbox on fixed samples of each kind it supports; read the outcome with retrieve_algorithm_test.

update_algorithm​

update_algorithm(algorithm_id: str, *, name: str | None = None, description: str | None = None, code: str | None = None, supports: Sequence[str] | None = None, is_active: bool | None = None) -> DriftAlgorithm

Change a custom drift algorithm and return it; only the fields given change.

is_active false keeps it out of drift analyses.

update_settings​

update_settings(model_id: str, *, enabled: bool | None = None, algorithms: Mapping[str, Any] | None = None, performance_drop_warning: float | None = None, performance_drop_alert: float | None = None, min_baseline_sample_size: int | None = None, min_window_sample_size: int | None = None, notifications: Mapping[str, Any] | None = None, schedule: Mapping[str, Any] | None = None) -> DriftSettings | None

Save a model's drift settings and return them.

Only the settings given change; the first save stores defaults for the rest.

Parameters

  • enabled (bool | None, optional): Drift settings on; required for the schedule to run.
  • algorithms (Mapping[str, Any] | None, optional): {"enabled": [names], "overrides": {name: {"warningThreshold", "alertThreshold", "params"}}}.
  • performance_drop_warning, performance_drop_alert (float | None, optional): How much worse than the baseline the model's performance may get (relative: 0.05 = 5% worse; lower accuracy or review score, higher mean absolute error) before prediction drift is a warning (default 0.05) or an alert (default 0.10); the warning must be smaller. A warning or alert makes the analysis one too.
  • min_baseline_sample_size (int | None, optional): Fewest reference rows a baseline needs (default 30).
  • min_window_sample_size (int | None, optional): Fewest predictions with recorded inputs an analysis window needs to be scored (at least 1; default 30).
  • notifications (Mapping[str, Any] | None, optional): {"email": bool, "recipients": [addresses], "slack": incoming webhook URL}. When an analysis finds drift (warning or alert) the model's owner is notified in Strongly; with email the recipients are emailed (the owner when there are none); with slack that webhook gets a message. An address that is not one, or a webhook pointing at localhost, an internal host or a private address, is refused. Each result records how it was announced (DriftResult.notification).
  • schedule (Mapping[str, Any] | None, optional): {"enabled": bool, "frequency": "hourly" | "daily" | "weekly", "timeWindowDays": int}`; it runs while `enabled (above) is true too.