Drift Detection
Drift detection for registry (traditional ML) models.
Access it as client.drift_detection on a Strongly client, or the same path on AsyncStrongly with await. All methods exist on both with identical signatures.
Methods
All methods
active_baseline
active_baseline(model_id: str) -> ReferenceBaseline | None
Return the active baseline of the model's active version.
(what drift compares that version with), or None when it has none.
analyze
analyze(*, model_id: str, window_days: int | None = None, window_start: str | None = None, window_end: str | None = None) -> DriftAnalysisRun
Start a drift analysis.
The active version's production predictions in the window (default the last 7 days) against its active baseline.
A version without a baseline is refused. The run's progress, and why it
failed when it produced no result, is the model's
latest_analysis.
analyze_status
analyze_status(job_id: str, *, model_id: str) -> DriftAnalysisRun
One drift analysis run by the job id analyze returned.
Its status (pending, running, completed, failed), what started it,
its window, its result in short when it completed, and its error when it failed.
create_algorithm
create_algorithm(*, name: str, code: str, supports: Sequence[str], description: str | None = None) -> DriftAlgorithm
Create a custom drift algorithm for your organization and return it.
Parameters
name(str): Its name, unique in your organization (a model's drift settings enable it by name).code(str): Python definingdetect_drift(reference_data, current_data); numpy, math, random, json and scipy.stats can be imported.supports(Sequence[str]):numericaland/orcategorical.description(str | None, optional): What it detects.
Raises
ConflictError: Another algorithm of your organization has this name.ValidationError: The name or code is empty, orsupportsnames another kind.
delete_algorithm
delete_algorithm(algorithm_id: str) -> None
Delete a custom drift algorithm, with its tests.
Raises
UnprocessableEntityError: A model's drift settings run it (resource-in-use, naming the models).
latest_result
latest_result(model_id: str) -> DriftResult | None
Return a model's most recent drift result, or None when it has never been analyzed.
Its rows are result_features.
list_algorithms
list_algorithms(*, search: str | None = None, limit: int | None = None) -> SyncPaginator[DriftAlgorithm]
Your organization's custom drift algorithms, newest first.
Parameters
search(str | None, optional): Case-insensitive match on the name or description.limit(int | None, optional): The most algorithms to return; every match when omitted.
list_analyses
list_analyses(model_id: str, *, search: str | None = None, sort: str | None = None, limit: int | None = None) -> SyncPaginator[DriftAnalysisRun]
List a model's drift analysis runs (its history), newest first.
Parameters
model_id(str): The registry model's ID.search(str | None, optional): Text to find in a run's status, trigger or error.sort(str | None, optional):-createdAt(default) ordurationMs.limit(int | None, optional): The most runs to return; all of them when omitted.
list_baselines
list_baselines(model_id: str, *, search: str | None = None, sort: str | None = None, limit: int | None = None) -> SyncPaginator[ReferenceBaseline]
Return a model's baselines (newest first), each with its inputs' names.
Build one with build_baseline.
Parameters
sort(str | None, optional):-createdAt(default),modelVersion,sampleSize,baselinePerformance.value.
list_predictions
list_predictions(model_id: str, *, source: str | None = None, start_date: str | None = None, end_date: str | None = None, search: str | None = None, sort: str | None = None, limit: int | None = None) -> SyncPaginator[PredictionRecord]
Return a registry model's recorded predictions.
(newest first), each with its actual when one was recorded.
Parameters
source(str | None, optional): Onlyproductionortest(the model page's Test tab) predictions.start_date, end_date(str | None, optional): ISO 8601 bounds on the prediction time.search(str | None, optional): Matches the prediction id, entity id or error, or the prediction exactly.sort(str | None, optional):-timestamp(default),latencyMs,modelVersion,successorconfidence(-for descending).limit(int | None, optional): Maximum number of predictions (default: all).
log_prediction
log_prediction(*, model_id: str, features: Mapping[str, Any], prediction: Any, probabilities: Sequence[float] | None = None, entity_id: str | None = None, model_version: int | None = None, latency_ms: float | None = None) -> str
Record a prediction a registry model made outside the platform.
(a copy served elsewhere), so drift includes it. Predictions the platform serves are recorded automatically.
Parameters
model_id(str): The registry model.features(Mapping[str, Any]): The input values, by feature name.prediction(Any): The model's output.probabilities(Sequence[float] | None, optional): Per-class probabilities; required for a classification model (the highest is its confidence).entity_id(str | None, optional): Your id for what was predicted; actuals can join on it.model_version(int | None, optional): The model version that made the prediction.latency_ms(float | None, optional): How long the prediction took.
Returns
str: The prediction id (actuals can join on it).
models
models(*, status: str | None = None, search: str | None = None, sort: str | None = None, limit: int | None = None) -> SyncPaginator[DriftModel]
Return the registry models you can see with their drift state.
The latest result (DriftModel.latest_drift) and the latest analysis run
(DriftModel.latest_analysis, with why it failed).
Parameters
status(str | None, optional): Only models whose latest result isok,warning,alertorerror, orno-datafor models never analyzed.search(str | None, optional): Matches name, description or framework.sort(str | None, optional):-latestDrift.calculatedAt(default),name,latestDrift.driftScore,latestDrift.sampleSize,updatedAt.limit(int | None, optional): Maximum number of models (default: all).
result_features
result_features(result_id: str, *, model_id: str, rows: str | None = None, search: str | None = None, sort: str | None = None, limit: int | None = None) -> SyncPaginator[DriftResultRow]
Return a drift result's rows, most drifted first.
One per feature (every per-feature algorithm's metric on it) and one per algorithm about the whole dataset.
Parameters
result_id(str): The result's id (DriftResult.id, or a model'slatest_drift.result_id).model_id(str): The model the result belongs to.rows(str | None, optional): Onlyfeatures, or onlydatasetrows.search(str | None, optional): Matches the feature name.sort(str | None, optional):-severity(default),score,featureName,order.limit(int | None, optional): Maximum number of rows (default: all).
results
results(model_id: str, *, status: str | None = None, search: str | None = None, sort: str | None = None, limit: int | None = None) -> SyncPaginator[DriftResult]
Return a model's drift results (newest first).
Parameters
status(str | None, optional): Onlyok,warning,alertorerrorresults.sort(str | None, optional):-calculatedAt(default),driftScore,sampleSize.limit(int | None, optional): Maximum number of results (default: all).
retrieve_algorithm
retrieve_algorithm(algorithm_id: str) -> DriftAlgorithm
Return a custom drift algorithm.
retrieve_algorithm_test
retrieve_algorithm_test(test_id: str, *, algorithm_id: str) -> DriftAlgorithmTest
Return a test of a custom drift algorithm, with its status and outcomes.
settings
settings(model_id: str) -> DriftSettings | None
Return a model's drift settings, or None when they were never saved.
test_algorithm
test_algorithm(algorithm_id: str) -> DriftAlgorithmTest
Test a custom drift algorithm; returns the test, pending.
A job runs its code in the drift sandbox on fixed samples of each kind
it supports; read the outcome with retrieve_algorithm_test.
update_algorithm
update_algorithm(algorithm_id: str, *, name: str | None = None, description: str | None = None, code: str | None = None, supports: Sequence[str] | None = None, is_active: bool | None = None) -> DriftAlgorithm
Change a custom drift algorithm and return it; only the fields given change.
is_active false keeps it out of drift analyses.
update_settings
update_settings(model_id: str, *, enabled: bool | None = None, algorithms: Mapping[str, Any] | None = None, performance_drop_warning: float | None = None, performance_drop_alert: float | None = None, min_baseline_sample_size: int | None = None, min_window_sample_size: int | None = None, notifications: Mapping[str, Any] | None = None, schedule: Mapping[str, Any] | None = None) -> DriftSettings | None
Save a model's drift settings and return them.
Only the settings given change; the first save stores defaults for the rest.
Parameters
enabled(bool | None, optional): Drift settings on; required for the schedule to run.algorithms(Mapping[str, Any] | None, optional):{"enabled": [names], "overrides": {name: {"warningThreshold", "alertThreshold", "params"}}}.performance_drop_warning, performance_drop_alert(float | None, optional): How much worse than the baseline the model's performance may get (relative: 0.05 = 5% worse; lower accuracy or review score, higher mean absolute error) before prediction drift is a warning (default 0.05) or an alert (default 0.10); the warning must be smaller. A warning or alert makes the analysis one too.min_baseline_sample_size(int | None, optional): Fewest reference rows a baseline needs (default 30).min_window_sample_size(int | None, optional): Fewest predictions with recorded inputs an analysis window needs to be scored (at least 1; default 30).notifications(Mapping[str, Any] | None, optional):{"email": bool, "recipients": [addresses], "slack": incoming webhook URL}. When an analysis finds drift (warningoralert) the model's owner is notified in Strongly; withemailthe recipients are emailed (the owner when there are none); withslackthat webhook gets a message. An address that is not one, or a webhook pointing at localhost, an internal host or a private address, is refused. Each result records how it was announced (DriftResult.notification).schedule(Mapping[str, Any] | None, optional):{"enabled": bool, "frequency": "hourly" | "daily" | "weekly", "timeWindowDays": int}`; it runs while `enabled(above) is true too.