Deploy and Scale
Deploying
Select Deploy on a model's page; it is offered while the model is Ready to Deploy, Stopped or Failed. The model builds its serving image (Building), starts (Deploying, Starting) and serves (Running), using the size and scaling saved on its Environment tab. Follow it on the Logs tab.
- Stop takes a running model down; it keeps its versions, settings, predictions and monitoring. Start brings a stopped model back.
- Delete removes the model, its versions, its deployment and its uploaded files. A model connected to apps is not deleted until it is disconnected; if its deployment cannot be deleted, the model is kept and the message says why.
Deploying can require governance approval when a governance solution covers ML models.
The Environment tab
The model's editors change how it is served here. Save keeps the settings for the model's next deploy; on a running model, Save & Restart saves them and redeploys it with them as a rolling update (the current pod serves until the new one is ready). Changing a running model's settings requires a restart.
Resource Configuration
- Environment: a pre-configured size (its vCPUs, memory and GPUs), or Custom Configuration to set each:
- CPU (vCPU): 1, 2, 4, 8 or 16;
- Memory (GB): 4, 8, 16, 32 or 64;
- GPU: none, or 1, 2, 4 or 8, with the GPU type;
- Disk Space (GB): 10, 20, 50, 100 or 200.
- Replicas: the pod count when the model is active (1 to 10). An always-on model holds this count; an on-demand model wakes from zero to it.
The size is the whole pod, exactly what it requests and is limited to.
Scaling Strategy
The same five strategies a self-hosted model has:
| Strategy | How it runs | Settings |
|---|---|---|
| Always-on (fixed replicas) | Holds Replicas pods all the time. Every prediction is answered at once. | |
| Always-on + auto-scaling | A Kubernetes HPA scales between min and max on CPU and memory use, checked every polling interval. Never scales to zero. | Min Replicas (1 to 20), Max Replicas (2 to 50), Polling Interval (s) (10 to 300, default 30), CPU Threshold (%) (default 70), Memory Threshold (%) (default 80) |
| Scheduled (runs during set hours) | Started with Replicas pods at each start time and stopped at each stop time, on the days you choose. | Days of Operation, Operating Hours (one or more start and stop times; Add Time Range), Timezone |
| On-demand (scales to zero) | Scales to zero when idle and wakes to Replicas on its first prediction. | Auto-shutdown after idle (minutes) (1 to 1,440, default 15) |
| On-demand + auto-scaling | Wakes from zero, then sizes the fleet to the live request load (in-flight predictions divided by the target), between min and max, and scales back to zero when idle. | Auto-shutdown after idle (minutes), Min Replicas, Max Replicas, Target Concurrency / Replica (default 3) |
On-demand suits low-traffic models that should not keep an idle pod. The first prediction after a scale-down waits for the pod to start (a cold start, typically 30 to 90 seconds for a scikit-learn model). The registry list shows a scaled-down on-demand model as Idle (on-demand).
A scheduled model is started and stopped by the platform as its Start and Stop buttons do. A start is checked against your budgets like any start, and a time that passes while the model is still building or starting is applied when it is running. Between its windows, a scheduled model is Stopped; you can still start or stop it by hand.
Scheduling and demand scaling apply to a deployed model as soon as they are saved; resources and spot from its next deploy or start (Save & Restart on a running model). A start always runs the model as its Environment tab is saved, whether you start it or its schedule does.
Spot capacity
Use Spot Instances runs the model on spot capacity, which is cheaper but can be reclaimed: the pod is then restarted on another node, and predictions in flight at that moment fail and should be retried. Fall back to on-demand when spot capacity is unavailable (on by default) prefers spot: the model runs on a spot node when one has room or a new node is added for it, and on on-demand capacity otherwise; turned off, the model waits for spot capacity.
Versions
Each model keeps its versions on the Versions tab, each with its status, description, size and creation time:
| Status | Meaning |
|---|---|
| Active & Running | The version the running model serves |
| Active | The version the model serves when deployed |
| Inactive | Kept; not served |
- Add Version uploads a new artifact as the next version (see Register a model).
- Set as Active makes a version active on a model that is not deployed.
- Deploy this version makes it active and deploys the model with it; this is also how you roll back to an earlier version. The current version serves until the new one is ready.
- Delete version removes an inactive version.
The active version is the one the model serves and the one drift analyzes. Each version keeps its own drift baseline, so a version you switch to is compared with its own reference data. A model trained on a feature service serves the features its own version trained on.
From Python
client.model_registry.deploy(model_id)
client.model_registry.stop(model_id)
client.model_registry.start(model_id)
The REST API's POST /api/v1/mlops/models/{id}/deploy takes the same size and scaling as the Environment tab: resources, autoShutdownMinutes (above 0 deploys on demand), schedulingMode (always_on, on_demand or scheduled), scheduleRules, scalingMode (fixed or auto), minReplicas, maxReplicas, targetConcurrency, useSpot and spotFallback. PATCH /api/v1/mlops/models/{id}/serving changes them without deploying. See the Model Registry API.