Spot Instance Support

Use AWS Spot Instances to reduce compute costs by 60-90% for suitable workloads.

Overview

Spot instances are spare EC2 capacity offered at significantly reduced prices. The trade-off is that AWS can reclaim them with 2 minutes notice when capacity is needed elsewhere.

Strongly AI supports spot instances through Karpenter NodePools. When you enable spot for a workload, it is scheduled on a spot node. If the spot instance is reclaimed, Kubernetes automatically reschedules the workload on a new node; persistent data (volumes) is not affected.

Enabling Spot

Per-Environment

Create an environment with capacity_type: spot to make workloads using that environment run on spot instances. The environment create form includes the same spot toggle as the deploy forms.

Per-Deployment (UI)

The "Use Spot Instances (save up to 70%)" toggle appears in the deploy and create forms for:

• Apps (Deploy App form)
• Workspaces (Create Workspace form)
• Fine-tuning jobs
• AutoML jobs
• Avatars
• Environments
• Self-hosted models (Resources step)

Enabling the toggle opens a confirmation dialog explaining the trade-offs, and shows a warning that spot instances can be interrupted and the workload will automatically restart on a new node.

Per-Deployment (REST API)

# Apps
POST /api/v1/apps/:id/deploy
{ "capacity_type": "spot" }        # or "useSpotInstances": "true"

# Workspaces
POST /api/v1/workspaces
{ "name": "...", "useSpotInstances": true }   # or "capacity_type": "spot"

# Self-hosted models
POST /api/v1/ai/models/:id/deploy
{ "use_spot": true, "spot_fallback": true }

Workload Compatibility

Workload	Spot?	Why
Apps	Yes (opt-in)	Stateless, auto-restart, PVC survives
Workspaces	Yes (opt-in)	Code on PVC survives, running state lost
Fine-tuning jobs	Yes (opt-in)	Training checkpoints let interrupted jobs resume
AutoML jobs	Yes (opt-in)	Batch jobs that can be re-run
Avatars	Yes (opt-in)	Redeployable
Workflows	No	Workflow workers are pinned to on-demand nodes
Addons	No	Databases run on the dedicated user-addons pool (no spot, never moved)
Self-hosted models	Yes (opt-in)	Stateless inference; callers retry; on-demand fallback by default

Self-Hosted Models on Spot

Self-hosted model inference is stateless per request, which makes it a safe spot workload:

1. On-demand fallback (default) -- with spot_fallback: true (the default, and the "Fall back to on-demand when spot capacity is unavailable (recommended)" checkbox in the deploy form), spot is requested as a scheduling preference: a spot node is provisioned when capacity exists and an on-demand node when it does not. Availability is never sacrificed for price. Setting spot_fallback: false pins strictly to spot -- the deployment waits until spot capacity is available.
2. Interruption recovery -- when a spot node is reclaimed, Kubernetes reschedules the model pods on a replacement node. In-flight requests fail and are retried by the callers.

Do not use spot for stateful serving-adjacent workloads (databases, addons) -- the compatibility table above still applies to those.

UI

In the self-hosted model deploy form (Resources step), toggle Use Spot Instances. When enabled, the Fall back to on-demand checkbox (checked by default) controls the preference-vs-pin behaviour.

REST

POST /api/v1/ai/models/:id/deploy
{
  "instance_type": "g5.xlarge",
  "replicas": 4,
  "use_spot": true,        # opt-in (default false)
  "spot_fallback": true    # prefer spot, fall back to on-demand (default true)
}

Admin Controls

Admins control spot availability through the Compute admin page (Admin > Compute, /admin/compute):

• Enable or disable the "Spot (60-90% cheaper, can be interrupted)" capacity type per workload pool. The default for every pool is on-demand only.
• Set resource limits per pool (pools are uncapped by default)

A workload only lands on spot when both the admin has enabled the spot capacity type for its pool and the deployment itself opted into spot.