Spot Instance Support

Use AWS Spot Instances to reduce compute costs by 60-90% for suitable workloads.

Overview

Spot instances are spare EC2 capacity offered at significantly reduced prices. The trade-off is that AWS can reclaim them with 2 minutes notice when capacity is needed elsewhere.

Strongly AI supports spot instances through Karpenter NodePools. When you enable spot for a workload, Karpenter schedules it on a spot node. If the spot instance is reclaimed, Kubernetes automatically reschedules the pod on a new node.

Enabling Spot

Per-Environment

Create an environment with capacity_type: spot to make all deployments using that environment run on spot instances.

Per-Deployment (UI)

When deploying an app, creating a workspace, or running a training job, toggle "Use Spot Instances" in the deployment form. A confirmation dialog explains the trade-offs.

Per-Deployment (REST API)

# Apps
POST /api/v1/apps/:id/deploy
{ "capacity_type": "spot" }

# Workspaces
POST /api/v1/workspaces
{ "name": "...", "useSpotInstances": true }

# Workflows
POST /api/v1/workflows/:id/execute
{ "capacity_type": "spot" }

# Self-hosted models
POST /api/v1/ai/models/:id/deploy
{ "use_spot": true, "spot_fallback": true }

Workload Compatibility

Workload	Spot?	Why
Apps	Yes	Stateless, auto-restart, PVC survives
Workspaces	Yes	Code on PVC survives, running state lost
Fine-tuning jobs	Yes	Checkpoint-resumable via S3
AutoML jobs	Yes	Fault-tolerant by design
Workflows	Yes	Workers restart, execution resumes
Builds	Yes (default)	Idempotent, auto-retry on on-demand if spot interrupted
Addons	Never	Databases need reliability
Self-hosted models	Yes (opt-in)	Stateless inference; callers retry; on-demand fallback by default
Model registry	Never	Serving needs reliability

Self-Hosted Models on Spot

Self-hosted model inference is stateless per request, which makes it a safe spot workload when three safeguards are in place (all built in):

1. On-demand fallback (default) — with spot_fallback: true (the default, and the "Fall back to on-demand" checkbox in the deploy form), spot is requested as a scheduling preference: Karpenter provisions a spot node when capacity exists and an on-demand node when it does not. Availability is never sacrificed for price. Setting spot_fallback: false pins strictly to spot — the deployment waits until spot capacity is available.
2. Graceful interruption — Karpenter's interruption controller watches the EC2 2-minute reclaim warning (via the cluster's interruption SQS queue) and cordons/drains the node before it disappears, provisioning a replacement immediately. In-flight requests fail fast and are retried by the callers.
3. No partial results — response integrity guards on multi-part outputs (e.g. page-count checks on document processing) reject truncated responses, so an interruption can only ever produce a retried request, never a corrupted result.

Do not use spot for stateful serving-adjacent pods (databases, addons, workflow workers holding run state) — the compatibility table above still applies to those.

UI

In the self-hosted model deploy form (Resources step), toggle Use Spot Instances. When enabled, the Fall back to on-demand checkbox (checked by default) controls the preference-vs-pin behaviour.

REST

POST /api/v1/ai/models/:id/deploy
{
  "instance_type": "g5.xlarge",
  "auto_shutdown_minutes": 10,
  "replicas": 4,
  "use_spot": true,        # opt-in (default false)
  "spot_fallback": true    # prefer spot, fall back to on-demand (default true)
}

Build Spot Retry

Builds default to spot instances for cost savings. If a build fails due to spot interruption (node eviction), the system automatically retries on on-demand instances. Other build failures (code errors, OOM) are NOT retried on a different instance type.

Detection: The system checks for pod eviction reasons (Evicted, Preempting) and spot-related messages in the pod status.

Cost Savings

Typical savings by instance type:

Instance	On-Demand	Spot	Savings
m5.xlarge (4 CPU, 16 GB)	~$0.192/hr	~$0.06/hr	69%
g5.xlarge (1 GPU, 16 GB)	~$1.006/hr	~$0.35/hr	65%
c5.2xlarge (8 CPU, 16 GB)	~$0.34/hr	~$0.10/hr	71%

Admin Controls

Admins control spot availability through the Compute admin page (/admin/compute):

• Enable/disable spot capacity per workload type (General, AI, MCP)
• Set resource limits for spot pools
• View spot vs on-demand breakdown in FinOps dashboard