Compute

Configure what hardware Karpenter is allowed to provision for platform workloads. Navigate to Admin > Compute (/admin/compute). This page is admin-only.

Architecture

The compute layer has two distinct levels:

+--------------------------------------------------+
|  Platform Scripts (Infrastructure Layer)          |
|  - Set up Karpenter NodePools and node classes    |
|  - Define the FULL pool of allowed instance types |
|  - Managed via platform repo scripts              |
|  - Changes require infrastructure access          |
+---------------------------+----------------------+
                            |
                            v
+--------------------------------------------------+
|  Compute Admin UI (Karpenter Config Layer)        |
|  - Choose instance categories, sizes, capacity    |
|  - Set each pool's maximum node size              |
|  - Set resource limits per workload pool          |
|  - Applied to the Karpenter NodePool for you      |
|  - No infrastructure access needed                |
+---------------------------+----------------------+
                            |
                            v
+--------------------------------------------------+
|  Pod Scheduling                                   |
|  - Karpenter provisions nodes within constraints  |
|  - Pods scheduled based on resource requests      |
|  - New pods use updated rules immediately         |
|  - Existing pods NOT affected                     |
+--------------------------------------------------+

Platform Scripts (Infrastructure Layer)

The platform repo (strongly-ai-platform) contains the base Karpenter configuration the platform installs with (platform/bootstrap/manifests/): the NodePools for every pool below plus spot variants of the general, MCP and AI pools, and the EC2 node classes they use. The k8s-ops-controller service also creates any Compute pool that is missing.

Once a pool exists, its Compute settings are its own: a platform upgrade creates pools that are missing but does not overwrite the ones you tuned here. The spot variants are not on the Compute page and keep the platform's settings.

Workload Pools

The Compute page shows one tab per workload pool:

Pool	What runs there	Behaviour
General Workloads	Movable platform workloads: workflows, agents, streaming, gateways, model registry	Consolidates when empty or underutilized
AI Workloads	GPU workloads: model deployments, GPU fine-tuning, avatars, GPU workspaces	Nodes are released when the workload stops
MCP Servers	Tool servers and MCP deployments	Small, stateless, movable
User Addons	User databases (MongoDB, Postgres, Redis, ...)	Never moved once placed; node released when the addon is deleted
User Workspaces	Interactive coding environments (Jupyter, IDEs)	Never moved while running; nodes released when workspaces stop
User Jobs	Batch jobs running user code (distributed workers, data forge, builds, scheduled jobs)	Never evicted mid-run; nodes reaped when jobs finish
User Apps	User-deployed applications	Pinned like addons; nodes released when apps are undeployed

What You Can Configure

For each pool:

Instance Categories

Toggle which EC2 instance families Karpenter can use. Each pool offers a subset of:

Category	Description
C (Compute)	CPU-optimized for compute-intensive workloads
M (General)	Balanced CPU/memory for general workloads
R (Memory)	Memory-optimized for data processing
T (Burstable)	Burstable for variable workloads, cost-effective
G (GPU)	T4/A10G GPUs for inference and training
P (GPU)	A100/V100 GPUs for large model training
INF (Inferentia)	AWS Inferentia for ML inference

The GPU categories (G, P, INF) are only available on the AI Workloads pool.

Instance Sizes

Maximum node size sets the largest node the pool may provision. Sizes run from the pool's smallest (small, or medium for AI Workloads) up to the maximum you choose, on this ladder:

small, medium, large, xlarge, 2xlarge, 4xlarge, 8xlarge, 12xlarge, 16xlarge, 24xlarge, 32xlarge, 48xlarge, 96xlarge

Until you change it, each pool keeps the maximum it always had:

Pool	Smallest	Default maximum
General Workloads	small	4xlarge
AI Workloads	medium	4xlarge
MCP Servers, User Addons	small	large
User Workspaces, User Jobs, User Apps	small	2xlarge

Raising the maximum switches on every size it adds; lowering it removes the sizes above it. Under Allowed sizes you can still switch individual sizes off.

Next to the maximum, the page shows what it lets the pool run, read from AWS for your region: the largest nodes allowed with their vCPU and memory and, on the AI pool, the largest GPU nodes and their GPUs. For example, with the AI maximum at 48xlarge it lists machines such as p5.48xlarge (8x H100), p5en.48xlarge (8x H200) and p6-b200.48xlarge (8x B200). A maximum above the default is marked, since every node the pool adds can be one of those shapes and is billed for as long as it runs.

Models that need large GPU nodes

Many self-hosted models need more than one GPU on a node: 70B-class models run on 4-GPU nodes such as g5.12xlarge or g6e.12xlarge, and the largest ones on 8-GPU nodes such as p4d.24xlarge or p5.48xlarge. None of these fit under the AI pool's default 4xlarge maximum.

When a model's node is larger than the AI pool allows, its deploy is refused before anything starts, with the reason, for example:

Qwen2.5 72B needs a 12xlarge node with 4 A10G; this platform's Compute settings allow AI nodes up to 4xlarge; an admin can raise it on the Compute page

To run it, raise the AI Workloads maximum node size to at least the size named, save, and deploy again. A deploy is refused the same way when the model's instance family is switched off for the pool.

Capacity Types

• On-Demand (reliable, higher cost) -- the default; every pool starts with on-demand only
• Spot (60-90% cheaper, can be interrupted) -- enable to allow spot-opted workloads in this pool

Pool Resource Limits

Pools are uncapped by default: Karpenter adds nodes as demand requires, with no ceiling. Turn off "Uncapped (scale freely)" to set a deliberate maximum total CPU, memory, and (for the AI pool) GPUs across all nodes in the pool.

Node Size (bin-packing)

An optional per-pool override that pins the pool to one node family and size. With it on, the first workload spins up a single node of that shape, later workloads pack onto it, and the next node is the same shape when it fills -- instead of a separate small node per workload. With it off, Karpenter picks the cheapest node that fits each workload. Apply the choice with Apply Node Configuration.

The override works inside the pool's size range:

• You can pin any size from the pool's smallest up to its saved maximum node size. To pin a bigger size, raise the maximum and save first.
• A workload too large for the pinned size gets a bigger node of the same family, never above the maximum.
• While the override is on, saving the pool's settings keeps it in place.
• Turning the override off puts back the sizes and families you saved for the pool, within your maximum.

The page lists the real instance types behind the pinned family and size, with their GPUs, vCPU and memory.

Live NodePool

At the top of each tab, Live NodePool shows what the cluster enforces right now for that pool: its sizes, families and capacity types. If they differ from your saved settings (for example after someone changed the NodePool outside this page), a badge says so; Save Settings applies your settings again.

How Changes Apply

1. Admin saves settings in the UI
2. The change is applied to the live Karpenter NodePool first (via the k8s-ops-controller service). If that apply fails, nothing is saved -- the stored settings can never diverge from the cluster
3. On success, settings are persisted in MongoDB (compute_settings collection)
4. Karpenter picks up changes immediately (the NodePool is a live resource)
5. New pods are scheduled based on the updated constraints
6. Existing pods are NOT affected until rescheduled

Important Notes

• Instance families are chosen from each pool's list (the GPU families only on AI Workloads); node sizes go up to the maximum you set, as far as 96xlarge
• Larger nodes cost more per node: check the largest shapes the page lists before raising a maximum, and consider a GPU limit under Pool Resource Limits on the AI pool
• The maximum applies to the on-demand pools on this page. The AI spot pool is not on this page and keeps sizes up to 4xlarge
• Running a workload on spot requires both the admin enabling the "spot" capacity type for its pool here and the workload itself opting into spot at deploy time (see Spot Instance Support)