Skip to main content

AI Models

AI Models are the model catalog. Use this resource to register third-party or self-hosted models, link a provider key (via apiKeyId), deploy/start/stop self-hosted models, and read status, logs, and permissions.

Access it as client.ai.models on a Strongly client, or the same path on AsyncStrongly with await. All methods exist on both with identical signatures.

Quick start​

from strongly import Strongly

client = Strongly()

# List (auto-paginates as you iterate) # filters: search, type, status, provider, model_type
for ai in client.ai.models.list():
print(ai.id)

Methods​

Core​

list​

list(*, search: str | None = None, type: str | None = None, status: str | None = None, provider: str | None = None, model_type: str | None = None, limit: int | None = None) -> SyncPaginator[AIModel]

List AI models with pagination, filtering, and search.

model_type takes one type or several, comma-separated (chat,multimodal,completion: the models an agent can run on).

create​

create(*, name: str, type: str, provider: str, vendor_model_id: str, model_type: str | None = None, description: str | None = None, capabilities: Sequence[str] | None = None, max_output_tokens: int | None = None, context_window: int | None = None, config: Mapping[str, Any] | None = None, prompt_storage: PromptStorage | None = None, **extra) -> AIModel

Create a new AI model.

Parameters

  • name (str): Model display name.

  • type (str): Model kind (e.g. third-party or self-hosted).

  • provider (str): Provider name.

  • vendor_model_id (str): The vendor's model identifier.

  • model_type (str | None, optional): Capability class (e.g. chat, multimodal, embedding). A certified model or prebuilt template brings its own; required for a model neither records, which is refused without it.

  • description (str | None, optional): Optional description.

  • capabilities (Sequence[str] | None, optional): Optional capability tags.

  • max_output_tokens (int | None, optional): Chat model: the most tokens it writes in one reply. Required (with context_window) where certification records none; where it does, at most its ceiling (a third-party model's certified cap, a self-hosted vLLM model's context window). Left out, a self-hosted vLLM model records a quarter of its context window.

  • context_window (int | None, optional): Chat model: the most tokens it reads. Required where certification records none.

  • config (Mapping[str, Any] | None, optional): Optional model configuration.

  • prompt_storage ({"stored", "encrypted", "none"} | None, optional): How the AI Gateway keeps each call's prompt and response: "stored" keeps them; "encrypted" keeps them privately, so only the person who made the call can read them back (usage, cost, latency and errors stay readable); "none" keeps only usage, no prompt or response. Default: "stored".

  • **extra (Any): Additional fields accepted by the route. Pass them snake_case (e.g. api_key_id, a saved key you may use; api_endpoint); they are converted to the camelCase the API expects. Already-camelCase keys also work.

    A self-hosted model is stored with its settings resolved. Created from a catalog entry (vendor_model_id names one, see list_prebuilt_templates) it takes every setting from the entry, and each of instance_type, resources (cpu, memory, gpu, disk), dockerfile, port, env_vars, scheduling_mode, auto_shutdown_minutes, schedule_rules, replicas, scaling_mode, min_replicas, max_replicas, target_concurrency, use_spot, spot_fallback given here is that one model's override (the catalog entry never changes). A custom model (no catalog entry) gives instance_type, resources.disk, dockerfile and port. Spot is off unless use_spot=True. Creating a model deploys nothing.

retrieve​

retrieve(model_id: str) -> AIModel

Get a single AI model by ID.

update​

update(model_id: str, *, name: str | None = None, type: str | None = None, provider: str | None = None, vendor_model_id: str | None = None, model_type: str | None = None, description: str | None = None, capabilities: Sequence[str] | None = None, max_output_tokens: int | None = None, context_window: int | None = None, config: Mapping[str, Any] | None = None, replicas: int | None = None, scheduling_mode: SchedulingMode | None = None, auto_shutdown_minutes: int | None = None, schedule_rules: Sequence[Mapping[str, Any]] | None = None, scaling_mode: ScalingMode | None = None, min_replicas: int | None = None, max_replicas: int | None = None, target_concurrency: int | None = None, prompt_storage: PromptStorage | None = None, **extra) -> AIModel

Update an AI model: the fields the model Settings tab saves.

For a deployed self-hosted model, replicas, the scheduling fields and the scaling fields change the running model without a redeploy. The resulting combination must be coherent or the call fails with a validation error and nothing changes:

  • scheduling_mode="on_demand" needs auto_shutdown_minutes (integer >= 1, the idle minutes before it scales to zero).
  • scheduling_mode="scheduled" needs schedule_rules.
  • scaling_mode is exactly "fixed" or "auto"; "auto" needs an on-demand model and min_replicas, max_replicas and target_concurrency.

prompt_storage ("stored", "encrypted" or "none") changes how the AI Gateway keeps the model's prompts and responses; the next call follows it.

CPU, memory, disk and GPU are changed with resize. **extra forwards additional fields accepted by the update route.

delete​

delete(model_id: str) -> None

Delete an AI model (refused while anything uses it).

A self-hosted model's build, deployment and images go with it; deleting one that is still building or deploying cancels it: its build stops and nothing is deployed.

Lifecycle & actions​

start​

start(model_id: str) -> AIModel

Start an AI model.

stop​

stop(model_id: str) -> AIModel

Stop an AI model.

restart​

restart(model_id: str) -> AIModel

Restart a running self-hosted AI model: its instances roll (requests in flight may fail).

A third-party model runs at its provider, so it has nothing to restart (ConflictError).

deploy​

deploy(model_id: str, *, instance_type: str | None = None, replicas: int | None = None, auto_shutdown_minutes: int | None = None, schedule_rules: Sequence[Mapping[str, Any]] | None = None, scaling_mode: ScalingMode | None = None, min_replicas: int | None = None, max_replicas: int | None = None, target_concurrency: int | None = None, env_vars: Mapping[str, str] | None = None, use_spot: bool | None = None, spot_fallback: bool | None = None, **extra) -> AIModel

Deploy an AI model (or activate a third-party one).

Self-hosted scheduling: auto_shutdown_minutes >= 1 makes the model on-demand (scales to zero after that many idle minutes, wakes on the next request); 0 or omitted is always-on; schedule_rules runs it in start/stop windows. replicas (integer >= 1, default 1) is the pod count while it runs.

scaling_mode is exactly "fixed" (wake to replicas) or "auto" (replicas follow live load between min_replicas and max_replicas, adding one per target_concurrency in-flight requests). "auto" needs an on-demand model and all three bounds; any invalid combination is refused with a validation error and nothing is deployed.

On an already deployed model this updates those settings and starts it; another instance_type, or a dockerfile, gpu, gpu_type, cpu, memory or volume_size, is refused with a validation error and nothing changes (resize with resize; a newer catalog version with update_to_catalog). **extra forwards other deploy fields (port, cpu, memory, gpu, volume_size, dockerfile, autoscaling). Every setting given is that one model's override, stored before it deploys (None is left out; to reset an override to its catalog default send it as null with update); the model deploys from its resolved settings: its catalog entry plus its overrides. CPU and memory not set are the room its instance gives one workload; a size above that room, or a gpu_type its instance does not carry, is refused naming the field and nothing is written or deployed.

A size is the whole workload: Strongly's helper services beside it (sign-in proxy, outbound proxy, data volume mounts, model cache upload, metrics) are taken out of it, never added on top (set 4 GB and the workload uses 4 GB in total). A size too small to hold them is refused naming them. A CPU, memory, GPU count or disk above the largest allowed machine's room for one workload raises APIError with status_code 400 naming the resource and both figures, and nothing is created or changed. Check client.compute.list_pools() allowed_sizes (max_cpu, max_memory_gib, max_disk_gib, max_gpus) before a large size.

Other​

add_member​

add_member(resource_id: str, *, user_id: str, role: str) -> Permissions

Share the resource with a user.

Parameters

  • resource_id (str): The resource's id.
  • user_id (str): The user to share it with.
  • role (str): "editor" (use and change it) or "user" (use it only; types without a use-only tier, such as knowledge bases, take "editor" only).

clear_cache​

clear_cache(model_id: str) -> None

Clear a model's cached responses (its response cache and semantic cache).

list_certified​

list_certified(*, provider: str | None = None, model_type: str | None = None, capability: str | None = None) -> list[CertifiedModel]

List the third-party models the platform has certified, to add as models.

Parameters

  • provider (str | None, optional): Only this provider's (openai, anthropic, ...).
  • model_type (str | None, optional): Only this type (chat, embedding, image, ...).
  • capability (str | None, optional): Only the models with this capability (vision, audio, ...).

list_prebuilt​

list_prebuilt(*, category: str | None = None, provider: str | None = None) -> list[PrebuiltTemplate]

List the self-hosted model templates you can deploy.

Parameters

  • category (str | None, optional): Only this category (llm, embedding, image, ...).
  • provider (str | None, optional): Only this provider's (meta, google, ...).

list_providers​

list_providers() -> list[ModelProvider]

List the third-party providers of the certified models, with how many each has.

logs​

logs(model_id: str, *, type: Literal['build', 'deployment', 'runtime'] | None = None, lines: int | None = None, since: str | None = None, container: str | None = None) -> list[ModelLogEntry]

Get a self-hosted model's log entries; a third-party model has none (409).

Parameters

  • model_id (str): The model ID.
  • type ({"build", "deployment", "runtime"} | None, optional): Which log: the running model's output (runtime) when omitted.
  • lines (int | None, optional): How many of the latest entries (100 when omitted).
  • since (str | None, optional): ISO date: only entries since then (runtime logs).
  • container (str | None, optional): Only this container's entries (runtime logs).

metrics​

metrics(model_id: str, *, time_range: Literal['1h', '6h', '24h', '7d'] | None = None) -> ModelLiveMetrics | ModelUsage

Get a model's metrics.

A self-hosted model answers its running replicas measured now (ModelLiveMetrics: CPU, memory, GPU, disk, network, replicas); a third-party model the requests the AI Gateway served for it over time_range (ModelUsage).

Parameters

  • model_id (str): The model ID.
  • time_range ({"1h", "6h", "24h", "7d"} | None, optional): A third-party model's window; 24h when omitted.

options​

options(model_id: str) -> ModelOptions

Get what a model offers (voices, languages, provider models), as it reports them.

overview​

overview() -> AIModelOverview

Get overview stats for AI models.

permissions​

permissions(resource_id: str) -> Permissions

Return who can reach the resource: its owner, members and their roles, and visibility.

Parameters

  • resource_id (str): The resource's id.

remove_member​

remove_member(resource_id: str, user_id: str) -> None

Stop sharing the resource with a user.

Parameters

  • resource_id (str): The resource's id.
  • user_id (str): The user to stop sharing it with.

resize​

resize(model_id: str, *, cpu: str | None = None, memory: str | None = None, disk: str | None = None, gpu: int | None = None, use_spot: bool | None = None, spot_fallback: bool | None = None) -> AIModel

Resize a deployed self-hosted model in place.

The running model restarts at exactly the new size (cpu e.g. "2", memory e.g. "8Gi", disk e.g. "100Gi", gpu a count). The quantities are Kubernetes ones; the model's page shows the same amounts in vCPU and GB ("8Gi" is 8 GB, 1 GB = 1024 MB). use_spot moves it on (True) or off (False) spot capacity the same way; with use_spot, spot_fallback True (the API's default) falls back to on-demand when spot is unavailable and False waits for spot. It is not deleted or redeployed. The model's budget is checked first: a refused resize raises APIError with status_code 402 and nothing changes. A GPU model's size must fit its machine (the pinned instance type, else any instance of its GPU type): above the room the largest such machine gives one workload (compute.list_pools() allowed_sizes of the ai pool) it is refused with 400 naming that room, and nothing changes.

A size is the whole workload: Strongly's helper services beside it (sign-in proxy, outbound proxy, data volume mounts, model cache upload, metrics) are taken out of it, never added on top (set 4 GB and the workload uses 4 GB in total). A size too small to hold them is refused naming them. A CPU, memory, GPU count or disk above the largest allowed machine's room for one workload raises APIError with status_code 400 naming the resource and both figures, and nothing is created or changed. Check client.compute.list_pools() allowed_sizes (max_cpu, max_memory_gib, max_disk_gib, max_gpus) before a large size.

set_visibility​

set_visibility(resource_id: str, visibility: str) -> Permissions

Make the resource public (every user can find and use it) or private.

Parameters

  • resource_id (str): The resource's id.
  • visibility (str): "public" or "private".

status​

status(model_id: str) -> AIModelStatus

Get model deployment status.

time_series​

time_series(model_id: str, *, window: SeriesWindow | None = None, granularity: SeriesGranularity | None = None) -> ModelUseSeries

Get a model's AI Gateway use over time: requests, errors, tokens, latency.

Counts the requests you may count (your own; an admin everyone's).

Every hour (24h) or UTC day (7d, 30d) of the window is a point, from the one its start falls in to the current one; a quiet point counts zero.

Parameters

  • model_id (str): The model ID.
  • window ({"24h", "7d", "30d"} | None, optional): The window. The API's default is "30d".
  • granularity ({"hourly", "daily"} | None, optional): hourly for 24h, daily otherwise; the window's when left out (another is refused).

update_to_catalog​

update_to_catalog(model_id: str, *, instance_type: str | None = None) -> AIModel

Move a self-hosted model to the newer version of its catalog entry.

Only when its newer_catalog_version is set; never automatic. The old deployment is replaced by one built from the catalog's Dockerfile (a Dockerfile of its own is replaced by it); the model keeps its other overrides (limits, scaling, spot choice, environment variables, disk) and its own instance. A catalog instance with a bigger GPU than the model runs on now is refused (ValidationError, nothing changed) unless instance_type names the instance. No newer version, or a model still building or deploying, is a ConflictError.