Skip to main content

Create an AutoML Job

Open MLOps > AutoML and select New AutoML Job. The page has five steps: Dataset, Preparation, Training, Advanced and Review. Next checks each step and lists anything still missing before it moves on.

Step 1: Dataset​

Job Name​

The job's name. It is shown on the jobs list and suggested as the published model's name.

Dataset Source​

Where the training rows come from. There are four sources.

Shared Volume​

A file in a code/data volume you can read.

  1. Pick the volume. Search by name, description or scope. Each volume shows how many data files it holds, or No data.
  2. Pick the file. Supported formats are CSV, JSON, Parquet and TSV.
  3. Change Volume goes back to the volume list.

Upload​

This uploads files into a shared volume. The job then trains from that volume as if you had picked it under Shared Volume.

  • Files: a CSV table. For image columns, also add the images the table names; their paths in the table are paths in the volume. You can upload several files at once.
  • Training table: when you upload more than one CSV, the one to train on.
  • Shared volume to keep the data in: an existing shared volume, or or create a new shared volume with a name you give.

Select Upload. When it finishes, the page switches to Shared Volume with the uploaded table selected.

Data Source​

The job reads rows straight from one of your data sources or add-ons when it runs. It connects the way an app does. Nothing is copied first, and every row is read.

  1. Choose Data source or Add-on, then pick one. Search by name, and sort by name or type.
  2. Under What to read from, say what to read. The fields depend on the kind of store:
StoreWhat you name
SQL databases: PostgreSQL, MySQL, MariaDB, SQL Server, Oracle, Snowflake, BigQuery, ClickHouse, Redshift, CockroachDB, TimescaleDB and similarA Table with an optional Schema (a Dataset for BigQuery), or a read-only Query. A statement that writes is refused.
MongoDBA Collection, with an optional JSON Filter or an Aggregation pipeline. A pipeline with $out or $merge is refused.
ElasticsearchAn Index, with an optional Query DSL
CouchDBA Collection, with an optional JSON Filter
Couchbase, ArangoDB, SurrealDBA Collection, or a read-only query (SQL++, AQL or SurrealQL)
DynamoDB, FirestoreA Table (DynamoDB) or Collection
Neo4j, NeptuneA read-only Cypher or openCypher query. Each returned column becomes a column.
TigerGraphA Vertex type. Every vertex of that type is read, with its id and attributes.
Vector stores: Milvus, Pinecone, Weaviate, Qdrant, Chroma, VespaA Collection (an Index for Pinecone, with an optional Namespace; namespace/documenttype for Vespa). Each vector becomes one column per dimension.
Google Sheets, Airtable, Baserow, NocoDB, SeaTable, GristA worksheet or table
Object storage: S3, MinIO, GCS, Azure BlobAn Object key or prefix. This is a CSV, TSV, Parquet or JSON (lines) file, or a prefix ending in / whose files (all one format) are read as one table.
RedisA Key pattern, such as customer:*. Each key is one row: a hash or a JSON value.
Kafka, PulsarA Topic. Every message from the first to the last when the job starts is read. Nothing is consumed.

Some types are not offered, and the page names them with the reason:

  • RabbitMQ and SQS: reading would remove messages from the apps that consume them.
  • MQTT: it keeps no history.
  • Memcached: it cannot list its keys.
  • Marqo: it returns documents only by id or search.
  • API keys for search services: these are not data.

The target column is checked when the job reads the data. If it is missing, the job fails and names the columns it found.

Feature Service​

This trains on features resolved point in time from a feature store.

  • Feature Store: a store you can read.
  • Feature Service: one of its services. The job trains on the newest version of the service.
  • Entity / label file: a CSV or Parquet file with:
    • the entity join keys;
    • an event_timestamp per row;
    • the target column.

The feature store resolves each row's features as of its event_timestamp, so no feature carries information from after that moment. The model then trains on the joined result. The published model records the feature service version it trained on, and it resolves those features at inference (see Models and features).

Output Volume​

This is the shared volume the trained model and its artifacts are written to. Pick an Existing shared volume or Create a new shared volume by name. Each job writes under its own folder:

PathHolds
automl/<job id>/model/The AutoGluon predictor
automl/<job id>/artifacts/leaderboard.csv, feature_importance.csv, training_summary.json

The model stays there, and publishing reads it from there.

Step 2: Preparation​

  • Target Column: the column to predict. When the page can read the dataset's columns, it lists them to pick from. Otherwise, type the name.

  • Feature Columns: the columns the model may read. All are checked by default. Uncheck any column that should be left out, such as:

    • an id;
    • a free-text note;
    • a column that would leak the answer.

    For a feature service, the features are the service's, and the entity keys and event_timestamp are what the join runs on, not features.

Step 3: Training​

Problem Type​

Tabular Data, Multimodal or Time Series. Choosing Time Series makes the predictor a Forecaster with the MASE metric.

Data Splits​

This applies to Tabular and Multimodal jobs. The bar shows the training, validation and test shares.

SplitDefaultRangeWhat it is for
Training70%The restThe rows the models learn from
Validation15%0 to 30%Rows the models are scored, ranked and stacked on. At 0, AutoGluon holds back its own share, or scores bagged models out of fold.
Test15%0 to 30%Rows no model trains or is picked on. The served model is scored on them once, for an unbiased Test Score. At 0, the job has no test score.

A classifier's splits keep each class's proportion. Rows without a target value are dropped first.

Predictor Type and Evaluation Metric​

The metric is what models are ranked by. The choices follow the predictor type, and switching the predictor picks one of the new type's metrics.

Predictor TypeEvaluation Metric choices (first is the default)
Binary Classifieraccuracy, roc_auc, f1, precision, recall, log_loss
Multiclass Classifieraccuracy, balanced_accuracy, f1_macro, f1_micro, f1_weighted
Regressorrmse, mae, mape, r2, median_absolute_error
Forecaster (Time Series)MASE, MAPE, sMAPE, RMSE, MAE, WAPE, WQL

Training Preset​

PresetDescriptionProblem types
Best QualityHighest accuracy, longest trainingAll
High QualityGood accuracy, moderate training timeAll
Good QualityBalanced accuracy and speedTabular only
Medium Quality (default)Fast training, reasonable accuracyAll
Optimize for DeploymentFast inference, smaller modelsTabular only

Time Limit (seconds)​

This is how long training may run: 60 to 86,400 seconds, with a default of 3,600. The page shows it as a duration. AutoGluon fits as many models as the time allows, so a short limit trains fewer of them.

Hardware Resources​

FieldDefaultRange
CPU (vCPU)41 to 32
Memory (GB)161 to 256
Disk (GB)2010 to 1,000
GPU Count00 to 8
  • GPU type: chosen when the GPU count is above 0.
  • Use Spot Instances: trains on spot capacity at a lower price. Spot capacity can be reclaimed. Once it is on, Fall back to on-demand when spot capacity is unavailable (checked by default) runs it on on-demand capacity when no spot is available; unchecked, it waits for spot. See Spot Instances.

Tabular models rarely need a GPU. A multimodal model with text or image columns trains far faster on one.

Step 4: Advanced​

Every advanced setting is optional, and the defaults are sensible. See Advanced settings.

Step 5: Review​

Review lists the job's details, data configuration, training configuration, training resources and advanced options. Select Start Training to start the job. Its page opens and follows it as it prepares, trains and evaluates (see Results and publishing).