Skip to main content

Results and Publishing

Following a job​

Open a job from the AutoML list to see its page. While it runs, the page shows its status and what it is doing:

  • Waiting to start
  • Preparing environment
  • Training models
  • Evaluating models

It also shows when the job started and was last updated. Stop ends a running job, and Delete removes a job.

A completed job​

The cards at the top show:

  • Validation Score: the served model's score on the validation rows, in the job's metric.
  • Test Score: shown when the job held back test rows.
  • Models Trained.
  • Training Time.
  • Best Model: the model the job serves. This is often a weighted ensemble.

Overview​

The Overview tab has these parts:

  • Dataset: rows, columns, the target, the features and the split.
  • Best Model: its validation and test score, its own fit time (as the leaderboard lists it), model size and metric. Publish to Registry is here.
  • Results: every metric, on the validation rows and on the test rows (with the row counts).
  • Results by class (classification jobs): on the validation rows and on the test rows, the confusion matrix (rows are the true class, columns the predicted class; correct predictions on the diagonal, mistakes highlighted) and each class's precision, recall, F1 and support (its number of rows).
  • Quick Links: View Training Experiment opens the job's experiment in Experiments: its runs, one per model trained, to compare them.

A time series job says that its model stays in the job's output volume, because the registry serves tabular and multimodal models.

Leaderboard​

Every model trained, ranked by validation score, with:

  • its test score;
  • its fit time;
  • its prediction time.

The served model is marked.

Feature Importance​

This shows how much the served model's score drops when each feature's values are shuffled (permutation importance). It is shown as a share of the total, measured on rows the model was not trained on:

  • usually its validation rows;
  • for bagged models validated out of fold, the test rows. Such a job without a test share has no ranking.

A negative share means shuffling the feature helped the score. Multimodal models have no feature importance. A time series ranks its covariates.

Configuration​

The settings the job ran with:

  • Training: problem and predictor type, preset, metric and time limit.
  • Hardware: CPU, memory, GPU and its type, disk, and spot or on-demand capacity.
  • Advanced settings for its kind of job:
    • Tabular: model families, ensemble, data handling, training behavior, tuning, hyperparameters and custom JSON.
    • Time series: its forecast settings.
    • Multimodal: its columns, backbone and ensembling.

Logs​

The Logs tab shows the Job log and the Training log. They refresh while the job runs, which the Live badge marks. You can refresh or download either log.

Publish a model​

Publish to Registry on a completed tabular or multimodal job registers its served model in the Model Registry:

  • Model Name: required, and unique among your models (for example customer-churn-predictor).
  • Description: optional.

Publishing can need approval when a governance solution covers ML models.

What the published model carries​

The published model carries what the job recorded:

  • its task (classification, multiclass or regression);
  • the metric;
  • the classes, in the order its probabilities come in;
  • its input columns and their types;
  • its validation and test results;
  • its leaderboard and its feature importance.

How it is served and monitored​

  • It is served from the job's output volume, by AutoGluon's tabular predictor or, for a multimodal model, its multimodal predictor. The framework is strongly-automl or strongly-automl-multimodal.
  • It gets a drift baseline, built automatically from the rows it was validated on, and its drift is analyzed daily from then on (see Baselines).
  • A model whose validation rows could not be saved says so. You then build its baseline from a file on its Monitoring tab.
  • A model trained on few rows can get a baseline smaller than drift needs. Its Monitoring tab says how many rows the baseline has and how many drift needs.

Using the published model​

Publishing also deploys the model: its registry page opens while it builds and starts, and once it is Running you can call it from:

  • its Test tab;
  • the REST API;
  • the Python SDK;
  • workflows;
  • apps.

A multimodal model's inputs:

  • Image inputs take the image itself, as base64 or a data: URL. On the Test tab you pick an image file.
  • Text inputs are free text.
  • Drift leaves text and image inputs out of its comparison, because each value is unique, and compares the model's other inputs.

A model trained on a feature service resolves its features at inference, so callers send only entity keys (see Models and features).

Time series models​

A time series model is not published. It stays in the job's output volume (automl/<job id>/model/), where a workspace can load it with AutoGluon's TimeSeriesPredictor.load.