Model (experiment) validation flow

The flow of validating a model in H2O Model Validation can be summarized in the following sequential steps:

• Step 1: Select model
• Step 2: Run validation tests
• Step 3: Review metrics

In the below sections, each step, in turn, is summarized.

Step 1: Select model

As the first step in the model validation flow, select a model (experiment). H2O Model Validation enables you to validate your models (experiments) in an established Driverless AI (DAI) instance connection.

• To learn how to connect your DAI instance to H2O Model Validation, see Create a connection

Step 2: Run validation tests

As the second step in the model validation flow, run one or multiple validation tests for a model. H2O Model Validation offers an array of validation tests to analyze the robustness and stability of Driverless AI (DAI) experiments (models). When selecting a validation test, each test offers several settings that you can adjust for total control over various factors of a validation test.

• Validation tests for an experiment:

  • Adversarial Similarity
  • Backtesting
  • Drift Detection
  • Size Dependency

Step 3: Review metrics

As the third and final step of the model validation flow, review validation test metrics. H2O Model Validation offers an array of metrics in graphs, charts, and heatmaps, enabling you to understand a validation test in detail.

• Metrics: Adversarial Similarity
• Metrics: Backtesting
• Metrics: Drift Detection
• Metrics: Size Dependency

Feedback

• Submit and view feedback for this page
• Send feedback about H2O Model Validation to cloud-feedback@h2o.ai