Skip to content

Settings: Backtesting

H2O Model Validation offers an array of settings for a Backtesting test. Below, each setting is described in turn.

Training dataset

Defines the training dataset H2O Model Validation uses during the validation test to assess model performance over time. H2O Model Validation splits the dataset by time to simulate a model refitting process while observing and recording the model's accuracy over time. During each split, training data before the specified date is used as the training dataset and after - as the test dataset.

Choose the time variable

Defines the time column in the training dataset that H2O Model Validation uses to split the training dataset during the validation

Number of splits

Defines the number of dates H2O Model Validation uses for the dataset splitting and model refitting process during the validation test. H2O Model Validation sets the specified number of dates in the past while being equally apart from one another to generate appropriate dataset splits.

Test period length

Defines the length of the test dataset for each Backtesting split, starting from a split date. By default (auto) and if a model test dataset is available, H2O Model Validation will use the length of the model test dataset. H2O Model Validation will use the model forecast horizon if you did not provide a test dataset when you built your model.

Units

Defines the data time unit H2O Model Validation uses for each test dataset split.

Training period length

Defines the length of the training dataset for each Backtesting split, ending at a split date. By default, H2O Model Validation specifies this setting as auto, which indicates that H2O Model Validation uses all the data up to a split date as the training dataset.

Units

Defines the data time unit H2O Model Validation uses for each training dataset split.

Remove validation experiments from DAI after finish

Determines if H2O Model Validation should delete the Driverless AI (DAI) artifacts, including experiments and datasets generated during the Backtesting validation test. By default, H2O Model Validation checks this setting (enables it), and accordingly, H2O Model Validation deletes all DAI artifacts because they are no longer needed after the validation test is complete.