H2O Eval Studio Pickled Model CLI demo

This example demonstrates how to interpret a pickled model using H2O Eval Studio from the command line.

[1]:
import os
import time

Get H2O Eval Studio command line interface help:

[3]:
!h2o-sonar --help
usage: h2o-sonar [-h] [--dataset DATASET] [--target-col TARGET_COL]
                 [--results-location RESULTS_LOCATION] [--model MODEL]
                 [--validset VALIDSET] [--testset TESTSET]
                 [--use_raw-features USE_RAW_FEATURES]
                 [--weight-col WEIGHT_COL] [--drop-cols DROP_COLS]
                 [--sample-num-rows SAMPLE_NUM_ROWS]
                 [--used-features USED_FEATURES] [--model-type {pickle,mojo}]
                 [--explainer EXPLAINER] [--explainers EXPLAINERS]
                 [--all-explainers] [--explainers-pars EXPLAINERS_PARS]
                 [--config-path CONFIG_PATH] [--config-type CONFIG_TYPE]
                 [--config-value CONFIG_VALUE]
                 [--encryption-key ENCRYPTION_KEY] [-d]
                 [--args-as-json-location ARGS_AS_JSON_LOCATION]
                 [--log-level {error,warning,info,debug}]
                 action entity

H2O Eval Studio Python library for Responsible AI.

H2O Eval Studio is Python package that enables a holistic, low-risk, human-interpretable,
fair, and trustable approach to machine learning by implementing various facets of
Responsible AI.

optional arguments per action and entity:

  show version:
                      show H2O Eval Studio version

  add config:
    --config-path     path to JSon or TOML file with H2O Eval Studio config to be changed
    --config-type     config item type: 'CONNECTION' or 'LICENSE'
    --config-value    config item value (serialized as JSon) to add to the config file
    --encryption-key  secret key to encrypt config fields with sensitive data
                      (alternatively set H2O_SONAR_ENCRYPTION_KEY environment variable)

  show config:
    --config-path     path to JSon or TOML file with H2O Eval Studio config
    --encryption-key  optional secret key to decrypt config fields with sensitive data
                      (alternatively set H2O_SONAR_ENCRYPTION_KEY environment variable)

  list explainers:
    --detailed        show detailed descriptors (only IDs are shown by default)
    --args-as-json-location
                      optional JSon file which overrides filtering CLI arguments

  describe explainer:
    --explainer       explainer ID

  run interpretation:
    --dataset         path to dataset
    --target-col      target column
    --model           path to the serialized model, URL or locator
    --results-location
                      optional path to the interpretation results location (directory)
    --validset        optional path to validation dataset
    --testset         optional path to test dataset
    --use_raw_features
                      force the use of transformed features in surrogate models
                      with 'false', by default the original (raw) features are used
    --weight-col      optional dataset column name with examples weights
    --drop-cols       optional list of dataset columns to drop
    --sample-num-rows
                      optional number of rows to sample from dataset (default: sample
                      based on the RAM size, 0 do not sample, >0 sample to the specified
                      number of rows)
    --all-explainers  run all explainers (only the most important are run by default)
    --used-features   optional comma separated list of features used by the model
    --model-type      optional model type: 'pickle' or 'mojo'
    --explainers      optional comma separated list of explainer IDs to be run
    --explainers-pars optional dictionary with explainer parameters
    --config-path     path to JSon or TOML file with H2O Eval Studio config to be changed
    --args-as-json-location
                      optional JSon file which overrides CLI arguments
    --log-level       optional log level: 'error', 'warning', 'info', 'debug'

  list interpretations:
    --results-location
                      path to directory, URL, location of interpretation results
    --log-level       optional log level: 'error', 'warning', 'info', 'debug'

positional arguments:
  action                action to take: 'list', 'run' or 'describe'
  entity                entity on which to perform the action:
                        'interpretation'(s) or 'explainer'(s)

optional arguments:
  -h, --help            show this help message and exit
  --dataset DATASET     location of the dataset
  --target-col TARGET_COL
                        target column
  --results-location RESULTS_LOCATION
                        location where to store the interpretation results
  --model MODEL         location of the model
  --validset VALIDSET   location of the validation dataset
  --testset TESTSET     location of the test dataset
  --use_raw-features USE_RAW_FEATURES
                        force the use of transformed features in surrogate
                        models with `false`
  --weight-col WEIGHT_COL
                        optional dataset column name with examples weights
  --drop-cols DROP_COLS
                        optional list of dataset columns to drop
  --sample-num-rows SAMPLE_NUM_ROWS
                        optional number of rows to sample from the dataset
  --used-features USED_FEATURES
                        optional comma separated list of features used by the
                        model
  --model-type {pickle,mojo}
                        model type: 'pickle' (.pkl) or 'mojo' (.mojo)
  --explainer EXPLAINER
                        ID of the explainer to describe
  --explainers EXPLAINERS
                        comma separated list of explainer IDs to be run (only
                        the most important explainers are run by default)
  --all-explainers      run all explainers (only the most important explainers
                        are run by default)
  --explainers-pars EXPLAINERS_PARS
                        optional dictionary with explainer parameters - the
                        dictionary key is explainer ID and value is dictionary
                        with parameters; parameter dictionary has parameter
                        name as the key and parameter value as the value
  --config-path CONFIG_PATH
                        path to JSon or TOML file with H2O Eval Studio configuration
                        to be used to override defaults - specify only items
                        you want to change (please refer to
                        h2o_sonar.config.H2oSonarConfig for more details)
  --config-type CONFIG_TYPE
                        configuration item type - 'CONNECTION' or 'LICENSE'
  --config-value CONFIG_VALUE
                        configuration item value represented either as
                        dictionary or as string with JSon serialization of the
                        configuration item - it is expected that the config
                        item is NOT encrypted
  --encryption-key ENCRYPTION_KEY
                        encryption key to be used for encrypting/decrypting
                        sensitive data in the configuration. If not specified,
                        shell environment variable H2O_SONAR_ENCRYPTION_KEY
                        with the encryption key is used.
  -d, --detailed        show detailed descriptors (only IDs are shown by
                        default)
  --args-as-json-location ARGS_AS_JSON_LOCATION
                        location of the JSon file with all command arguments
                        (replacing command line arguments) allowing to load
                        them from the filesystem
  --log-level {error,warning,info,debug}
                        log level

examples:

  h2o-sonar --help
  h2o-sonar show version
  h2o-sonar list explainers
  h2o-sonar list explainers --detailed
  h2o-sonar describe explainer
    --explainer=h2o_sonar.explainers.dia_explainer.DiaExplainer
  h2o-sonar run interpretation
    --dataset=dataset.csv
    --target-col=PROFIT
    --results-location=/home/user/results
    --model=model.pickle
    --all-explainers
  h2o-sonar run interpretation
    --dataset=dataset.csv
    --target-col=PROFIT
    --results-location=/home/user/results
    --model=model.pickle
    --used-features=FEATURE_1,FEATURE_2,FEATURE_3
    --explainers=h2o_sonar.explainers.dia_explainer.DiaExplainer
    --explainers-pars=
      "{'h2o_sonar.explainers.dia_explainer.DiaExplainer':{'cut_off': 0.5}}"
    --drop_cols=COLUMN_1,COLUMN_2,COLUMN_3
  h2o-sonar run interpretation
    --args-as-json-location=h2o-sonar-args.json
  h2o-sonar list interpretations --results-location=/home/user/results

H2O Eval Studio JSon configuration example:
  {
    "h2o_host": "192.168.1.210",
    "h2o_port": 57561,
    "h2o_auto_start": true
  }

Interpretation arguments JSon file example - see interpret.py::run_interpretation():
  {
    "dataset": "dataset.csv",
    "model": "model.pickle",
    "target_col": "PROFIT",
    "results_location": "./results"
  }

Explainer listing arguments JSon file example - see interpret.py::list_explainers():
  {
    "experiment_types": ["regression"],
    "explanation_scopes": ["local_scope"],
    "keywords": ["explains-fairness"],
    "explainer_filter": [{"filter_by": "filter-name", "value": "v"}]
  }

Pickled model to be interpreted by the library:

[4]:
dataset_path = "../../data/creditcard.csv"
target_column = "\"default payment next month\""

# path to sklearn GBM model
model_path = "../../data/models/creditcard-binomial-sklearn-gbm.pkl"

results_path = f"/tmp/{time.time()}"
os.mkdir(results_path)

Run new interpretation using command line interface:

[8]:
!h2o-sonar run interpretation \
  --dataset={dataset_path} \
  --model={model_path} \
  --target-col={target_column} \
  --results-location={results_path}
/home/dvorka/h/mli/git/h2o-sonar/.venv/lib/python3.8/site-packages/shap/utils/_clustering.py:35: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def _pt_shuffle_rec(i, indexes, index_mask, partition_tree, M, pos):
/home/dvorka/h/mli/git/h2o-sonar/.venv/lib/python3.8/site-packages/shap/utils/_clustering.py:54: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def delta_minimization_order(all_masks, max_swap_size=100, num_passes=2):
/home/dvorka/h/mli/git/h2o-sonar/.venv/lib/python3.8/site-packages/shap/utils/_clustering.py:63: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def _reverse_window(order, start, length):
/home/dvorka/h/mli/git/h2o-sonar/.venv/lib/python3.8/site-packages/shap/utils/_clustering.py:69: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def _reverse_window_score_gain(masks, order, start, length):
/home/dvorka/h/mli/git/h2o-sonar/.venv/lib/python3.8/site-packages/shap/utils/_clustering.py:77: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def _mask_delta_score(m1, m2):
/home/dvorka/h/mli/git/h2o-sonar/.venv/lib/python3.8/site-packages/shap/links.py:5: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def identity(x):
/home/dvorka/h/mli/git/h2o-sonar/.venv/lib/python3.8/site-packages/shap/links.py:10: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def _identity_inverse(x):
/home/dvorka/h/mli/git/h2o-sonar/.venv/lib/python3.8/site-packages/shap/links.py:15: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def logit(x):
/home/dvorka/h/mli/git/h2o-sonar/.venv/lib/python3.8/site-packages/shap/links.py:20: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def _logit_inverse(x):
/home/dvorka/h/mli/git/h2o-sonar/.venv/lib/python3.8/site-packages/shap/utils/_masked_model.py:363: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def _build_fixed_single_output(averaged_outs, last_outs, outputs, batch_positions, varying_rows, num_varying_rows, link, linearizing_weights):
/home/dvorka/h/mli/git/h2o-sonar/.venv/lib/python3.8/site-packages/shap/utils/_masked_model.py:385: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def _build_fixed_multi_output(averaged_outs, last_outs, outputs, batch_positions, varying_rows, num_varying_rows, link, linearizing_weights):
/home/dvorka/h/mli/git/h2o-sonar/.venv/lib/python3.8/site-packages/shap/utils/_masked_model.py:428: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def _init_masks(cluster_matrix, M, indices_row_pos, indptr):
/home/dvorka/h/mli/git/h2o-sonar/.venv/lib/python3.8/site-packages/shap/utils/_masked_model.py:439: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def _rec_fill_masks(cluster_matrix, indices_row_pos, indptr, indices, M, ind):
/home/dvorka/h/mli/git/h2o-sonar/.venv/lib/python3.8/site-packages/shap/maskers/_tabular.py:186: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def _single_delta_mask(dind, masked_inputs, last_mask, data, x, noop_code):
/home/dvorka/h/mli/git/h2o-sonar/.venv/lib/python3.8/site-packages/shap/maskers/_tabular.py:197: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def _delta_masking(masks, x, curr_delta_inds, varying_rows_out,
/home/dvorka/h/mli/git/h2o-sonar/.venv/lib/python3.8/site-packages/shap/maskers/_image.py:175: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def _jit_build_partition_tree(xmin, xmax, ymin, ymax, zmin, zmax, total_ywidth, total_zwidth, M, clustering, q):
/home/dvorka/h/mli/git/h2o-sonar/.venv/lib/python3.8/site-packages/shap/explainers/_partition.py:676: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
  def lower_credit(i, value, M, values, clustering):
"is" with a literal. Did you mean "=="?
"is" with a literal. Did you mean "=="?
"is" with a literal. Did you mean "=="?
"is" with a literal. Did you mean "=="?
The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
"is not" with a literal. Did you mean "!="?
"is not" with a literal. Did you mean "!="?
Stratified/random sampler: loading the original dataset '../../data/creditcard.csv' for sampling...
Stratified/random sampler:   -> did NO sampling as the sampling limit is smaller than the number of rows in the dataset: 10000 <= 25000
Trying to unpickle estimator DummyClassifier from version 1.1.2 when using version 1.2.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
Trying to unpickle estimator DecisionTreeRegressor from version 1.1.2 when using version 1.2.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
Trying to unpickle estimator GradientBoostingClassifier from version 1.1.2 when using version 1.2.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
Preparing and checking DIA features (None): dataset=     | BILL_AMT5  PAY_5  EDUCATION  PAY_AMT4  BILL_AMT4  PAY_AMT3  BILL_AMT3  PAY_2    SEX  PAY_AMT6  …  PAY_AMT5  default payment next month  PAY_AMT2  BILL_AMT2  PAY_4
     |     int32  int32      int32     int32      int32     int32      int32  int32  int32     int32        int32                        int8     int32      int32  int32
---- + ---------  -----  ---------  --------  ---------  --------  ---------  -----  -----  --------     --------  --------------------------  --------  ---------  -----
   0 |         0     -2          2         0          0         0        689      2      2         0  …         0                           1       689       3102     -1
   1 |      3455      0          2      1000       3272      1000       2682      2      2      2000  …         0                           1      1000       1725      0
   2 |     14948      0          2      1000      14331      1000      13559      0      2      5000  …      1000                           0      1500      14027      0
   3 |     28959      0          2      1100      28314      1200      49291      0      2      1000  …      1069                           0      2019      48233      0
   4 |     19146      0          2      9000      20940     10000      35835      0      1       679  …       689                           0     36681       5670      0
   5 |     19619      0          1      1000      19394       657      57608      0      1       800  …      1000                           0      1815      57069      0
   6 |    483003      0          1     20239     542653     38000     445007      0      1     13770  …     13750                           0     40000     412023      0
   7 |      -159      0          2       581        221         0        601     -1      2      1542  …      1687                           0       601        380      0
   8 |     11793      0          3      1000      12211       432      12108      0      2      1000  …      1000                           0         0      14096      0
   9 |     13007     -1          3     13007          0         0          0     -2      1         0  …      1122                           0         0          0     -2
  10 |      1828      0          3       300       2513        50       5535      0      2        66  …      3738                           0        12       9787      0
  11 |     22287     -1          1     22301       8517      8583       9966     -1      2      3640  …         0                           0      9966      21670     -1
  12 |      6500     -1          2      6500       6500      6500       6500      0      2         0  …      2870                           0      6500       6500     -1
  13 |     36137      0          2      3000      66782      3000      65701      2      1         0  …      1500                           1         0      67369      0
  14 |     56875      0          1      3000      59696      3000      63561      0      1      3000  …      3000                           0      3000      67060      0
   … |         …      …          …         …          …         …          …      …      …         …  …         …                           …         …          …      …
9995 |         0     -2          1         0          0         0          0     -2      2      1419  …         0                           0         0        241     -2
9996 |         0     -2          2         0          0         0          0     -2      2         0  …         0                           0         0          0     -2
9997 |    176717      0          3     27080     151078     10000     142520      0      1      4200  …     10017                           0      5000     144085      0
9998 |         0     -2          2         0          0         0          0      2      2         0  …         0                           1         0        780     -2
9999 |     19255      0          2      3000      19506      3000      19750      0      1      3000  …      3000                           0      3000      20715      0
[10000 rows x 25 columns]
 dataset_meta={
  "shape": "(10000, 25)",
  "row_count": 10000,
  "column_names": [
    "ID",
    "LIMIT_BAL",
    "SEX",
    "EDUCATION",
    "MARRIAGE",
    "AGE",
    "PAY_0",
    "PAY_2",
    "PAY_3",
    "PAY_4",
    "PAY_5",
    "PAY_6",
    "BILL_AMT1",
    "BILL_AMT2",
    "BILL_AMT3",
    "BILL_AMT4",
    "BILL_AMT5",
    "BILL_AMT6",
    "PAY_AMT1",
    "PAY_AMT2",
    "PAY_AMT3",
    "PAY_AMT4",
    "PAY_AMT5",
    "PAY_AMT6",
    "default payment next month"
  ],
  "column_types": [
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int"
  ],
  "column_uniques": [
    10000,
    72,
    2,
    7,
    4,
    54,
    11,
    11,
    11,
    11,
    10,
    10,
    8371,
    8215,
    8072,
    7913,
    7764,
    7550,
    3763,
    3581,
    3305,
    3247,
    3258,
    3174,
    2
  ],
  "columns_cat": [],
  "columns_num": [],
  "file_path": "../../data/creditcard.csv",
  "file_name": "",
  "file_size": 944719,
  "missing_values": [
    "",
    "?",
    "None",
    "nan",
    "NA",
    "N/A",
    "unknown",
    "inf",
    "-inf",
    "1.7976931348623157e+308",
    "-1.7976931348623157e+308"
  ],
  "columns_meta": [
    {
      "name": "ID",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": true,
      "is_numeric": true,
      "is_categorical": false,
      "count": 10000,
      "frequency": 0,
      "unique": 10000,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "LIMIT_BAL",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 72,
      "frequency": 0,
      "unique": 72,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "SEX",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 2,
      "frequency": 0,
      "unique": 2,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "EDUCATION",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 7,
      "frequency": 0,
      "unique": 7,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "MARRIAGE",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 4,
      "frequency": 0,
      "unique": 4,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "AGE",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 54,
      "frequency": 0,
      "unique": 54,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_0",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 11,
      "frequency": 0,
      "unique": 11,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_2",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 11,
      "frequency": 0,
      "unique": 11,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_3",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 11,
      "frequency": 0,
      "unique": 11,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_4",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 11,
      "frequency": 0,
      "unique": 11,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_5",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 10,
      "frequency": 0,
      "unique": 10,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_6",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 10,
      "frequency": 0,
      "unique": 10,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "BILL_AMT1",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 8371,
      "frequency": 0,
      "unique": 8371,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "BILL_AMT2",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 8215,
      "frequency": 0,
      "unique": 8215,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "BILL_AMT3",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 8072,
      "frequency": 0,
      "unique": 8072,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "BILL_AMT4",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 7913,
      "frequency": 0,
      "unique": 7913,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "BILL_AMT5",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 7764,
      "frequency": 0,
      "unique": 7764,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "BILL_AMT6",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 7550,
      "frequency": 0,
      "unique": 7550,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_AMT1",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 3763,
      "frequency": 0,
      "unique": 3763,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_AMT2",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 3581,
      "frequency": 0,
      "unique": 3581,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_AMT3",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 3305,
      "frequency": 0,
      "unique": 3305,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_AMT4",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 3247,
      "frequency": 0,
      "unique": 3247,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_AMT5",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 3258,
      "frequency": 0,
      "unique": 3258,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_AMT6",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 3174,
      "frequency": 0,
      "unique": 3174,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "default payment next month",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 2,
      "frequency": 0,
      "unique": 2,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    }
  ],
  "original_dataset_sampled": false,
  "original_dataset_path": "",
  "original_dataset_size": 0,
  "original_dataset_shape": [
    10000,
    25
  ]
}
Using dataset ENTITY to prepare DIA features: column_names=['ID', 'LIMIT_BAL', 'SEX', 'EDUCATION', 'MARRIAGE', 'AGE', 'PAY_0', 'PAY_2', 'PAY_3', 'PAY_4', 'PAY_5', 'PAY_6', 'BILL_AMT1', 'BILL_AMT2', 'BILL_AMT3', 'BILL_AMT4', 'BILL_AMT5', 'BILL_AMT6', 'PAY_AMT1', 'PAY_AMT2', 'PAY_AMT3', 'PAY_AMT4', 'PAY_AMT5', 'PAY_AMT6', 'default payment next month'] column_uniques=[10000, 72, 2, 7, 4, 54, 11, 11, 11, 11, 10, 10, 8371, 8215, 8072, 7913, 7764, 7550, 3763, 3581, 3305, 3247, 3258, 3174, 2]
DIA group columns prepared using dataset ENTITY: {'PAY_0', 'PAY_2', 'SEX', 'PAY_5', 'PAY_6', 'PAY_3', 'EDUCATION', 'default payment next month', 'MARRIAGE', 'PAY_4'}
DIA group columns to SKIP: {'default payment next month', 'model_pred'}
DIA group columns as BOOLs: [<h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7fb21e8f0100>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7fb21e8f0160>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7fb21e8f01c0>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7fb21e8f0220>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7fb21e8f0280>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7fb21e8f02e0>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7fb21e8f0340>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7fb21e8f03a0>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7fb21e8f0400>]
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
Checking whether there is an H2O instance running at http://localhost:49539 ..... not found.
Attempting to start a local H2O server...
  Java Version: openjdk version "10" 2018-03-20; OpenJDK Runtime Environment 18.3 (build 10+44); OpenJDK 64-Bit Server VM 18.3 (build 10+44, mixed mode)
  Starting server from /home/dvorka/h/mli/git/h2o-sonar/.venv/lib/python3.8/site-packages/hmli/backend/bin/hmli.jar
  Ice root: /tmp/tmpbg2ozvca
  JVM stdout: /tmp/tmpbg2ozvca/hmli_dvorka_started_from_python.out
  JVM stderr: /tmp/tmpbg2ozvca/hmli_dvorka_started_from_python.err
  Server is running at http://127.0.0.1:49539
Connecting to H2O server at http://127.0.0.1:49539 ... successful.
Warning: Your H2O cluster version is too old (1 year, 6 months and 4 days)!Please download and install the latest version from http://hmli.ai/download/
--------------------------  ----------------------------------------------------------------
H2O_cluster_uptime:         00 secs
H2O_cluster_timezone:       Europe/Prague
H2O_data_parsing_timezone:  UTC
H2O_cluster_version:        3.34.0.7
H2O_cluster_version_age:    1 year, 6 months and 4 days !!!
H2O_cluster_name:           H2O_from_python_dvorka_pxcffc
H2O_cluster_total_nodes:    1
H2O_cluster_free_memory:    4 Gb
H2O_cluster_total_cores:    16
H2O_cluster_allowed_cores:  16
H2O_cluster_status:         locked, healthy
H2O_connection_url:         http://127.0.0.1:49539
H2O_connection_proxy:       {"http": null, "https": null}
H2O_internal_security:      False
H2O_API_Extensions:         XGBoost, Algos, MLI, MLI-Driver, Core V3, Core V4, TargetEncoder
Python_version:             3.8.10 final
--------------------------  ----------------------------------------------------------------
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
Connecting to H2O server at http://localhost:49539 ... successful.
Warning: Your H2O cluster version is too old (1 year, 6 months and 4 days)!Please download and install the latest version from http://hmli.ai/download/
--------------------------  ----------------------------------------------------------------
H2O_cluster_uptime:         01 secs
H2O_cluster_timezone:       Europe/Prague
H2O_data_parsing_timezone:  UTC
H2O_cluster_version:        3.34.0.7
H2O_cluster_version_age:    1 year, 6 months and 4 days !!!
H2O_cluster_name:           H2O_from_python_dvorka_pxcffc
H2O_cluster_total_nodes:    1
H2O_cluster_free_memory:    4 Gb
H2O_cluster_total_cores:    16
H2O_cluster_allowed_cores:  16
H2O_cluster_status:         locked, healthy
H2O_connection_url:         http://localhost:49539
H2O_connection_proxy:       {"http": null, "https": null}
H2O_internal_security:      False
H2O_API_Extensions:         XGBoost, Algos, MLI, MLI-Driver, Core V3, Core V4, TargetEncoder
Python_version:             3.8.10 final
--------------------------  ----------------------------------------------------------------
Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%
Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%
Response is numeric, so the regression model will be trained. However, the cardinality is equaled to two, so if you want to train a classification model, convert the response column to categorical before training.
drf Model Build progress: |██████████████████████████████████████████████████████ (done)| 100%
Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%
Export File progress: |██████████████████████████████████████████████████████████ (done)| 100%
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
Connecting to H2O server at http://localhost:49539 ... successful.
Warning: Your H2O cluster version is too old (1 year, 6 months and 4 days)!Please download and install the latest version from http://hmli.ai/download/
--------------------------  ----------------------------------------------------------------
H2O_cluster_uptime:         03 secs
H2O_cluster_timezone:       Europe/Prague
H2O_data_parsing_timezone:  UTC
H2O_cluster_version:        3.34.0.7
H2O_cluster_version_age:    1 year, 6 months and 4 days !!!
H2O_cluster_name:           H2O_from_python_dvorka_pxcffc
H2O_cluster_total_nodes:    1
H2O_cluster_free_memory:    4 Gb
H2O_cluster_total_cores:    16
H2O_cluster_allowed_cores:  16
H2O_cluster_status:         locked, healthy
H2O_connection_url:         http://localhost:49539
H2O_connection_proxy:       {"http": null, "https": null}
H2O_internal_security:      False
H2O_API_Extensions:         XGBoost, Algos, MLI, MLI-Driver, Core V3, Core V4, TargetEncoder
Python_version:             3.8.10 final
--------------------------  ----------------------------------------------------------------
Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%
Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%
Response is numeric, so the regression model will be trained. However, the cardinality is equaled to two, so if you want to train a classification model, convert the response column to categorical before training.
drf Model Build progress: |██████████████████████████████████████████████████████ (done)| 100%
Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%
Export File progress: |██████████████████████████████████████████████████████████ (done)| 100%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 30.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 40.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 50.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 50.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 50.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 50.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 50.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 60.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 60.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 60.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 60.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 60.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 70.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 70.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 70.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 70.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 70.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 80.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 80.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 80.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 80.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 90.0%
More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`). Consider using `matplotlib.pyplot.close()`.
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 90.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 90.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 90.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 90.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 100.0%
h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer: progress 100.0%
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 10.0%
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 20.0%
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 20.0%
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 20.0%
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 30.0%
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 30.0%
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 30.0%
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 40.0%
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 40.0%
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 50.0%
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 50.0%
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 60.0%
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 60.0%
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 60.0%
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 70.0%
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 70.0%
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 70.0%
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 80.0%
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 80.0%
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 90.0%
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 90.0%
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 90.0%
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 90.0%
h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer: progress 100.0%
H2O session _sid_aa21 closed.

Interpretation FINISHED with the following results:
  Results directory:
    file:///tmp/1687763984.958333
  HTML report:
    file:///tmp/1687763984.958333/h2o-sonar/mli_experiment_17dccab1-23ba-4446-9650-73a0d6f8f000/interpretation.html
[9]:
!tree $results_path
/tmp/1687763984.958333
├── h2o-sonar
│   └── mli_experiment_17dccab1-23ba-4446-9650-73a0d6f8f000
│       ├── explainer_h2o_sonar_explainers_dia_explainer_DiaExplainer_cc22295f-6458-4a0e-917d-7d890a76bac9
│       │   ├── global_disparate_impact_analysis
│       │   │   ├── text_plain
│       │   │   │   └── explanation.txt
│       │   │   └── text_plain.meta
│       │   ├── global_html_fragment
│       │   │   ├── text_html
│       │   │   │   ├── dia-0-accuracy.png
│       │   │   │   ├── dia-0-adverse_impact.png
│       │   │   │   ├── dia-0-false_discovery_rate.png
│       │   │   │   ├── dia-0-false_negative_rate.png
│       │   │   │   ├── dia-0-false_omissions_rate.png
│       │   │   │   ├── dia-0-false_positive_rate.png
│       │   │   │   ├── dia-0-negative_predicted_value.png
│       │   │   │   ├── dia-0-n.png
│       │   │   │   ├── dia-0-precision.png
│       │   │   │   ├── dia-0-specificity.png
│       │   │   │   ├── dia-0-true_positive_rate.png
│       │   │   │   ├── dia-1-accuracy.png
│       │   │   │   ├── dia-1-adverse_impact.png
│       │   │   │   ├── dia-1-false_discovery_rate.png
│       │   │   │   ├── dia-1-false_negative_rate.png
│       │   │   │   ├── dia-1-false_omissions_rate.png
│       │   │   │   ├── dia-1-false_positive_rate.png
│       │   │   │   ├── dia-1-negative_predicted_value.png
│       │   │   │   ├── dia-1-n.png
│       │   │   │   ├── dia-1-precision.png
│       │   │   │   ├── dia-1-specificity.png
│       │   │   │   ├── dia-1-true_positive_rate.png
│       │   │   │   ├── dia-2-accuracy.png
│       │   │   │   ├── dia-2-adverse_impact.png
│       │   │   │   ├── dia-2-false_discovery_rate.png
│       │   │   │   ├── dia-2-false_negative_rate.png
│       │   │   │   ├── dia-2-false_omissions_rate.png
│       │   │   │   ├── dia-2-false_positive_rate.png
│       │   │   │   ├── dia-2-negative_predicted_value.png
│       │   │   │   ├── dia-2-n.png
│       │   │   │   ├── dia-2-precision.png
│       │   │   │   ├── dia-2-specificity.png
│       │   │   │   ├── dia-2-true_positive_rate.png
│       │   │   │   ├── dia-3-accuracy.png
│       │   │   │   ├── dia-3-adverse_impact.png
│       │   │   │   ├── dia-3-false_discovery_rate.png
│       │   │   │   ├── dia-3-false_negative_rate.png
│       │   │   │   ├── dia-3-false_omissions_rate.png
│       │   │   │   ├── dia-3-false_positive_rate.png
│       │   │   │   ├── dia-3-negative_predicted_value.png
│       │   │   │   ├── dia-3-n.png
│       │   │   │   ├── dia-3-precision.png
│       │   │   │   ├── dia-3-specificity.png
│       │   │   │   ├── dia-3-true_positive_rate.png
│       │   │   │   ├── dia-4-accuracy.png
│       │   │   │   ├── dia-4-adverse_impact.png
│       │   │   │   ├── dia-4-false_discovery_rate.png
│       │   │   │   ├── dia-4-false_negative_rate.png
│       │   │   │   ├── dia-4-false_omissions_rate.png
│       │   │   │   ├── dia-4-false_positive_rate.png
│       │   │   │   ├── dia-4-negative_predicted_value.png
│       │   │   │   ├── dia-4-n.png
│       │   │   │   ├── dia-4-precision.png
│       │   │   │   ├── dia-4-specificity.png
│       │   │   │   ├── dia-4-true_positive_rate.png
│       │   │   │   ├── dia-5-accuracy.png
│       │   │   │   ├── dia-5-adverse_impact.png
│       │   │   │   ├── dia-5-false_discovery_rate.png
│       │   │   │   ├── dia-5-false_negative_rate.png
│       │   │   │   ├── dia-5-false_omissions_rate.png
│       │   │   │   ├── dia-5-false_positive_rate.png
│       │   │   │   ├── dia-5-negative_predicted_value.png
│       │   │   │   ├── dia-5-n.png
│       │   │   │   ├── dia-5-precision.png
│       │   │   │   ├── dia-5-specificity.png
│       │   │   │   ├── dia-5-true_positive_rate.png
│       │   │   │   ├── dia-6-accuracy.png
│       │   │   │   ├── dia-6-adverse_impact.png
│       │   │   │   ├── dia-6-false_discovery_rate.png
│       │   │   │   ├── dia-6-false_negative_rate.png
│       │   │   │   ├── dia-6-false_omissions_rate.png
│       │   │   │   ├── dia-6-false_positive_rate.png
│       │   │   │   ├── dia-6-negative_predicted_value.png
│       │   │   │   ├── dia-6-n.png
│       │   │   │   ├── dia-6-precision.png
│       │   │   │   ├── dia-6-specificity.png
│       │   │   │   ├── dia-6-true_positive_rate.png
│       │   │   │   ├── dia-7-accuracy.png
│       │   │   │   ├── dia-7-adverse_impact.png
│       │   │   │   ├── dia-7-false_discovery_rate.png
│       │   │   │   ├── dia-7-false_negative_rate.png
│       │   │   │   ├── dia-7-false_omissions_rate.png
│       │   │   │   ├── dia-7-false_positive_rate.png
│       │   │   │   ├── dia-7-negative_predicted_value.png
│       │   │   │   ├── dia-7-n.png
│       │   │   │   ├── dia-7-precision.png
│       │   │   │   ├── dia-7-specificity.png
│       │   │   │   ├── dia-7-true_positive_rate.png
│       │   │   │   ├── dia-8-accuracy.png
│       │   │   │   ├── dia-8-adverse_impact.png
│       │   │   │   ├── dia-8-false_discovery_rate.png
│       │   │   │   ├── dia-8-false_negative_rate.png
│       │   │   │   ├── dia-8-false_omissions_rate.png
│       │   │   │   ├── dia-8-false_positive_rate.png
│       │   │   │   ├── dia-8-negative_predicted_value.png
│       │   │   │   ├── dia-8-n.png
│       │   │   │   ├── dia-8-precision.png
│       │   │   │   ├── dia-8-specificity.png
│       │   │   │   ├── dia-8-true_positive_rate.png
│       │   │   │   └── explanation.html
│       │   │   └── text_html.meta
│       │   ├── log
│       │   │   └── explainer_run_cc22295f-6458-4a0e-917d-7d890a76bac9.log
│       │   ├── model_problems
│       │   │   └── problems_and_actions.json
│       │   ├── result_descriptor.json
│       │   └── work
│       │       ├── dia_entity.json
│       │       ├── EDUCATION
│       │       │   ├── 0
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 1
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 2
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 3
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 4
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 5
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 6
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   └── metrics.jay
│       │       ├── MARRIAGE
│       │       │   ├── 0
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 1
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 2
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 3
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   └── metrics.jay
│       │       ├── PAY_0
│       │       │   ├── 0
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 1
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 10
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 2
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 3
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 4
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 5
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 6
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 7
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 8
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 9
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   └── metrics.jay
│       │       ├── PAY_2
│       │       │   ├── 0
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 1
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 10
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 2
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 3
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 4
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 5
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 6
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 7
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 8
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 9
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   └── metrics.jay
│       │       ├── PAY_3
│       │       │   ├── 0
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 1
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 10
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 2
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 3
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 4
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 5
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 6
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 7
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 8
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 9
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   └── metrics.jay
│       │       ├── PAY_4
│       │       │   ├── 0
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 1
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 10
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 2
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 3
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 4
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 5
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 6
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 7
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 8
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 9
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   └── metrics.jay
│       │       ├── PAY_5
│       │       │   ├── 0
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 1
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 2
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 3
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 4
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 5
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 6
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 7
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 8
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 9
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   └── metrics.jay
│       │       ├── PAY_6
│       │       │   ├── 0
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 1
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 2
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 3
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 4
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 5
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 6
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 7
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 8
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 9
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   └── metrics.jay
│       │       └── SEX
│       │           ├── 0
│       │           │   ├── cm.jay
│       │           │   ├── disparity.jay
│       │           │   ├── me_smd.jay
│       │           │   └── parity.jay
│       │           ├── 1
│       │           │   ├── cm.jay
│       │           │   ├── disparity.jay
│       │           │   ├── me_smd.jay
│       │           │   └── parity.jay
│       │           └── metrics.jay
│       ├── explainer_h2o_sonar_explainers_dt_surrogate_explainer_DecisionTreeSurrogateExplainer_5f33d98a-d416-46b5-a0b2-2597907a4c01
│       │   ├── global_custom_archive
│       │   │   ├── application_zip
│       │   │   │   └── explanation.zip
│       │   │   └── application_zip.meta
│       │   ├── global_decision_tree
│       │   │   ├── application_json
│       │   │   │   ├── dt_class_0.json
│       │   │   │   └── explanation.json
│       │   │   └── application_json.meta
│       │   ├── global_html_fragment
│       │   │   ├── text_html
│       │   │   │   ├── dt-class-0.png
│       │   │   │   └── explanation.html
│       │   │   └── text_html.meta
│       │   ├── local_decision_tree
│       │   │   ├── application_json
│       │   │   │   └── explanation.json
│       │   │   └── application_json.meta
│       │   ├── log
│       │   │   └── explainer_run_5f33d98a-d416-46b5-a0b2-2597907a4c01.log
│       │   ├── model_problems
│       │   │   └── problems_and_actions.json
│       │   ├── result_descriptor.json
│       │   └── work
│       │       ├── dt-class-0.dot
│       │       ├── dt-class-0.dot.pdf
│       │       ├── dtModel.json
│       │       ├── dtpaths_frame.bin
│       │       ├── dtPathsFrame.csv
│       │       ├── dtsurr_mojo.zip
│       │       ├── dtSurrogate.json
│       │       └── dt_surrogate_rules.zip
│       ├── explainer_h2o_sonar_explainers_pd_ice_explainer_PdIceExplainer_9033431a-034f-46a4-9c4d-f90f9a9468ba
│       │   ├── global_html_fragment
│       │   │   ├── text_html
│       │   │   │   ├── explanation.html
│       │   │   │   ├── pd-feature-0-class-0.png
│       │   │   │   ├── pd-feature-1-class-0.png
│       │   │   │   ├── pd-feature-2-class-0.png
│       │   │   │   ├── pd-feature-3-class-0.png
│       │   │   │   ├── pd-feature-4-class-0.png
│       │   │   │   ├── pd-feature-5-class-0.png
│       │   │   │   ├── pd-feature-6-class-0.png
│       │   │   │   ├── pd-feature-7-class-0.png
│       │   │   │   ├── pd-feature-8-class-0.png
│       │   │   │   └── pd-feature-9-class-0.png
│       │   │   └── text_html.meta
│       │   ├── global_partial_dependence
│       │   │   ├── application_json
│       │   │   │   ├── explanation.json
│       │   │   │   ├── pd_feature_0_class_0.json
│       │   │   │   ├── pd_feature_1_class_0.json
│       │   │   │   ├── pd_feature_2_class_0.json
│       │   │   │   ├── pd_feature_3_class_0.json
│       │   │   │   ├── pd_feature_4_class_0.json
│       │   │   │   ├── pd_feature_5_class_0.json
│       │   │   │   ├── pd_feature_6_class_0.json
│       │   │   │   ├── pd_feature_7_class_0.json
│       │   │   │   ├── pd_feature_8_class_0.json
│       │   │   │   └── pd_feature_9_class_0.json
│       │   │   └── application_json.meta
│       │   ├── local_individual_conditional_explanation
│       │   │   ├── application_vnd_h2oai_json_datatable_jay
│       │   │   │   ├── explanation.json
│       │   │   │   ├── ice_feature_0_class_0.jay
│       │   │   │   ├── ice_feature_1_class_0.jay
│       │   │   │   ├── ice_feature_2_class_0.jay
│       │   │   │   ├── ice_feature_3_class_0.jay
│       │   │   │   ├── ice_feature_4_class_0.jay
│       │   │   │   ├── ice_feature_5_class_0.jay
│       │   │   │   ├── ice_feature_6_class_0.jay
│       │   │   │   ├── ice_feature_7_class_0.jay
│       │   │   │   ├── ice_feature_8_class_0.jay
│       │   │   │   ├── ice_feature_9_class_0.jay
│       │   │   │   └── y_hat.jay
│       │   │   └── application_vnd_h2oai_json_datatable_jay.meta
│       │   ├── log
│       │   │   └── explainer_run_9033431a-034f-46a4-9c4d-f90f9a9468ba.log
│       │   ├── model_problems
│       │   │   └── problems_and_actions.json
│       │   ├── result_descriptor.json
│       │   └── work
│       │       ├── h2o_sonar-ice-dai-model-10.jay
│       │       ├── h2o_sonar-ice-dai-model-1.jay
│       │       ├── h2o_sonar-ice-dai-model-2.jay
│       │       ├── h2o_sonar-ice-dai-model-3.jay
│       │       ├── h2o_sonar-ice-dai-model-4.jay
│       │       ├── h2o_sonar-ice-dai-model-5.jay
│       │       ├── h2o_sonar-ice-dai-model-6.jay
│       │       ├── h2o_sonar-ice-dai-model-7.jay
│       │       ├── h2o_sonar-ice-dai-model-8.jay
│       │       ├── h2o_sonar-ice-dai-model-9.jay
│       │       ├── h2o_sonar-ice-dai-model.json
│       │       ├── h2o_sonar-pd-dai-model.json
│       │       └── mli_dataset_y_hat.jay
│       ├── explainer_h2o_sonar_explainers_residual_dt_surrogate_explainer_ResidualDecisionTreeSurrogateExplainer_2b79d59a-4de0-49fa-b3d8-1444703ad29c
│       │   ├── global_custom_archive
│       │   │   ├── application_zip
│       │   │   │   └── explanation.zip
│       │   │   └── application_zip.meta
│       │   ├── global_decision_tree
│       │   │   ├── application_json
│       │   │   │   ├── dt_class_0.json
│       │   │   │   └── explanation.json
│       │   │   └── application_json.meta
│       │   ├── global_html_fragment
│       │   │   ├── text_html
│       │   │   │   ├── dt-class-0.png
│       │   │   │   └── explanation.html
│       │   │   └── text_html.meta
│       │   ├── local_decision_tree
│       │   │   ├── application_json
│       │   │   │   └── explanation.json
│       │   │   └── application_json.meta
│       │   ├── log
│       │   │   └── explainer_run_2b79d59a-4de0-49fa-b3d8-1444703ad29c.log
│       │   ├── model_problems
│       │   │   └── problems_and_actions.json
│       │   ├── result_descriptor.json
│       │   └── work
│       │       ├── dt-class-0.dot
│       │       ├── dt-class-0.dot.pdf
│       │       ├── dtModel.json
│       │       ├── dtpaths_frame.bin
│       │       ├── dtPathsFrame.csv
│       │       ├── dtsurr_mojo.zip
│       │       ├── dtSurrogate.json
│       │       └── dt_surrogate_rules.zip
│       ├── explainer_h2o_sonar_explainers_summary_shap_explainer_SummaryShapleyExplainer_f075f921-4200-4ed6-a904-35da068784fc
│       │   ├── global_html_fragment
│       │   │   ├── text_html
│       │   │   │   ├── explanation.html
│       │   │   │   ├── feature_0_class_0.png
│       │   │   │   ├── feature_10_class_0.png
│       │   │   │   ├── feature_11_class_0.png
│       │   │   │   ├── feature_12_class_0.png
│       │   │   │   ├── feature_13_class_0.png
│       │   │   │   ├── feature_14_class_0.png
│       │   │   │   ├── feature_15_class_0.png
│       │   │   │   ├── feature_16_class_0.png
│       │   │   │   ├── feature_17_class_0.png
│       │   │   │   ├── feature_18_class_0.png
│       │   │   │   ├── feature_19_class_0.png
│       │   │   │   ├── feature_1_class_0.png
│       │   │   │   ├── feature_20_class_0.png
│       │   │   │   ├── feature_21_class_0.png
│       │   │   │   ├── feature_22_class_0.png
│       │   │   │   ├── feature_23_class_0.png
│       │   │   │   ├── feature_2_class_0.png
│       │   │   │   ├── feature_3_class_0.png
│       │   │   │   ├── feature_4_class_0.png
│       │   │   │   ├── feature_5_class_0.png
│       │   │   │   ├── feature_6_class_0.png
│       │   │   │   ├── feature_7_class_0.png
│       │   │   │   ├── feature_8_class_0.png
│       │   │   │   ├── feature_9_class_0.png
│       │   │   │   └── shapley-class-0.png
│       │   │   └── text_html.meta
│       │   ├── global_summary_feature_importance
│       │   │   ├── application_json
│       │   │   │   ├── explanation.json
│       │   │   │   ├── feature_0_class_0.png
│       │   │   │   ├── feature_10_class_0.png
│       │   │   │   ├── feature_11_class_0.png
│       │   │   │   ├── feature_12_class_0.png
│       │   │   │   ├── feature_13_class_0.png
│       │   │   │   ├── feature_14_class_0.png
│       │   │   │   ├── feature_15_class_0.png
│       │   │   │   ├── feature_16_class_0.png
│       │   │   │   ├── feature_17_class_0.png
│       │   │   │   ├── feature_18_class_0.png
│       │   │   │   ├── feature_19_class_0.png
│       │   │   │   ├── feature_1_class_0.png
│       │   │   │   ├── feature_20_class_0.png
│       │   │   │   ├── feature_21_class_0.png
│       │   │   │   ├── feature_22_class_0.png
│       │   │   │   ├── feature_23_class_0.png
│       │   │   │   ├── feature_2_class_0.png
│       │   │   │   ├── feature_3_class_0.png
│       │   │   │   ├── feature_4_class_0.png
│       │   │   │   ├── feature_5_class_0.png
│       │   │   │   ├── feature_6_class_0.png
│       │   │   │   ├── feature_7_class_0.png
│       │   │   │   ├── feature_8_class_0.png
│       │   │   │   ├── feature_9_class_0.png
│       │   │   │   ├── summary_feature_importance_class_0_offset_0.json
│       │   │   │   ├── summary_feature_importance_class_0_offset_1.json
│       │   │   │   └── summary_feature_importance_class_0_offset_2.json
│       │   │   ├── application_json.meta
│       │   │   ├── application_vnd_h2oai_json_datatable_jay
│       │   │   │   ├── explanation.json
│       │   │   │   └── summary_feature_importance_class_0.jay
│       │   │   ├── application_vnd_h2oai_json_datatable_jay.meta
│       │   │   ├── text_markdown
│       │   │   │   ├── explanation.md
│       │   │   │   └── shapley-class-0.png
│       │   │   └── text_markdown.meta
│       │   ├── log
│       │   │   └── explainer_run_f075f921-4200-4ed6-a904-35da068784fc.log
│       │   ├── model_problems
│       │   │   └── problems_and_actions.json
│       │   ├── result_descriptor.json
│       │   └── work
│       │       ├── raw_shapley_contribs_class_0.jay
│       │       ├── raw_shapley_contribs_index.json
│       │       ├── report.md
│       │       └── shapley-class-0.png
│       ├── explainers_parameters.json
│       ├── interpretation.html
│       └── interpretation.json
├── h2o-sonar.html
└── h2o-sonar.log

140 directories, 593 files

See interpretation.html ^ for interpretation results.

[ ]: