H2O Sonar Pickled Model CLI demo

This example demonstrates how to interpret a pickled model using H2O Sonar from the command line.

[6]:
import os
import time

Get H2O Sonar command line interface help:

[7]:
!h2o-sonar --help
usage: h2o-sonar [-h] [--dataset DATASET] [--target-col TARGET_COL]
                 [--results-location RESULTS_LOCATION]
                 [--results-formats RESULTS_FORMATS] [--model MODEL]
                 [--validset VALIDSET] [--testset TESTSET]
                 [--use_raw-features USE_RAW_FEATURES]
                 [--weight-col WEIGHT_COL] [--drop-cols DROP_COLS]
                 [--sample-num-rows SAMPLE_NUM_ROWS]
                 [--used-features USED_FEATURES] [--model-type {pickle,mojo}]
                 [--explainer EXPLAINER] [--explainers EXPLAINERS]
                 [--all-explainers] [--explainers-pars EXPLAINERS_PARS]
                 [--config-path CONFIG_PATH] [--config-type CONFIG_TYPE]
                 [--config-value CONFIG_VALUE]
                 [--encryption-key ENCRYPTION_KEY] [-d]
                 [--interpretation INTERPRETATION]
                 [--collection-name COLLECTION_NAME] [--upload-to UPLOAD_TO]
                 [--args-as-json-location ARGS_AS_JSON_LOCATION]
                 [--log-level {error,warning,info,debug}]
                 action entity

H2O Sonar Python library for Responsible AI.

H2O Sonar is Python package that enables a holistic, low-risk, human-interpretable,
fair, and trustable approach to machine learning by implementing various facets of
Responsible AI.

model, dataset, validset or testset handle schema:

  resource:connection:"<connection_configuration_name>":key:<resource_key>
  [:version:<resource_version>]

optional arguments per action and entity:

  show version:
                      show H2O Sonar version

  add config:
    --config-path     path to JSon or TOML file with H2O Sonar config to be changed
    --config-type     config item type: 'CONNECTION' or 'LICENSE'
    --config-value    config item value (serialized as JSon) to add to the config file
    --encryption-key  secret key to encrypt config fields with sensitive data
                      (alternatively set H2O_SONAR_ENCRYPTION_KEY environment variable)

  show config:
    --config-path     path to JSon or TOML file with H2O Sonar config
    --encryption-key  optional secret key to decrypt config fields with sensitive data
                      (alternatively set H2O_SONAR_ENCRYPTION_KEY environment variable)

  list explainers:
    --detailed        show detailed descriptors (only IDs are shown by default)
    --args-as-json-location
                      optional JSon file which overrides filtering CLI arguments

  describe explainer:
    --explainer       explainer ID

  run interpretation:
    --dataset         path to dataset
    --target-col      target column
    --model           path to the serialized model, URL or locator
    --results-location
                      optional path to the interpretation results location (directory)
    --validset        optional path to validation dataset
    --testset         optional path to test dataset
    --use_raw_features
                      force the use of transformed features in surrogate models
                      with 'false', by default the original (raw) features are used
    --weight-col      optional dataset column name with examples weights
    --drop-cols       optional list of dataset columns to drop
    --sample-num-rows
                      optional number of rows to sample from dataset (default: sample
                      based on the RAM size, 0 do not sample, >0 sample to the specified
                      number of rows)
    --all-explainers  run all explainers (only the most important are run by default)
    --used-features   optional comma separated list of features used by the model
    --model-type      optional model type: 'pickle' or 'mojo'
    --explainers      optional comma separated list of explainer IDs to be run
    --explainers-pars optional dictionary with explainer parameters
    --config-path     path to JSon or TOML file with H2O Sonar config to be changed
    --encryption-key  secret key to encrypt config fields with sensitive data
                      (alternatively set H2O_SONAR_ENCRYPTION_KEY environment variable)
    --upload-to       optional (h2oGPT Enterprise) connection key from the configuration
                      where the report is to be uploaded in order to talk to it
    --args-as-json-location
                      optional JSon file which overrides CLI arguments
    --log-level       optional log level: 'error', 'warning', 'info', 'debug'

  list interpretations:
    --results-location
                      path to directory, URL, location of interpretation results
    --log-level       optional log level: 'error', 'warning', 'info', 'debug'

  upload interpretations:
    --interpretation  path to the interpretation report PDF or HTML file
    --upload-to       (h2oGPT Enterprise) connection key from the configuration
                      where the report is to be uploaded in order to talk to it
    --collection-name optional name of the collection where the report is to be uploaded
    --config-path     path to JSon or TOML file with H2O Sonar config to be changed
    --encryption-key  secret key to encrypt config fields with sensitive data
                      (alternatively set H2O_SONAR_ENCRYPTION_KEY environment variable)

positional arguments:
  action                action to take: 'list', 'run' or 'describe'
  entity                entity on which to perform the action:
                        'interpretation'(s) or 'explainer'(s)

options:
  -h, --help            show this help message and exit
  --dataset DATASET     location of the dataset
  --target-col TARGET_COL
                        target column
  --results-location RESULTS_LOCATION
                        location where to store the interpretation results
  --results-formats RESULTS_FORMATS
                        comma separated list of MIME types of the
                        interpretation results to create
  --model MODEL         location of the model
  --validset VALIDSET   location of the validation dataset
  --testset TESTSET     location of the test dataset
  --use_raw-features USE_RAW_FEATURES
                        force the use of transformed features in surrogate
                        models with `false`
  --weight-col WEIGHT_COL
                        optional dataset column name with examples weights
  --drop-cols DROP_COLS
                        optional list of dataset columns to drop
  --sample-num-rows SAMPLE_NUM_ROWS
                        optional number of rows to sample from the dataset
  --used-features USED_FEATURES
                        optional comma separated list of features used by the
                        model
  --model-type {pickle,mojo}
                        model type: 'pickle' (.pkl) or 'mojo' (.mojo)
  --explainer EXPLAINER
                        ID of the explainer to describe
  --explainers EXPLAINERS
                        comma separated list of explainer IDs to be run (only
                        the most important explainers are run by default)
  --all-explainers      run all explainers (only the most important explainers
                        are run by default)
  --explainers-pars EXPLAINERS_PARS
                        optional dictionary with explainer parameters - the
                        dictionary key is explainer ID and value is dictionary
                        with parameters; parameter dictionary has parameter
                        name as the key and parameter value as the value
  --config-path CONFIG_PATH
                        path to JSon or TOML file with H2O Sonar configuration
                        to be used to override defaults - specify only items
                        you want to change (please refer to
                        h2o_sonar.config.H2oSonarConfig for more details)
  --config-type CONFIG_TYPE
                        configuration item type - 'CONNECTION' or 'LICENSE'
  --config-value CONFIG_VALUE
                        configuration item value represented either as
                        dictionary or as string with JSon serialization of the
                        configuration item - it is expected that the config
                        item is NOT encrypted
  --encryption-key ENCRYPTION_KEY
                        encryption key to be used for encrypting/decrypting
                        sensitive data in the configuration. If not specified,
                        shell environment variable H2O_SONAR_ENCRYPTION_KEY
                        with the encryption key is used.
  -d, --detailed        show detailed descriptors (only IDs are shown by
                        default)
  --interpretation INTERPRETATION
                        path to the interpretation report (PDF or HTML) to be
                        uploaded
  --collection-name COLLECTION_NAME
                        optional (h2oGPT Enterprise) collection name where to
                        upload the interpretation report
  --upload-to UPLOAD_TO
                        optional (h2oGPT Enterprise) connection key from the
                        configuration where the report is to be uploaded in
                        order to talk to it
  --args-as-json-location ARGS_AS_JSON_LOCATION
                        location of the JSon file with all command arguments
                        (replacing command line arguments) allowing to load
                        them from the filesystem
  --log-level {error,warning,info,debug}
                        log level

examples:

  h2o-sonar --help
  h2o-sonar show version
  h2o-sonar list explainers
  h2o-sonar list explainers --detailed
  h2o-sonar describe explainer
    --explainer=h2o_sonar.explainers.dia_explainer.DiaExplainer
  h2o-sonar run interpretation
    --dataset=dataset.csv
    --target-col=PROFIT
    --results-location=/home/user/results
    --model=model.pickle
    --all-explainers
  h2o-sonar run interpretation
    --dataset=dataset.csv
    --target-col=PROFIT
    --results-location=/home/user/results
    --model=model.pickle
    --used-features=FEATURE_1,FEATURE_2,FEATURE_3
    --explainers=h2o_sonar.explainers.dia_explainer.DiaExplainer
    --explainers-pars=
      "{'h2o_sonar.explainers.dia_explainer.DiaExplainer':{'cut_off': 0.5}}"
    --drop_cols=COLUMN_1,COLUMN_2,COLUMN_3
    --config-path=h2o-sonar-config.json
    --upload-to=4cff6fc9-f49a-4dda-aeb5-2c42e9f12807
  h2o-sonar run interpretation
    --args-as-json-location=h2o-sonar-args.json
  h2o-sonar list interpretations --results-location=/home/user/results
  h2o-sonar upload interpretation
    --interpretation=./results/h2o-sonar/mli-experiment/interpretation-detailed.html
    --upload-to=4cff6fc9-f49a-4dda-aeb5-2c42e9f12807

H2O Sonar JSon configuration example:
  {
    "h2o_host": "192.168.0.1",
    "h2o_port": 57561,
    "h2o_auto_start": true,
    "connections": [
      {
        "key": "4cff6fc9-f49a-4dda-aeb5-2c42e9f12807",
        "connection_type": "H2O_GPT_E",
        "name": "H2O GPT Enterprise",
        "description": "H2O GPT Enterprise service.",
        "server_url": "https://h2ogpte.h2o.ai",
        "token": {
          "encrypted": "gAAA3LcKQ7x_X...gnKsBqVdNydYTlk8nyQ=="
        },
        "token_use_type": "API_KEY"
      }
    ]
  }

Interpretation arguments JSon file example - see interpret.py::run_interpretation():
  {
    "dataset": "dataset.csv",
    "model": "model.pickle",
    "target_col": "PROFIT",
    "results_location": "./results"
  }

Explainer listing arguments JSon file example - see interpret.py::list_explainers():
  {
    "experiment_types": ["regression"],
    "explanation_scopes": ["local_scope"],
    "keywords": ["explains-fairness"],
    "explainer_filter": [{"filter_by": "filter-name", "value": "v"}]
  }

Pickled model to be interpreted by the library:

[8]:
dataset_path = "../../data/predictive/creditcard.csv"
target_column = "\"default payment next month\""

# path to sklearn GBM model - update model to match your Scikit learn version
model_path = "../../data/predictive/models/creditcard-binomial-sklearn-1.8.0-gbm.pkl"

results_path = f"/tmp/{time.time()}"
os.mkdir(results_path)

Run new interpretation using command line interface:

[9]:
!h2o-sonar run interpretation \
  --dataset={dataset_path} \
  --model={model_path} \
  --target-col={target_column} \
  --results-location={results_path}
/home/user/h/mli/git/h2o-sonar-FLOSS/.venv/lib/python3.11/site-packages/ragas/metrics/__init__.py:1: LangChainDeprecationWarning: As of langchain-core 0.3.0, LangChain uses pydantic v2 internally. The langchain_core.pydantic_v1 module was a compatibility shim for pydantic v1, and should no longer be used. Please update the code to import from Pydantic directly.

For example, replace imports like: `from langchain_core.pydantic_v1 import BaseModel`
with: `from pydantic import BaseModel`
or the v1 compatibility namespace if you are working in a code base that has not been fully upgraded to pydantic 2 yet.         from pydantic.v1 import BaseModel

  from ragas.metrics._answer_correctness import AnswerCorrectness, answer_correctness
/home/user/h/mli/git/h2o-sonar-FLOSS/.venv/lib/python3.11/site-packages/ragas/metrics/__init__.py:4: LangChainDeprecationWarning: As of langchain-core 0.3.0, LangChain uses pydantic v2 internally. The langchain.pydantic_v1 module was a compatibility shim for pydantic v1, and should no longer be used. Please update the code to import from Pydantic directly.

For example, replace imports like: `from langchain.pydantic_v1 import BaseModel`
with: `from pydantic import BaseModel`
or the v1 compatibility namespace if you are working in a code base that has not been fully upgraded to pydantic 2 yet.         from pydantic.v1 import BaseModel

  from ragas.metrics._context_entities_recall import (
Stratified/random sampler: loading the original dataset '../../data/predictive/creditcard.csv' for sampling...
Stratified/random sampler:   -> did NO sampling as the sampling limit is smaller than the number of rows in the dataset: 10000 <= 25000
Preparing and checking DIA features (None): dataset=     |   SEX     ID    AGE  PAY_5  BILL_AMT2  PAY_AMT1  MARRIAGE  PAY_0  PAY_AMT2  BILL_AMT3  …  PAY_AMT3  PAY_4  BILL_AMT6  EDUCATION  BILL_AMT4
     | int32  int32  int32  int32      int32     int32     int32  int32     int32      int32        int32  int32      int32      int32      int32
---- + -----  -----  -----  -----  ---------  --------  --------  -----  --------  ---------     --------  -----  ---------  ---------  ---------
   0 |     2      1     24     -2       3102         0         1     -2       689        689  …         0     -1          0          2          0
   1 |     2      2     26      0       1725         0         2     -1      1000       2682  …      1000      0       3261          2       3272
   2 |     2      3     34      0      14027      1518         2      0      1500      13559  …      1000      0      15549          2      14331
   3 |     2      4     37      0      48233      2000         1      1      2019      49291  …      1200      0      29547          2      28314
   4 |     1      5     57      0       5670      2000         1      2     36681      35835  …     10000      0      19131          2      20940
   5 |     1      6     37      0      57069      2500         2      3      1815      57608  …       657      0      20024          1      19394
   6 |     1      7     29      0     412023     55000         2      4     40000     445007  …     38000      0     473944          1     542653
   7 |     2      8     23      0        380       380         2      5       601        601  …         0      0        567          2        221
   8 |     2      9     28      0      14096      3329         1      6         0      12108  …       432      0       3719          3      12211
   9 |     1     10     35     -1          0         0         2      7         0          0  …         0     -2      13912          3          0
  10 |     2     11     34      0       9787      2306         2      8        12       5535  …        50      0       3731          3       2513
  11 |     2     12     51     -1      21670     21818         2     -1      9966       9966  …      8583     -1      13668          1       8517
  12 |     2     13     41     -1       6500      1000         2     -1      6500       6500  …      6500     -1       2870          2       6500
  13 |     1     14     30      0      67369      3200         2      1         0      65701  …      3000      0      36894          2      66782
  14 |     1     15     29      0      67060      3000         2      0      3000      63561  …      3000      0      55512          1      59696
   … |     …      …      …      …          …         …         …      …         …          …  …         …      …          …          …          …
9995 |     2   9996     31     -2        241       241         2      1         0          0  …         0     -2          0          1          0
9996 |     2   9997     37     -2          0         0         2     -2         0          0  …         0     -2          0          2          0
9997 |     1   9998     44      0     144085      6437         1      0      5000     142520  …     10000      0     168431          3     151078
9998 |     2   9999     26     -2        780         0         2     -1         0          0  …         0     -2          0          2          0
9999 |     1  10000     36      0      20715      3000         1      0      3000      19750  …      3000      0      17479          2      19506
[10000 rows x 25 columns]
 dataset_meta={
  "shape": "(10000, 25)",
  "row_count": 10000,
  "column_names": [
    "ID",
    "LIMIT_BAL",
    "SEX",
    "EDUCATION",
    "MARRIAGE",
    "AGE",
    "PAY_0",
    "PAY_2",
    "PAY_3",
    "PAY_4",
    "PAY_5",
    "PAY_6",
    "BILL_AMT1",
    "BILL_AMT2",
    "BILL_AMT3",
    "BILL_AMT4",
    "BILL_AMT5",
    "BILL_AMT6",
    "PAY_AMT1",
    "PAY_AMT2",
    "PAY_AMT3",
    "PAY_AMT4",
    "PAY_AMT5",
    "PAY_AMT6",
    "default payment next month"
  ],
  "column_types": [
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int",
    "int"
  ],
  "column_uniques": [
    10000,
    72,
    2,
    7,
    4,
    54,
    11,
    11,
    11,
    11,
    10,
    10,
    8371,
    8215,
    8072,
    7913,
    7764,
    7550,
    3763,
    3581,
    3305,
    3247,
    3258,
    3174,
    2
  ],
  "columns_cat": [],
  "columns_num": [],
  "file_path": "../../data/predictive/creditcard.csv",
  "file_name": "",
  "file_size": 944719,
  "missing_values": [
    "",
    "?",
    "None",
    "nan",
    "NA",
    "N/A",
    "unknown",
    "inf",
    "-inf",
    "1.7976931348623157e+308",
    "-1.7976931348623157e+308"
  ],
  "columns_meta": [
    {
      "name": "ID",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": true,
      "is_numeric": true,
      "is_categorical": false,
      "count": 10000,
      "frequency": 0,
      "unique": 10000,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "LIMIT_BAL",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 72,
      "frequency": 0,
      "unique": 72,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "SEX",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 2,
      "frequency": 0,
      "unique": 2,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "EDUCATION",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 7,
      "frequency": 0,
      "unique": 7,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "MARRIAGE",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 4,
      "frequency": 0,
      "unique": 4,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "AGE",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 54,
      "frequency": 0,
      "unique": 54,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_0",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 11,
      "frequency": 0,
      "unique": 11,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_2",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 11,
      "frequency": 0,
      "unique": 11,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_3",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 11,
      "frequency": 0,
      "unique": 11,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_4",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 11,
      "frequency": 0,
      "unique": 11,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_5",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 10,
      "frequency": 0,
      "unique": 10,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_6",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 10,
      "frequency": 0,
      "unique": 10,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "BILL_AMT1",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 8371,
      "frequency": 0,
      "unique": 8371,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "BILL_AMT2",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 8215,
      "frequency": 0,
      "unique": 8215,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "BILL_AMT3",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 8072,
      "frequency": 0,
      "unique": 8072,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "BILL_AMT4",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 7913,
      "frequency": 0,
      "unique": 7913,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "BILL_AMT5",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 7764,
      "frequency": 0,
      "unique": 7764,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "BILL_AMT6",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 7550,
      "frequency": 0,
      "unique": 7550,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_AMT1",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 3763,
      "frequency": 0,
      "unique": 3763,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_AMT2",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 3581,
      "frequency": 0,
      "unique": 3581,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_AMT3",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 3305,
      "frequency": 0,
      "unique": 3305,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_AMT4",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 3247,
      "frequency": 0,
      "unique": 3247,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_AMT5",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 3258,
      "frequency": 0,
      "unique": 3258,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "PAY_AMT6",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 3174,
      "frequency": 0,
      "unique": 3174,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    },
    {
      "name": "default payment next month",
      "data_type": "int",
      "logical_types": [],
      "format": "",
      "is_id": false,
      "is_numeric": true,
      "is_categorical": false,
      "count": 2,
      "frequency": 0,
      "unique": 2,
      "max": null,
      "min": null,
      "mean": null,
      "std": null,
      "histogram_counts": [],
      "histogram_ticks": []
    }
  ],
  "original_dataset_sampled": false,
  "original_dataset_path": "",
  "original_dataset_size": 0,
  "original_dataset_shape": [
    10000,
    25
  ]
}
Using dataset ENTITY to prepare DIA features: column_names=['ID', 'LIMIT_BAL', 'SEX', 'EDUCATION', 'MARRIAGE', 'AGE', 'PAY_0', 'PAY_2', 'PAY_3', 'PAY_4', 'PAY_5', 'PAY_6', 'BILL_AMT1', 'BILL_AMT2', 'BILL_AMT3', 'BILL_AMT4', 'BILL_AMT5', 'BILL_AMT6', 'PAY_AMT1', 'PAY_AMT2', 'PAY_AMT3', 'PAY_AMT4', 'PAY_AMT5', 'PAY_AMT6', 'default payment next month'] column_uniques=[10000, 72, 2, 7, 4, 54, 11, 11, 11, 11, 10, 10, 8371, 8215, 8072, 7913, 7764, 7550, 3763, 3581, 3305, 3247, 3258, 3174, 2]
DIA group columns prepared using dataset ENTITY: {'SEX', 'PAY_0', 'PAY_3', 'PAY_4', 'PAY_5', 'PAY_2', 'PAY_6', 'default payment next month', 'MARRIAGE', 'EDUCATION'}
DIA group columns to SKIP: {'model_pred', 'default payment next month'}
DIA group columns as BOOLs: [<h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7f8a08468090>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7f8a08469d90>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7f8a08468790>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7f8a08469050>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7f8a084686d0>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7f8a08469f90>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7f8a08469390>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7f8a08468e10>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7f8a08469310>]
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
Checking whether there is an H2O instance running at http://localhost:54321..... not found.
Attempting to start a local H2O server...
  Java Version: openjdk version "10" 2018-03-20; OpenJDK Runtime Environment 18.3 (build 10+44); OpenJDK 64-Bit Server VM 18.3 (build 10+44, mixed mode)
  Starting server from /home/user/h/mli/git/h2o-sonar-FLOSS/.venv/lib/python3.11/site-packages/h2o/backend/bin/h2o.jar
  Ice root: /tmp/tmphs6w297_
  JVM stdout: /tmp/tmphs6w297_/h2o_user_started_from_python.out
  JVM stderr: /tmp/tmphs6w297_/h2o_user_started_from_python.err
  Server is running at http://127.0.0.1:54321
 successful.o H2O server at http://127.0.0.1:54321 ...
--------------------------  -----------------------------
H2O_cluster_uptime:         01 secs
H2O_cluster_timezone:       Europe/Prague
H2O_data_parsing_timezone:  UTC
H2O_cluster_version:        3.46.0.9
H2O_cluster_version_age:    2 months and 4 days
H2O_cluster_name:           H2O_from_python_user_oilgyy
H2O_cluster_total_nodes:    1
H2O_cluster_free_memory:    4 Gb
H2O_cluster_total_cores:    16
H2O_cluster_allowed_cores:  16
H2O_cluster_status:         locked, healthy
H2O_connection_url:         http://127.0.0.1:54321
H2O_connection_proxy:       {"http": null, "https": null}
H2O_internal_security:      False
Python_version:             3.11.11 final
--------------------------  -----------------------------
 successful.o H2O server at http://localhost:54321 ...
--------------------------  -----------------------------
H2O_cluster_uptime:         01 secs
H2O_cluster_timezone:       Europe/Prague
H2O_data_parsing_timezone:  UTC
H2O_cluster_version:        3.46.0.9
H2O_cluster_version_age:    2 months and 4 days
H2O_cluster_name:           H2O_from_python_user_oilgyy
H2O_cluster_total_nodes:    1
H2O_cluster_free_memory:    4 Gb
H2O_cluster_total_cores:    16
H2O_cluster_allowed_cores:  16
H2O_cluster_status:         locked, healthy
H2O_connection_url:         http://localhost:54321
H2O_connection_proxy:       {"http": null, "https": null}
H2O_internal_security:      False
Python_version:             3.11.11 final
--------------------------  -----------------------------
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
Connecting to H2O server at http://localhost:54321 ... successful.
--------------------------  -----------------------------
H2O_cluster_uptime:         01 secs
H2O_cluster_timezone:       Europe/Prague
H2O_data_parsing_timezone:  UTC
H2O_cluster_version:        3.46.0.9
H2O_cluster_version_age:    2 months and 4 days
H2O_cluster_name:           H2O_from_python_user_oilgyy
H2O_cluster_total_nodes:    1
H2O_cluster_free_memory:    4 Gb
H2O_cluster_total_cores:    16
H2O_cluster_allowed_cores:  16
H2O_cluster_status:         locked, healthy
H2O_connection_url:         http://localhost:54321
H2O_connection_proxy:       {"http": null, "https": null}
H2O_internal_security:      False
Python_version:             3.11.11 final
--------------------------  -----------------------------
Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%
Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%
drf Model Build progress: |██████████████████████████████████████████████████████ (done)| 100%
Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%
Converting H2O frame to pandas dataframe using single-thread.  For faster conversion using multi-thread, install polars and pyarrow and use it as pandas_df = h2o_df.as_data_frame(use_multi_thread=True)

X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
Connecting to H2O server at http://localhost:54321 ... successful.
--------------------------  -----------------------------
H2O_cluster_uptime:         04 secs
H2O_cluster_timezone:       Europe/Prague
H2O_data_parsing_timezone:  UTC
H2O_cluster_version:        3.46.0.9
H2O_cluster_version_age:    2 months and 4 days
H2O_cluster_name:           H2O_from_python_user_oilgyy
H2O_cluster_total_nodes:    1
H2O_cluster_free_memory:    4 Gb
H2O_cluster_total_cores:    16
H2O_cluster_allowed_cores:  16
H2O_cluster_status:         locked, healthy
H2O_connection_url:         http://localhost:54321
H2O_connection_proxy:       {"http": null, "https": null}
H2O_internal_security:      False
Python_version:             3.11.11 final
--------------------------  -----------------------------
Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%
Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%
We have detected that your response column has only 2 unique values (0/1). If you wish to train a binary model instead of a regression model, convert your target column to categorical before training.
drf Model Build progress: |██████████████████████████████████████████████████████ (done)| 100%
Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%
Converting H2O frame to pandas dataframe using single-thread.  For faster conversion using multi-thread, install polars and pyarrow and use it as pandas_df = h2o_df.as_data_frame(use_multi_thread=True)

More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`). Consider using `matplotlib.pyplot.close()`.
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names
X does not have valid feature names, but GradientBoostingClassifier was fitted with feature names

Interpretation FINISHED.
  Results directory:
    file:///tmp/1769698076.7775896
  Interpretations index:
    file:///tmp/1769698076.7775896/h2o-sonar.html
  HTML report:
    file:///tmp/1769698076.7775896/h2o-sonar/mli_experiment_ffbb1cd8-c442-40ef-95db-234cc7ba82c2/interpretation.html
Closing connection _sid_8b32 at exit
H2O session _sid_8b32 closed.
Closing connection _sid_b813 at exit
H2O session _sid_b813 closed.
Closing connection _sid_b3ae at exit
H2O session _sid_b3ae closed.
[10]:
!tree $results_path
/tmp/1769698076.7775896
├── h2o-sonar
│   └── mli_experiment_ffbb1cd8-c442-40ef-95db-234cc7ba82c2
│       ├── explainer_h2o_sonar_explainers_dia_explainer_DiaExplainer_86f4e7c7-37d1-47a2-b1cc-0c16f03ec1d7
│       │   ├── global_disparate_impact_analysis
│       │   │   ├── text_plain
│       │   │   │   └── explanation.txt
│       │   │   └── text_plain.meta
│       │   ├── global_html_fragment
│       │   │   ├── text_html
│       │   │   │   ├── dia-0-accuracy.png
│       │   │   │   ├── dia-0-adverse_impact.png
│       │   │   │   ├── dia-0-false_discovery_rate.png
│       │   │   │   ├── dia-0-false_negative_rate.png
│       │   │   │   ├── dia-0-false_omissions_rate.png
│       │   │   │   ├── dia-0-false_positive_rate.png
│       │   │   │   ├── dia-0-negative_predicted_value.png
│       │   │   │   ├── dia-0-n.png
│       │   │   │   ├── dia-0-precision.png
│       │   │   │   ├── dia-0-specificity.png
│       │   │   │   ├── dia-0-true_positive_rate.png
│       │   │   │   ├── dia-1-accuracy.png
│       │   │   │   ├── dia-1-adverse_impact.png
│       │   │   │   ├── dia-1-false_discovery_rate.png
│       │   │   │   ├── dia-1-false_negative_rate.png
│       │   │   │   ├── dia-1-false_omissions_rate.png
│       │   │   │   ├── dia-1-false_positive_rate.png
│       │   │   │   ├── dia-1-negative_predicted_value.png
│       │   │   │   ├── dia-1-n.png
│       │   │   │   ├── dia-1-precision.png
│       │   │   │   ├── dia-1-specificity.png
│       │   │   │   ├── dia-1-true_positive_rate.png
│       │   │   │   ├── dia-2-accuracy.png
│       │   │   │   ├── dia-2-adverse_impact.png
│       │   │   │   ├── dia-2-false_discovery_rate.png
│       │   │   │   ├── dia-2-false_negative_rate.png
│       │   │   │   ├── dia-2-false_omissions_rate.png
│       │   │   │   ├── dia-2-false_positive_rate.png
│       │   │   │   ├── dia-2-negative_predicted_value.png
│       │   │   │   ├── dia-2-n.png
│       │   │   │   ├── dia-2-precision.png
│       │   │   │   ├── dia-2-specificity.png
│       │   │   │   ├── dia-2-true_positive_rate.png
│       │   │   │   ├── dia-3-accuracy.png
│       │   │   │   ├── dia-3-adverse_impact.png
│       │   │   │   ├── dia-3-false_discovery_rate.png
│       │   │   │   ├── dia-3-false_negative_rate.png
│       │   │   │   ├── dia-3-false_omissions_rate.png
│       │   │   │   ├── dia-3-false_positive_rate.png
│       │   │   │   ├── dia-3-negative_predicted_value.png
│       │   │   │   ├── dia-3-n.png
│       │   │   │   ├── dia-3-precision.png
│       │   │   │   ├── dia-3-specificity.png
│       │   │   │   ├── dia-3-true_positive_rate.png
│       │   │   │   ├── dia-4-accuracy.png
│       │   │   │   ├── dia-4-adverse_impact.png
│       │   │   │   ├── dia-4-false_discovery_rate.png
│       │   │   │   ├── dia-4-false_negative_rate.png
│       │   │   │   ├── dia-4-false_omissions_rate.png
│       │   │   │   ├── dia-4-false_positive_rate.png
│       │   │   │   ├── dia-4-negative_predicted_value.png
│       │   │   │   ├── dia-4-n.png
│       │   │   │   ├── dia-4-precision.png
│       │   │   │   ├── dia-4-specificity.png
│       │   │   │   ├── dia-4-true_positive_rate.png
│       │   │   │   ├── dia-5-accuracy.png
│       │   │   │   ├── dia-5-adverse_impact.png
│       │   │   │   ├── dia-5-false_discovery_rate.png
│       │   │   │   ├── dia-5-false_negative_rate.png
│       │   │   │   ├── dia-5-false_omissions_rate.png
│       │   │   │   ├── dia-5-false_positive_rate.png
│       │   │   │   ├── dia-5-negative_predicted_value.png
│       │   │   │   ├── dia-5-n.png
│       │   │   │   ├── dia-5-precision.png
│       │   │   │   ├── dia-5-specificity.png
│       │   │   │   ├── dia-5-true_positive_rate.png
│       │   │   │   ├── dia-6-accuracy.png
│       │   │   │   ├── dia-6-adverse_impact.png
│       │   │   │   ├── dia-6-false_discovery_rate.png
│       │   │   │   ├── dia-6-false_negative_rate.png
│       │   │   │   ├── dia-6-false_omissions_rate.png
│       │   │   │   ├── dia-6-false_positive_rate.png
│       │   │   │   ├── dia-6-negative_predicted_value.png
│       │   │   │   ├── dia-6-n.png
│       │   │   │   ├── dia-6-precision.png
│       │   │   │   ├── dia-6-specificity.png
│       │   │   │   ├── dia-6-true_positive_rate.png
│       │   │   │   ├── dia-7-accuracy.png
│       │   │   │   ├── dia-7-adverse_impact.png
│       │   │   │   ├── dia-7-false_discovery_rate.png
│       │   │   │   ├── dia-7-false_negative_rate.png
│       │   │   │   ├── dia-7-false_omissions_rate.png
│       │   │   │   ├── dia-7-false_positive_rate.png
│       │   │   │   ├── dia-7-negative_predicted_value.png
│       │   │   │   ├── dia-7-n.png
│       │   │   │   ├── dia-7-precision.png
│       │   │   │   ├── dia-7-specificity.png
│       │   │   │   ├── dia-7-true_positive_rate.png
│       │   │   │   ├── dia-8-accuracy.png
│       │   │   │   ├── dia-8-adverse_impact.png
│       │   │   │   ├── dia-8-false_discovery_rate.png
│       │   │   │   ├── dia-8-false_negative_rate.png
│       │   │   │   ├── dia-8-false_omissions_rate.png
│       │   │   │   ├── dia-8-false_positive_rate.png
│       │   │   │   ├── dia-8-negative_predicted_value.png
│       │   │   │   ├── dia-8-n.png
│       │   │   │   ├── dia-8-precision.png
│       │   │   │   ├── dia-8-specificity.png
│       │   │   │   ├── dia-8-true_positive_rate.png
│       │   │   │   └── explanation.html
│       │   │   └── text_html.meta
│       │   ├── insights
│       │   │   └── insights_and_actions.json
│       │   ├── log
│       │   │   └── explainer_run_86f4e7c7-37d1-47a2-b1cc-0c16f03ec1d7.log
│       │   ├── problems
│       │   │   └── problems_and_actions.json
│       │   ├── result_descriptor.json
│       │   └── work
│       │       ├── dia_entity.json
│       │       ├── EDUCATION
│       │       │   ├── 0
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 1
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 2
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 3
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 4
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 5
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 6
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   └── metrics.jay
│       │       ├── MARRIAGE
│       │       │   ├── 0
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 1
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 2
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 3
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   └── metrics.jay
│       │       ├── PAY_0
│       │       │   ├── 0
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 1
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 10
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 2
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 3
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 4
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 5
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 6
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 7
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 8
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 9
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   └── metrics.jay
│       │       ├── PAY_2
│       │       │   ├── 0
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 1
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 10
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 2
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 3
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 4
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 5
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 6
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 7
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 8
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 9
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   └── metrics.jay
│       │       ├── PAY_3
│       │       │   ├── 0
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 1
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 10
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 2
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 3
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 4
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 5
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 6
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 7
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 8
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 9
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   └── metrics.jay
│       │       ├── PAY_4
│       │       │   ├── 0
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 1
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 10
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 2
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 3
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 4
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 5
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 6
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 7
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 8
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 9
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   └── metrics.jay
│       │       ├── PAY_5
│       │       │   ├── 0
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 1
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 2
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 3
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 4
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 5
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 6
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 7
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 8
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 9
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   └── metrics.jay
│       │       ├── PAY_6
│       │       │   ├── 0
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 1
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 2
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 3
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 4
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 5
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 6
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 7
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 8
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   ├── 9
│       │       │   │   ├── cm.jay
│       │       │   │   ├── disparity.jay
│       │       │   │   ├── me_smd.jay
│       │       │   │   └── parity.jay
│       │       │   └── metrics.jay
│       │       └── SEX
│       │           ├── 0
│       │           │   ├── cm.jay
│       │           │   ├── disparity.jay
│       │           │   ├── me_smd.jay
│       │           │   └── parity.jay
│       │           ├── 1
│       │           │   ├── cm.jay
│       │           │   ├── disparity.jay
│       │           │   ├── me_smd.jay
│       │           │   └── parity.jay
│       │           └── metrics.jay
│       ├── explainer_h2o_sonar_explainers_dt_surrogate_explainer_DecisionTreeSurrogateExplainer_f03ade5e-92e1-475b-93fc-b52a2b5b3385
│       │   ├── global_custom_archive
│       │   │   ├── application_zip
│       │   │   │   └── explanation.zip
│       │   │   └── application_zip.meta
│       │   ├── global_decision_tree
│       │   │   ├── application_json
│       │   │   │   ├── dt_class_0.json
│       │   │   │   └── explanation.json
│       │   │   └── application_json.meta
│       │   ├── global_html_fragment
│       │   │   ├── text_html
│       │   │   │   ├── dt-class-0.png
│       │   │   │   └── explanation.html
│       │   │   └── text_html.meta
│       │   ├── insights
│       │   │   └── insights_and_actions.json
│       │   ├── local_decision_tree
│       │   │   ├── application_json
│       │   │   │   └── explanation.json
│       │   │   └── application_json.meta
│       │   ├── log
│       │   │   └── explainer_run_f03ade5e-92e1-475b-93fc-b52a2b5b3385.log
│       │   ├── problems
│       │   │   └── problems_and_actions.json
│       │   ├── result_descriptor.json
│       │   └── work
│       │       ├── dt-class-0.dot
│       │       ├── dt-class-0.dot.pdf
│       │       ├── dtModel.json
│       │       ├── dtpaths_frame.bin
│       │       ├── dtPathsFrame.csv
│       │       ├── dtsurr_mojo.zip
│       │       ├── dtSurrogate.json
│       │       └── dt_surrogate_rules.zip
│       ├── explainer_h2o_sonar_explainers_pd_ice_explainer_PdIceExplainer_a60801bf-aae6-479d-9af8-11dc089aa988
│       │   ├── global_html_fragment
│       │   │   ├── text_html
│       │   │   │   ├── explanation.html
│       │   │   │   ├── pd-feature-0-class-0.png
│       │   │   │   ├── pd-feature-1-class-0.png
│       │   │   │   ├── pd-feature-2-class-0.png
│       │   │   │   ├── pd-feature-3-class-0.png
│       │   │   │   ├── pd-feature-4-class-0.png
│       │   │   │   ├── pd-feature-5-class-0.png
│       │   │   │   ├── pd-feature-6-class-0.png
│       │   │   │   ├── pd-feature-7-class-0.png
│       │   │   │   ├── pd-feature-8-class-0.png
│       │   │   │   └── pd-feature-9-class-0.png
│       │   │   └── text_html.meta
│       │   ├── global_partial_dependence
│       │   │   ├── application_json
│       │   │   │   ├── explanation.json
│       │   │   │   ├── pd_feature_0_class_0.json
│       │   │   │   ├── pd_feature_1_class_0.json
│       │   │   │   ├── pd_feature_2_class_0.json
│       │   │   │   ├── pd_feature_3_class_0.json
│       │   │   │   ├── pd_feature_4_class_0.json
│       │   │   │   ├── pd_feature_5_class_0.json
│       │   │   │   ├── pd_feature_6_class_0.json
│       │   │   │   ├── pd_feature_7_class_0.json
│       │   │   │   ├── pd_feature_8_class_0.json
│       │   │   │   └── pd_feature_9_class_0.json
│       │   │   └── application_json.meta
│       │   ├── insights
│       │   │   └── insights_and_actions.json
│       │   ├── local_individual_conditional_explanation
│       │   │   ├── application_vnd_h2oai_json_datatable_jay
│       │   │   │   ├── explanation.json
│       │   │   │   ├── ice_feature_0_class_0.jay
│       │   │   │   ├── ice_feature_1_class_0.jay
│       │   │   │   ├── ice_feature_2_class_0.jay
│       │   │   │   ├── ice_feature_3_class_0.jay
│       │   │   │   ├── ice_feature_4_class_0.jay
│       │   │   │   ├── ice_feature_5_class_0.jay
│       │   │   │   ├── ice_feature_6_class_0.jay
│       │   │   │   ├── ice_feature_7_class_0.jay
│       │   │   │   ├── ice_feature_8_class_0.jay
│       │   │   │   ├── ice_feature_9_class_0.jay
│       │   │   │   └── y_hat.jay
│       │   │   └── application_vnd_h2oai_json_datatable_jay.meta
│       │   ├── log
│       │   │   └── explainer_run_a60801bf-aae6-479d-9af8-11dc089aa988.log
│       │   ├── problems
│       │   │   └── problems_and_actions.json
│       │   ├── result_descriptor.json
│       │   └── work
│       │       ├── h2o_sonar-ice-dai-model-10.jay
│       │       ├── h2o_sonar-ice-dai-model-1.jay
│       │       ├── h2o_sonar-ice-dai-model-2.jay
│       │       ├── h2o_sonar-ice-dai-model-3.jay
│       │       ├── h2o_sonar-ice-dai-model-4.jay
│       │       ├── h2o_sonar-ice-dai-model-5.jay
│       │       ├── h2o_sonar-ice-dai-model-6.jay
│       │       ├── h2o_sonar-ice-dai-model-7.jay
│       │       ├── h2o_sonar-ice-dai-model-8.jay
│       │       ├── h2o_sonar-ice-dai-model-9.jay
│       │       ├── h2o_sonar-ice-dai-model.json
│       │       ├── h2o_sonar-pd-dai-model.json
│       │       └── mli_dataset_y_hat.jay
│       ├── explainer_h2o_sonar_explainers_residual_dt_surrogate_explainer_ResidualDecisionTreeSurrogateExplainer_2a0a6169-0847-4f2e-aebd-2c81be908b69
│       │   ├── global_custom_archive
│       │   │   ├── application_zip
│       │   │   │   └── explanation.zip
│       │   │   └── application_zip.meta
│       │   ├── global_decision_tree
│       │   │   ├── application_json
│       │   │   │   ├── dt_class_0.json
│       │   │   │   └── explanation.json
│       │   │   └── application_json.meta
│       │   ├── global_html_fragment
│       │   │   ├── text_html
│       │   │   │   ├── dt-class-0.png
│       │   │   │   └── explanation.html
│       │   │   └── text_html.meta
│       │   ├── insights
│       │   │   └── insights_and_actions.json
│       │   ├── local_decision_tree
│       │   │   ├── application_json
│       │   │   │   └── explanation.json
│       │   │   └── application_json.meta
│       │   ├── log
│       │   │   └── explainer_run_2a0a6169-0847-4f2e-aebd-2c81be908b69.log
│       │   ├── problems
│       │   │   └── problems_and_actions.json
│       │   ├── result_descriptor.json
│       │   └── work
│       │       ├── dt-class-0.dot
│       │       ├── dt-class-0.dot.pdf
│       │       ├── dtModel.json
│       │       ├── dtpaths_frame.bin
│       │       ├── dtPathsFrame.csv
│       │       ├── dtsurr_mojo.zip
│       │       ├── dtSurrogate.json
│       │       └── dt_surrogate_rules.zip
│       ├── explainer_h2o_sonar_explainers_summary_shap_explainer_SummaryShapleyExplainer_9db2bad1-f7b2-42a1-85de-6a5e1b8f8510
│       │   ├── global_html_fragment
│       │   │   ├── text_html
│       │   │   │   ├── explanation.html
│       │   │   │   ├── feature_0_class_0.png
│       │   │   │   ├── feature_10_class_0.png
│       │   │   │   ├── feature_11_class_0.png
│       │   │   │   ├── feature_12_class_0.png
│       │   │   │   ├── feature_13_class_0.png
│       │   │   │   ├── feature_14_class_0.png
│       │   │   │   ├── feature_15_class_0.png
│       │   │   │   ├── feature_16_class_0.png
│       │   │   │   ├── feature_17_class_0.png
│       │   │   │   ├── feature_18_class_0.png
│       │   │   │   ├── feature_19_class_0.png
│       │   │   │   ├── feature_1_class_0.png
│       │   │   │   ├── feature_20_class_0.png
│       │   │   │   ├── feature_21_class_0.png
│       │   │   │   ├── feature_22_class_0.png
│       │   │   │   ├── feature_23_class_0.png
│       │   │   │   ├── feature_2_class_0.png
│       │   │   │   ├── feature_3_class_0.png
│       │   │   │   ├── feature_4_class_0.png
│       │   │   │   ├── feature_5_class_0.png
│       │   │   │   ├── feature_6_class_0.png
│       │   │   │   ├── feature_7_class_0.png
│       │   │   │   ├── feature_8_class_0.png
│       │   │   │   ├── feature_9_class_0.png
│       │   │   │   └── shapley-class-0.png
│       │   │   └── text_html.meta
│       │   ├── global_summary_feature_importance
│       │   │   ├── application_json
│       │   │   │   ├── explanation.json
│       │   │   │   ├── feature_0_class_0.png
│       │   │   │   ├── feature_10_class_0.png
│       │   │   │   ├── feature_11_class_0.png
│       │   │   │   ├── feature_12_class_0.png
│       │   │   │   ├── feature_13_class_0.png
│       │   │   │   ├── feature_14_class_0.png
│       │   │   │   ├── feature_15_class_0.png
│       │   │   │   ├── feature_16_class_0.png
│       │   │   │   ├── feature_17_class_0.png
│       │   │   │   ├── feature_18_class_0.png
│       │   │   │   ├── feature_19_class_0.png
│       │   │   │   ├── feature_1_class_0.png
│       │   │   │   ├── feature_20_class_0.png
│       │   │   │   ├── feature_21_class_0.png
│       │   │   │   ├── feature_22_class_0.png
│       │   │   │   ├── feature_23_class_0.png
│       │   │   │   ├── feature_2_class_0.png
│       │   │   │   ├── feature_3_class_0.png
│       │   │   │   ├── feature_4_class_0.png
│       │   │   │   ├── feature_5_class_0.png
│       │   │   │   ├── feature_6_class_0.png
│       │   │   │   ├── feature_7_class_0.png
│       │   │   │   ├── feature_8_class_0.png
│       │   │   │   ├── feature_9_class_0.png
│       │   │   │   ├── summary_feature_importance_class_0_offset_0.json
│       │   │   │   ├── summary_feature_importance_class_0_offset_1.json
│       │   │   │   └── summary_feature_importance_class_0_offset_2.json
│       │   │   ├── application_json.meta
│       │   │   ├── application_vnd_h2oai_json_datatable_jay
│       │   │   │   ├── explanation.json
│       │   │   │   └── summary_feature_importance_class_0.jay
│       │   │   ├── application_vnd_h2oai_json_datatable_jay.meta
│       │   │   ├── text_markdown
│       │   │   │   ├── explanation.md
│       │   │   │   └── shapley-class-0.png
│       │   │   └── text_markdown.meta
│       │   ├── insights
│       │   │   └── insights_and_actions.json
│       │   ├── log
│       │   │   └── explainer_run_9db2bad1-f7b2-42a1-85de-6a5e1b8f8510.log
│       │   ├── problems
│       │   │   └── problems_and_actions.json
│       │   ├── result_descriptor.json
│       │   └── work
│       │       ├── raw_shapley_contribs_class_0.jay
│       │       ├── raw_shapley_contribs_index.json
│       │       ├── report.md
│       │       └── shapley-class-0.png
│       ├── interpretation.html
│       ├── interpretation.json
│       └── tmp
├── h2o-sonar.html
└── h2o-sonar.log

146 directories, 597 files

See interpretation.html ^ for interpretation results.

[ ]: