H2O Sonar CLI demo
This example demonstrates how to interpret a model using H2O Sonar from the command line:
by running all compatible explainers
by running selected explainers
[1]:
import os
import time
Get H2O Sonar command line interface help:
[2]:
!h2o-sonar --help
usage: h2o-sonar [-h] [--dataset DATASET] [--target-col TARGET_COL]
[--results-location RESULTS_LOCATION]
[--results-formats RESULTS_FORMATS] [--model MODEL]
[--validset VALIDSET] [--testset TESTSET]
[--use_raw-features USE_RAW_FEATURES]
[--weight-col WEIGHT_COL] [--drop-cols DROP_COLS]
[--sample-num-rows SAMPLE_NUM_ROWS]
[--used-features USED_FEATURES] [--model-type {pickle,mojo}]
[--explainer EXPLAINER] [--explainers EXPLAINERS]
[--all-explainers] [--explainers-pars EXPLAINERS_PARS]
[--config-path CONFIG_PATH] [--config-type CONFIG_TYPE]
[--config-value CONFIG_VALUE]
[--encryption-key ENCRYPTION_KEY] [-d]
[--interpretation INTERPRETATION]
[--collection-name COLLECTION_NAME] [--upload-to UPLOAD_TO]
[--args-as-json-location ARGS_AS_JSON_LOCATION]
[--log-level {error,warning,info,debug}]
action entity
H2O Sonar Python library for Responsible AI.
H2O Sonar is Python package that enables a holistic, low-risk, human-interpretable,
fair, and trustable approach to machine learning by implementing various facets of
Responsible AI.
model, dataset, validset or testset handle schema:
resource:connection:"<connection_configuration_name>":key:<resource_key>
[:version:<resource_version>]
optional arguments per action and entity:
show version:
show H2O Sonar version
add config:
--config-path path to JSon or TOML file with H2O Sonar config to be changed
--config-type config item type: 'CONNECTION' or 'LICENSE'
--config-value config item value (serialized as JSon) to add to the config file
--encryption-key secret key to encrypt config fields with sensitive data
(alternatively set H2O_SONAR_ENCRYPTION_KEY environment variable)
show config:
--config-path path to JSon or TOML file with H2O Sonar config
--encryption-key optional secret key to decrypt config fields with sensitive data
(alternatively set H2O_SONAR_ENCRYPTION_KEY environment variable)
list explainers:
--detailed show detailed descriptors (only IDs are shown by default)
--args-as-json-location
optional JSon file which overrides filtering CLI arguments
describe explainer:
--explainer explainer ID
run interpretation:
--dataset path to dataset
--target-col target column
--model path to the serialized model, URL or locator
--results-location
optional path to the interpretation results location (directory)
--validset optional path to validation dataset
--testset optional path to test dataset
--use_raw_features
force the use of transformed features in surrogate models
with 'false', by default the original (raw) features are used
--weight-col optional dataset column name with examples weights
--drop-cols optional list of dataset columns to drop
--sample-num-rows
optional number of rows to sample from dataset (default: sample
based on the RAM size, 0 do not sample, >0 sample to the specified
number of rows)
--all-explainers run all explainers (only the most important are run by default)
--used-features optional comma separated list of features used by the model
--model-type optional model type: 'pickle' or 'mojo'
--explainers optional comma separated list of explainer IDs to be run
--explainers-pars optional dictionary with explainer parameters
--config-path path to JSon or TOML file with H2O Sonar config to be changed
--encryption-key secret key to encrypt config fields with sensitive data
(alternatively set H2O_SONAR_ENCRYPTION_KEY environment variable)
--upload-to optional (h2oGPT Enterprise) connection key from the configuration
where the report is to be uploaded in order to talk to it
--args-as-json-location
optional JSon file which overrides CLI arguments
--log-level optional log level: 'error', 'warning', 'info', 'debug'
list interpretations:
--results-location
path to directory, URL, location of interpretation results
--log-level optional log level: 'error', 'warning', 'info', 'debug'
upload interpretations:
--interpretation path to the interpretation report PDF or HTML file
--upload-to (h2oGPT Enterprise) connection key from the configuration
where the report is to be uploaded in order to talk to it
--collection-name optional name of the collection where the report is to be uploaded
--config-path path to JSon or TOML file with H2O Sonar config to be changed
--encryption-key secret key to encrypt config fields with sensitive data
(alternatively set H2O_SONAR_ENCRYPTION_KEY environment variable)
positional arguments:
action action to take: 'list', 'run' or 'describe'
entity entity on which to perform the action:
'interpretation'(s) or 'explainer'(s)
options:
-h, --help show this help message and exit
--dataset DATASET location of the dataset
--target-col TARGET_COL
target column
--results-location RESULTS_LOCATION
location where to store the interpretation results
--results-formats RESULTS_FORMATS
comma separated list of MIME types of the
interpretation results to create
--model MODEL location of the model
--validset VALIDSET location of the validation dataset
--testset TESTSET location of the test dataset
--use_raw-features USE_RAW_FEATURES
force the use of transformed features in surrogate
models with `false`
--weight-col WEIGHT_COL
optional dataset column name with examples weights
--drop-cols DROP_COLS
optional list of dataset columns to drop
--sample-num-rows SAMPLE_NUM_ROWS
optional number of rows to sample from the dataset
--used-features USED_FEATURES
optional comma separated list of features used by the
model
--model-type {pickle,mojo}
model type: 'pickle' (.pkl) or 'mojo' (.mojo)
--explainer EXPLAINER
ID of the explainer to describe
--explainers EXPLAINERS
comma separated list of explainer IDs to be run (only
the most important explainers are run by default)
--all-explainers run all explainers (only the most important explainers
are run by default)
--explainers-pars EXPLAINERS_PARS
optional dictionary with explainer parameters - the
dictionary key is explainer ID and value is dictionary
with parameters; parameter dictionary has parameter
name as the key and parameter value as the value
--config-path CONFIG_PATH
path to JSon or TOML file with H2O Sonar configuration
to be used to override defaults - specify only items
you want to change (please refer to
h2o_sonar.config.H2oSonarConfig for more details)
--config-type CONFIG_TYPE
configuration item type - 'CONNECTION' or 'LICENSE'
--config-value CONFIG_VALUE
configuration item value represented either as
dictionary or as string with JSon serialization of the
configuration item - it is expected that the config
item is NOT encrypted
--encryption-key ENCRYPTION_KEY
encryption key to be used for encrypting/decrypting
sensitive data in the configuration. If not specified,
shell environment variable H2O_SONAR_ENCRYPTION_KEY
with the encryption key is used.
-d, --detailed show detailed descriptors (only IDs are shown by
default)
--interpretation INTERPRETATION
path to the interpretation report (PDF or HTML) to be
uploaded
--collection-name COLLECTION_NAME
optional (h2oGPT Enterprise) collection name where to
upload the interpretation report
--upload-to UPLOAD_TO
optional (h2oGPT Enterprise) connection key from the
configuration where the report is to be uploaded in
order to talk to it
--args-as-json-location ARGS_AS_JSON_LOCATION
location of the JSon file with all command arguments
(replacing command line arguments) allowing to load
them from the filesystem
--log-level {error,warning,info,debug}
log level
examples:
h2o-sonar --help
h2o-sonar show version
h2o-sonar list explainers
h2o-sonar list explainers --detailed
h2o-sonar describe explainer
--explainer=h2o_sonar.explainers.dia_explainer.DiaExplainer
h2o-sonar run interpretation
--dataset=dataset.csv
--target-col=PROFIT
--results-location=/home/user/results
--model=model.pickle
--all-explainers
h2o-sonar run interpretation
--dataset=dataset.csv
--target-col=PROFIT
--results-location=/home/user/results
--model=model.pickle
--used-features=FEATURE_1,FEATURE_2,FEATURE_3
--explainers=h2o_sonar.explainers.dia_explainer.DiaExplainer
--explainers-pars=
"{'h2o_sonar.explainers.dia_explainer.DiaExplainer':{'cut_off': 0.5}}"
--drop_cols=COLUMN_1,COLUMN_2,COLUMN_3
--config-path=h2o-sonar-config.json
--upload-to=4cff6fc9-f49a-4dda-aeb5-2c42e9f12807
h2o-sonar run interpretation
--args-as-json-location=h2o-sonar-args.json
h2o-sonar list interpretations --results-location=/home/user/results
h2o-sonar upload interpretation
--interpretation=./results/h2o-sonar/mli-experiment/interpretation-detailed.html
--upload-to=4cff6fc9-f49a-4dda-aeb5-2c42e9f12807
H2O Sonar JSon configuration example:
{
"h2o_host": "192.168.0.1",
"h2o_port": 57561,
"h2o_auto_start": true,
"connections": [
{
"key": "4cff6fc9-f49a-4dda-aeb5-2c42e9f12807",
"connection_type": "H2O_GPT_E",
"name": "H2O GPT Enterprise",
"description": "H2O GPT Enterprise service.",
"server_url": "https://h2ogpte.h2o.ai",
"token": {
"encrypted": "gAAA3LcKQ7x_X...gnKsBqVdNydYTlk8nyQ=="
},
"token_use_type": "API_KEY"
}
]
}
Interpretation arguments JSon file example - see interpret.py::run_interpretation():
{
"dataset": "dataset.csv",
"model": "model.pickle",
"target_col": "PROFIT",
"results_location": "./results"
}
Explainer listing arguments JSon file example - see interpret.py::list_explainers():
{
"experiment_types": ["regression"],
"explanation_scopes": ["local_scope"],
"keywords": ["explains-fairness"],
"explainer_filter": [{"filter_by": "filter-name", "value": "v"}]
}
Driverless AI MOJO model to be interpreted by the library:
[3]:
dataset_path = "../../data/predictive/creditcard.csv"
target_column = "\"default payment next month\""
# specify path to Driverless AI MOJO
model_path = "../../data/predictive/models/creditcard-binomial.mojo"
results_path = f"/tmp/{time.time()}"
os.mkdir(results_path)
Run new interpretation with all compatible explainers using command line interface:
[4]:
!h2o-sonar run interpretation \
--dataset={dataset_path} \
--model={model_path} \
--target-col={target_column} \
--results-location={results_path}
/home/user/h/mli/git/h2o-sonar-FLOSS/.venv/lib/python3.11/site-packages/ragas/metrics/__init__.py:1: LangChainDeprecationWarning: As of langchain-core 0.3.0, LangChain uses pydantic v2 internally. The langchain_core.pydantic_v1 module was a compatibility shim for pydantic v1, and should no longer be used. Please update the code to import from Pydantic directly.
For example, replace imports like: `from langchain_core.pydantic_v1 import BaseModel`
with: `from pydantic import BaseModel`
or the v1 compatibility namespace if you are working in a code base that has not been fully upgraded to pydantic 2 yet. from pydantic.v1 import BaseModel
from ragas.metrics._answer_correctness import AnswerCorrectness, answer_correctness
/home/user/h/mli/git/h2o-sonar-FLOSS/.venv/lib/python3.11/site-packages/ragas/metrics/__init__.py:4: LangChainDeprecationWarning: As of langchain-core 0.3.0, LangChain uses pydantic v2 internally. The langchain.pydantic_v1 module was a compatibility shim for pydantic v1, and should no longer be used. Please update the code to import from Pydantic directly.
For example, replace imports like: `from langchain.pydantic_v1 import BaseModel`
with: `from pydantic import BaseModel`
or the v1 compatibility namespace if you are working in a code base that has not been fully upgraded to pydantic 2 yet. from pydantic.v1 import BaseModel
from ragas.metrics._context_entities_recall import (
Stratified/random sampler: loading the original dataset '../../data/predictive/creditcard.csv' for sampling...
Stratified/random sampler: -> did NO sampling as the sampling limit is smaller than the number of rows in the dataset: 10000 <= 25000
Preparing and checking DIA features (None): dataset= | PAY_AMT1 PAY_5 ID EDUCATION BILL_AMT1 PAY_3 BILL_AMT2 default payment next month LIMIT_BAL MARRIAGE … BILL_AMT5 PAY_2 PAY_AMT6 PAY_6 AGE
| int32 int32 int32 int32 int32 int32 int32 int8 int32 int32 int32 int32 int32 int32 int32
---- + -------- ----- ----- --------- --------- ----- --------- -------------------------- --------- -------- --------- ----- -------- ----- -----
0 | 0 -2 1 2 3913 -1 3102 1 20000 1 … 0 2 0 -2 24
1 | 0 0 2 2 2682 0 1725 1 120000 2 … 3455 2 2000 2 26
2 | 1518 0 3 2 29239 0 14027 0 90000 2 … 14948 0 5000 0 34
3 | 2000 0 4 2 46990 0 48233 0 50000 1 … 28959 0 1000 0 37
4 | 2000 0 5 2 8617 -1 5670 0 50000 1 … 19146 0 679 0 57
5 | 2500 0 6 1 64400 0 57069 0 50000 2 … 19619 0 800 0 37
6 | 55000 0 7 1 367965 0 412023 0 500000 2 … 483003 0 13770 0 29
7 | 380 0 8 2 11876 -1 380 0 100000 2 … -159 -1 1542 -1 23
8 | 3329 0 9 3 11285 2 14096 0 140000 1 … 11793 0 1000 0 28
9 | 0 -1 10 3 0 -2 0 0 20000 2 … 13007 -2 0 -1 35
10 | 2306 0 11 3 11073 2 9787 0 200000 2 … 1828 0 66 -1 34
11 | 21818 -1 12 1 12261 -1 21670 0 260000 2 … 22287 -1 3640 2 51
12 | 1000 -1 13 2 12137 -1 6500 0 630000 2 … 6500 0 0 -1 41
13 | 3200 0 14 2 65802 2 67369 1 70000 2 … 36137 2 0 2 30
14 | 3000 0 15 1 70887 0 67060 0 250000 2 … 56875 0 3000 0 29
… | … … … … … … … … … … … … … … … …
9995 | 241 -2 9996 1 0 -2 241 0 140000 2 … 0 -2 1419 -2 31
9996 | 0 -2 9997 2 3946 -2 0 0 80000 2 … 0 -2 0 -2 37
9997 | 6437 0 9998 3 138877 0 144085 0 200000 1 … 176717 0 4200 0 44
9998 | 0 -2 9999 2 780 2 780 1 80000 2 … 0 2 0 -2 26
9999 | 3000 0 10000 2 19505 0 20715 0 230000 1 … 19255 0 3000 0 36
[10000 rows x 25 columns]
dataset_meta={
"shape": "(10000, 25)",
"row_count": 10000,
"column_names": [
"ID",
"LIMIT_BAL",
"SEX",
"EDUCATION",
"MARRIAGE",
"AGE",
"PAY_0",
"PAY_2",
"PAY_3",
"PAY_4",
"PAY_5",
"PAY_6",
"BILL_AMT1",
"BILL_AMT2",
"BILL_AMT3",
"BILL_AMT4",
"BILL_AMT5",
"BILL_AMT6",
"PAY_AMT1",
"PAY_AMT2",
"PAY_AMT3",
"PAY_AMT4",
"PAY_AMT5",
"PAY_AMT6",
"default payment next month"
],
"column_types": [
"int",
"int",
"int",
"int",
"int",
"int",
"int",
"int",
"int",
"int",
"int",
"int",
"int",
"int",
"int",
"int",
"int",
"int",
"int",
"int",
"int",
"int",
"int",
"int",
"int"
],
"column_uniques": [
10000,
72,
2,
7,
4,
54,
11,
11,
11,
11,
10,
10,
8371,
8215,
8072,
7913,
7764,
7550,
3763,
3581,
3305,
3247,
3258,
3174,
2
],
"columns_cat": [],
"columns_num": [],
"file_path": "../../data/predictive/creditcard.csv",
"file_name": "",
"file_size": 944719,
"missing_values": [
"",
"?",
"None",
"nan",
"NA",
"N/A",
"unknown",
"inf",
"-inf",
"1.7976931348623157e+308",
"-1.7976931348623157e+308"
],
"columns_meta": [
{
"name": "ID",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": true,
"is_numeric": true,
"is_categorical": false,
"count": 10000,
"frequency": 0,
"unique": 10000,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "LIMIT_BAL",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 72,
"frequency": 0,
"unique": 72,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "SEX",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 2,
"frequency": 0,
"unique": 2,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "EDUCATION",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 7,
"frequency": 0,
"unique": 7,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "MARRIAGE",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 4,
"frequency": 0,
"unique": 4,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "AGE",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 54,
"frequency": 0,
"unique": 54,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "PAY_0",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 11,
"frequency": 0,
"unique": 11,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "PAY_2",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 11,
"frequency": 0,
"unique": 11,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "PAY_3",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 11,
"frequency": 0,
"unique": 11,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "PAY_4",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 11,
"frequency": 0,
"unique": 11,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "PAY_5",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 10,
"frequency": 0,
"unique": 10,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "PAY_6",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 10,
"frequency": 0,
"unique": 10,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "BILL_AMT1",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 8371,
"frequency": 0,
"unique": 8371,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "BILL_AMT2",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 8215,
"frequency": 0,
"unique": 8215,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "BILL_AMT3",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 8072,
"frequency": 0,
"unique": 8072,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "BILL_AMT4",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 7913,
"frequency": 0,
"unique": 7913,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "BILL_AMT5",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 7764,
"frequency": 0,
"unique": 7764,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "BILL_AMT6",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 7550,
"frequency": 0,
"unique": 7550,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "PAY_AMT1",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 3763,
"frequency": 0,
"unique": 3763,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "PAY_AMT2",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 3581,
"frequency": 0,
"unique": 3581,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "PAY_AMT3",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 3305,
"frequency": 0,
"unique": 3305,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "PAY_AMT4",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 3247,
"frequency": 0,
"unique": 3247,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "PAY_AMT5",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 3258,
"frequency": 0,
"unique": 3258,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "PAY_AMT6",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 3174,
"frequency": 0,
"unique": 3174,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
},
{
"name": "default payment next month",
"data_type": "int",
"logical_types": [],
"format": "",
"is_id": false,
"is_numeric": true,
"is_categorical": false,
"count": 2,
"frequency": 0,
"unique": 2,
"max": null,
"min": null,
"mean": null,
"std": null,
"histogram_counts": [],
"histogram_ticks": []
}
],
"original_dataset_sampled": false,
"original_dataset_path": "",
"original_dataset_size": 0,
"original_dataset_shape": [
10000,
25
]
}
Using dataset ENTITY to prepare DIA features: column_names=['ID', 'LIMIT_BAL', 'SEX', 'EDUCATION', 'MARRIAGE', 'AGE', 'PAY_0', 'PAY_2', 'PAY_3', 'PAY_4', 'PAY_5', 'PAY_6', 'BILL_AMT1', 'BILL_AMT2', 'BILL_AMT3', 'BILL_AMT4', 'BILL_AMT5', 'BILL_AMT6', 'PAY_AMT1', 'PAY_AMT2', 'PAY_AMT3', 'PAY_AMT4', 'PAY_AMT5', 'PAY_AMT6', 'default payment next month'] column_uniques=[10000, 72, 2, 7, 4, 54, 11, 11, 11, 11, 10, 10, 8371, 8215, 8072, 7913, 7764, 7550, 3763, 3581, 3305, 3247, 3258, 3174, 2]
DIA group columns prepared using dataset ENTITY: {'PAY_5', 'EDUCATION', 'MARRIAGE', 'PAY_6', 'PAY_0', 'SEX', 'PAY_3', 'PAY_4', 'default payment next month', 'PAY_2'}
DIA group columns to SKIP: {'model_pred', 'default payment next month'}
DIA group columns as BOOLs: [<h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7f111aa117d0>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7f111aa06750>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7f111aa066d0>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7f111aa05510>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7f111aa05f90>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7f111aa06810>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7f111aa06790>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7f111aa06910>, <h2o_sonar.methods.fairness._dia.BoolEntry object at 0x7f111aa06990>]
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
Checking whether there is an H2O instance running at http://localhost:54324..... not found.
Attempting to start a local H2O server...
Java Version: openjdk version "10" 2018-03-20; OpenJDK Runtime Environment 18.3 (build 10+44); OpenJDK 64-Bit Server VM 18.3 (build 10+44, mixed mode)
Starting server from /home/user/h/mli/git/h2o-sonar-FLOSS/.venv/lib/python3.11/site-packages/h2o/backend/bin/h2o.jar
Ice root: /tmp/tmpiwy5gqbf
JVM stdout: /tmp/tmpiwy5gqbf/h2o_user_started_from_python.out
JVM stderr: /tmp/tmpiwy5gqbf/h2o_user_started_from_python.err
Server is running at http://127.0.0.1:54324
successful.o H2O server at http://127.0.0.1:54324 ...
-------------------------- -----------------------------
H2O_cluster_uptime: 01 secs
H2O_cluster_timezone: Europe/Prague
H2O_data_parsing_timezone: UTC
H2O_cluster_version: 3.46.0.9
H2O_cluster_version_age: 2 months and 4 days
H2O_cluster_name: H2O_from_python_user_2xryhy
H2O_cluster_total_nodes: 1
H2O_cluster_free_memory: 4 Gb
H2O_cluster_total_cores: 16
H2O_cluster_allowed_cores: 16
H2O_cluster_status: locked, healthy
H2O_connection_url: http://127.0.0.1:54324
H2O_connection_proxy: {"http": null, "https": null}
H2O_internal_security: False
Python_version: 3.11.11 final
-------------------------- -----------------------------
Connecting to H2O server at http://localhost:54324 ... successful.
-------------------------- -----------------------------
H2O_cluster_uptime: 01 secs
H2O_cluster_timezone: Europe/Prague
H2O_data_parsing_timezone: UTC
H2O_cluster_version: 3.46.0.9
H2O_cluster_version_age: 2 months and 4 days
H2O_cluster_name: H2O_from_python_user_2xryhy
H2O_cluster_total_nodes: 1
H2O_cluster_free_memory: 4 Gb
H2O_cluster_total_cores: 16
H2O_cluster_allowed_cores: 16
H2O_cluster_status: locked, healthy
H2O_connection_url: http://localhost:54324
H2O_connection_proxy: {"http": null, "https": null}
H2O_internal_security: False
Python_version: 3.11.11 final
-------------------------- -----------------------------
Connecting to H2O server at http://localhost:54324 ... successful.
-------------------------- -----------------------------
H2O_cluster_uptime: 01 secs
H2O_cluster_timezone: Europe/Prague
H2O_data_parsing_timezone: UTC
H2O_cluster_version: 3.46.0.9
H2O_cluster_version_age: 2 months and 4 days
H2O_cluster_name: H2O_from_python_user_2xryhy
H2O_cluster_total_nodes: 1
H2O_cluster_free_memory: 4 Gb
H2O_cluster_total_cores: 16
H2O_cluster_allowed_cores: 16
H2O_cluster_status: locked, healthy
H2O_connection_url: http://localhost:54324
H2O_connection_proxy: {"http": null, "https": null}
H2O_internal_security: False
Python_version: 3.11.11 final
-------------------------- -----------------------------
Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%
Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%
drf Model Build progress: |██████████████████████████████████████████████████████ (done)| 100%
Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%
Converting H2O frame to pandas dataframe using single-thread. For faster conversion using multi-thread, install polars and pyarrow and use it as pandas_df = h2o_df.as_data_frame(use_multi_thread=True)
Connecting to H2O server at http://localhost:54324 ... successful.
-------------------------- -----------------------------
H2O_cluster_uptime: 04 secs
H2O_cluster_timezone: Europe/Prague
H2O_data_parsing_timezone: UTC
H2O_cluster_version: 3.46.0.9
H2O_cluster_version_age: 2 months and 4 days
H2O_cluster_name: H2O_from_python_user_2xryhy
H2O_cluster_total_nodes: 1
H2O_cluster_free_memory: 4 Gb
H2O_cluster_total_cores: 16
H2O_cluster_allowed_cores: 16
H2O_cluster_status: locked, healthy
H2O_connection_url: http://localhost:54324
H2O_connection_proxy: {"http": null, "https": null}
H2O_internal_security: False
Python_version: 3.11.11 final
-------------------------- -----------------------------
Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%
Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%
drf Model Build progress: |██████████████████████████████████████████████████████ (done)| 100%
Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%
Converting H2O frame to pandas dataframe using single-thread. For faster conversion using multi-thread, install polars and pyarrow and use it as pandas_df = h2o_df.as_data_frame(use_multi_thread=True)
More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`). Consider using `matplotlib.pyplot.close()`.
Interpretation FINISHED.
Results directory:
file:///tmp/1769698782.3007452
Interpretations index:
file:///tmp/1769698782.3007452/h2o-sonar.html
HTML report:
file:///tmp/1769698782.3007452/h2o-sonar/mli_experiment_acc39269-6aea-45c7-89cf-37cf4316abb9/interpretation.html
Closing connection _sid_b519 at exit
H2O session _sid_b519 closed.
Closing connection _sid_b075 at exit
H2O session _sid_b075 closed.
[5]:
!tree $results_path
/tmp/1769698782.3007452
├── h2o-sonar
│ └── mli_experiment_acc39269-6aea-45c7-89cf-37cf4316abb9
│ ├── explainer_h2o_sonar_explainers_dia_explainer_DiaExplainer_eda4ed7a-6ef3-4edb-a229-a698e3ecb0ca
│ │ ├── global_disparate_impact_analysis
│ │ │ ├── text_plain
│ │ │ │ └── explanation.txt
│ │ │ └── text_plain.meta
│ │ ├── global_html_fragment
│ │ │ ├── text_html
│ │ │ │ ├── dia-0-accuracy.png
│ │ │ │ ├── dia-0-adverse_impact.png
│ │ │ │ ├── dia-0-false_discovery_rate.png
│ │ │ │ ├── dia-0-false_negative_rate.png
│ │ │ │ ├── dia-0-false_omissions_rate.png
│ │ │ │ ├── dia-0-false_positive_rate.png
│ │ │ │ ├── dia-0-negative_predicted_value.png
│ │ │ │ ├── dia-0-n.png
│ │ │ │ ├── dia-0-precision.png
│ │ │ │ ├── dia-0-specificity.png
│ │ │ │ ├── dia-0-true_positive_rate.png
│ │ │ │ ├── dia-1-accuracy.png
│ │ │ │ ├── dia-1-adverse_impact.png
│ │ │ │ ├── dia-1-false_discovery_rate.png
│ │ │ │ ├── dia-1-false_negative_rate.png
│ │ │ │ ├── dia-1-false_omissions_rate.png
│ │ │ │ ├── dia-1-false_positive_rate.png
│ │ │ │ ├── dia-1-negative_predicted_value.png
│ │ │ │ ├── dia-1-n.png
│ │ │ │ ├── dia-1-precision.png
│ │ │ │ ├── dia-1-specificity.png
│ │ │ │ ├── dia-1-true_positive_rate.png
│ │ │ │ ├── dia-2-accuracy.png
│ │ │ │ ├── dia-2-adverse_impact.png
│ │ │ │ ├── dia-2-false_discovery_rate.png
│ │ │ │ ├── dia-2-false_negative_rate.png
│ │ │ │ ├── dia-2-false_omissions_rate.png
│ │ │ │ ├── dia-2-false_positive_rate.png
│ │ │ │ ├── dia-2-negative_predicted_value.png
│ │ │ │ ├── dia-2-n.png
│ │ │ │ ├── dia-2-precision.png
│ │ │ │ ├── dia-2-specificity.png
│ │ │ │ ├── dia-2-true_positive_rate.png
│ │ │ │ ├── dia-3-accuracy.png
│ │ │ │ ├── dia-3-adverse_impact.png
│ │ │ │ ├── dia-3-false_discovery_rate.png
│ │ │ │ ├── dia-3-false_negative_rate.png
│ │ │ │ ├── dia-3-false_omissions_rate.png
│ │ │ │ ├── dia-3-false_positive_rate.png
│ │ │ │ ├── dia-3-negative_predicted_value.png
│ │ │ │ ├── dia-3-n.png
│ │ │ │ ├── dia-3-precision.png
│ │ │ │ ├── dia-3-specificity.png
│ │ │ │ ├── dia-3-true_positive_rate.png
│ │ │ │ ├── dia-4-accuracy.png
│ │ │ │ ├── dia-4-adverse_impact.png
│ │ │ │ ├── dia-4-false_discovery_rate.png
│ │ │ │ ├── dia-4-false_negative_rate.png
│ │ │ │ ├── dia-4-false_omissions_rate.png
│ │ │ │ ├── dia-4-false_positive_rate.png
│ │ │ │ ├── dia-4-negative_predicted_value.png
│ │ │ │ ├── dia-4-n.png
│ │ │ │ ├── dia-4-precision.png
│ │ │ │ ├── dia-4-specificity.png
│ │ │ │ ├── dia-4-true_positive_rate.png
│ │ │ │ ├── dia-5-accuracy.png
│ │ │ │ ├── dia-5-adverse_impact.png
│ │ │ │ ├── dia-5-false_discovery_rate.png
│ │ │ │ ├── dia-5-false_negative_rate.png
│ │ │ │ ├── dia-5-false_omissions_rate.png
│ │ │ │ ├── dia-5-false_positive_rate.png
│ │ │ │ ├── dia-5-negative_predicted_value.png
│ │ │ │ ├── dia-5-n.png
│ │ │ │ ├── dia-5-precision.png
│ │ │ │ ├── dia-5-specificity.png
│ │ │ │ ├── dia-5-true_positive_rate.png
│ │ │ │ ├── dia-6-accuracy.png
│ │ │ │ ├── dia-6-adverse_impact.png
│ │ │ │ ├── dia-6-false_discovery_rate.png
│ │ │ │ ├── dia-6-false_negative_rate.png
│ │ │ │ ├── dia-6-false_omissions_rate.png
│ │ │ │ ├── dia-6-false_positive_rate.png
│ │ │ │ ├── dia-6-negative_predicted_value.png
│ │ │ │ ├── dia-6-n.png
│ │ │ │ ├── dia-6-precision.png
│ │ │ │ ├── dia-6-specificity.png
│ │ │ │ ├── dia-6-true_positive_rate.png
│ │ │ │ ├── dia-7-accuracy.png
│ │ │ │ ├── dia-7-adverse_impact.png
│ │ │ │ ├── dia-7-false_discovery_rate.png
│ │ │ │ ├── dia-7-false_negative_rate.png
│ │ │ │ ├── dia-7-false_omissions_rate.png
│ │ │ │ ├── dia-7-false_positive_rate.png
│ │ │ │ ├── dia-7-negative_predicted_value.png
│ │ │ │ ├── dia-7-n.png
│ │ │ │ ├── dia-7-precision.png
│ │ │ │ ├── dia-7-specificity.png
│ │ │ │ ├── dia-7-true_positive_rate.png
│ │ │ │ ├── dia-8-accuracy.png
│ │ │ │ ├── dia-8-adverse_impact.png
│ │ │ │ ├── dia-8-false_discovery_rate.png
│ │ │ │ ├── dia-8-false_negative_rate.png
│ │ │ │ ├── dia-8-false_omissions_rate.png
│ │ │ │ ├── dia-8-false_positive_rate.png
│ │ │ │ ├── dia-8-negative_predicted_value.png
│ │ │ │ ├── dia-8-n.png
│ │ │ │ ├── dia-8-precision.png
│ │ │ │ ├── dia-8-specificity.png
│ │ │ │ ├── dia-8-true_positive_rate.png
│ │ │ │ └── explanation.html
│ │ │ └── text_html.meta
│ │ ├── insights
│ │ │ └── insights_and_actions.json
│ │ ├── log
│ │ │ └── explainer_run_eda4ed7a-6ef3-4edb-a229-a698e3ecb0ca.log
│ │ ├── problems
│ │ │ └── problems_and_actions.json
│ │ ├── result_descriptor.json
│ │ └── work
│ │ ├── dia_entity.json
│ │ ├── EDUCATION
│ │ │ ├── 0
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 1
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 2
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 3
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 4
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 5
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 6
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ └── metrics.jay
│ │ ├── MARRIAGE
│ │ │ ├── 0
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 1
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 2
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 3
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ └── metrics.jay
│ │ ├── PAY_0
│ │ │ ├── 0
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 1
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 10
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 2
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 3
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 4
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 5
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 6
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 7
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 8
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 9
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ └── metrics.jay
│ │ ├── PAY_2
│ │ │ ├── 0
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 1
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 10
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 2
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 3
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 4
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 5
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 6
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 7
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 8
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 9
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ └── metrics.jay
│ │ ├── PAY_3
│ │ │ ├── 0
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 1
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 10
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 2
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 3
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 4
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 5
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 6
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 7
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 8
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 9
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ └── metrics.jay
│ │ ├── PAY_4
│ │ │ ├── 0
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 1
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 10
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 2
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 3
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 4
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 5
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 6
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 7
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 8
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 9
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ └── metrics.jay
│ │ ├── PAY_5
│ │ │ ├── 0
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 1
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 2
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 3
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 4
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 5
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 6
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 7
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 8
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 9
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ └── metrics.jay
│ │ ├── PAY_6
│ │ │ ├── 0
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 1
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 2
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 3
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 4
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 5
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 6
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 7
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 8
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ ├── 9
│ │ │ │ ├── cm.jay
│ │ │ │ ├── disparity.jay
│ │ │ │ ├── me_smd.jay
│ │ │ │ └── parity.jay
│ │ │ └── metrics.jay
│ │ └── SEX
│ │ ├── 0
│ │ │ ├── cm.jay
│ │ │ ├── disparity.jay
│ │ │ ├── me_smd.jay
│ │ │ └── parity.jay
│ │ ├── 1
│ │ │ ├── cm.jay
│ │ │ ├── disparity.jay
│ │ │ ├── me_smd.jay
│ │ │ └── parity.jay
│ │ └── metrics.jay
│ ├── explainer_h2o_sonar_explainers_dt_surrogate_explainer_DecisionTreeSurrogateExplainer_1cdc0473-0680-4a19-9057-c95e5d7ab193
│ │ ├── global_custom_archive
│ │ │ ├── application_zip
│ │ │ │ └── explanation.zip
│ │ │ └── application_zip.meta
│ │ ├── global_decision_tree
│ │ │ ├── application_json
│ │ │ │ ├── dt_class_0.json
│ │ │ │ └── explanation.json
│ │ │ └── application_json.meta
│ │ ├── global_html_fragment
│ │ │ ├── text_html
│ │ │ │ ├── dt-class-0.png
│ │ │ │ └── explanation.html
│ │ │ └── text_html.meta
│ │ ├── insights
│ │ │ └── insights_and_actions.json
│ │ ├── local_decision_tree
│ │ │ ├── application_json
│ │ │ │ └── explanation.json
│ │ │ └── application_json.meta
│ │ ├── log
│ │ │ └── explainer_run_1cdc0473-0680-4a19-9057-c95e5d7ab193.log
│ │ ├── problems
│ │ │ └── problems_and_actions.json
│ │ ├── result_descriptor.json
│ │ └── work
│ │ ├── dt-class-0.dot
│ │ ├── dt-class-0.dot.pdf
│ │ ├── dtModel.json
│ │ ├── dtpaths_frame.bin
│ │ ├── dtPathsFrame.csv
│ │ ├── dtsurr_mojo.zip
│ │ ├── dtSurrogate.json
│ │ └── dt_surrogate_rules.zip
│ ├── explainer_h2o_sonar_explainers_fi_naive_shapley_explainer_NaiveShapleyMojoFeatureImportanceExplainer_4da226a0-4a0d-42bf-87cd-ae0d4f5f7be4
│ │ ├── global_feature_importance
│ │ │ ├── application_json
│ │ │ │ ├── explanation.json
│ │ │ │ └── feature_importance_class_0.json
│ │ │ ├── application_json.meta
│ │ │ ├── application_vnd_h2oai_json_csv
│ │ │ │ ├── explanation.json
│ │ │ │ └── feature_importance_class_0.csv
│ │ │ ├── application_vnd_h2oai_json_csv.meta
│ │ │ ├── application_vnd_h2oai_json_datatable_jay
│ │ │ │ ├── explanation.json
│ │ │ │ └── feature_importance_class_0.jay
│ │ │ └── application_vnd_h2oai_json_datatable_jay.meta
│ │ ├── global_html_fragment
│ │ │ ├── text_html
│ │ │ │ ├── explanation.html
│ │ │ │ └── fi-class-0.png
│ │ │ └── text_html.meta
│ │ ├── insights
│ │ │ └── insights_and_actions.json
│ │ ├── local_feature_importance
│ │ │ ├── application_vnd_h2oai_json_datatable_jay
│ │ │ │ ├── explanation.json
│ │ │ │ ├── feature_importance_class_0.jay
│ │ │ │ └── y_hat.bin
│ │ │ └── application_vnd_h2oai_json_datatable_jay.meta
│ │ ├── log
│ │ │ └── explainer_run_4da226a0-4a0d-42bf-87cd-ae0d4f5f7be4.log
│ │ ├── problems
│ │ │ └── problems_and_actions.json
│ │ ├── result_descriptor.json
│ │ └── work
│ │ ├── shapley_formatted_orig_feat.zip
│ │ ├── shapley.orig.feat.bin
│ │ ├── shapley.orig.feat.csv
│ │ └── y_hat.bin
│ ├── explainer_h2o_sonar_explainers_pd_ice_explainer_PdIceExplainer_39381a2e-a060-43ae-8ce3-50389789a2cb
│ │ ├── global_html_fragment
│ │ │ ├── text_html
│ │ │ │ ├── explanation.html
│ │ │ │ ├── pd-feature-0-class-0.png
│ │ │ │ ├── pd-feature-1-class-0.png
│ │ │ │ ├── pd-feature-2-class-0.png
│ │ │ │ ├── pd-feature-3-class-0.png
│ │ │ │ ├── pd-feature-4-class-0.png
│ │ │ │ ├── pd-feature-5-class-0.png
│ │ │ │ ├── pd-feature-6-class-0.png
│ │ │ │ ├── pd-feature-7-class-0.png
│ │ │ │ ├── pd-feature-8-class-0.png
│ │ │ │ └── pd-feature-9-class-0.png
│ │ │ └── text_html.meta
│ │ ├── global_partial_dependence
│ │ │ ├── application_json
│ │ │ │ ├── explanation.json
│ │ │ │ ├── pd_feature_0_class_0.json
│ │ │ │ ├── pd_feature_1_class_0.json
│ │ │ │ ├── pd_feature_2_class_0.json
│ │ │ │ ├── pd_feature_3_class_0.json
│ │ │ │ ├── pd_feature_4_class_0.json
│ │ │ │ ├── pd_feature_5_class_0.json
│ │ │ │ ├── pd_feature_6_class_0.json
│ │ │ │ ├── pd_feature_7_class_0.json
│ │ │ │ ├── pd_feature_8_class_0.json
│ │ │ │ └── pd_feature_9_class_0.json
│ │ │ └── application_json.meta
│ │ ├── insights
│ │ │ └── insights_and_actions.json
│ │ ├── local_individual_conditional_explanation
│ │ │ ├── application_vnd_h2oai_json_datatable_jay
│ │ │ │ ├── explanation.json
│ │ │ │ ├── ice_feature_0_class_0.jay
│ │ │ │ ├── ice_feature_1_class_0.jay
│ │ │ │ ├── ice_feature_2_class_0.jay
│ │ │ │ ├── ice_feature_3_class_0.jay
│ │ │ │ ├── ice_feature_4_class_0.jay
│ │ │ │ ├── ice_feature_5_class_0.jay
│ │ │ │ ├── ice_feature_6_class_0.jay
│ │ │ │ ├── ice_feature_7_class_0.jay
│ │ │ │ ├── ice_feature_8_class_0.jay
│ │ │ │ ├── ice_feature_9_class_0.jay
│ │ │ │ └── y_hat.jay
│ │ │ └── application_vnd_h2oai_json_datatable_jay.meta
│ │ ├── log
│ │ │ └── explainer_run_39381a2e-a060-43ae-8ce3-50389789a2cb.log
│ │ ├── problems
│ │ │ └── problems_and_actions.json
│ │ ├── result_descriptor.json
│ │ └── work
│ │ ├── h2o_sonar-ice-dai-model-10.jay
│ │ ├── h2o_sonar-ice-dai-model-1.jay
│ │ ├── h2o_sonar-ice-dai-model-2.jay
│ │ ├── h2o_sonar-ice-dai-model-3.jay
│ │ ├── h2o_sonar-ice-dai-model-4.jay
│ │ ├── h2o_sonar-ice-dai-model-5.jay
│ │ ├── h2o_sonar-ice-dai-model-6.jay
│ │ ├── h2o_sonar-ice-dai-model-7.jay
│ │ ├── h2o_sonar-ice-dai-model-8.jay
│ │ ├── h2o_sonar-ice-dai-model-9.jay
│ │ ├── h2o_sonar-ice-dai-model.json
│ │ ├── h2o_sonar-pd-dai-model.json
│ │ └── mli_dataset_y_hat.jay
│ ├── explainer_h2o_sonar_explainers_residual_dt_surrogate_explainer_ResidualDecisionTreeSurrogateExplainer_aa5cd513-a249-4380-898f-bdea329dc5e4
│ │ ├── global_custom_archive
│ │ │ ├── application_zip
│ │ │ │ └── explanation.zip
│ │ │ └── application_zip.meta
│ │ ├── global_decision_tree
│ │ │ ├── application_json
│ │ │ │ ├── dt_class_0.json
│ │ │ │ └── explanation.json
│ │ │ └── application_json.meta
│ │ ├── global_html_fragment
│ │ │ ├── text_html
│ │ │ │ ├── dt-class-0.png
│ │ │ │ └── explanation.html
│ │ │ └── text_html.meta
│ │ ├── insights
│ │ │ └── insights_and_actions.json
│ │ ├── local_decision_tree
│ │ │ ├── application_json
│ │ │ │ └── explanation.json
│ │ │ └── application_json.meta
│ │ ├── log
│ │ │ └── explainer_run_aa5cd513-a249-4380-898f-bdea329dc5e4.log
│ │ ├── problems
│ │ │ └── problems_and_actions.json
│ │ ├── result_descriptor.json
│ │ └── work
│ │ ├── dt-class-0.dot
│ │ ├── dt-class-0.dot.pdf
│ │ ├── dtModel.json
│ │ ├── dtpaths_frame.bin
│ │ ├── dtPathsFrame.csv
│ │ ├── dtsurr_mojo.zip
│ │ ├── dtSurrogate.json
│ │ └── dt_surrogate_rules.zip
│ ├── explainer_h2o_sonar_explainers_summary_shap_explainer_SummaryShapleyExplainer_f833eb2e-787d-4db3-8a93-a2f5ce69324c
│ │ ├── global_html_fragment
│ │ │ ├── text_html
│ │ │ │ ├── explanation.html
│ │ │ │ ├── feature_0_class_0.png
│ │ │ │ ├── feature_10_class_0.png
│ │ │ │ ├── feature_11_class_0.png
│ │ │ │ ├── feature_12_class_0.png
│ │ │ │ ├── feature_13_class_0.png
│ │ │ │ ├── feature_14_class_0.png
│ │ │ │ ├── feature_15_class_0.png
│ │ │ │ ├── feature_16_class_0.png
│ │ │ │ ├── feature_17_class_0.png
│ │ │ │ ├── feature_18_class_0.png
│ │ │ │ ├── feature_19_class_0.png
│ │ │ │ ├── feature_1_class_0.png
│ │ │ │ ├── feature_20_class_0.png
│ │ │ │ ├── feature_21_class_0.png
│ │ │ │ ├── feature_22_class_0.png
│ │ │ │ ├── feature_2_class_0.png
│ │ │ │ ├── feature_3_class_0.png
│ │ │ │ ├── feature_4_class_0.png
│ │ │ │ ├── feature_5_class_0.png
│ │ │ │ ├── feature_6_class_0.png
│ │ │ │ ├── feature_7_class_0.png
│ │ │ │ ├── feature_8_class_0.png
│ │ │ │ ├── feature_9_class_0.png
│ │ │ │ └── shapley-class-0.png
│ │ │ └── text_html.meta
│ │ ├── global_summary_feature_importance
│ │ │ ├── application_json
│ │ │ │ ├── explanation.json
│ │ │ │ ├── feature_0_class_0.png
│ │ │ │ ├── feature_10_class_0.png
│ │ │ │ ├── feature_11_class_0.png
│ │ │ │ ├── feature_12_class_0.png
│ │ │ │ ├── feature_13_class_0.png
│ │ │ │ ├── feature_14_class_0.png
│ │ │ │ ├── feature_15_class_0.png
│ │ │ │ ├── feature_16_class_0.png
│ │ │ │ ├── feature_17_class_0.png
│ │ │ │ ├── feature_18_class_0.png
│ │ │ │ ├── feature_19_class_0.png
│ │ │ │ ├── feature_1_class_0.png
│ │ │ │ ├── feature_20_class_0.png
│ │ │ │ ├── feature_21_class_0.png
│ │ │ │ ├── feature_22_class_0.png
│ │ │ │ ├── feature_2_class_0.png
│ │ │ │ ├── feature_3_class_0.png
│ │ │ │ ├── feature_4_class_0.png
│ │ │ │ ├── feature_5_class_0.png
│ │ │ │ ├── feature_6_class_0.png
│ │ │ │ ├── feature_7_class_0.png
│ │ │ │ ├── feature_8_class_0.png
│ │ │ │ ├── feature_9_class_0.png
│ │ │ │ ├── summary_feature_importance_class_0_offset_0.json
│ │ │ │ ├── summary_feature_importance_class_0_offset_1.json
│ │ │ │ └── summary_feature_importance_class_0_offset_2.json
│ │ │ ├── application_json.meta
│ │ │ ├── application_vnd_h2oai_json_datatable_jay
│ │ │ │ ├── explanation.json
│ │ │ │ └── summary_feature_importance_class_0.jay
│ │ │ ├── application_vnd_h2oai_json_datatable_jay.meta
│ │ │ ├── text_markdown
│ │ │ │ ├── explanation.md
│ │ │ │ └── shapley-class-0.png
│ │ │ └── text_markdown.meta
│ │ ├── insights
│ │ │ └── insights_and_actions.json
│ │ ├── log
│ │ │ └── explainer_run_f833eb2e-787d-4db3-8a93-a2f5ce69324c.log
│ │ ├── problems
│ │ │ └── problems_and_actions.json
│ │ ├── result_descriptor.json
│ │ └── work
│ │ ├── raw_shapley_contribs_class_0.jay
│ │ ├── raw_shapley_contribs_index.json
│ │ ├── report.md
│ │ └── shapley-class-0.png
│ ├── explainer_h2o_sonar_explainers_transformed_fi_shapley_explainer_ShapleyMojoTransformedFeatureImportanceExplainer_4b1a325d-ffbc-4e37-9671-954416d46a17
│ │ ├── global_feature_importance
│ │ │ ├── application_json
│ │ │ │ ├── explanation.json
│ │ │ │ └── feature_importance_class_0.json
│ │ │ ├── application_json.meta
│ │ │ ├── application_vnd_h2oai_json_csv
│ │ │ │ ├── explanation.json
│ │ │ │ └── feature_importance_class_0.csv
│ │ │ ├── application_vnd_h2oai_json_csv.meta
│ │ │ ├── application_vnd_h2oai_json_datatable_jay
│ │ │ │ ├── explanation.json
│ │ │ │ └── feature_importance_class_0.jay
│ │ │ └── application_vnd_h2oai_json_datatable_jay.meta
│ │ ├── global_html_fragment
│ │ │ ├── text_html
│ │ │ │ ├── explanation.html
│ │ │ │ └── fi-class-0.png
│ │ │ └── text_html.meta
│ │ ├── insights
│ │ │ └── insights_and_actions.json
│ │ ├── local_feature_importance
│ │ │ ├── application_vnd_h2oai_json_datatable_jay
│ │ │ │ ├── explanation.json
│ │ │ │ └── feature_importance_class_0.jay
│ │ │ └── application_vnd_h2oai_json_datatable_jay.meta
│ │ ├── log
│ │ │ └── explainer_run_4b1a325d-ffbc-4e37-9671-954416d46a17.log
│ │ ├── problems
│ │ │ └── problems_and_actions.json
│ │ ├── result_descriptor.json
│ │ └── work
│ │ ├── shapley.bin
│ │ ├── shapley.csv
│ │ └── shapley_formatted.zip
│ ├── interpretation.html
│ ├── interpretation.json
│ └── tmp
├── h2o-sonar.html
└── h2o-sonar.log
172 directories, 641 files
See interpretation.html ^ for interpretation results.
Run new interpretation with one particular explainer using command line interface:
[6]:
!h2o-sonar list explainers
/home/user/h/mli/git/h2o-sonar-FLOSS/.venv/lib/python3.11/site-packages/ragas/metrics/__init__.py:1: LangChainDeprecationWarning: As of langchain-core 0.3.0, LangChain uses pydantic v2 internally. The langchain_core.pydantic_v1 module was a compatibility shim for pydantic v1, and should no longer be used. Please update the code to import from Pydantic directly.
For example, replace imports like: `from langchain_core.pydantic_v1 import BaseModel`
with: `from pydantic import BaseModel`
or the v1 compatibility namespace if you are working in a code base that has not been fully upgraded to pydantic 2 yet. from pydantic.v1 import BaseModel
from ragas.metrics._answer_correctness import AnswerCorrectness, answer_correctness
/home/user/h/mli/git/h2o-sonar-FLOSS/.venv/lib/python3.11/site-packages/ragas/metrics/__init__.py:4: LangChainDeprecationWarning: As of langchain-core 0.3.0, LangChain uses pydantic v2 internally. The langchain.pydantic_v1 module was a compatibility shim for pydantic v1, and should no longer be used. Please update the code to import from Pydantic directly.
For example, replace imports like: `from langchain.pydantic_v1 import BaseModel`
with: `from pydantic import BaseModel`
or the v1 compatibility namespace if you are working in a code base that has not been fully upgraded to pydantic 2 yet. from pydantic.v1 import BaseModel
from ragas.metrics._context_entities_recall import (
{
"explainers": [
"h2o_sonar.evaluators.agent_sanity_check_evaluator.AgentSanityCheckEvaluator",
"h2o_sonar.explainers.fi_naive_shapley_explainer.NaiveShapleyMojoFeatureImportanceExplainer",
"h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer",
"h2o_sonar.explainers.dt_surrogate_explainer.DecisionTreeSurrogateExplainer",
"h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer",
"h2o_sonar.explainers.dia_explainer.DiaExplainer",
"h2o_sonar.explainers.transformed_fi_shapley_explainer.ShapleyMojoTransformedFeatureImportanceExplainer",
"h2o_sonar.explainers.residual_dt_surrogate_explainer.ResidualDecisionTreeSurrogateExplainer",
"h2o_sonar.explainers.residual_pd_ice_explainer.ResidualPdIceExplainer",
"h2o_sonar.explainers.fi_kernel_shap_explainer.KernelShapFeatureImportanceExplainer",
"h2o_sonar.explainers.pd_2_features_explainer.PdFor2FeaturesExplainer",
"h2o_sonar.explainers.friedman_h_statistic_explainer.FriedmanHStatisticExplainer",
"h2o_sonar.explainers.morris_sa_explainer.MorrisSensitivityAnalysisExplainer",
"h2o_sonar.explainers.dataset_and_model_insights_explainer.DatasetAndModelInsightsExplainer",
"h2o_sonar.explainers.drift_explainer.DriftDetectionExplainer",
"h2o_sonar.explainers.adversarial_similarity_explainer.AdversarialSimilarityExplainer",
"h2o_sonar.explainers.size_dependency_explainer.SizeDependencyExplainer",
"h2o_sonar.explainers.segment_performance_explainer.SegmentPerformanceExplainer",
"h2o_sonar.explainers.calibration_score_explainer.CalibrationScoreExplainer",
"h2o_sonar.explainers.backtesting_explainer.BacktestingExplainer",
"h2o_sonar.evaluators.answer_semantic_similarity_per_sentence_evaluator.AnswerSemanticSimilarityPerSentenceEvaluator",
"h2o_sonar.evaluators.rag_ragas_evaluator.RagasEvaluator",
"h2o_sonar.evaluators.rag_tokens_presence_evaluator.RagStrStrEvaluator",
"h2o_sonar.evaluators.answer_accuracy_evaluator.AnswerAccuracyEvaluator",
"h2o_sonar.evaluators.rag_answer_correctness_evaluator.AnswerCorrectnessEvaluator",
"h2o_sonar.evaluators.rag_answer_similarity_evaluator.AnswerSemanticSimilarityEvaluator",
"h2o_sonar.evaluators.rag_chunk_relevancy_evaluator.ContextChunkRelevancyEvaluator",
"h2o_sonar.evaluators.rag_context_relevancy_evaluator.ContextRelevancyEvaluator",
"h2o_sonar.evaluators.rag_answer_relevancy_evaluator.AnswerRelevancyEvaluator",
"h2o_sonar.evaluators.rag_answer_relevancy_no_judge_evaluator.RagAnswerRelevancyNoJudgeEvaluator",
"h2o_sonar.evaluators.rag_context_precision_evaluator.ContextPrecisionEvaluator",
"h2o_sonar.evaluators.rag_context_recall_evaluator.ContextRecallEvaluator",
"h2o_sonar.evaluators.rag_context_mean_reciprocal_rank_evaluator.MeanReciprocalRankEvaluator",
"h2o_sonar.evaluators.rag_faithfulness_evaluator.FaithfulnessEvaluator",
"h2o_sonar.evaluators.pii_leakage_evaluator.PiiLeakageEvaluator",
"h2o_sonar.evaluators.json_schema_evaluator.JSONSchemaEvaluator",
"h2o_sonar.evaluators.sensitive_data_leakage_evaluator.SensitiveDataLeakageEvaluator",
"h2o_sonar.evaluators.toxicity_evaluator.ToxicityEvaluator",
"h2o_sonar.evaluators.fairness_bias_evaluator.FairnessBiasEvaluator",
"h2o_sonar.evaluators.contact_information_byop_evaluator.ContactInformationByopEvaluator",
"h2o_sonar.evaluators.language_mismatch_byop_evaluator.LanguageMismatchByopEvaluator",
"h2o_sonar.evaluators.looping_detection_evaluator.LoopingDetectionEvaluator",
"h2o_sonar.evaluators.parameterizable_byop_evaluator.ParameterizableByopEvaluator",
"h2o_sonar.evaluators.perplexity_evaluator.PerplexityEvaluator",
"h2o_sonar.evaluators.sexism_byop_evaluator.SexismByopEvaluator",
"h2o_sonar.evaluators.stereotype_byop_evaluator.StereotypeByopEvaluator",
"h2o_sonar.evaluators.summarization_byop_evaluator.SummarizationByopEvaluator",
"h2o_sonar.evaluators.rag_groundedness_evaluator.RagGroundednessEvaluator",
"h2o_sonar.evaluators.rag_hallucination_evaluator.RagHallucinationEvaluator",
"h2o_sonar.evaluators.bertscore_evaluator.BertscoreEvaluator",
"h2o_sonar.evaluators.bleu_evaluator.BleuEvaluator",
"h2o_sonar.evaluators.rouge_evaluator.RougeEvaluator",
"h2o_sonar.evaluators.self_consistency_evaluator.SelfConsistencyEvaluator",
"h2o_sonar.evaluators.classification_evaluator.ClassificationEvaluator",
"h2o_sonar.evaluators.gptscore_summary_with_reference_evaluator.GptScoreSummaryWithReferenceEvaluator",
"h2o_sonar.evaluators.gptscore_summary_without_reference_evaluator.GptScoreSummaryWithoutReferenceEvaluator",
"h2o_sonar.evaluators.gptscore_question_answering_evaluator.GptScoreQuestionAnsweringEvaluator",
"h2o_sonar.evaluators.gptscore_machine_translation_evaluator.GptScoreMachineTranslationEvaluator",
"h2o_sonar.evaluators.encoding_guardrail_evaluator.EncodingGuardrailEvaluator",
"h2o_sonar.evaluators.agentic_fact_check_evaluator.FactCheckAgenticEvaluator"
]
}
[ ]:
results_path = f"/tmp/{time.time()}"
os.mkdir(results_path)
!h2o-sonar run interpretation \
--explainers=h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer,h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer \
--dataset={dataset_path} \
--model={model_path} \
--target-col={target_column} \
--results-location={results_path}
[ ]:
!tree $results_path
[ ]: