{ "cells": [ { "cell_type": "markdown", "id": "bf925586-9e47-4588-b544-0faa8bc4cf7b", "metadata": {}, "source": [ "# H2O Sonar CLI demo\n", "\n", "This example demonstrates how to interpret a model using H2O Sonar from the **command line**:\n", "\n", "1. by running all compatible explainers\n", "1. by running selected explainers" ] }, { "cell_type": "code", "execution_count": 1, "id": "0da787e4-964b-486b-a67e-19efdc8c48cd", "metadata": {}, "outputs": [], "source": [ "import os\n", "import time" ] }, { "cell_type": "markdown", "id": "94fa1d12-1fed-446c-b153-9ae4a4eb2f99", "metadata": {}, "source": [ "Get **H2O Sonar** command line interface help:" ] }, { "cell_type": "code", "execution_count": 2, "id": "7259cacf-44f5-4c81-934b-ec68ac759e8e", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "usage: h2o-sonar [-h] [--dataset DATASET] [--target-col TARGET_COL]\n", " [--results-location RESULTS_LOCATION]\n", " [--results-formats RESULTS_FORMATS] [--model MODEL]\n", " [--validset VALIDSET] [--testset TESTSET]\n", " [--use_raw-features USE_RAW_FEATURES]\n", " [--weight-col WEIGHT_COL] [--drop-cols DROP_COLS]\n", " [--sample-num-rows SAMPLE_NUM_ROWS]\n", " [--used-features USED_FEATURES] [--model-type {pickle,mojo}]\n", " [--explainer EXPLAINER] [--explainers EXPLAINERS]\n", " [--all-explainers] [--explainers-pars EXPLAINERS_PARS]\n", " [--config-path CONFIG_PATH] [--config-type CONFIG_TYPE]\n", " [--config-value CONFIG_VALUE]\n", " [--encryption-key ENCRYPTION_KEY] [-d]\n", " [--interpretation INTERPRETATION]\n", " [--collection-name COLLECTION_NAME] [--upload-to UPLOAD_TO]\n", " [--args-as-json-location ARGS_AS_JSON_LOCATION]\n", " [--log-level {error,warning,info,debug}]\n", " action entity\n", "\n", "H2O Sonar Python library for Responsible AI.\n", "\n", "H2O Sonar is Python package that enables a holistic, low-risk, human-interpretable,\n", "fair, and trustable approach to machine learning by implementing various facets of\n", "Responsible AI.\n", "\n", "model, dataset, validset or testset handle schema:\n", "\n", " resource:connection:\"\":key:\n", " [:version:]\n", "\n", "optional arguments per action and entity:\n", "\n", " show version:\n", " show H2O Sonar version\n", "\n", " add config:\n", " --config-path path to JSon or TOML file with H2O Sonar config to be changed\n", " --config-type config item type: 'CONNECTION' or 'LICENSE'\n", " --config-value config item value (serialized as JSon) to add to the config file\n", " --encryption-key secret key to encrypt config fields with sensitive data\n", " (alternatively set H2O_SONAR_ENCRYPTION_KEY environment variable)\n", "\n", " show config:\n", " --config-path path to JSon or TOML file with H2O Sonar config\n", " --encryption-key optional secret key to decrypt config fields with sensitive data\n", " (alternatively set H2O_SONAR_ENCRYPTION_KEY environment variable)\n", "\n", " list explainers:\n", " --detailed show detailed descriptors (only IDs are shown by default)\n", " --args-as-json-location\n", " optional JSon file which overrides filtering CLI arguments\n", "\n", " describe explainer:\n", " --explainer explainer ID\n", "\n", " run interpretation:\n", " --dataset path to dataset\n", " --target-col target column\n", " --model path to the serialized model, URL or locator\n", " --results-location\n", " optional path to the interpretation results location (directory)\n", " --validset optional path to validation dataset\n", " --testset optional path to test dataset\n", " --use_raw_features\n", " force the use of transformed features in surrogate models\n", " with 'false', by default the original (raw) features are used\n", " --weight-col optional dataset column name with examples weights\n", " --drop-cols optional list of dataset columns to drop\n", " --sample-num-rows\n", " optional number of rows to sample from dataset (default: sample\n", " based on the RAM size, 0 do not sample, >0 sample to the specified\n", " number of rows)\n", " --all-explainers run all explainers (only the most important are run by default)\n", " --used-features optional comma separated list of features used by the model\n", " --model-type optional model type: 'pickle' or 'mojo'\n", " --explainers optional comma separated list of explainer IDs to be run\n", " --explainers-pars optional dictionary with explainer parameters\n", " --config-path path to JSon or TOML file with H2O Sonar config to be changed\n", " --encryption-key secret key to encrypt config fields with sensitive data\n", " (alternatively set H2O_SONAR_ENCRYPTION_KEY environment variable)\n", " --upload-to optional (h2oGPT Enterprise) connection key from the configuration\n", " where the report is to be uploaded in order to talk to it\n", " --args-as-json-location\n", " optional JSon file which overrides CLI arguments\n", " --log-level optional log level: 'error', 'warning', 'info', 'debug'\n", "\n", " list interpretations:\n", " --results-location\n", " path to directory, URL, location of interpretation results\n", " --log-level optional log level: 'error', 'warning', 'info', 'debug'\n", "\n", " upload interpretations:\n", " --interpretation path to the interpretation report PDF or HTML file\n", " --upload-to (h2oGPT Enterprise) connection key from the configuration\n", " where the report is to be uploaded in order to talk to it\n", " --collection-name optional name of the collection where the report is to be uploaded\n", " --config-path path to JSon or TOML file with H2O Sonar config to be changed\n", " --encryption-key secret key to encrypt config fields with sensitive data\n", " (alternatively set H2O_SONAR_ENCRYPTION_KEY environment variable)\n", "\n", "positional arguments:\n", " action action to take: 'list', 'run' or 'describe'\n", " entity entity on which to perform the action:\n", " 'interpretation'(s) or 'explainer'(s)\n", "\n", "options:\n", " -h, --help show this help message and exit\n", " --dataset DATASET location of the dataset\n", " --target-col TARGET_COL\n", " target column\n", " --results-location RESULTS_LOCATION\n", " location where to store the interpretation results\n", " --results-formats RESULTS_FORMATS\n", " comma separated list of MIME types of the\n", " interpretation results to create\n", " --model MODEL location of the model\n", " --validset VALIDSET location of the validation dataset\n", " --testset TESTSET location of the test dataset\n", " --use_raw-features USE_RAW_FEATURES\n", " force the use of transformed features in surrogate\n", " models with `false`\n", " --weight-col WEIGHT_COL\n", " optional dataset column name with examples weights\n", " --drop-cols DROP_COLS\n", " optional list of dataset columns to drop\n", " --sample-num-rows SAMPLE_NUM_ROWS\n", " optional number of rows to sample from the dataset\n", " --used-features USED_FEATURES\n", " optional comma separated list of features used by the\n", " model\n", " --model-type {pickle,mojo}\n", " model type: 'pickle' (.pkl) or 'mojo' (.mojo)\n", " --explainer EXPLAINER\n", " ID of the explainer to describe\n", " --explainers EXPLAINERS\n", " comma separated list of explainer IDs to be run (only\n", " the most important explainers are run by default)\n", " --all-explainers run all explainers (only the most important explainers\n", " are run by default)\n", " --explainers-pars EXPLAINERS_PARS\n", " optional dictionary with explainer parameters - the\n", " dictionary key is explainer ID and value is dictionary\n", " with parameters; parameter dictionary has parameter\n", " name as the key and parameter value as the value\n", " --config-path CONFIG_PATH\n", " path to JSon or TOML file with H2O Sonar configuration\n", " to be used to override defaults - specify only items\n", " you want to change (please refer to\n", " h2o_sonar.config.H2oSonarConfig for more details)\n", " --config-type CONFIG_TYPE\n", " configuration item type - 'CONNECTION' or 'LICENSE'\n", " --config-value CONFIG_VALUE\n", " configuration item value represented either as\n", " dictionary or as string with JSon serialization of the\n", " configuration item - it is expected that the config\n", " item is NOT encrypted\n", " --encryption-key ENCRYPTION_KEY\n", " encryption key to be used for encrypting/decrypting\n", " sensitive data in the configuration. If not specified,\n", " shell environment variable H2O_SONAR_ENCRYPTION_KEY\n", " with the encryption key is used.\n", " -d, --detailed show detailed descriptors (only IDs are shown by\n", " default)\n", " --interpretation INTERPRETATION\n", " path to the interpretation report (PDF or HTML) to be\n", " uploaded\n", " --collection-name COLLECTION_NAME\n", " optional (h2oGPT Enterprise) collection name where to\n", " upload the interpretation report\n", " --upload-to UPLOAD_TO\n", " optional (h2oGPT Enterprise) connection key from the\n", " configuration where the report is to be uploaded in\n", " order to talk to it\n", " --args-as-json-location ARGS_AS_JSON_LOCATION\n", " location of the JSon file with all command arguments\n", " (replacing command line arguments) allowing to load\n", " them from the filesystem\n", " --log-level {error,warning,info,debug}\n", " log level\n", "\n", "examples:\n", "\n", " h2o-sonar --help\n", " h2o-sonar show version\n", " h2o-sonar list explainers\n", " h2o-sonar list explainers --detailed\n", " h2o-sonar describe explainer\n", " --explainer=h2o_sonar.explainers.dia_explainer.DiaExplainer\n", " h2o-sonar run interpretation\n", " --dataset=dataset.csv\n", " --target-col=PROFIT\n", " --results-location=/home/user/results\n", " --model=model.pickle\n", " --all-explainers\n", " h2o-sonar run interpretation\n", " --dataset=dataset.csv\n", " --target-col=PROFIT\n", " --results-location=/home/user/results\n", " --model=model.pickle\n", " --used-features=FEATURE_1,FEATURE_2,FEATURE_3\n", " --explainers=h2o_sonar.explainers.dia_explainer.DiaExplainer\n", " --explainers-pars=\n", " \"{'h2o_sonar.explainers.dia_explainer.DiaExplainer':{'cut_off': 0.5}}\"\n", " --drop_cols=COLUMN_1,COLUMN_2,COLUMN_3\n", " --config-path=h2o-sonar-config.json\n", " --upload-to=4cff6fc9-f49a-4dda-aeb5-2c42e9f12807\n", " h2o-sonar run interpretation\n", " --args-as-json-location=h2o-sonar-args.json\n", " h2o-sonar list interpretations --results-location=/home/user/results\n", " h2o-sonar upload interpretation\n", " --interpretation=./results/h2o-sonar/mli-experiment/interpretation-detailed.html\n", " --upload-to=4cff6fc9-f49a-4dda-aeb5-2c42e9f12807\n", "\n", "H2O Sonar JSon configuration example:\n", " {\n", " \"h2o_host\": \"192.168.0.1\",\n", " \"h2o_port\": 57561,\n", " \"h2o_auto_start\": true,\n", " \"connections\": [\n", " {\n", " \"key\": \"4cff6fc9-f49a-4dda-aeb5-2c42e9f12807\",\n", " \"connection_type\": \"H2O_GPT_E\",\n", " \"name\": \"H2O GPT Enterprise\",\n", " \"description\": \"H2O GPT Enterprise service.\",\n", " \"server_url\": \"https://h2ogpte.h2o.ai\",\n", " \"token\": {\n", " \"encrypted\": \"gAAA3LcKQ7x_X...gnKsBqVdNydYTlk8nyQ==\"\n", " },\n", " \"token_use_type\": \"API_KEY\"\n", " }\n", " ]\n", " }\n", "\n", "Interpretation arguments JSon file example - see interpret.py::run_interpretation():\n", " {\n", " \"dataset\": \"dataset.csv\",\n", " \"model\": \"model.pickle\",\n", " \"target_col\": \"PROFIT\",\n", " \"results_location\": \"./results\"\n", " }\n", "\n", "Explainer listing arguments JSon file example - see interpret.py::list_explainers():\n", " {\n", " \"experiment_types\": [\"regression\"],\n", " \"explanation_scopes\": [\"local_scope\"],\n", " \"keywords\": [\"explains-fairness\"],\n", " \"explainer_filter\": [{\"filter_by\": \"filter-name\", \"value\": \"v\"}]\n", " }\n" ] } ], "source": [ "!h2o-sonar --help" ] }, { "cell_type": "markdown", "id": "7e593ebe-33aa-4a58-b6d1-7c35767690e7", "metadata": {}, "source": [ "Driverless AI **MOJO model** to be interpreted by the library:" ] }, { "cell_type": "code", "execution_count": 3, "id": "7934b7a8-f86a-44ca-8ce7-f0b5be4b696e", "metadata": {}, "outputs": [], "source": [ "dataset_path = \"../../data/predictive/creditcard.csv\"\n", "target_column = \"\\\"default payment next month\\\"\"\n", "\n", "# specify path to Driverless AI MOJO\n", "model_path = \"../../data/predictive/models/creditcard-binomial.mojo\"\n", "\n", "results_path = f\"/tmp/{time.time()}\"\n", "os.mkdir(results_path)" ] }, { "cell_type": "markdown", "id": "308379a8-c1cd-48ca-ab71-38c51e897935", "metadata": {}, "source": [ "Run new interpretation with **all compatible explainers** using command line interface:" ] }, { "cell_type": "code", "execution_count": 4, "id": "c8697b39-06b0-4bf5-a481-f17fc7baaf6d", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/home/user/h/mli/git/h2o-sonar-FLOSS/.venv/lib/python3.11/site-packages/ragas/metrics/__init__.py:1: LangChainDeprecationWarning: As of langchain-core 0.3.0, LangChain uses pydantic v2 internally. The langchain_core.pydantic_v1 module was a compatibility shim for pydantic v1, and should no longer be used. Please update the code to import from Pydantic directly.\n", "\n", "For example, replace imports like: `from langchain_core.pydantic_v1 import BaseModel`\n", "with: `from pydantic import BaseModel`\n", "or the v1 compatibility namespace if you are working in a code base that has not been fully upgraded to pydantic 2 yet. \tfrom pydantic.v1 import BaseModel\n", "\n", " from ragas.metrics._answer_correctness import AnswerCorrectness, answer_correctness\n", "/home/user/h/mli/git/h2o-sonar-FLOSS/.venv/lib/python3.11/site-packages/ragas/metrics/__init__.py:4: LangChainDeprecationWarning: As of langchain-core 0.3.0, LangChain uses pydantic v2 internally. The langchain.pydantic_v1 module was a compatibility shim for pydantic v1, and should no longer be used. Please update the code to import from Pydantic directly.\n", "\n", "For example, replace imports like: `from langchain.pydantic_v1 import BaseModel`\n", "with: `from pydantic import BaseModel`\n", "or the v1 compatibility namespace if you are working in a code base that has not been fully upgraded to pydantic 2 yet. \tfrom pydantic.v1 import BaseModel\n", "\n", " from ragas.metrics._context_entities_recall import (\n", "Stratified/random sampler: loading the original dataset '../../data/predictive/creditcard.csv' for sampling...\n", "Stratified/random sampler: -> did NO sampling as the sampling limit is smaller than the number of rows in the dataset: 10000 <= 25000\n", "Preparing and checking DIA features (None): dataset= | PAY_AMT1 PAY_5 ID EDUCATION BILL_AMT1 PAY_3 BILL_AMT2 default payment next month LIMIT_BAL MARRIAGE … BILL_AMT5 PAY_2 PAY_AMT6 PAY_6 AGE\n", " | int32 int32 int32 int32 int32 int32 int32 int8 int32 int32 int32 int32 int32 int32 int32\n", "---- + -------- ----- ----- --------- --------- ----- --------- -------------------------- --------- -------- --------- ----- -------- ----- -----\n", " 0 | 0 -2 1 2 3913 -1 3102 1 20000 1 … 0 2 0 -2 24\n", " 1 | 0 0 2 2 2682 0 1725 1 120000 2 … 3455 2 2000 2 26\n", " 2 | 1518 0 3 2 29239 0 14027 0 90000 2 … 14948 0 5000 0 34\n", " 3 | 2000 0 4 2 46990 0 48233 0 50000 1 … 28959 0 1000 0 37\n", " 4 | 2000 0 5 2 8617 -1 5670 0 50000 1 … 19146 0 679 0 57\n", " 5 | 2500 0 6 1 64400 0 57069 0 50000 2 … 19619 0 800 0 37\n", " 6 | 55000 0 7 1 367965 0 412023 0 500000 2 … 483003 0 13770 0 29\n", " 7 | 380 0 8 2 11876 -1 380 0 100000 2 … -159 -1 1542 -1 23\n", " 8 | 3329 0 9 3 11285 2 14096 0 140000 1 … 11793 0 1000 0 28\n", " 9 | 0 -1 10 3 0 -2 0 0 20000 2 … 13007 -2 0 -1 35\n", " 10 | 2306 0 11 3 11073 2 9787 0 200000 2 … 1828 0 66 -1 34\n", " 11 | 21818 -1 12 1 12261 -1 21670 0 260000 2 … 22287 -1 3640 2 51\n", " 12 | 1000 -1 13 2 12137 -1 6500 0 630000 2 … 6500 0 0 -1 41\n", " 13 | 3200 0 14 2 65802 2 67369 1 70000 2 … 36137 2 0 2 30\n", " 14 | 3000 0 15 1 70887 0 67060 0 250000 2 … 56875 0 3000 0 29\n", " … | … … … … … … … … … … … … … … … …\n", "9995 | 241 -2 9996 1 0 -2 241 0 140000 2 … 0 -2 1419 -2 31\n", "9996 | 0 -2 9997 2 3946 -2 0 0 80000 2 … 0 -2 0 -2 37\n", "9997 | 6437 0 9998 3 138877 0 144085 0 200000 1 … 176717 0 4200 0 44\n", "9998 | 0 -2 9999 2 780 2 780 1 80000 2 … 0 2 0 -2 26\n", "9999 | 3000 0 10000 2 19505 0 20715 0 230000 1 … 19255 0 3000 0 36\n", "[10000 rows x 25 columns]\n", " dataset_meta={\n", " \"shape\": \"(10000, 25)\",\n", " \"row_count\": 10000,\n", " \"column_names\": [\n", " \"ID\",\n", " \"LIMIT_BAL\",\n", " \"SEX\",\n", " \"EDUCATION\",\n", " \"MARRIAGE\",\n", " \"AGE\",\n", " \"PAY_0\",\n", " \"PAY_2\",\n", " \"PAY_3\",\n", " \"PAY_4\",\n", " \"PAY_5\",\n", " \"PAY_6\",\n", " \"BILL_AMT1\",\n", " \"BILL_AMT2\",\n", " \"BILL_AMT3\",\n", " \"BILL_AMT4\",\n", " \"BILL_AMT5\",\n", " \"BILL_AMT6\",\n", " \"PAY_AMT1\",\n", " \"PAY_AMT2\",\n", " \"PAY_AMT3\",\n", " \"PAY_AMT4\",\n", " \"PAY_AMT5\",\n", " \"PAY_AMT6\",\n", " \"default payment next month\"\n", " ],\n", " \"column_types\": [\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\",\n", " \"int\"\n", " ],\n", " \"column_uniques\": [\n", " 10000,\n", " 72,\n", " 2,\n", " 7,\n", " 4,\n", " 54,\n", " 11,\n", " 11,\n", " 11,\n", " 11,\n", " 10,\n", " 10,\n", " 8371,\n", " 8215,\n", " 8072,\n", " 7913,\n", " 7764,\n", " 7550,\n", " 3763,\n", " 3581,\n", " 3305,\n", " 3247,\n", " 3258,\n", " 3174,\n", " 2\n", " ],\n", " \"columns_cat\": [],\n", " \"columns_num\": [],\n", " \"file_path\": \"../../data/predictive/creditcard.csv\",\n", " \"file_name\": \"\",\n", " \"file_size\": 944719,\n", " \"missing_values\": [\n", " \"\",\n", " \"?\",\n", " \"None\",\n", " \"nan\",\n", " \"NA\",\n", " \"N/A\",\n", " \"unknown\",\n", " \"inf\",\n", " \"-inf\",\n", " \"1.7976931348623157e+308\",\n", " \"-1.7976931348623157e+308\"\n", " ],\n", " \"columns_meta\": [\n", " {\n", " \"name\": \"ID\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": true,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 10000,\n", " \"frequency\": 0,\n", " \"unique\": 10000,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"LIMIT_BAL\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 72,\n", " \"frequency\": 0,\n", " \"unique\": 72,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"SEX\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 2,\n", " \"frequency\": 0,\n", " \"unique\": 2,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"EDUCATION\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 7,\n", " \"frequency\": 0,\n", " \"unique\": 7,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"MARRIAGE\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 4,\n", " \"frequency\": 0,\n", " \"unique\": 4,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"AGE\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 54,\n", " \"frequency\": 0,\n", " \"unique\": 54,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"PAY_0\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 11,\n", " \"frequency\": 0,\n", " \"unique\": 11,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"PAY_2\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 11,\n", " \"frequency\": 0,\n", " \"unique\": 11,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"PAY_3\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 11,\n", " \"frequency\": 0,\n", " \"unique\": 11,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"PAY_4\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 11,\n", " \"frequency\": 0,\n", " \"unique\": 11,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"PAY_5\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 10,\n", " \"frequency\": 0,\n", " \"unique\": 10,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"PAY_6\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 10,\n", " \"frequency\": 0,\n", " \"unique\": 10,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"BILL_AMT1\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 8371,\n", " \"frequency\": 0,\n", " \"unique\": 8371,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"BILL_AMT2\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 8215,\n", " \"frequency\": 0,\n", " \"unique\": 8215,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"BILL_AMT3\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 8072,\n", " \"frequency\": 0,\n", " \"unique\": 8072,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"BILL_AMT4\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 7913,\n", " \"frequency\": 0,\n", " \"unique\": 7913,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"BILL_AMT5\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 7764,\n", " \"frequency\": 0,\n", " \"unique\": 7764,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"BILL_AMT6\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 7550,\n", " \"frequency\": 0,\n", " \"unique\": 7550,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"PAY_AMT1\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 3763,\n", " \"frequency\": 0,\n", " \"unique\": 3763,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"PAY_AMT2\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 3581,\n", " \"frequency\": 0,\n", " \"unique\": 3581,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"PAY_AMT3\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 3305,\n", " \"frequency\": 0,\n", " \"unique\": 3305,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"PAY_AMT4\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 3247,\n", " \"frequency\": 0,\n", " \"unique\": 3247,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"PAY_AMT5\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 3258,\n", " \"frequency\": 0,\n", " \"unique\": 3258,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"PAY_AMT6\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 3174,\n", " \"frequency\": 0,\n", " \"unique\": 3174,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " },\n", " {\n", " \"name\": \"default payment next month\",\n", " \"data_type\": \"int\",\n", " \"logical_types\": [],\n", " \"format\": \"\",\n", " \"is_id\": false,\n", " \"is_numeric\": true,\n", " \"is_categorical\": false,\n", " \"count\": 2,\n", " \"frequency\": 0,\n", " \"unique\": 2,\n", " \"max\": null,\n", " \"min\": null,\n", " \"mean\": null,\n", " \"std\": null,\n", " \"histogram_counts\": [],\n", " \"histogram_ticks\": []\n", " }\n", " ],\n", " \"original_dataset_sampled\": false,\n", " \"original_dataset_path\": \"\",\n", " \"original_dataset_size\": 0,\n", " \"original_dataset_shape\": [\n", " 10000,\n", " 25\n", " ]\n", "}\n", "Using dataset ENTITY to prepare DIA features: column_names=['ID', 'LIMIT_BAL', 'SEX', 'EDUCATION', 'MARRIAGE', 'AGE', 'PAY_0', 'PAY_2', 'PAY_3', 'PAY_4', 'PAY_5', 'PAY_6', 'BILL_AMT1', 'BILL_AMT2', 'BILL_AMT3', 'BILL_AMT4', 'BILL_AMT5', 'BILL_AMT6', 'PAY_AMT1', 'PAY_AMT2', 'PAY_AMT3', 'PAY_AMT4', 'PAY_AMT5', 'PAY_AMT6', 'default payment next month'] column_uniques=[10000, 72, 2, 7, 4, 54, 11, 11, 11, 11, 10, 10, 8371, 8215, 8072, 7913, 7764, 7550, 3763, 3581, 3305, 3247, 3258, 3174, 2]\n", "DIA group columns prepared using dataset ENTITY: {'PAY_5', 'EDUCATION', 'MARRIAGE', 'PAY_6', 'PAY_0', 'SEX', 'PAY_3', 'PAY_4', 'default payment next month', 'PAY_2'}\n", "DIA group columns to SKIP: {'model_pred', 'default payment next month'}\n", "DIA group columns as BOOLs: [, , , , , , , , ]\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().\n", "Checking whether there is an H2O instance running at http://localhost:54324..... not found.\n", "Attempting to start a local H2O server...\n", " Java Version: openjdk version \"10\" 2018-03-20; OpenJDK Runtime Environment 18.3 (build 10+44); OpenJDK 64-Bit Server VM 18.3 (build 10+44, mixed mode)\n", " Starting server from /home/user/h/mli/git/h2o-sonar-FLOSS/.venv/lib/python3.11/site-packages/h2o/backend/bin/h2o.jar\n", " Ice root: /tmp/tmpiwy5gqbf\n", " JVM stdout: /tmp/tmpiwy5gqbf/h2o_user_started_from_python.out\n", " JVM stderr: /tmp/tmpiwy5gqbf/h2o_user_started_from_python.err\n", " Server is running at http://127.0.0.1:54324\n", " successful.o H2O server at http://127.0.0.1:54324 ...\n", "-------------------------- -----------------------------\n", "H2O_cluster_uptime: 01 secs\n", "H2O_cluster_timezone: Europe/Prague\n", "H2O_data_parsing_timezone: UTC\n", "H2O_cluster_version: 3.46.0.9\n", "H2O_cluster_version_age: 2 months and 4 days\n", "H2O_cluster_name: H2O_from_python_user_2xryhy\n", "H2O_cluster_total_nodes: 1\n", "H2O_cluster_free_memory: 4 Gb\n", "H2O_cluster_total_cores: 16\n", "H2O_cluster_allowed_cores: 16\n", "H2O_cluster_status: locked, healthy\n", "H2O_connection_url: http://127.0.0.1:54324\n", "H2O_connection_proxy: {\"http\": null, \"https\": null}\n", "H2O_internal_security: False\n", "Python_version: 3.11.11 final\n", "-------------------------- -----------------------------\n", "Connecting to H2O server at http://localhost:54324 ... successful.\n", "-------------------------- -----------------------------\n", "H2O_cluster_uptime: 01 secs\n", "H2O_cluster_timezone: Europe/Prague\n", "H2O_data_parsing_timezone: UTC\n", "H2O_cluster_version: 3.46.0.9\n", "H2O_cluster_version_age: 2 months and 4 days\n", "H2O_cluster_name: H2O_from_python_user_2xryhy\n", "H2O_cluster_total_nodes: 1\n", "H2O_cluster_free_memory: 4 Gb\n", "H2O_cluster_total_cores: 16\n", "H2O_cluster_allowed_cores: 16\n", "H2O_cluster_status: locked, healthy\n", "H2O_connection_url: http://localhost:54324\n", "H2O_connection_proxy: {\"http\": null, \"https\": null}\n", "H2O_internal_security: False\n", "Python_version: 3.11.11 final\n", "-------------------------- -----------------------------\n", "Connecting to H2O server at http://localhost:54324 ... successful.\n", "-------------------------- -----------------------------\n", "H2O_cluster_uptime: 01 secs\n", "H2O_cluster_timezone: Europe/Prague\n", "H2O_data_parsing_timezone: UTC\n", "H2O_cluster_version: 3.46.0.9\n", "H2O_cluster_version_age: 2 months and 4 days\n", "H2O_cluster_name: H2O_from_python_user_2xryhy\n", "H2O_cluster_total_nodes: 1\n", "H2O_cluster_free_memory: 4 Gb\n", "H2O_cluster_total_cores: 16\n", "H2O_cluster_allowed_cores: 16\n", "H2O_cluster_status: locked, healthy\n", "H2O_connection_url: http://localhost:54324\n", "H2O_connection_proxy: {\"http\": null, \"https\": null}\n", "H2O_internal_security: False\n", "Python_version: 3.11.11 final\n", "-------------------------- -----------------------------\n", "Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%\n", "Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%\n", "drf Model Build progress: |██████████████████████████████████████████████████████ (done)| 100%\n", "Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%\n", "Converting H2O frame to pandas dataframe using single-thread. For faster conversion using multi-thread, install polars and pyarrow and use it as pandas_df = h2o_df.as_data_frame(use_multi_thread=True)\n", "\n", "Connecting to H2O server at http://localhost:54324 ... successful.\n", "-------------------------- -----------------------------\n", "H2O_cluster_uptime: 04 secs\n", "H2O_cluster_timezone: Europe/Prague\n", "H2O_data_parsing_timezone: UTC\n", "H2O_cluster_version: 3.46.0.9\n", "H2O_cluster_version_age: 2 months and 4 days\n", "H2O_cluster_name: H2O_from_python_user_2xryhy\n", "H2O_cluster_total_nodes: 1\n", "H2O_cluster_free_memory: 4 Gb\n", "H2O_cluster_total_cores: 16\n", "H2O_cluster_allowed_cores: 16\n", "H2O_cluster_status: locked, healthy\n", "H2O_connection_url: http://localhost:54324\n", "H2O_connection_proxy: {\"http\": null, \"https\": null}\n", "H2O_internal_security: False\n", "Python_version: 3.11.11 final\n", "-------------------------- -----------------------------\n", "Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%\n", "Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%\n", "drf Model Build progress: |██████████████████████████████████████████████████████ (done)| 100%\n", "Parse progress: |████████████████████████████████████████████████████████████████ (done)| 100%\n", "Converting H2O frame to pandas dataframe using single-thread. For faster conversion using multi-thread, install polars and pyarrow and use it as pandas_df = h2o_df.as_data_frame(use_multi_thread=True)\n", "\n", "More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`). Consider using `matplotlib.pyplot.close()`.\n", "\n", "Interpretation FINISHED.\n", " Results directory:\n", " file:///tmp/1769698782.3007452\n", " Interpretations index:\n", " file:///tmp/1769698782.3007452/h2o-sonar.html\n", " HTML report:\n", " file:///tmp/1769698782.3007452/h2o-sonar/mli_experiment_acc39269-6aea-45c7-89cf-37cf4316abb9/interpretation.html\n", "Closing connection _sid_b519 at exit\n", "H2O session _sid_b519 closed.\n", "Closing connection _sid_b075 at exit\n", "H2O session _sid_b075 closed.\n" ] } ], "source": [ "!h2o-sonar run interpretation \\\n", " --dataset={dataset_path} \\\n", " --model={model_path} \\\n", " --target-col={target_column} \\\n", " --results-location={results_path} " ] }, { "cell_type": "code", "execution_count": 5, "id": "feef75bc-eb9c-4d24-8fd2-10c7dedee5a7", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[01;34m/tmp/1769698782.3007452\u001b[00m\n", "├── \u001b[01;34mh2o-sonar\u001b[00m\n", "│   └── \u001b[01;34mmli_experiment_acc39269-6aea-45c7-89cf-37cf4316abb9\u001b[00m\n", "│   ├── \u001b[01;34mexplainer_h2o_sonar_explainers_dia_explainer_DiaExplainer_eda4ed7a-6ef3-4edb-a229-a698e3ecb0ca\u001b[00m\n", "│   │   ├── \u001b[01;34mglobal_disparate_impact_analysis\u001b[00m\n", "│   │   │   ├── \u001b[01;34mtext_plain\u001b[00m\n", "│   │   │   │   └── explanation.txt\n", "│   │   │   └── text_plain.meta\n", "│   │   ├── \u001b[01;34mglobal_html_fragment\u001b[00m\n", "│   │   │   ├── \u001b[01;34mtext_html\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-0-accuracy.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-0-adverse_impact.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-0-false_discovery_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-0-false_negative_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-0-false_omissions_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-0-false_positive_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-0-negative_predicted_value.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-0-n.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-0-precision.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-0-specificity.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-0-true_positive_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-1-accuracy.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-1-adverse_impact.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-1-false_discovery_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-1-false_negative_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-1-false_omissions_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-1-false_positive_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-1-negative_predicted_value.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-1-n.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-1-precision.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-1-specificity.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-1-true_positive_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-2-accuracy.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-2-adverse_impact.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-2-false_discovery_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-2-false_negative_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-2-false_omissions_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-2-false_positive_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-2-negative_predicted_value.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-2-n.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-2-precision.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-2-specificity.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-2-true_positive_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-3-accuracy.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-3-adverse_impact.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-3-false_discovery_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-3-false_negative_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-3-false_omissions_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-3-false_positive_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-3-negative_predicted_value.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-3-n.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-3-precision.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-3-specificity.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-3-true_positive_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-4-accuracy.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-4-adverse_impact.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-4-false_discovery_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-4-false_negative_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-4-false_omissions_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-4-false_positive_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-4-negative_predicted_value.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-4-n.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-4-precision.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-4-specificity.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-4-true_positive_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-5-accuracy.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-5-adverse_impact.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-5-false_discovery_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-5-false_negative_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-5-false_omissions_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-5-false_positive_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-5-negative_predicted_value.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-5-n.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-5-precision.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-5-specificity.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-5-true_positive_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-6-accuracy.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-6-adverse_impact.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-6-false_discovery_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-6-false_negative_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-6-false_omissions_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-6-false_positive_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-6-negative_predicted_value.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-6-n.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-6-precision.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-6-specificity.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-6-true_positive_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-7-accuracy.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-7-adverse_impact.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-7-false_discovery_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-7-false_negative_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-7-false_omissions_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-7-false_positive_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-7-negative_predicted_value.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-7-n.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-7-precision.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-7-specificity.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-7-true_positive_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-8-accuracy.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-8-adverse_impact.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-8-false_discovery_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-8-false_negative_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-8-false_omissions_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-8-false_positive_rate.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-8-negative_predicted_value.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-8-n.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-8-precision.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-8-specificity.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdia-8-true_positive_rate.png\u001b[00m\n", "│   │   │   │   └── explanation.html\n", "│   │   │   └── text_html.meta\n", "│   │   ├── \u001b[01;34minsights\u001b[00m\n", "│   │   │   └── insights_and_actions.json\n", "│   │   ├── \u001b[01;34mlog\u001b[00m\n", "│   │   │   └── explainer_run_eda4ed7a-6ef3-4edb-a229-a698e3ecb0ca.log\n", "│   │   ├── \u001b[01;34mproblems\u001b[00m\n", "│   │   │   └── problems_and_actions.json\n", "│   │   ├── result_descriptor.json\n", "│   │   └── \u001b[01;34mwork\u001b[00m\n", "│   │   ├── dia_entity.json\n", "│   │   ├── \u001b[01;34mEDUCATION\u001b[00m\n", "│   │   │   ├── \u001b[01;34m0\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m1\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m2\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m3\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m4\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m5\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m6\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   └── metrics.jay\n", "│   │   ├── \u001b[01;34mMARRIAGE\u001b[00m\n", "│   │   │   ├── \u001b[01;34m0\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m1\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m2\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m3\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   └── metrics.jay\n", "│   │   ├── \u001b[01;34mPAY_0\u001b[00m\n", "│   │   │   ├── \u001b[01;34m0\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m1\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m10\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m2\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m3\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m4\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m5\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m6\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m7\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m8\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m9\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   └── metrics.jay\n", "│   │   ├── \u001b[01;34mPAY_2\u001b[00m\n", "│   │   │   ├── \u001b[01;34m0\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m1\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m10\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m2\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m3\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m4\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m5\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m6\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m7\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m8\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m9\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   └── metrics.jay\n", "│   │   ├── \u001b[01;34mPAY_3\u001b[00m\n", "│   │   │   ├── \u001b[01;34m0\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m1\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m10\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m2\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m3\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m4\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m5\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m6\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m7\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m8\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m9\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   └── metrics.jay\n", "│   │   ├── \u001b[01;34mPAY_4\u001b[00m\n", "│   │   │   ├── \u001b[01;34m0\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m1\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m10\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m2\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m3\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m4\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m5\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m6\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m7\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m8\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m9\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   └── metrics.jay\n", "│   │   ├── \u001b[01;34mPAY_5\u001b[00m\n", "│   │   │   ├── \u001b[01;34m0\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m1\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m2\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m3\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m4\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m5\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m6\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m7\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m8\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m9\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   └── metrics.jay\n", "│   │   ├── \u001b[01;34mPAY_6\u001b[00m\n", "│   │   │   ├── \u001b[01;34m0\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m1\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m2\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m3\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m4\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m5\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m6\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m7\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m8\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   ├── \u001b[01;34m9\u001b[00m\n", "│   │   │   │   ├── cm.jay\n", "│   │   │   │   ├── disparity.jay\n", "│   │   │   │   ├── me_smd.jay\n", "│   │   │   │   └── parity.jay\n", "│   │   │   └── metrics.jay\n", "│   │   └── \u001b[01;34mSEX\u001b[00m\n", "│   │   ├── \u001b[01;34m0\u001b[00m\n", "│   │   │   ├── cm.jay\n", "│   │   │   ├── disparity.jay\n", "│   │   │   ├── me_smd.jay\n", "│   │   │   └── parity.jay\n", "│   │   ├── \u001b[01;34m1\u001b[00m\n", "│   │   │   ├── cm.jay\n", "│   │   │   ├── disparity.jay\n", "│   │   │   ├── me_smd.jay\n", "│   │   │   └── parity.jay\n", "│   │   └── metrics.jay\n", "│   ├── \u001b[01;34mexplainer_h2o_sonar_explainers_dt_surrogate_explainer_DecisionTreeSurrogateExplainer_1cdc0473-0680-4a19-9057-c95e5d7ab193\u001b[00m\n", "│   │   ├── \u001b[01;34mglobal_custom_archive\u001b[00m\n", "│   │   │   ├── \u001b[01;34mapplication_zip\u001b[00m\n", "│   │   │   │   └── \u001b[01;31mexplanation.zip\u001b[00m\n", "│   │   │   └── application_zip.meta\n", "│   │   ├── \u001b[01;34mglobal_decision_tree\u001b[00m\n", "│   │   │   ├── \u001b[01;34mapplication_json\u001b[00m\n", "│   │   │   │   ├── dt_class_0.json\n", "│   │   │   │   └── explanation.json\n", "│   │   │   └── application_json.meta\n", "│   │   ├── \u001b[01;34mglobal_html_fragment\u001b[00m\n", "│   │   │   ├── \u001b[01;34mtext_html\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdt-class-0.png\u001b[00m\n", "│   │   │   │   └── explanation.html\n", "│   │   │   └── text_html.meta\n", "│   │   ├── \u001b[01;34minsights\u001b[00m\n", "│   │   │   └── insights_and_actions.json\n", "│   │   ├── \u001b[01;34mlocal_decision_tree\u001b[00m\n", "│   │   │   ├── \u001b[01;34mapplication_json\u001b[00m\n", "│   │   │   │   └── explanation.json\n", "│   │   │   └── application_json.meta\n", "│   │   ├── \u001b[01;34mlog\u001b[00m\n", "│   │   │   └── explainer_run_1cdc0473-0680-4a19-9057-c95e5d7ab193.log\n", "│   │   ├── \u001b[01;34mproblems\u001b[00m\n", "│   │   │   └── problems_and_actions.json\n", "│   │   ├── result_descriptor.json\n", "│   │   └── \u001b[01;34mwork\u001b[00m\n", "│   │   ├── dt-class-0.dot\n", "│   │   ├── dt-class-0.dot.pdf\n", "│   │   ├── dtModel.json\n", "│   │   ├── dtpaths_frame.bin\n", "│   │   ├── dtPathsFrame.csv\n", "│   │   ├── \u001b[01;31mdtsurr_mojo.zip\u001b[00m\n", "│   │   ├── dtSurrogate.json\n", "│   │   └── \u001b[01;31mdt_surrogate_rules.zip\u001b[00m\n", "│   ├── \u001b[01;34mexplainer_h2o_sonar_explainers_fi_naive_shapley_explainer_NaiveShapleyMojoFeatureImportanceExplainer_4da226a0-4a0d-42bf-87cd-ae0d4f5f7be4\u001b[00m\n", "│   │   ├── \u001b[01;34mglobal_feature_importance\u001b[00m\n", "│   │   │   ├── \u001b[01;34mapplication_json\u001b[00m\n", "│   │   │   │   ├── explanation.json\n", "│   │   │   │   └── feature_importance_class_0.json\n", "│   │   │   ├── application_json.meta\n", "│   │   │   ├── \u001b[01;34mapplication_vnd_h2oai_json_csv\u001b[00m\n", "│   │   │   │   ├── explanation.json\n", "│   │   │   │   └── feature_importance_class_0.csv\n", "│   │   │   ├── application_vnd_h2oai_json_csv.meta\n", "│   │   │   ├── \u001b[01;34mapplication_vnd_h2oai_json_datatable_jay\u001b[00m\n", "│   │   │   │   ├── explanation.json\n", "│   │   │   │   └── feature_importance_class_0.jay\n", "│   │   │   └── application_vnd_h2oai_json_datatable_jay.meta\n", "│   │   ├── \u001b[01;34mglobal_html_fragment\u001b[00m\n", "│   │   │   ├── \u001b[01;34mtext_html\u001b[00m\n", "│   │   │   │   ├── explanation.html\n", "│   │   │   │   └── \u001b[01;35mfi-class-0.png\u001b[00m\n", "│   │   │   └── text_html.meta\n", "│   │   ├── \u001b[01;34minsights\u001b[00m\n", "│   │   │   └── insights_and_actions.json\n", "│   │   ├── \u001b[01;34mlocal_feature_importance\u001b[00m\n", "│   │   │   ├── \u001b[01;34mapplication_vnd_h2oai_json_datatable_jay\u001b[00m\n", "│   │   │   │   ├── explanation.json\n", "│   │   │   │   ├── feature_importance_class_0.jay\n", "│   │   │   │   └── y_hat.bin\n", "│   │   │   └── application_vnd_h2oai_json_datatable_jay.meta\n", "│   │   ├── \u001b[01;34mlog\u001b[00m\n", "│   │   │   └── explainer_run_4da226a0-4a0d-42bf-87cd-ae0d4f5f7be4.log\n", "│   │   ├── \u001b[01;34mproblems\u001b[00m\n", "│   │   │   └── problems_and_actions.json\n", "│   │   ├── result_descriptor.json\n", "│   │   └── \u001b[01;34mwork\u001b[00m\n", "│   │   ├── \u001b[01;31mshapley_formatted_orig_feat.zip\u001b[00m\n", "│   │   ├── shapley.orig.feat.bin\n", "│   │   ├── shapley.orig.feat.csv\n", "│   │   └── y_hat.bin\n", "│   ├── \u001b[01;34mexplainer_h2o_sonar_explainers_pd_ice_explainer_PdIceExplainer_39381a2e-a060-43ae-8ce3-50389789a2cb\u001b[00m\n", "│   │   ├── \u001b[01;34mglobal_html_fragment\u001b[00m\n", "│   │   │   ├── \u001b[01;34mtext_html\u001b[00m\n", "│   │   │   │   ├── explanation.html\n", "│   │   │   │   ├── \u001b[01;35mpd-feature-0-class-0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mpd-feature-1-class-0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mpd-feature-2-class-0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mpd-feature-3-class-0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mpd-feature-4-class-0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mpd-feature-5-class-0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mpd-feature-6-class-0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mpd-feature-7-class-0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mpd-feature-8-class-0.png\u001b[00m\n", "│   │   │   │   └── \u001b[01;35mpd-feature-9-class-0.png\u001b[00m\n", "│   │   │   └── text_html.meta\n", "│   │   ├── \u001b[01;34mglobal_partial_dependence\u001b[00m\n", "│   │   │   ├── \u001b[01;34mapplication_json\u001b[00m\n", "│   │   │   │   ├── explanation.json\n", "│   │   │   │   ├── pd_feature_0_class_0.json\n", "│   │   │   │   ├── pd_feature_1_class_0.json\n", "│   │   │   │   ├── pd_feature_2_class_0.json\n", "│   │   │   │   ├── pd_feature_3_class_0.json\n", "│   │   │   │   ├── pd_feature_4_class_0.json\n", "│   │   │   │   ├── pd_feature_5_class_0.json\n", "│   │   │   │   ├── pd_feature_6_class_0.json\n", "│   │   │   │   ├── pd_feature_7_class_0.json\n", "│   │   │   │   ├── pd_feature_8_class_0.json\n", "│   │   │   │   └── pd_feature_9_class_0.json\n", "│   │   │   └── application_json.meta\n", "│   │   ├── \u001b[01;34minsights\u001b[00m\n", "│   │   │   └── insights_and_actions.json\n", "│   │   ├── \u001b[01;34mlocal_individual_conditional_explanation\u001b[00m\n", "│   │   │   ├── \u001b[01;34mapplication_vnd_h2oai_json_datatable_jay\u001b[00m\n", "│   │   │   │   ├── explanation.json\n", "│   │   │   │   ├── ice_feature_0_class_0.jay\n", "│   │   │   │   ├── ice_feature_1_class_0.jay\n", "│   │   │   │   ├── ice_feature_2_class_0.jay\n", "│   │   │   │   ├── ice_feature_3_class_0.jay\n", "│   │   │   │   ├── ice_feature_4_class_0.jay\n", "│   │   │   │   ├── ice_feature_5_class_0.jay\n", "│   │   │   │   ├── ice_feature_6_class_0.jay\n", "│   │   │   │   ├── ice_feature_7_class_0.jay\n", "│   │   │   │   ├── ice_feature_8_class_0.jay\n", "│   │   │   │   ├── ice_feature_9_class_0.jay\n", "│   │   │   │   └── y_hat.jay\n", "│   │   │   └── application_vnd_h2oai_json_datatable_jay.meta\n", "│   │   ├── \u001b[01;34mlog\u001b[00m\n", "│   │   │   └── explainer_run_39381a2e-a060-43ae-8ce3-50389789a2cb.log\n", "│   │   ├── \u001b[01;34mproblems\u001b[00m\n", "│   │   │   └── problems_and_actions.json\n", "│   │   ├── result_descriptor.json\n", "│   │   └── \u001b[01;34mwork\u001b[00m\n", "│   │   ├── h2o_sonar-ice-dai-model-10.jay\n", "│   │   ├── h2o_sonar-ice-dai-model-1.jay\n", "│   │   ├── h2o_sonar-ice-dai-model-2.jay\n", "│   │   ├── h2o_sonar-ice-dai-model-3.jay\n", "│   │   ├── h2o_sonar-ice-dai-model-4.jay\n", "│   │   ├── h2o_sonar-ice-dai-model-5.jay\n", "│   │   ├── h2o_sonar-ice-dai-model-6.jay\n", "│   │   ├── h2o_sonar-ice-dai-model-7.jay\n", "│   │   ├── h2o_sonar-ice-dai-model-8.jay\n", "│   │   ├── h2o_sonar-ice-dai-model-9.jay\n", "│   │   ├── h2o_sonar-ice-dai-model.json\n", "│   │   ├── h2o_sonar-pd-dai-model.json\n", "│   │   └── mli_dataset_y_hat.jay\n", "│   ├── \u001b[01;34mexplainer_h2o_sonar_explainers_residual_dt_surrogate_explainer_ResidualDecisionTreeSurrogateExplainer_aa5cd513-a249-4380-898f-bdea329dc5e4\u001b[00m\n", "│   │   ├── \u001b[01;34mglobal_custom_archive\u001b[00m\n", "│   │   │   ├── \u001b[01;34mapplication_zip\u001b[00m\n", "│   │   │   │   └── \u001b[01;31mexplanation.zip\u001b[00m\n", "│   │   │   └── application_zip.meta\n", "│   │   ├── \u001b[01;34mglobal_decision_tree\u001b[00m\n", "│   │   │   ├── \u001b[01;34mapplication_json\u001b[00m\n", "│   │   │   │   ├── dt_class_0.json\n", "│   │   │   │   └── explanation.json\n", "│   │   │   └── application_json.meta\n", "│   │   ├── \u001b[01;34mglobal_html_fragment\u001b[00m\n", "│   │   │   ├── \u001b[01;34mtext_html\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mdt-class-0.png\u001b[00m\n", "│   │   │   │   └── explanation.html\n", "│   │   │   └── text_html.meta\n", "│   │   ├── \u001b[01;34minsights\u001b[00m\n", "│   │   │   └── insights_and_actions.json\n", "│   │   ├── \u001b[01;34mlocal_decision_tree\u001b[00m\n", "│   │   │   ├── \u001b[01;34mapplication_json\u001b[00m\n", "│   │   │   │   └── explanation.json\n", "│   │   │   └── application_json.meta\n", "│   │   ├── \u001b[01;34mlog\u001b[00m\n", "│   │   │   └── explainer_run_aa5cd513-a249-4380-898f-bdea329dc5e4.log\n", "│   │   ├── \u001b[01;34mproblems\u001b[00m\n", "│   │   │   └── problems_and_actions.json\n", "│   │   ├── result_descriptor.json\n", "│   │   └── \u001b[01;34mwork\u001b[00m\n", "│   │   ├── dt-class-0.dot\n", "│   │   ├── dt-class-0.dot.pdf\n", "│   │   ├── dtModel.json\n", "│   │   ├── dtpaths_frame.bin\n", "│   │   ├── dtPathsFrame.csv\n", "│   │   ├── \u001b[01;31mdtsurr_mojo.zip\u001b[00m\n", "│   │   ├── dtSurrogate.json\n", "│   │   └── \u001b[01;31mdt_surrogate_rules.zip\u001b[00m\n", "│   ├── \u001b[01;34mexplainer_h2o_sonar_explainers_summary_shap_explainer_SummaryShapleyExplainer_f833eb2e-787d-4db3-8a93-a2f5ce69324c\u001b[00m\n", "│   │   ├── \u001b[01;34mglobal_html_fragment\u001b[00m\n", "│   │   │   ├── \u001b[01;34mtext_html\u001b[00m\n", "│   │   │   │   ├── explanation.html\n", "│   │   │   │   ├── \u001b[01;35mfeature_0_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_10_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_11_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_12_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_13_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_14_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_15_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_16_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_17_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_18_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_19_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_1_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_20_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_21_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_22_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_2_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_3_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_4_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_5_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_6_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_7_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_8_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_9_class_0.png\u001b[00m\n", "│   │   │   │   └── \u001b[01;35mshapley-class-0.png\u001b[00m\n", "│   │   │   └── text_html.meta\n", "│   │   ├── \u001b[01;34mglobal_summary_feature_importance\u001b[00m\n", "│   │   │   ├── \u001b[01;34mapplication_json\u001b[00m\n", "│   │   │   │   ├── explanation.json\n", "│   │   │   │   ├── \u001b[01;35mfeature_0_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_10_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_11_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_12_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_13_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_14_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_15_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_16_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_17_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_18_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_19_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_1_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_20_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_21_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_22_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_2_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_3_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_4_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_5_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_6_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_7_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_8_class_0.png\u001b[00m\n", "│   │   │   │   ├── \u001b[01;35mfeature_9_class_0.png\u001b[00m\n", "│   │   │   │   ├── summary_feature_importance_class_0_offset_0.json\n", "│   │   │   │   ├── summary_feature_importance_class_0_offset_1.json\n", "│   │   │   │   └── summary_feature_importance_class_0_offset_2.json\n", "│   │   │   ├── application_json.meta\n", "│   │   │   ├── \u001b[01;34mapplication_vnd_h2oai_json_datatable_jay\u001b[00m\n", "│   │   │   │   ├── explanation.json\n", "│   │   │   │   └── summary_feature_importance_class_0.jay\n", "│   │   │   ├── application_vnd_h2oai_json_datatable_jay.meta\n", "│   │   │   ├── \u001b[01;34mtext_markdown\u001b[00m\n", "│   │   │   │   ├── explanation.md\n", "│   │   │   │   └── \u001b[01;35mshapley-class-0.png\u001b[00m\n", "│   │   │   └── text_markdown.meta\n", "│   │   ├── \u001b[01;34minsights\u001b[00m\n", "│   │   │   └── insights_and_actions.json\n", "│   │   ├── \u001b[01;34mlog\u001b[00m\n", "│   │   │   └── explainer_run_f833eb2e-787d-4db3-8a93-a2f5ce69324c.log\n", "│   │   ├── \u001b[01;34mproblems\u001b[00m\n", "│   │   │   └── problems_and_actions.json\n", "│   │   ├── result_descriptor.json\n", "│   │   └── \u001b[01;34mwork\u001b[00m\n", "│   │   ├── raw_shapley_contribs_class_0.jay\n", "│   │   ├── raw_shapley_contribs_index.json\n", "│   │   ├── report.md\n", "│   │   └── \u001b[01;35mshapley-class-0.png\u001b[00m\n", "│   ├── \u001b[01;34mexplainer_h2o_sonar_explainers_transformed_fi_shapley_explainer_ShapleyMojoTransformedFeatureImportanceExplainer_4b1a325d-ffbc-4e37-9671-954416d46a17\u001b[00m\n", "│   │   ├── \u001b[01;34mglobal_feature_importance\u001b[00m\n", "│   │   │   ├── \u001b[01;34mapplication_json\u001b[00m\n", "│   │   │   │   ├── explanation.json\n", "│   │   │   │   └── feature_importance_class_0.json\n", "│   │   │   ├── application_json.meta\n", "│   │   │   ├── \u001b[01;34mapplication_vnd_h2oai_json_csv\u001b[00m\n", "│   │   │   │   ├── explanation.json\n", "│   │   │   │   └── feature_importance_class_0.csv\n", "│   │   │   ├── application_vnd_h2oai_json_csv.meta\n", "│   │   │   ├── \u001b[01;34mapplication_vnd_h2oai_json_datatable_jay\u001b[00m\n", "│   │   │   │   ├── explanation.json\n", "│   │   │   │   └── feature_importance_class_0.jay\n", "│   │   │   └── application_vnd_h2oai_json_datatable_jay.meta\n", "│   │   ├── \u001b[01;34mglobal_html_fragment\u001b[00m\n", "│   │   │   ├── \u001b[01;34mtext_html\u001b[00m\n", "│   │   │   │   ├── explanation.html\n", "│   │   │   │   └── \u001b[01;35mfi-class-0.png\u001b[00m\n", "│   │   │   └── text_html.meta\n", "│   │   ├── \u001b[01;34minsights\u001b[00m\n", "│   │   │   └── insights_and_actions.json\n", "│   │   ├── \u001b[01;34mlocal_feature_importance\u001b[00m\n", "│   │   │   ├── \u001b[01;34mapplication_vnd_h2oai_json_datatable_jay\u001b[00m\n", "│   │   │   │   ├── explanation.json\n", "│   │   │   │   └── feature_importance_class_0.jay\n", "│   │   │   └── application_vnd_h2oai_json_datatable_jay.meta\n", "│   │   ├── \u001b[01;34mlog\u001b[00m\n", "│   │   │   └── explainer_run_4b1a325d-ffbc-4e37-9671-954416d46a17.log\n", "│   │   ├── \u001b[01;34mproblems\u001b[00m\n", "│   │   │   └── problems_and_actions.json\n", "│   │   ├── result_descriptor.json\n", "│   │   └── \u001b[01;34mwork\u001b[00m\n", "│   │   ├── shapley.bin\n", "│   │   ├── shapley.csv\n", "│   │   └── \u001b[01;31mshapley_formatted.zip\u001b[00m\n", "│   ├── interpretation.html\n", "│   ├── interpretation.json\n", "│   └── \u001b[01;34mtmp\u001b[00m\n", "├── h2o-sonar.html\n", "└── h2o-sonar.log\n", "\n", "172 directories, 641 files\n" ] } ], "source": [ "!tree $results_path" ] }, { "cell_type": "markdown", "id": "8524e3ac-caf6-4844-9cd3-60ae757022e2", "metadata": {}, "source": [ "See `interpretation.html` ^ for interpretation **results**." ] }, { "cell_type": "markdown", "id": "429716d7-774b-4678-b6ed-cd7c6dfed7ac", "metadata": { "tags": [] }, "source": [ "Run new interpretation with **one particular explainer** using command line interface:" ] }, { "cell_type": "code", "execution_count": 6, "id": "07def439-b4ad-4fb9-bc26-a3f553ce6a92", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "/home/user/h/mli/git/h2o-sonar-FLOSS/.venv/lib/python3.11/site-packages/ragas/metrics/__init__.py:1: LangChainDeprecationWarning: As of langchain-core 0.3.0, LangChain uses pydantic v2 internally. The langchain_core.pydantic_v1 module was a compatibility shim for pydantic v1, and should no longer be used. Please update the code to import from Pydantic directly.\n", "\n", "For example, replace imports like: `from langchain_core.pydantic_v1 import BaseModel`\n", "with: `from pydantic import BaseModel`\n", "or the v1 compatibility namespace if you are working in a code base that has not been fully upgraded to pydantic 2 yet. \tfrom pydantic.v1 import BaseModel\n", "\n", " from ragas.metrics._answer_correctness import AnswerCorrectness, answer_correctness\n", "/home/user/h/mli/git/h2o-sonar-FLOSS/.venv/lib/python3.11/site-packages/ragas/metrics/__init__.py:4: LangChainDeprecationWarning: As of langchain-core 0.3.0, LangChain uses pydantic v2 internally. The langchain.pydantic_v1 module was a compatibility shim for pydantic v1, and should no longer be used. Please update the code to import from Pydantic directly.\n", "\n", "For example, replace imports like: `from langchain.pydantic_v1 import BaseModel`\n", "with: `from pydantic import BaseModel`\n", "or the v1 compatibility namespace if you are working in a code base that has not been fully upgraded to pydantic 2 yet. \tfrom pydantic.v1 import BaseModel\n", "\n", " from ragas.metrics._context_entities_recall import (\n", "{\n", " \"explainers\": [\n", " \"h2o_sonar.evaluators.agent_sanity_check_evaluator.AgentSanityCheckEvaluator\",\n", " \"h2o_sonar.explainers.fi_naive_shapley_explainer.NaiveShapleyMojoFeatureImportanceExplainer\",\n", " \"h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer\",\n", " \"h2o_sonar.explainers.dt_surrogate_explainer.DecisionTreeSurrogateExplainer\",\n", " \"h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer\",\n", " \"h2o_sonar.explainers.dia_explainer.DiaExplainer\",\n", " \"h2o_sonar.explainers.transformed_fi_shapley_explainer.ShapleyMojoTransformedFeatureImportanceExplainer\",\n", " \"h2o_sonar.explainers.residual_dt_surrogate_explainer.ResidualDecisionTreeSurrogateExplainer\",\n", " \"h2o_sonar.explainers.residual_pd_ice_explainer.ResidualPdIceExplainer\",\n", " \"h2o_sonar.explainers.fi_kernel_shap_explainer.KernelShapFeatureImportanceExplainer\",\n", " \"h2o_sonar.explainers.pd_2_features_explainer.PdFor2FeaturesExplainer\",\n", " \"h2o_sonar.explainers.friedman_h_statistic_explainer.FriedmanHStatisticExplainer\",\n", " \"h2o_sonar.explainers.morris_sa_explainer.MorrisSensitivityAnalysisExplainer\",\n", " \"h2o_sonar.explainers.dataset_and_model_insights_explainer.DatasetAndModelInsightsExplainer\",\n", " \"h2o_sonar.explainers.drift_explainer.DriftDetectionExplainer\",\n", " \"h2o_sonar.explainers.adversarial_similarity_explainer.AdversarialSimilarityExplainer\",\n", " \"h2o_sonar.explainers.size_dependency_explainer.SizeDependencyExplainer\",\n", " \"h2o_sonar.explainers.segment_performance_explainer.SegmentPerformanceExplainer\",\n", " \"h2o_sonar.explainers.calibration_score_explainer.CalibrationScoreExplainer\",\n", " \"h2o_sonar.explainers.backtesting_explainer.BacktestingExplainer\",\n", " \"h2o_sonar.evaluators.answer_semantic_similarity_per_sentence_evaluator.AnswerSemanticSimilarityPerSentenceEvaluator\",\n", " \"h2o_sonar.evaluators.rag_ragas_evaluator.RagasEvaluator\",\n", " \"h2o_sonar.evaluators.rag_tokens_presence_evaluator.RagStrStrEvaluator\",\n", " \"h2o_sonar.evaluators.answer_accuracy_evaluator.AnswerAccuracyEvaluator\",\n", " \"h2o_sonar.evaluators.rag_answer_correctness_evaluator.AnswerCorrectnessEvaluator\",\n", " \"h2o_sonar.evaluators.rag_answer_similarity_evaluator.AnswerSemanticSimilarityEvaluator\",\n", " \"h2o_sonar.evaluators.rag_chunk_relevancy_evaluator.ContextChunkRelevancyEvaluator\",\n", " \"h2o_sonar.evaluators.rag_context_relevancy_evaluator.ContextRelevancyEvaluator\",\n", " \"h2o_sonar.evaluators.rag_answer_relevancy_evaluator.AnswerRelevancyEvaluator\",\n", " \"h2o_sonar.evaluators.rag_answer_relevancy_no_judge_evaluator.RagAnswerRelevancyNoJudgeEvaluator\",\n", " \"h2o_sonar.evaluators.rag_context_precision_evaluator.ContextPrecisionEvaluator\",\n", " \"h2o_sonar.evaluators.rag_context_recall_evaluator.ContextRecallEvaluator\",\n", " \"h2o_sonar.evaluators.rag_context_mean_reciprocal_rank_evaluator.MeanReciprocalRankEvaluator\",\n", " \"h2o_sonar.evaluators.rag_faithfulness_evaluator.FaithfulnessEvaluator\",\n", " \"h2o_sonar.evaluators.pii_leakage_evaluator.PiiLeakageEvaluator\",\n", " \"h2o_sonar.evaluators.json_schema_evaluator.JSONSchemaEvaluator\",\n", " \"h2o_sonar.evaluators.sensitive_data_leakage_evaluator.SensitiveDataLeakageEvaluator\",\n", " \"h2o_sonar.evaluators.toxicity_evaluator.ToxicityEvaluator\",\n", " \"h2o_sonar.evaluators.fairness_bias_evaluator.FairnessBiasEvaluator\",\n", " \"h2o_sonar.evaluators.contact_information_byop_evaluator.ContactInformationByopEvaluator\",\n", " \"h2o_sonar.evaluators.language_mismatch_byop_evaluator.LanguageMismatchByopEvaluator\",\n", " \"h2o_sonar.evaluators.looping_detection_evaluator.LoopingDetectionEvaluator\",\n", " \"h2o_sonar.evaluators.parameterizable_byop_evaluator.ParameterizableByopEvaluator\",\n", " \"h2o_sonar.evaluators.perplexity_evaluator.PerplexityEvaluator\",\n", " \"h2o_sonar.evaluators.sexism_byop_evaluator.SexismByopEvaluator\",\n", " \"h2o_sonar.evaluators.stereotype_byop_evaluator.StereotypeByopEvaluator\",\n", " \"h2o_sonar.evaluators.summarization_byop_evaluator.SummarizationByopEvaluator\",\n", " \"h2o_sonar.evaluators.rag_groundedness_evaluator.RagGroundednessEvaluator\",\n", " \"h2o_sonar.evaluators.rag_hallucination_evaluator.RagHallucinationEvaluator\",\n", " \"h2o_sonar.evaluators.bertscore_evaluator.BertscoreEvaluator\",\n", " \"h2o_sonar.evaluators.bleu_evaluator.BleuEvaluator\",\n", " \"h2o_sonar.evaluators.rouge_evaluator.RougeEvaluator\",\n", " \"h2o_sonar.evaluators.self_consistency_evaluator.SelfConsistencyEvaluator\",\n", " \"h2o_sonar.evaluators.classification_evaluator.ClassificationEvaluator\",\n", " \"h2o_sonar.evaluators.gptscore_summary_with_reference_evaluator.GptScoreSummaryWithReferenceEvaluator\",\n", " \"h2o_sonar.evaluators.gptscore_summary_without_reference_evaluator.GptScoreSummaryWithoutReferenceEvaluator\",\n", " \"h2o_sonar.evaluators.gptscore_question_answering_evaluator.GptScoreQuestionAnsweringEvaluator\",\n", " \"h2o_sonar.evaluators.gptscore_machine_translation_evaluator.GptScoreMachineTranslationEvaluator\",\n", " \"h2o_sonar.evaluators.encoding_guardrail_evaluator.EncodingGuardrailEvaluator\",\n", " \"h2o_sonar.evaluators.agentic_fact_check_evaluator.FactCheckAgenticEvaluator\"\n", " ]\n", "}\n" ] } ], "source": [ "!h2o-sonar list explainers" ] }, { "cell_type": "code", "execution_count": null, "id": "64164921-bab9-4857-b8e4-1f2d301a5c26", "metadata": {}, "outputs": [], "source": [ "results_path = f\"/tmp/{time.time()}\"\n", "os.mkdir(results_path)\n", "\n", "!h2o-sonar run interpretation \\\n", " --explainers=h2o_sonar.explainers.summary_shap_explainer.SummaryShapleyExplainer,h2o_sonar.explainers.pd_ice_explainer.PdIceExplainer \\\n", " --dataset={dataset_path} \\\n", " --model={model_path} \\\n", " --target-col={target_column} \\\n", " --results-location={results_path} " ] }, { "cell_type": "code", "execution_count": null, "id": "b3da4e02-ba3b-4610-b1b2-e1515dbb389c", "metadata": {}, "outputs": [], "source": [ "!tree $results_path" ] }, { "cell_type": "code", "execution_count": null, "id": "9585d46c-dffd-40d5-aae8-e7fcc821f52c", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "H2O Sonar", "language": "python", "name": "h2o-sonar" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.11" } }, "nbformat": 4, "nbformat_minor": 5 }