Library Configuration

Global H2O Eval Studio configuration can be either managed using the Python API or stored to a JSon file whose path can be passed using --config-path parameter when running an interpretation from the command line:

• connections

  • Configuration of connections used by H2O Eval Studio runtime and/or explainers to access required services. Note that certain fields with sensitive data are encrypted using an encryption key which might be either specified in H2O_SONAR_ENCRYPTION_KEY environment variable or using CLI argument or Python API argument.

• licenses

  • Configuration of licenses used by H2O Eval Studio and/or explainers to use a license protected functionality. Note that certain fields with sensitive data are encrypted using an encryption key which might be either specified in H2O_SONAR_ENCRYPTION_KEY environment variable or using CLI argument or Python API argument.

• h2o_host

  • Host address of the H2O-3 server (localhost by default).

• h2o_port

  • Port of the H2O-3 server (12349 by default).

• h2o_auto_start

  • True to automatically start H2O-3 server in case that interpretation will run explainer which needs it. False to be used either when no H2O-3 based explainers are run or when H2O Eval Studio can use H2O-3 server managed by the user.

• h2o_min_mem_size

  • Minimum memory to be used by H2O-3 server started by H2O Eval Studio (2GB by default).

• h2o_max_mem_size

  • Maximum memory to be used by H2O-3 server started by H2O Eval Studio (4GB by default).

• custom_explainers

  • Custom explainers to be loaded and registered on H2O Eval Studio start.

• look_end_feel

  • Interpretation report look and feel.

• per_explainer_logger

  • Whether to create a logger for each explainer, or whether to log everything to one central H2O Eval Studio log.

• create_html_representation

  • Whether to create HTML representations for each explainer (used to build HTML report). For instance, the creation of HTML representations is disabled when H2O Eval Studio is running within Driverless AI as they are not needed.

Connection Configuration

Configuration of a connection has the following fields:

• key

  • Unique connection key. If key is not specified, then it will be automatically generated by H2O Eval Studio. This key is used to identify and/or reference the connection.

• connection_type

  • Connection type - one of: DRIVERLESS_AI, DRIVERLESS_AI_AIEM, DRIVERLESS_STEAM and H2O_3.

• name

  • Custom connection name.

• description

  • Custom connection description.

• server_url

  • Server URL. For example:

    • https://enginemanager.cloud.h2o.ai/ … H2O AIEM hosted Driverless AI.

    • https://steam.cloud.h2o.ai/ … H2O Enterprise Steam hosted Driverless AI.

    • https://host:12345/ … a standalone Driverless AI installation.

    • https://playground.h2ogpte.h2o.ai/ … Enterprise h2oGPT server.

• server_id

  • Server ID used to identify the server in case it does not have a fixed URL. For example:

    • "new-dai-engine-42" … ID assigned by H2O AIEM to Driverless AI.

    • "my-driverless-ai" … custom name given to Driverless AI in H2O Steam.

• auth_server_url

  • Authentication server URL.

• environment_url

  • Environment URL used in the H2O token provider authentication method. For example:

    • https://cloud.h2o.ai/ … H2O.ai cloud.

    • https://cloud-qa.acme.com/ … an on-premise QA cloud.

    • https://cloud-dev.acme.com/ … a on-premise development cloud.

• realm_name

  • Realm name (authentication domain) or workspace in case of Driverless AI hosted by H2O AIEM connection.

• client_id

  • Client ID.

• token

  • A client refresh, access or a secret token:

    • H2O AIEM:

      • Get/generate token: H2O.ai Cloud > User > CLI & API Access > API Token. Use REFRESH_TOKEN as the token_use_type.

    • H2O Enterprise Steam:

      • Get/generate token: H2O.ai Cloud > Enterprise Steam > Configurations > Personal Access Token > Get token. Use REFRESH_TOKEN as the token_use_type.

• token_use_type

  • Client use type like REFRESH_TOKEN, ACCESS_TOKEN or SECRET.

• username

  • Username to use in case of username/password authentication.

• password

  • Password to use in case of username/password authentication.

H2O AIEM hosted Driverless AI server connection example:

{
   "key": "462399f5-dd3b-41ee-89e4-e8acb25fc654",
   "connection_type": "DRIVERLESS_AI_AIEM",
   "name": "My H2O AIEM hosted Driverless AI ",
   "description": "Driverless AI server hosted by H2O Enterprise AIEM.",
   "auth_server_url": "",
   "environment_url": "https://cloud.h2o.ai",
   "server_url": "https://enginemanager.cloud.h2o.ai/",
   "server_id": "new-dai-engine-42",
   "realm_name": "",
   "client_id": "",
   "token": {
      "encrypted": "gAAAAABlE...91VeGYkIKUmwghe18f8wAoSTp0W"
   },
   "token_use_type": "REFRESH_TOKEN",
   "username": "firstname.lastname@h2o.ai",
   "password": {
      "encrypted": ""
   }
}

H2O Enterprise Steam hosted Driverless AI server connection example:

{
   "key": "4a26db24-af84-4c95-adea-fb4cb9364954",
   "connection_type": "DRIVERLESS_AI_STEAM",
   "name": "My H2O Steam hosted Driverless AI ",
   "description": "Driverless AI server hosted by H2O Enterprise Steam.",
   "auth_server_url": "",
   "environment_url": "",
   "server_url": "https://steam.cloud.h2o.ai/",
   "server_id": "model-validation",
   "realm_name": "",
   "client_id": "",
   "token": {
      "encrypted": "gAAAA...teqm8ErNuwBnxJuISI"
   },
   "token_use_type": "ACCESS_TOKEN",
   "username": "firstname.lastname@h2o.ai",
   "password": {
      "encrypted": ""
   }
}

Local Driverless AI server connection example:

{
   "key": "4b46c18a-e596-4fa8-8638-055dfd52d37c",
   "connection_type": "DRIVERLESS_AI",
   "name": "Local Driverless AI server",
   "description": "Driverless AI server running on the localhost.",
   "auth_server_url": "",
   "environment_url": "",
   "server_url": "http://localhost:12345",
   "server_id": "",
   "realm_name": "",
   "client_id": "",
   "token": {
      "encrypted": ""
   },
   "token_use_type": "",
   "username": "h2oai",
   "password": {
      "encrypted": "gAAAAABlEuFMcWiaK-E3Ccq...m8nUzRZ7UvwRag3DdMrg=="
   }
}

Enterprise h2oGPT server connection example:

{
   "key": "4b46c18a-e596-4fa8-8638-055dfd52d37c",
   "connection_type": "H2O_GPT_E",
   "name": "H2O GPT Enterprise",
   "description": "H2O GPT Enterprise server.",
   "auth_server_url": "",
   "environment_url": "",
   "server_url": "https://playground.h2ogpte.h2o.ai",
   "server_id": "",
   "realm_name": "",
   "client_id": "",
   "token": {
      "encrypted": "gAAAAABlEuFMcWiaK-E3Ccq...m8nUzRZ7UvwRag3DdMrg=="
   },
   "token_use_type": "API_KEY",
   "username": "",
   "password": {
      "encrypted": ""
   }
}

License Configuration

Configuration of a license has the following fields:

• key

  • Unique connection key. If the key is not specified, then it will be automatically generated.

• product

  • Product name of the product whose license is being configured.

• name

  • License name.

• description

  • License description.

• license

  • License key. In case that both license and license file are specified, then the license (key) is used.

• license_file

  • License file path.

Driverless AI license configuration example:

{
   "key": "4b46c18a-e596-4fa8-8638-055dfd52d37c",
   "name": "Driverless AI license",
   "description": "This is my Driverless AI license.",
   "product": "DRIVERLESS_AI",
   "license": {
      "encrypted": "gAAAAABkTfy24XO8SA-F7L...tSIkOikmPil66mVRIJ_MRK97ZIiPsw=="
   },
   "license_file": "/home/user/.driverlessai/license.sig"
}

Library Configuration Examples

H2O Eval Studio configuration example with encrypted fields:

{
    "h2o_host": "192.168.1.210",
    "h2o_port": 57561,
    "h2o_auto_start": true,
    "h2o_min_mem_size": "2G",
    "h2o_max_mem_size": "4G",
    "custom_explainers": [
          "example_morris_sa_explainer::ExampleMorrisSensitivityAnalysisExplainer"
    ],
    "look_and_feel": "h2o_sonar",
    "per_explainer_logger": true,
    "create_html_representations": true,
    "connections": [
        {
           "key": "4b46c18a-e596-4fa8-8638-055dfd52d37c",
           "connection_type": "DRIVERLESS_AI",
           "name": "Local Driverless AI server",
           "description": "Driverless AI server running on the localhost.",
           "auth_server_url": "",
           "environment_url": "",
           "server_url": "http://localhost:12345",
           "server_id": "",
           "realm_name": "",
           "client_id": "",
           "token": {
              "encrypted": ""
           },
           "token_use_type": "",
           "username": "sonaruser",
           "password": {
              "encrypted": "gAAAAABlEuFMcWiaK-E3Ccq...m8nUzRZ7UvwRag3DdMrg=="
           }
    ],
    "licenses": [
        {
            "key": "my-license-key",
            "product": "DRIVERLESS_AI",
            "name": "Driverless AI license",
            "description": "This is my Driverless AI license.",
            "license": {
                "encrypted": "gAAAAABkTfy24XO8SA-F7L...tSIkOikmPil66mVRIJ_MRK97ZIiPsw=="
            },
            "license_file": "/home/user/.driverlessai/license.sig"
        }
    ]
}

H2O Eval Studio configuration example with decrypted fields:

{
    "h2o_host": "192.168.1.210",
    "h2o_port": 57561,
    "h2o_auto_start": true,
    "h2o_min_mem_size": "2G",
    "h2o_max_mem_size": "4G",
    "custom_explainers": [
          "example_morris_sa_explainer::ExampleMorrisSensitivityAnalysisExplainer"
    ],
    "look_and_feel": "h2o_sonar",
    "per_explainer_logger": true,
    "create_html_representations": true,
    "connections": [
        {
           "key": "4b46c18a-e596-4fa8-8638-055dfd52d37c",
           "connection_type": "DRIVERLESS_AI",
           "name": "Local Driverless AI server",
           "description": "Driverless AI server running on the localhost.",
           "auth_server_url": "",
           "environment_url": "",
           "server_url": "http://localhost:12345",
           "server_id": "",
           "realm_name": "",
           "client_id": "",
           "token": {
              "encrypted": ""
           },
           "token_use_type": "",
           "username": "sonaruser",
           "password": "my-password"
        }
    ],
    "licenses": [
        {
            "key": "my-license-key",
            "product": "DRIVERLESS_AI",
            "name": "Driverless AI license",
            "description": "This is my Driverless AI license.",
            "license": "Ngkflu...MAo=",
            "license_file": "/home/user/.driverlessai/license.sig"
        }
    ]
}

See also:

• h2o_sonar.config module

Library Configuration Python API

H2O Eval Studio provides API allowing to manage its configuration programmatically. It is typically used to manage configuration items which include sensitive data and therefore are encrypted.

Example: create a new configuration instance and save it for later use.

my_aiem_connection = config.ConnectionConfig(
    connection_type=h2o_sonar_config.ConnectionConfigType.DRIVERLESS_AI_AIEM.name,
    name="My H2O AIEM hosted Driverless AI ",
    description="Driverless AI server hosted by H2O Enterprise AIEM.",
    server_url="https://enginemanager.cloud.h2o.ai/",
    server_id="new-dai-engine-42",
    environment_url="https://cloud.h2o.ai",
    username="firstname.lastname@h2o.ai",
    token=os.getenv(KEY_AIEM_REFRESH_TOKEN_CLOUD),
    token_use_type=h2o_sonar_config.TokenUseType.REFRESH_TOKEN.name,
)

my_steam_connection = config.ConnectionConfig(
    connection_type=h2o_sonar_config.ConnectionConfigType.DRIVERLESS_AI_STEAM.name,
    name="My H2O Steam hosted Driverless AI ",
    description="Driverless AI server hosted by H2O Enterprise Steam.",
    server_id="model-validation",
    server_url="https://steam.cloud.h2o.ai/",
    username="firstname.lastname@h2o.ai",
    token=os.getenv(KEY_STEAM_TOKEN_CLOUD),
    token_use_type=h2o_sonar_config.TokenUseType.ACCESS_TOKEN.name,
)

my_dai_connection = config.ConnectionConfig(
    connection_type=h2o_sonar_config.ConnectionConfigType.DRIVERLESS_AI.name,
    name="My Driverless AI server",
    description="Driverless AI server running on the localhost.",
    server_url="http://host:12345",
    username="h2oai",
    password=os.getenv(KEY_DAI_PASSWD),
)

my_h2ogpt_enterprise_connection = config.ConnectionConfig(
    connection_type=h2o_sonar_config.ConnectionConfigType.H2O_GPT_E.name,
    name="H2O GPT Enterprise",
    description="H2O GPT Enterprise server.",
    server_url="https://playground.h2ogpte.h2o.ai",
    token=os.getenv(KEY_H2OGPT_ENTERPRISE_API_KEY),
    token_use_type=h2o_sonar_config.TokenUseType.API_KEY.name,
)

my_license = config.LicenseConfig(
    key="my-license-key",
    product=config.ProductLicenseConfig.DRIVERLESS_AI.name,
    name="My license name",
    description="My license description.",
    license=license_value,
    license_file="/tmp/license-of-my-product.txt",
)

sonar_cfg = config.H2oSonarConfig(
    connections=[my_aiem_connection, my_steam_connection, my_dai_connection],
    licenses=[my_license],
)

sonar_cfg.save(config_path=str(json_config_path), encryption_key=encryption_key)

See also:

• h2o_sonar.interpret::add_config_item()

• h2o_sonar.interpret::get_config()

Library Configuration CLI API

H2O Eval Studio provides API allowing to manage its configuration programmatically. It is typically used to manage configuration items which include sensitive data and therefore are encrypted.

Example: create a new configuration instance and save it for later use.

$ h2o-sonar add config
--config-type CONNECTION
--config-value '{
     "connection_type": "DRIVERLESS_AI",
     "name": "Local Driverless AI server",
     "id": "local-driverless-ai-server",
     "description": "Driverless AI server running on the localhost.",
     "auth_server_url": "",
     "server_url": "http://localhost:12345",
     "username": "h2oai",
     "password": "M1P3ssw0rd"
 }'
--config-path "./h2o-sonar-config.json"
--encryption-key "secret-key-to-decrypt-encrypted-config-fields"

Where:

• --config-type

  • … is one of CONNECTION or LICENSE.