h2o_sonar.lib.container package

Submodules

h2o_sonar.lib.container.explainer_container module

class h2o_sonar.lib.container.explainer_container.AsyncLocalContainer(max_workers: int | None = 3)

Bases: ExplainerContainer

Local container which runs interpretations asynchronously.

OPT_PER_INTERPRETATION_LOGGER: bool = True

TYPE_ID = 'ASYNC_LOCAL_EXPLAINER_CONTAINER'

property executor

gc()

Free system resources:

shutdowns process pool(s)
runs garbage collector
clears temporary files

run_interpretation(dataset: str | Path | ExplainableDataset | Frame | DataFrame, model: str | Path | ExplainableModel | Any, models: list[str | Path | ExplainableModel | Any], target_col: str, explainers: list[str | ExplainerToRun] | None = None, explainer_keywords: list[str] | None = None, validset: str | Path | ExplainableDataset | Frame | DataFrame | None = None, testset: str | Path | ExplainableDataset | Frame | DataFrame | None = None, use_raw_features: bool = True, used_features: list | None = None, weight_col: str = '', prediction_col: str = '', drop_cols: list | None = None, sample_num_rows: int | None = 0, sampler: DatasetSampler | None = None, results_location: str | Path | dict | None = None, results_formats: list[str] | None = None, progress_callback: AbstractProgressCallbackContext | None = None, run_asynchronously: bool = False, run_explainers_in_parallel: bool = False, key: str = '', extra_params: list | None = None) → Interpretation

setup(results_location: str | Any = '', persistence_api: PersistenceApi | None = None, persistence_type: PersistenceType | None = PersistenceType.file_system, logger=None, log_level: int | None = None)

Setup explainer container.

Parameters:

results_locationstr | Any: Optional directory where can container (interpretation, explainers, …) store the data. If not specified, then in-memory store is used and logging to stdout/stderr is used.
persistence_apip10s.PersistenceApi | None: Instance of the persistence API allowing to create various persistence types (like file-system or DB)
persistence_typep10s.Persistence | None: Persistence type to be used by this container - default is the file-system persistence with the root in the current directory.
loggerloggers.SonarLogger | None: Optional custom logger.
log_levelint | None: Interpretation log level.

class h2o_sonar.lib.container.explainer_container.ContainerRegistry(singleton_create_key)

Bases: object

Explainer container factory and registry class is able to instantiate containers using symbolic names.

container_types: dict

containers: dict

create_container(container_type_id): Get instance of the container for given container type.

get_container_types() → list

list_containers() → dict

classmethod register_container_type(container_type: type[ExplainerContainer], container_type_id: str)

classmethod registry()

resolve_container(container: str | ExplainerContainer | None = None)

class h2o_sonar.lib.container.explainer_container.DependencyTreeNode(explainer_class, *children)

Bases: object

add_child(node)

get_node_by_id(eid) → DependencyTreeNode | None

str_lines()

to_list(result: list, queue_result: dict, parent_queue: int = -1) → tuple[list, dict]

class h2o_sonar.lib.container.explainer_container.ExplainerContainer

Bases: ABC

Explainer container interface provides execution runtime independent APIs and functions which enable explainers to run “anywhere” via explainer container implementations for various runtimes.

Explainer gets reference to explainer container on its instantiation. Explainer container in turn contains references to public dataset, model and persistence APIs.

Explainer container can be instantiated with various combinations of mock/real/DB-based/… model/dataset/persistence APIs.

TYPE_ID = ''

delete_interpretation(mli_key: str)

property explainers_registry: Registry of explainers registered within the container.

gc()

Free system resources:

shutdowns process pool(s)
runs garbage collector
clears temporary files

get_explainer(explainer_id) → ExplainerDescriptor

get_explainer_class(explainer_id) → type[Explainer]

get_explainer_job_keys_by_id()

get_explainer_job_statuses(mli_key: str): Get explainer job status for every explainer which was run within the interpretation.

get_explainer_local_result()

get_explainer_log_path()

get_explainer_metadata()

get_explainer_result_path()

get_explainer_snapshot_path()

get_interpretation_params(mli_key: str)

get_interpretation_status(mli_key: str): Did interpretation succeeded and failed?

hot_deploy_explainer(explainer_descriptor: str)

Import custom BYOE recipe explainer class from given Python module. It is expected that the module is installed and/or on PYTHONPATH.

Parameters:

explainer_descriptorstr: Explainer descriptor with format ... for example ....

Returns:

Type[Explainer]: New explainer type.

list_explainers(experiment_types: list[str] | None = None, explanation_scopes: list[str] | None = None, model_meta: ExplainableModelMeta | None = None, keywords: list[str] | None = None, explainer_filter: list[FilterEntry] | None = None, portable: bool = False) → list[ExplainerDescriptor]

List and filter explainers.

Returns:

list[ExplainerDescriptor]: List of explainer descriptors which match the filter.

list_interpretations(username: str = 'h2o-sonar') → list[str]

load_interpretations(interpretation_key: str, username: str = 'h2o-sonar') → Interpretation

logger: SonarLogger

persistence_type: PersistenceType

register_configured_explainers(): Register custom BYOE recipe explainers which are configured in H2O Sonar configuration.

register_explainer(explainer_class: type[Explainer], explainer_id: str = '', extra_params: dict | None = None)

run_interpretation(dataset: str | Path | ExplainableDataset | Frame | DataFrame, model: str | Path | ExplainableModel | Any, models: list[str | Path | ExplainableModel | Any], target_col: str, explainers: list[str | ExplainerToRun] | None = None, explainer_keywords: list[str] | None = None, validset: str | Path | ExplainableDataset | ResourceHandle | Frame | DataFrame | ExplainableDatasetHandle | None = None, testset: str | Path | ExplainableDataset | ResourceHandle | Frame | DataFrame | ExplainableDatasetHandle | None = None, use_raw_features: bool = True, used_features: list | None = None, weight_col: str = '', prediction_col: str = '', drop_cols: list | None = None, sample_num_rows: int | None = 0, sampler: DatasetSampler | None = None, results_location: str | Path | dict | None = None, results_formats: list[str] | None = None, progress_callback: AbstractProgressCallbackContext | None = None, run_asynchronously: bool = False, run_explainers_in_parallel: bool = False, key: str = '', logger: SonarLogger | None = None, extra_params: list | None = None) → Interpretation

Setup explainer container.

Parameters:

results_locationstr | Any: Optional directory where can container (interpretation, explainers, …) store the data. If not specified, then in-memory store is used and logging to stdout/stderr is used.
persistence_apip10s.PersistenceApi | None: Instance of the persistence API allowing to create various persistence types (like file-system or DB)
persistence_typep10s.Persistence | None: Persistence type to be used by this container - default is the file-system persistence with the root in the current directory.
loggerloggers.SonarLogger | None: Optional custom logger.
log_levelint | None: Interpretation log level.

unregister_explainer(explainer_id: str, extra_params: dict | None = None) → str

update_explainer_result()

class h2o_sonar.lib.container.explainer_container.ExplainerExecutor

Bases: ABC

Executor of the interpretation and its explainers.

class h2o_sonar.lib.container.explainer_container.LocalExplainerContainer

Bases: ExplainerContainer

Local explainer container enables explainers to run on local machine.

TYPE_ID = 'LOCAL_EXPLAINER_CONTAINER'

Setup explainer container.

Parameters:

results_locationstr | Any: Optional directory where can container (interpretation, explainers, …) store the data. If not specified, then in-memory store is used and logging to stdout/stderr is used.
persistence_apip10s.PersistenceApi | None: Instance of the persistence API allowing to create various persistence types (like file-system or DB)
persistence_typep10s.Persistence | None: Persistence type to be used by this container - default is the file-system persistence with the root in the current directory.
loggerloggers.SonarLogger | None: Optional custom logger.
log_levelint | None: Interpretation log level.

class h2o_sonar.lib.container.explainer_container.ParallelExplainerExecutor

Bases: ExplainerExecutor

Run interpretation explainers in parallel.

class PriorityQueues

Bases: object

Explainer executor priority queues.

HIGH_PRIORITY = 1

LOW_PRIORITY = 3

MEDIUM_PRIORITY = 2

classmethod ensure_valid_queue_plan(execution_plan: list, queue_plan: dict, container_logger)

In 3 priority queues design, task from a higher priority queue cannot depend on task from a lower priority queue.

Parameters:

execution_planlist: Execution plan which gives sequence of explainers to be executed.
queue_plandict: Queue plan to be verified (and eventually fixed).
container_logger: Container logger.

classmethod get_queue_for_explainer(explainer_keywords: list) → int

Get execution queue (high priority, medium priority, low priority) for explainer.

Parameters:

explainer_keywordslist: Explainer keywords where it requests the queue

Returns:

Priority queue ID.

class h2o_sonar.lib.container.explainer_container.SequentialExplainerExecutor(container: ExplainerContainer, explainers_registry: ExplainerRegistry, log_level, container_logger)

Bases: ExplainerExecutor

Run interpretation explainers sequentially.

This is “for loop style” explainers execution executor - no multithreading, no multiprocessing, no async - explainers are executed sequentially within the same process. This is straightforward, deadlock/race conditions free and system resources considerate approach.

run(interpretation: Interpretation) → Interpretation

username: str

h2o_sonar.lib.container.explainer_container.register_ootb_explainers(container: ExplainerContainer)

h2o_sonar.lib.container package

Submodules

h2o_sonar.lib.container.explainer_container module

Module contents