h2o_sonar package
Subpackages
- h2o_sonar.explainers package
- Submodules
- h2o_sonar.explainers.adversarial_similarity_explainer module
AdversarialSimilarityExplainer
AdversarialSimilarityExplainer.CLASS_ONE_AND_ONLY
AdversarialSimilarityExplainer.DEFAULT_DROP_COLS
AdversarialSimilarityExplainer.DEFAULT_SHAPLEY_VALUES
AdversarialSimilarityExplainer.PARAM_DROP_COLS
AdversarialSimilarityExplainer.PARAM_SHAPLEY_VALUES
AdversarialSimilarityExplainer.PARAM_WORKER
AdversarialSimilarityExplainer.PLOT_TITLE
AdversarialSimilarityExplainer.Result
AdversarialSimilarityExplainer.check_compatibility()
AdversarialSimilarityExplainer.explain()
AdversarialSimilarityExplainer.get_result()
AdversarialSimilarityExplainer.setup()
- h2o_sonar.explainers.backtesting_explainer module
BacktestingExplainer
BacktestingExplainer.DEFAULT_CUSTOM_DATES
BacktestingExplainer.DEFAULT_FORECAST_PERIOD_UNIT
BacktestingExplainer.DEFAULT_NUMBER_OF_FORECAST_PERIODS
BacktestingExplainer.DEFAULT_NUMBER_OF_SPLITS
BacktestingExplainer.DEFAULT_NUMBER_OF_TRAINING_PERIODS
BacktestingExplainer.DEFAULT_SPLIT_TYPE
BacktestingExplainer.DEFAULT_TIME_COLUMN
BacktestingExplainer.DEFAULT_TRAINING_PERIOD_UNIT
BacktestingExplainer.OPT_SPLIT_TYPE_AUTO
BacktestingExplainer.OPT_SPLIT_TYPE_CUSTOM
BacktestingExplainer.PARAM_CUSTOM_DATES
BacktestingExplainer.PARAM_FORECAST_PERIOD_UNIT
BacktestingExplainer.PARAM_NUMBER_OF_FORECAST_PERIODS
BacktestingExplainer.PARAM_NUMBER_OF_SPLITS
BacktestingExplainer.PARAM_NUMBER_OF_TRAINING_PERIODS
BacktestingExplainer.PARAM_PLOT_TYPE
BacktestingExplainer.PARAM_SPLIT_TYPE
BacktestingExplainer.PARAM_TIME_COLUMN
BacktestingExplainer.PARAM_TRAINING_PERIOD_UNIT
BacktestingExplainer.PARAM_WORKER
BacktestingExplainer.check_compatibility()
BacktestingExplainer.explain()
BacktestingExplainer.get_result()
BacktestingExplainer.setup()
- h2o_sonar.explainers.calibration_score_explainer module
CalibrationScoreExplainer
CalibrationScoreExplainer.COL_PROB_PRED
CalibrationScoreExplainer.COL_PROB_TRUE
CalibrationScoreExplainer.COL_SCORE
CalibrationScoreExplainer.COL_TARGET
CalibrationScoreExplainer.DEFAULT_BIN_STRATEGY
CalibrationScoreExplainer.DEFAULT_NUMBER_OF_BINS
CalibrationScoreExplainer.KEY_BRIER_SCORE
CalibrationScoreExplainer.KEY_CALIBRATION_CURVE
CalibrationScoreExplainer.KEY_CLASSES_LABELS
CalibrationScoreExplainer.KEY_CLASSES_LEGENDS
CalibrationScoreExplainer.KEY_DATA
CalibrationScoreExplainer.KEY_PLOTS_PATHS
CalibrationScoreExplainer.OPT_BIN_STRATEGY_QUANTILE
CalibrationScoreExplainer.OPT_BIN_STRATEGY_UNIFORM
CalibrationScoreExplainer.PARAM_BIN_STRATEGY
CalibrationScoreExplainer.PARAM_NUMBER_OF_BINS
CalibrationScoreExplainer.PARAM_WORKER
CalibrationScoreExplainer.RESULT_FILE_JSON
CalibrationScoreExplainer.Result
CalibrationScoreExplainer.check_compatibility()
CalibrationScoreExplainer.explain()
CalibrationScoreExplainer.get_result()
CalibrationScoreExplainer.normalize_to_gom()
CalibrationScoreExplainer.setup()
- h2o_sonar.explainers.dataset_and_model_insights_explainer module
- h2o_sonar.explainers.dia_explainer module
DiaArgs
DiaExplainer
DiaExplainer.PARAM_CUT_OFF
DiaExplainer.PARAM_FAST_APPROX
DiaExplainer.PARAM_FEATURES
DiaExplainer.PARAM_FEATURE_NAME
DiaExplainer.PARAM_FEATURE_SUMMARIES
DiaExplainer.PARAM_MAXIMIZE_METRIC
DiaExplainer.PARAM_MAX_CARD
DiaExplainer.PARAM_MIN_CARD
DiaExplainer.PARAM_NAME
DiaExplainer.PARAM_NUM_CARD
DiaExplainer.PARAM_SAMPLE_SIZE
DiaExplainer.PARAM_USE_HOLDOUT_PREDS
DiaExplainer.check_compatibility()
DiaExplainer.explain()
DiaExplainer.get_entry_constants()
DiaExplainer.get_max_metric()
DiaExplainer.get_result()
DiaExplainer.is_enabled()
DiaExplainer.setup()
- h2o_sonar.explainers.drift_explainer module
DriftDetectionExplainer
DriftDetectionExplainer.DEFAULT_DRIFT_THRESHOLD
DriftDetectionExplainer.DEFAULT_DROP_COLS
DriftDetectionExplainer.PARAM_DRIFT_THRESHOLD
DriftDetectionExplainer.PARAM_DROP_COLS
DriftDetectionExplainer.PARAM_WORKER
DriftDetectionExplainer.check_compatibility()
DriftDetectionExplainer.explain()
DriftDetectionExplainer.get_result()
DriftDetectionExplainer.setup()
- h2o_sonar.explainers.dt_surrogate_explainer module
DecisionTreeConstants
DecisionTreeConstants.CAT_ENCODING_DICT
DecisionTreeConstants.CAT_ENCODING_LIST
DecisionTreeConstants.COLUMN_DAI_PREDICT
DecisionTreeConstants.COLUMN_DT_PATH
DecisionTreeConstants.COLUMN_MODEL_PRED
DecisionTreeConstants.COLUMN_ORIG_PRED
DecisionTreeConstants.DEFAULT_NFOLDS
DecisionTreeConstants.DEFAULT_TREE_DEPTH
DecisionTreeConstants.DIR_DT_SURROGATE
DecisionTreeConstants.ENC_AUTO
DecisionTreeConstants.ENC_ENUM_LTD
DecisionTreeConstants.ENC_LE
DecisionTreeConstants.ENC_ONE_HOT
DecisionTreeConstants.ENC_SORT
DecisionTreeConstants.FILE_DEFAULT_DETAILS
DecisionTreeConstants.FILE_DEFAULT_TREE
DecisionTreeConstants.FILE_DRF_VAR_IMP
DecisionTreeConstants.FILE_METRICS_DT
DecisionTreeConstants.FILE_METRICS_DT_MULTI_PREFIX
DecisionTreeConstants.FILE_WORK_DT
DecisionTreeConstants.FILE_WORK_DT_MULTI_PREFIX
DecisionTreeConstants.H2O_ENCODING_NAMES
DecisionTreeConstants.H2O_ENC_AUTO
DecisionTreeConstants.H2O_ENC_ENUM_LTD
DecisionTreeConstants.H2O_ENC_LE
DecisionTreeConstants.H2O_ENC_ONE_HOT
DecisionTreeConstants.H2O_ENC_SORT
DecisionTreeConstants.KEY_LABELS_MAP
DecisionTreeConstants.SEED
DecisionTreeSurrogateExplainer
DecisionTreeSurrogateExplainer.PARAM_CAT_ENCODING
DecisionTreeSurrogateExplainer.PARAM_DEBUG_RESIDUALS
DecisionTreeSurrogateExplainer.PARAM_DEBUG_RESIDUALS_CLASS
DecisionTreeSurrogateExplainer.PARAM_DT_DEPTH
DecisionTreeSurrogateExplainer.PARAM_NFOLDS
DecisionTreeSurrogateExplainer.PARAM_QBIN_COLS
DecisionTreeSurrogateExplainer.PARAM_QBIN_COUNT
DecisionTreeSurrogateExplainer.check_compatibility()
DecisionTreeSurrogateExplainer.explain()
DecisionTreeSurrogateExplainer.explain_local()
DecisionTreeSurrogateExplainer.get_result()
DecisionTreeSurrogateExplainer.is_enabled()
DecisionTreeSurrogateExplainer.setup()
- h2o_sonar.explainers.fi_kernel_shap_explainer module
KernelShapFeatureImportanceExplainer
KernelShapFeatureImportanceExplainer.OPT_BIN_1_CLASS
KernelShapFeatureImportanceExplainer.PARAM_FAST_APPROX
KernelShapFeatureImportanceExplainer.PARAM_L1
KernelShapFeatureImportanceExplainer.PARAM_LEAKAGE_WARN_THRESHOLD
KernelShapFeatureImportanceExplainer.PARAM_MAXRUNTIME
KernelShapFeatureImportanceExplainer.PARAM_NSAMPLE
KernelShapFeatureImportanceExplainer.check_compatibility()
KernelShapFeatureImportanceExplainer.explain()
KernelShapFeatureImportanceExplainer.explain_problems()
KernelShapFeatureImportanceExplainer.get_result()
KernelShapFeatureImportanceExplainer.setup()
- h2o_sonar.explainers.fi_naive_shapley_explainer module
NaiveShapleyMojoFeatureImportanceExplainer
NaiveShapleyMojoFeatureImportanceExplainer.DEFAULT_FAST_APPROX
NaiveShapleyMojoFeatureImportanceExplainer.OPT_BIN_1_CLASS
NaiveShapleyMojoFeatureImportanceExplainer.PARAM_FAST_APPROX
NaiveShapleyMojoFeatureImportanceExplainer.PARAM_LEAKAGE_WARN_THRESHOLD
NaiveShapleyMojoFeatureImportanceExplainer.PARAM_SAMPLE_SIZE
NaiveShapleyMojoFeatureImportanceExplainer.PREFIX_CONTRIB
NaiveShapleyMojoFeatureImportanceExplainer.check_compatibility()
NaiveShapleyMojoFeatureImportanceExplainer.explain()
NaiveShapleyMojoFeatureImportanceExplainer.explain_local()
NaiveShapleyMojoFeatureImportanceExplainer.explain_problems()
NaiveShapleyMojoFeatureImportanceExplainer.get_result()
NaiveShapleyMojoFeatureImportanceExplainer.setup()
- h2o_sonar.explainers.friedman_h_statistic_explainer module
- h2o_sonar.explainers.morris_sa_explainer module
- h2o_sonar.explainers.pd_2_features_explainer module
PdFor2FeaturesArgs
PdFor2FeaturesExplainer
PdFor2FeaturesExplainer.FILE_Y_HAT
PdFor2FeaturesExplainer.GRID_RESOLUTION
PdFor2FeaturesExplainer.MAX_FEATURES
PdFor2FeaturesExplainer.OPT_ICE_1_FRAME_ENABLED
PdFor2FeaturesExplainer.PARAM_FEATURES
PdFor2FeaturesExplainer.PARAM_GRID_RESOLUTION
PdFor2FeaturesExplainer.PARAM_MAX_FEATURES
PdFor2FeaturesExplainer.PARAM_OOR_GRID_RESOLUTION
PdFor2FeaturesExplainer.PARAM_PLOT_TYPE
PdFor2FeaturesExplainer.PARAM_QTILE_BINS
PdFor2FeaturesExplainer.PARAM_QTILE_GRID_RESOLUTION
PdFor2FeaturesExplainer.PARAM_SAMPLE_SIZE
PdFor2FeaturesExplainer.PROGRESS_MAX
PdFor2FeaturesExplainer.PROGRESS_MIN
PdFor2FeaturesExplainer.SAMPLE_SIZE
PdFor2FeaturesExplainer.check_compatibility()
PdFor2FeaturesExplainer.explain()
PdFor2FeaturesExplainer.get_result()
PdFor2FeaturesExplainer.normalize()
PdFor2FeaturesExplainer.setup()
- h2o_sonar.explainers.pd_ice_explainer module
PdIceArgs
PdIceExplainer
PdIceExplainer.FILE_ICE_JSON
PdIceExplainer.FILE_PD_JSON
PdIceExplainer.FILE_Y_HAT
PdIceExplainer.GRID_RESOLUTION
PdIceExplainer.KEY_BINS
PdIceExplainer.KEY_LABELS_MAP
PdIceExplainer.MAX_FEATURES
PdIceExplainer.NUMCAT_NUM_CHART
PdIceExplainer.NUMCAT_THRESHOLD
PdIceExplainer.OPT_ICE_1_FRAME_ENABLED
PdIceExplainer.PARAM_CENTER
PdIceExplainer.PARAM_DEBUG_RESIDUALS
PdIceExplainer.PARAM_FEATURES
PdIceExplainer.PARAM_GRID_RESOLUTION
PdIceExplainer.PARAM_HISTOGRAMS
PdIceExplainer.PARAM_MAX_FEATURES
PdIceExplainer.PARAM_NUMCAT_NUM_CHART
PdIceExplainer.PARAM_NUMCAT_THRESHOLD
PdIceExplainer.PARAM_OOR_GRID_RESOLUTION
PdIceExplainer.PARAM_QTILE_BINS
PdIceExplainer.PARAM_QTILE_GRID_RESOLUTION
PdIceExplainer.PARAM_SAMPLE_SIZE
PdIceExplainer.PARAM_SORT_BINS
PdIceExplainer.PROGRESS_MAX
PdIceExplainer.PROGRESS_MIN
PdIceExplainer.SAMPLE_SIZE
PdIceExplainer.UPDATE_PARAM_NUMCAT_OVERRIDE
PdIceExplainer.UPDATE_SCOPE_NUMCAT
PdIceExplainer.UPDATE_TYPE_ADD_FEATURE
PdIceExplainer.UPDATE_TYPE_ADD_NUMCAT
PdIceExplainer.check_compatibility()
PdIceExplainer.explain()
PdIceExplainer.explain_global()
PdIceExplainer.explain_local()
PdIceExplainer.get_result()
PdIceExplainer.normalize_data()
PdIceExplainer.setup()
- h2o_sonar.explainers.residual_dt_surrogate_explainer module
ResidualDecisionTreeSurrogateExplainer
ResidualDecisionTreeSurrogateExplainer.PARAM_CAT_ENCODING
ResidualDecisionTreeSurrogateExplainer.PARAM_DEBUG_RESIDUALS
ResidualDecisionTreeSurrogateExplainer.PARAM_DEBUG_RESIDUALS_CLASS
ResidualDecisionTreeSurrogateExplainer.PARAM_DT_DEPTH
ResidualDecisionTreeSurrogateExplainer.PARAM_NFOLDS
ResidualDecisionTreeSurrogateExplainer.PARAM_QBIN_COLS
ResidualDecisionTreeSurrogateExplainer.PARAM_QBIN_COUNT
ResidualDecisionTreeSurrogateExplainer.check_compatibility()
ResidualDecisionTreeSurrogateExplainer.explain()
ResidualDecisionTreeSurrogateExplainer.explain_problems()
ResidualDecisionTreeSurrogateExplainer.get_result()
ResidualDecisionTreeSurrogateExplainer.is_enabled()
ResidualDecisionTreeSurrogateExplainer.setup()
- h2o_sonar.explainers.residual_pd_ice_explainer module
ResidualPdIceExplainer
ResidualPdIceExplainer.PARAM_CENTER
ResidualPdIceExplainer.PARAM_DEBUG_RESIDUALS
ResidualPdIceExplainer.PARAM_FEATURES
ResidualPdIceExplainer.PARAM_GRID_RESOLUTION
ResidualPdIceExplainer.PARAM_HISTOGRAMS
ResidualPdIceExplainer.PARAM_MAX_FEATURES
ResidualPdIceExplainer.PARAM_NUMCAT_NUM_CHART
ResidualPdIceExplainer.PARAM_NUMCAT_THRESHOLD
ResidualPdIceExplainer.PARAM_OOR_GRID_RESOLUTION
ResidualPdIceExplainer.PARAM_QTILE_BINS
ResidualPdIceExplainer.PARAM_QTILE_GRID_RESOLUTION
ResidualPdIceExplainer.PARAM_SAMPLE_SIZE
ResidualPdIceExplainer.PARAM_SORT_BINS
ResidualPdIceExplainer.check_compatibility()
ResidualPdIceExplainer.explain()
ResidualPdIceExplainer.explain_problems()
ResidualPdIceExplainer.get_result()
ResidualPdIceExplainer.setup()
- h2o_sonar.explainers.segment_performance_explainer module
SegmentPerformanceExplainer
SegmentPerformanceExplainer.DEFAULT_DROP_COLS
SegmentPerformanceExplainer.DEFAULT_NUMBER_OF_BINS
SegmentPerformanceExplainer.DEFAULT_PRECISION
SegmentPerformanceExplainer.PARAM_DROP_COLS
SegmentPerformanceExplainer.PARAM_NUMBER_OF_BINS
SegmentPerformanceExplainer.PARAM_PRECISION
SegmentPerformanceExplainer.PARAM_WORKER
SegmentPerformanceExplainer.RESULT_FILE_CSV
SegmentPerformanceExplainer.Result
SegmentPerformanceExplainer.check_compatibility()
SegmentPerformanceExplainer.explain()
SegmentPerformanceExplainer.get_result()
SegmentPerformanceExplainer.normalize_scatter_plot()
SegmentPerformanceExplainer.normalize_to_gom()
SegmentPerformanceExplainer.setup()
- h2o_sonar.explainers.size_dependency_explainer module
SizeDependencyExplainer
SizeDependencyExplainer.DEFAULT_NUMBER_OF_SPLITS
SizeDependencyExplainer.DEFAULT_TIME_COLUMN
SizeDependencyExplainer.DEFAULT_WORKER_CLEANUP
SizeDependencyExplainer.PARAM_NUMBER_OF_SPLITS
SizeDependencyExplainer.PARAM_PLOT_TYPE
SizeDependencyExplainer.PARAM_TIME_COLUMN
SizeDependencyExplainer.PARAM_WORKER
SizeDependencyExplainer.PARAM_WORKER_CLEANUP
SizeDependencyExplainer.check_compatibility()
SizeDependencyExplainer.explain()
SizeDependencyExplainer.get_result()
SizeDependencyExplainer.setup()
- h2o_sonar.explainers.summary_shap_explainer module
PdIceArgs
PdIceExplainer
PdIceExplainer.FILE_ICE_JSON
PdIceExplainer.FILE_PD_JSON
PdIceExplainer.FILE_Y_HAT
PdIceExplainer.GRID_RESOLUTION
PdIceExplainer.KEY_BINS
PdIceExplainer.KEY_LABELS_MAP
PdIceExplainer.MAX_FEATURES
PdIceExplainer.NUMCAT_NUM_CHART
PdIceExplainer.NUMCAT_THRESHOLD
PdIceExplainer.OPT_ICE_1_FRAME_ENABLED
PdIceExplainer.PARAM_CENTER
PdIceExplainer.PARAM_DEBUG_RESIDUALS
PdIceExplainer.PARAM_FEATURES
PdIceExplainer.PARAM_GRID_RESOLUTION
PdIceExplainer.PARAM_HISTOGRAMS
PdIceExplainer.PARAM_MAX_FEATURES
PdIceExplainer.PARAM_NUMCAT_NUM_CHART
PdIceExplainer.PARAM_NUMCAT_THRESHOLD
PdIceExplainer.PARAM_OOR_GRID_RESOLUTION
PdIceExplainer.PARAM_QTILE_BINS
PdIceExplainer.PARAM_QTILE_GRID_RESOLUTION
PdIceExplainer.PARAM_SAMPLE_SIZE
PdIceExplainer.PARAM_SORT_BINS
PdIceExplainer.PROGRESS_MAX
PdIceExplainer.PROGRESS_MIN
PdIceExplainer.SAMPLE_SIZE
PdIceExplainer.UPDATE_PARAM_NUMCAT_OVERRIDE
PdIceExplainer.UPDATE_SCOPE_NUMCAT
PdIceExplainer.UPDATE_TYPE_ADD_FEATURE
PdIceExplainer.UPDATE_TYPE_ADD_NUMCAT
PdIceExplainer.check_compatibility()
PdIceExplainer.dataset_api
PdIceExplainer.dataset_meta
PdIceExplainer.explain()
PdIceExplainer.explain_global()
PdIceExplainer.explain_local()
PdIceExplainer.explainer_deps
PdIceExplainer.explainer_params
PdIceExplainer.explainer_params_as_str
PdIceExplainer.get_result()
PdIceExplainer.is_on_demand_pd
PdIceExplainer.key
PdIceExplainer.log_name
PdIceExplainer.logger
PdIceExplainer.mli_key
PdIceExplainer.model
PdIceExplainer.model_api
PdIceExplainer.model_meta
PdIceExplainer.normalize_data()
PdIceExplainer.params
PdIceExplainer.persistence
PdIceExplainer.problematic_features
PdIceExplainer.setup()
PdIceExplainer.testset_meta
PdIceExplainer.validset_meta
- h2o_sonar.explainers.transformed_fi_shapley_explainer module
ShapleyMojoTransformedFeatureImportanceExplainer
ShapleyMojoTransformedFeatureImportanceExplainer.DEFAULT_FAST_APPROX
ShapleyMojoTransformedFeatureImportanceExplainer.OPT_BIN_1_CLASS
ShapleyMojoTransformedFeatureImportanceExplainer.check_compatibility()
ShapleyMojoTransformedFeatureImportanceExplainer.explain()
ShapleyMojoTransformedFeatureImportanceExplainer.get_result()
ShapleyMojoTransformedFeatureImportanceExplainer.setup()
- h2o_sonar.explainers.adversarial_similarity_explainer module
- Module contents
- Submodules
- h2o_sonar.evaluators package
- Submodules
- h2o_sonar.evaluators.abc_byop_evaluator module
AbcByopEvaluator
AbcByopEvaluator.Classes
AbcByopEvaluator.IDENTIFIER_ACTUAL_OUTPUT
AbcByopEvaluator.IDENTIFIER_CONTEXT
AbcByopEvaluator.IDENTIFIER_EXPECTED_OUTPUT
AbcByopEvaluator.IDENTIFIER_INPUT
AbcByopEvaluator.KEY_ANSWER
AbcByopEvaluator.KEY_ERROR
AbcByopEvaluator.KEY_PARSED_ANSWER
AbcByopEvaluator.KEY_PROMPT
AbcByopEvaluator.PARAM_JUDGE_HOST
AbcByopEvaluator.PARAM_JUDGE_MODEL
AbcByopEvaluator.check_compatibility()
AbcByopEvaluator.evaluate()
AbcByopEvaluator.get_result()
AbcByopEvaluator.judge
AbcByopEvaluator.setup()
- h2o_sonar.evaluators.bleu_evaluator module
- h2o_sonar.evaluators.classification_evaluator module
- h2o_sonar.evaluators.contact_information_byop_evaluator module
- h2o_sonar.evaluators.fairness_bias_evaluator module
- h2o_sonar.evaluators.gptscore_evaluator module
- h2o_sonar.evaluators.gptscore_machine_translation_evaluator module
- h2o_sonar.evaluators.gptscore_question_answering_evaluator module
GptScoreQuestionAnsweringEvaluator
GptScoreQuestionAnsweringEvaluator.METRIC_CORRECTNESS
GptScoreQuestionAnsweringEvaluator.METRIC_ENGAGEMENT
GptScoreQuestionAnsweringEvaluator.METRIC_FLUENCY
GptScoreQuestionAnsweringEvaluator.METRIC_INTEREST
GptScoreQuestionAnsweringEvaluator.METRIC_RELEVANCE
GptScoreQuestionAnsweringEvaluator.METRIC_SEMANTICALLY_APPROPRIATE
GptScoreQuestionAnsweringEvaluator.METRIC_SPECIFIC
GptScoreQuestionAnsweringEvaluator.METRIC_UNDERSTANDABILITY
- h2o_sonar.evaluators.gptscore_summary_without_reference_evaluator module
GptScoreSummaryWithoutReferenceEvaluator
GptScoreSummaryWithoutReferenceEvaluator.METRIC_COHERENCE
GptScoreSummaryWithoutReferenceEvaluator.METRIC_CONSISTENCY
GptScoreSummaryWithoutReferenceEvaluator.METRIC_FACTUALITY
GptScoreSummaryWithoutReferenceEvaluator.METRIC_FLUENCY
GptScoreSummaryWithoutReferenceEvaluator.METRIC_INFORMATIVENESS
GptScoreSummaryWithoutReferenceEvaluator.METRIC_RELEVANCE
GptScoreSummaryWithoutReferenceEvaluator.METRIC_SEMANTIC_COVERAGE
- h2o_sonar.evaluators.gptscore_summary_with_reference_evaluator module
GptScoreSummaryWithReferenceEvaluator
GptScoreSummaryWithReferenceEvaluator.METRIC_COHERENCE
GptScoreSummaryWithReferenceEvaluator.METRIC_FACTUALITY
GptScoreSummaryWithReferenceEvaluator.METRIC_FLUENCY
GptScoreSummaryWithReferenceEvaluator.METRIC_INFORMATIVENESS
GptScoreSummaryWithReferenceEvaluator.METRIC_RELEVANCE
GptScoreSummaryWithReferenceEvaluator.METRIC_SEMANTIC_COVERAGE
- h2o_sonar.evaluators.language_mismatch_byop_evaluator module
- h2o_sonar.evaluators.parameterizable_byop_evaluator module
- h2o_sonar.evaluators.perplexity_evaluator module
- h2o_sonar.evaluators.pii_leakage_evaluator module
PiiLeakageEvaluator
PiiLeakageEvaluator.DEFAULT_EVAL_RC
PiiLeakageEvaluator.PARAM_EVAL_RC
PiiLeakageEvaluator.check_compatibility()
PiiLeakageEvaluator.check_creditcard_leakage()
PiiLeakageEvaluator.check_email_leakage()
PiiLeakageEvaluator.check_ssn_leakage()
PiiLeakageEvaluator.evaluate()
PiiLeakageEvaluator.get_result()
PiiLeakageEvaluator.setup()
- h2o_sonar.evaluators.rag_answer_correctness_evaluator module
- h2o_sonar.evaluators.rag_answer_relevancy_evaluator module
- h2o_sonar.evaluators.rag_answer_relevancy_no_judge_evaluator module
RagAnswerRelevancyNoJudgeEvaluator
RagAnswerRelevancyNoJudgeEvaluator.COL_ACTUAL_OUTPUT
RagAnswerRelevancyNoJudgeEvaluator.COL_CONTEXT
RagAnswerRelevancyNoJudgeEvaluator.COL_EXPECTED_OUTPUT
RagAnswerRelevancyNoJudgeEvaluator.COL_INPUT
RagAnswerRelevancyNoJudgeEvaluator.COL_MODEL
RagAnswerRelevancyNoJudgeEvaluator.COL_SCORE
RagAnswerRelevancyNoJudgeEvaluator.METRIC_ANSWER_RELEVANCY
RagAnswerRelevancyNoJudgeEvaluator.check_compatibility()
RagAnswerRelevancyNoJudgeEvaluator.evaluate()
RagAnswerRelevancyNoJudgeEvaluator.get_result()
RagAnswerRelevancyNoJudgeEvaluator.setup()
RagAnswerRelevancyNoJudgeEvaluator.split_sentences()
- h2o_sonar.evaluators.rag_answer_similarity_evaluator module
- h2o_sonar.evaluators.rag_chunk_relevancy_evaluator module
ContextChunkRelevancyEvaluator
ContextChunkRelevancyEvaluator.COL_ACTUAL_OUTPUT
ContextChunkRelevancyEvaluator.COL_CONTEXT
ContextChunkRelevancyEvaluator.COL_EXPECTED_OUTPUT
ContextChunkRelevancyEvaluator.COL_INPUT
ContextChunkRelevancyEvaluator.COL_MODEL
ContextChunkRelevancyEvaluator.COL_SCORE
ContextChunkRelevancyEvaluator.METRIC_PRECISION_RELEVANCY
ContextChunkRelevancyEvaluator.METRIC_RECALL_RELEVANCY
ContextChunkRelevancyEvaluator.check_compatibility()
ContextChunkRelevancyEvaluator.evaluate()
ContextChunkRelevancyEvaluator.get_result()
ContextChunkRelevancyEvaluator.setup()
ContextChunkRelevancyEvaluator.split_sentences()
- h2o_sonar.evaluators.rag_context_precision_evaluator module
- h2o_sonar.evaluators.rag_context_recall_evaluator module
- h2o_sonar.evaluators.rag_context_relevancy_evaluator module
- h2o_sonar.evaluators.rag_faithfulness_evaluator module
- h2o_sonar.evaluators.rag_groundedness_evaluator module
RagGroundednessEvaluator
RagGroundednessEvaluator.COL_ACTUAL_OUTPUT
RagGroundednessEvaluator.COL_CONTEXT
RagGroundednessEvaluator.COL_EXPECTED_OUTPUT
RagGroundednessEvaluator.COL_INPUT
RagGroundednessEvaluator.COL_MODEL
RagGroundednessEvaluator.COL_SCORE
RagGroundednessEvaluator.METRIC_GROUNDEDNESS
RagGroundednessEvaluator.check_compatibility()
RagGroundednessEvaluator.evaluate()
RagGroundednessEvaluator.get_result()
RagGroundednessEvaluator.setup()
RagGroundednessEvaluator.split_sentences()
- h2o_sonar.evaluators.rag_hallucination_evaluator module
RagHallucinationEvaluator
RagHallucinationEvaluator.COL_ACTUAL_OUTPUT
RagHallucinationEvaluator.COL_CONTEXT
RagHallucinationEvaluator.COL_EXPECTED_OUTPUT
RagHallucinationEvaluator.COL_INPUT
RagHallucinationEvaluator.COL_MODEL
RagHallucinationEvaluator.COL_SCORE
RagHallucinationEvaluator.DEFAULT_METRIC_THRESHOLD
RagHallucinationEvaluator.METRIC_HALLUCINATION
RagHallucinationEvaluator.check_compatibility()
RagHallucinationEvaluator.evaluate()
RagHallucinationEvaluator.get_result()
RagHallucinationEvaluator.setup()
- h2o_sonar.evaluators.rag_ragas_evaluator module
RagasEvaluator
RagasEvaluator.KEY_ANSWER
RagasEvaluator.KEY_CONTEXTS
RagasEvaluator.KEY_GROUND_TRUTHS
RagasEvaluator.KEY_QUESTION
RagasEvaluator.METRIC_ANSWER_CORRECTNESS
RagasEvaluator.METRIC_ANSWER_RELEVANCY
RagasEvaluator.METRIC_ANSWER_SIMILARITY
RagasEvaluator.METRIC_CONTEXT_PRECISION
RagasEvaluator.METRIC_CONTEXT_RECALL
RagasEvaluator.METRIC_CONTEXT_RELEVANCY
RagasEvaluator.METRIC_FAITHFULNESS
RagasEvaluator.METRIC_META_ANSWER_CORRECTNESS
RagasEvaluator.METRIC_META_ANSWER_RELEVANCY
RagasEvaluator.METRIC_META_ANSWER_SIMILARITY
RagasEvaluator.METRIC_META_CONTEXT_PRECISION
RagasEvaluator.METRIC_META_CONTEXT_RECALL
RagasEvaluator.METRIC_META_CONTEXT_RELEVANCY
RagasEvaluator.METRIC_META_FAITHFULNESS
RagasEvaluator.METRIC_META_RAGAS
RagasEvaluator.METRIC_RAGAS
RagasEvaluator.check_compatibility()
RagasEvaluator.eval_custom_metrics()
RagasEvaluator.evaluate()
RagasEvaluator.get_result()
RagasEvaluator.setup()
- h2o_sonar.evaluators.rag_tokens_presence_evaluator module
- h2o_sonar.evaluators.rouge_evaluator module
- h2o_sonar.evaluators.sensitive_data_leakage_evaluator module
- h2o_sonar.evaluators.sexism_byop_evaluator module
- h2o_sonar.evaluators.stereotype_byop_evaluator module
- h2o_sonar.evaluators.summarization_byop_evaluator module
- h2o_sonar.evaluators.summarization_evaluator module
SummarizationEvaluator
SummarizationEvaluator.KEY_COMPLETENESS
SummarizationEvaluator.KEY_FAITHFULNESS_CONV
SummarizationEvaluator.KEY_FAITHFULNESS_ZS
SummarizationEvaluator.calculate_scores()
SummarizationEvaluator.check_compatibility()
SummarizationEvaluator.evaluate()
SummarizationEvaluator.get_result()
SummarizationEvaluator.setup()
SummarizationEvaluator.split_sentences()
SummarizationEvaluator.summac_faith_score1()
SummarizationEvaluator.summac_faith_score2()
SummarizationEvaluator.summary_completeness_batch()
load_summac()
pairwise_distances_wrapper()
segment_calc()
- h2o_sonar.evaluators.toxicity_evaluator module
ToxicityEvaluator
ToxicityEvaluator.DEFAULT_TOXICITY_METRIC_THRESHOLD
ToxicityEvaluator.METRIC_IDENTITY_ATTACK
ToxicityEvaluator.METRIC_INSULT
ToxicityEvaluator.METRIC_OBSCENE
ToxicityEvaluator.METRIC_SEVERE_TOXICITY
ToxicityEvaluator.METRIC_THREAT
ToxicityEvaluator.METRIC_TOXICITY
ToxicityEvaluator.check_compatibility()
ToxicityEvaluator.evaluate()
ToxicityEvaluator.get_result()
ToxicityEvaluator.setup()
- h2o_sonar.evaluators.abc_byop_evaluator module
- Module contents
- Submodules
- h2o_sonar.lib package
- Subpackages
- h2o_sonar.lib.api package
- Submodules
- h2o_sonar.lib.api.commons module
Branding
CommonInterpretationParams
CommonInterpretationParams.PARAM_DATASET
CommonInterpretationParams.PARAM_DROP_COLS
CommonInterpretationParams.PARAM_MODEL
CommonInterpretationParams.PARAM_MODELS
CommonInterpretationParams.PARAM_PREDICTION_COL
CommonInterpretationParams.PARAM_RESULTS_LOCATION
CommonInterpretationParams.PARAM_SAMPLE_NUM_ROWS
CommonInterpretationParams.PARAM_TARGET_COL
CommonInterpretationParams.PARAM_TESTSET
CommonInterpretationParams.PARAM_USED_FEATURES
CommonInterpretationParams.PARAM_USE_RAW_FEATURES
CommonInterpretationParams.PARAM_VALIDSET
CommonInterpretationParams.PARAM_WEIGHT_COL
CommonInterpretationParams.clone()
CommonInterpretationParams.describe_config_item()
CommonInterpretationParams.describe_config_items()
CommonInterpretationParams.dump()
CommonInterpretationParams.load()
CommonInterpretationParams.to_dict()
ConfigItem
EvaluatorParamType
EvaluatorToRun
ExperimentType
ExplainerFilter
ExplainerJobStatus
ExplainerJobStatus.ABORTED_BY_RESTART
ExplainerJobStatus.ABORTED_BY_USER
ExplainerJobStatus.CANCELLED
ExplainerJobStatus.FAILED
ExplainerJobStatus.FINISHED
ExplainerJobStatus.IN_PROGRESS
ExplainerJobStatus.RUNNING
ExplainerJobStatus.SCHEDULED
ExplainerJobStatus.SUCCESS
ExplainerJobStatus.SYNCING
ExplainerJobStatus.TIMED_OUT
ExplainerJobStatus.UNKNOWN
ExplainerJobStatus.from_int()
ExplainerJobStatus.is_job_failed()
ExplainerJobStatus.is_job_finished()
ExplainerJobStatus.is_job_running()
ExplainerJobStatus.to_string()
ExplainerParamKey
ExplainerParamKey.KEY_ALL_EXPLAINERS_PARAMS
ExplainerParamKey.KEY_DATASET
ExplainerParamKey.KEY_DESCR_PATH
ExplainerParamKey.KEY_EXPERIMENT_TYPE
ExplainerParamKey.KEY_E_DEPS
ExplainerParamKey.KEY_E_ID
ExplainerParamKey.KEY_E_JOB_KEY
ExplainerParamKey.KEY_E_PARAMS
ExplainerParamKey.KEY_FEATURES_META
ExplainerParamKey.KEY_I_DATA_PATH
ExplainerParamKey.KEY_KWARGS
ExplainerParamKey.KEY_LEGACY_I_PARAMS
ExplainerParamKey.KEY_MODEL
ExplainerParamKey.KEY_MODEL_TYPE
ExplainerParamKey.KEY_ON_DEMAND
ExplainerParamKey.KEY_ON_DEMAND_MLI_KEY
ExplainerParamKey.KEY_ON_DEMAND_PARAMS
ExplainerParamKey.KEY_PARAMS
ExplainerParamKey.KEY_RUN_KEY
ExplainerParamKey.KEY_TESTSET
ExplainerParamKey.KEY_USER
ExplainerParamKey.KEY_VALIDSET
ExplainerParamKey.KEY_WORKER_NAME
ExplainerParamType
ExplainerToRun
ExplanationScope
FilterEntry
InterpretationParamType
Keyword
KeywordGroup
KeywordGroups
LlmModelHostType
LookAndFeel
LookAndFeel.BLUE_THEME
LookAndFeel.COLORMAP_BLUE_2_RED
LookAndFeel.COLORMAP_WHITE_2_BLACK
LookAndFeel.COLORMAP_YELLOW_2_BLACK
LookAndFeel.COLOR_BLACK
LookAndFeel.COLOR_DAI_GREEN
LookAndFeel.COLOR_H2OAI_YELLOW
LookAndFeel.COLOR_HOT_ORANGE
LookAndFeel.COLOR_MATPLOTLIB_BLUE
LookAndFeel.COLOR_RED
LookAndFeel.COLOR_WHITE
LookAndFeel.DRIVERLESS_AI_THEME
LookAndFeel.FORMAT_HEXA
LookAndFeel.H2O_SONAR_THEME
LookAndFeel.KEY_LF
LookAndFeel.THEME_2_BG_COLOR
LookAndFeel.THEME_2_COLORMAP
LookAndFeel.THEME_2_FG_COLOR
LookAndFeel.THEME_2_LINE_COLOR
LookAndFeel.get_bg_color()
LookAndFeel.get_colormap()
LookAndFeel.get_fg_color()
LookAndFeel.get_line_color()
MetricMeta
MetricMeta.DATA_TYPE_SECONDS
MetricMeta.KEY_DATA_TYPE
MetricMeta.KEY_DESCRIPTION
MetricMeta.KEY_DISPLAY_FORMAT
MetricMeta.KEY_DISPLAY_NAME
MetricMeta.KEY_EXCLUDE
MetricMeta.KEY_HIGHER_IS_BETTER
MetricMeta.KEY_IS_PRIMARY_METRIC
MetricMeta.KEY_KEY
MetricMeta.KEY_PARENT_METRIC
MetricMeta.KEY_THRESHOLD
MetricMeta.KEY_VALUE_ENUM
MetricMeta.KEY_VALUE_RANGE
MetricMeta.copy()
MetricMeta.dump()
MetricMeta.from_dict()
MetricMeta.load()
MetricMeta.to_dict()
MetricMeta.to_md()
MetricsMeta
MetricsMeta.KEY_META
MetricsMeta.add_metric()
MetricsMeta.contains()
MetricsMeta.copy_with_overrides()
MetricsMeta.dump()
MetricsMeta.from_dict()
MetricsMeta.get_metric()
MetricsMeta.get_metric_description()
MetricsMeta.get_metric_keys()
MetricsMeta.get_primary_metric()
MetricsMeta.get_threshold()
MetricsMeta.is_higher_better()
MetricsMeta.is_metric_passed()
MetricsMeta.load()
MetricsMeta.set_threshold()
MetricsMeta.size()
MetricsMeta.to_dict()
MetricsMeta.to_list()
MimeType
MimeType.EXT_CSV
MimeType.EXT_DATATABLE
MimeType.EXT_DOCX
MimeType.EXT_HTML
MimeType.EXT_JPG
MimeType.EXT_JSON
MimeType.EXT_MARKDOWN
MimeType.EXT_PNG
MimeType.EXT_SVG
MimeType.EXT_TEXT
MimeType.EXT_ZIP
MimeType.MIME_CSV
MimeType.MIME_DATATABLE
MimeType.MIME_DOCX
MimeType.MIME_EVALSTUDIO_MARKDOWN
MimeType.MIME_HTML
MimeType.MIME_IMAGE
MimeType.MIME_JPG
MimeType.MIME_JSON
MimeType.MIME_JSON_CSV
MimeType.MIME_JSON_DATATABLE
MimeType.MIME_MARKDOWN
MimeType.MIME_MODEL_PIPELINE
MimeType.MIME_PDF
MimeType.MIME_PNG
MimeType.MIME_SVG
MimeType.MIME_TEXT
MimeType.MIME_ZIP
MimeType.ext_for_mime()
ModelTypeExplanation
Param
ParamType
PerturbationIntensity
PerturbatorToRun
ResourceHandle
ResourceLocatorType
SafeJavaScript
SemVer
UpdateGlobalExplanation
UpdateGlobalExplanation.OPT_CLASS
UpdateGlobalExplanation.OPT_FEATURE
UpdateGlobalExplanation.OPT_INHERIT
UpdateGlobalExplanation.OPT_MERGE
UpdateGlobalExplanation.OPT_REPLACE
UpdateGlobalExplanation.OPT_REQUEST
UpdateGlobalExplanation.PARAMS_SOURCE
UpdateGlobalExplanation.UPDATE_MODE
UpdateGlobalExplanation.UPDATE_SCOPE
add_string_list()
base_pkg()
generate_key()
harmonic_mean()
is_ncname()
is_port_used()
is_valid_key()
- h2o_sonar.lib.api.datasets module
DatasetApi
ExplainableColumnMeta
ExplainableDataset
ExplainableDataset.COL_BIAS
ExplainableDataset.KEY_DATA
ExplainableDataset.KEY_METADATA
ExplainableDataset.data
ExplainableDataset.frame_2_datatable()
ExplainableDataset.frame_2_numpy()
ExplainableDataset.frame_2_pandas()
ExplainableDataset.is_bias_col()
ExplainableDataset.meta
ExplainableDataset.prepare()
ExplainableDataset.sample()
ExplainableDataset.to_dict()
ExplainableDataset.to_json()
ExplainableDataset.transform()
ExplainableDatasetHandle
ExplainableDatasetMeta
ExplainableDatasetMeta.KEY_COLUMNS_CAT
ExplainableDatasetMeta.KEY_COLUMNS_META
ExplainableDatasetMeta.KEY_COLUMNS_NUM
ExplainableDatasetMeta.KEY_COLUMN_NAMES
ExplainableDatasetMeta.KEY_COLUMN_TYPES
ExplainableDatasetMeta.KEY_COLUMN_UNIQUES
ExplainableDatasetMeta.KEY_FILE_NAME
ExplainableDatasetMeta.KEY_FILE_PATH
ExplainableDatasetMeta.KEY_FILE_SIZE
ExplainableDatasetMeta.KEY_MISSING_VALUES
ExplainableDatasetMeta.KEY_ORIGINAL_DATASET_PATH
ExplainableDatasetMeta.KEY_ORIGINAL_DATASET_SAMPLED
ExplainableDatasetMeta.KEY_ORIGINAL_DATASET_SHAPE
ExplainableDatasetMeta.KEY_ORIGINAL_DATASET_SIZE
ExplainableDatasetMeta.KEY_ROW_COUNT
ExplainableDatasetMeta.KEY_SHAPE
ExplainableDatasetMeta.copy()
ExplainableDatasetMeta.get_column_meta()
ExplainableDatasetMeta.has_column()
ExplainableDatasetMeta.is_categorical_column()
ExplainableDatasetMeta.is_numeric_column()
ExplainableDatasetMeta.to_dict()
ExplainableDatasetMeta.to_json()
ExplainableDatasetType
ExplainableDatatableDataset
LlmDataset
LlmDataset.COLUMNS
LlmDataset.COL_ACTUAL_DURATION
LlmDataset.COL_ACTUAL_OUTPUT
LlmDataset.COL_CATEGORIES
LlmDataset.COL_CONTEXT
LlmDataset.COL_CORPUS
LlmDataset.COL_COST
LlmDataset.COL_EXPECTED_OUTPUT
LlmDataset.COL_INPUT
LlmDataset.COL_MODEL_KEY
LlmDataset.COL_OUTPUT_CONDITION
LlmDataset.COL_OUTPUT_CONSTRAINTS
LlmDataset.COL_RELATIONSHIPS
LlmDataset.COL_TEST_KEY
LlmDataset.KEY_ACTUAL_DURATION
LlmDataset.KEY_ACTUAL_OUTPUT
LlmDataset.KEY_CATEGORIES
LlmDataset.KEY_CONTEXT
LlmDataset.KEY_CORPUS
LlmDataset.KEY_COST
LlmDataset.KEY_EXPECTED_OUTPUT
LlmDataset.KEY_INPUT
LlmDataset.KEY_INPUTS
LlmDataset.KEY_KEY
LlmDataset.KEY_MODEL_KEY
LlmDataset.KEY_OUTPUT_CONDITION
LlmDataset.KEY_OUTPUT_CONSTRAINTS
LlmDataset.KEY_RELATIONSHIPS
LlmDataset.KEY_TC_KEY
LlmDataset.KEY_TEST_KEY
LlmDataset.LlmDatasetRow
LlmDataset.add_input()
LlmDataset.from_datatable_dict()
LlmDataset.from_datatable_json_enc_col()
LlmDataset.from_dict()
LlmDataset.load_from_json()
LlmDataset.merge()
LlmDataset.perturb()
LlmDataset.prompts()
LlmDataset.save_as_json()
LlmDataset.shape()
LlmDataset.stats()
LlmDataset.to_datatable()
LlmDataset.to_datatable_dict()
LlmDataset.to_dict()
LlmEvalResults
LlmEvalResults.COL_ACTUAL_OUTPUT_META
LlmEvalResults.KEY_RESULTS
LlmEvalResults.LlmEvalResultRow
LlmEvalResults.add_result()
LlmEvalResults.from_dict()
LlmEvalResults.load_from_json()
LlmEvalResults.prompts()
LlmEvalResults.save_as_json()
LlmEvalResults.shape()
LlmEvalResults.to_datatable()
LlmEvalResults.to_datatable_dict()
LlmEvalResults.to_dict()
LlmEvalResults.to_llm_dataset()
LlmInputRel
LlmInputRelTargetType
LlmInputRelType
LlmPromptCategories
LlmPromptCategories.classification
LlmPromptCategories.coding
LlmPromptCategories.evaluation
LlmPromptCategories.facts
LlmPromptCategories.harm
LlmPromptCategories.knowledge
LlmPromptCategories.math
LlmPromptCategories.planning
LlmPromptCategories.question_answering
LlmPromptCategories.reasoning
LlmPromptCategories.recommendation
LlmPromptCategories.summarization
LlmPromptCategories.troubleshooting
LlmPromptCategories.unknown
LlmPromptCategories.writing
filter_importance_greater_than_zero()
- h2o_sonar.lib.api.explainers module
Explainer
Explainer.ARG_EXPLAINER_PARAMS
Explainer.EXPLAINERS_PURPOSES
Explainer.KEYWORD_COMPLIANCE_TEST
Explainer.KEYWORD_DEFAULT
Explainer.KEYWORD_EVALUATES_LLM
Explainer.KEYWORD_EVALUATES_RAG
Explainer.KEYWORD_EXPLAINS_APPROX_BEHAVIOR
Explainer.KEYWORD_EXPLAINS_DATASET
Explainer.KEYWORD_EXPLAINS_FAIRNESS
Explainer.KEYWORD_EXPLAINS_FEATURE_BEHAVIOR
Explainer.KEYWORD_EXPLAINS_MODEL_DEBUGGING
Explainer.KEYWORD_EXPLAINS_O_FEATURE_IMPORTANCE
Explainer.KEYWORD_EXPLAINS_T_FEATURE_IMPORTANCE
Explainer.KEYWORD_EXPLAINS_UNKNOWN
Explainer.KEYWORD_H2O_MODEL_VALIDATION
Explainer.KEYWORD_H2O_SONAR
Explainer.KEYWORD_IS_FAST
Explainer.KEYWORD_IS_SLOW
Explainer.KEYWORD_LLM
Explainer.KEYWORD_MOCK
Explainer.KEYWORD_NLP
Explainer.KEYWORD_PREFIX_CAPABILITY
Explainer.KEYWORD_PREFIX_EXPLAINS
Explainer.KEYWORD_PROXY
Explainer.KEYWORD_REQUIRES_H2O3
Explainer.KEYWORD_REQUIRES_OPENAI_KEY
Explainer.KEYWORD_RQ_AA
Explainer.KEYWORD_RQ_C
Explainer.KEYWORD_RQ_EA
Explainer.KEYWORD_RQ_J
Explainer.KEYWORD_RQ_P
Explainer.KEYWORD_RQ_RC
Explainer.KEYWORD_TEMPLATE
Explainer.KEYWORD_UNLISTED
Explainer.add_insight()
Explainer.add_problem()
Explainer.as_descriptor()
Explainer.brief_description
Explainer.can_explain()
Explainer.check_compatibility()
Explainer.check_required_modules()
Explainer.class_brief_description()
Explainer.class_description()
Explainer.class_display_name()
Explainer.class_name
Explainer.class_tagline()
Explainer.create_explanation_workdir_archive()
Explainer.dependencies
Explainer.depends_on()
Explainer.description
Explainer.destroy()
Explainer.display_name
Explainer.evaluator_id()
Explainer.exlainer_params_as_dict()
Explainer.expected_custom_class
Explainer.explain()
Explainer.explain_global()
Explainer.explain_insights()
Explainer.explain_local()
Explainer.explain_problems()
Explainer.explainer_id()
Explainer.explainer_version()
Explainer.explains_binary()
Explainer.explains_multiclass()
Explainer.explains_regression()
Explainer.explanations
Explainer.fit()
Explainer.get_explanations()
Explainer.get_result()
Explainer.has_explanation_scopes()
Explainer.has_explanation_types()
Explainer.has_explanations()
Explainer.has_model_type_explanations()
Explainer.is_enabled()
Explainer.is_iid()
Explainer.is_image()
Explainer.is_llm()
Explainer.is_rag()
Explainer.is_time_series()
Explainer.is_unsupervised()
Explainer.keywords
Explainer.load()
Explainer.load_descriptor()
Explainer.metrics_meta()
Explainer.parameters()
Explainer.priority()
Explainer.report_progress()
Explainer.requires_model()
Explainer.requires_predict_method()
Explainer.requires_preloaded_predictor()
Explainer.run_explain()
Explainer.run_explain_global()
Explainer.run_explain_local()
Explainer.run_fit()
Explainer.save()
Explainer.save_descriptor()
Explainer.setup()
Explainer.supports_dataset_locator()
Explainer.supports_model_locator()
Explainer.tagline
Explainer.validate_explanations()
Explainer.working_dir
ExplainerArgs
ExplainerDescriptor
ExplainerDescriptor.KEY_BRIEF_DESCRIPTION
ExplainerDescriptor.KEY_CAN_EXPLAIN
ExplainerDescriptor.KEY_DESCRIPTION
ExplainerDescriptor.KEY_DISPLAY_NAME
ExplainerDescriptor.KEY_EXPLANATIONS
ExplainerDescriptor.KEY_EXPLANATION_SCOPES
ExplainerDescriptor.KEY_ID
ExplainerDescriptor.KEY_KEYWORDS
ExplainerDescriptor.KEY_METRICS_META
ExplainerDescriptor.KEY_MODEL_TYPES
ExplainerDescriptor.KEY_NAME
ExplainerDescriptor.KEY_PARAMETERS
ExplainerDescriptor.KEY_TAGLINE
ExplainerDescriptor.clone()
ExplainerDescriptor.dump()
ExplainerDescriptor.load()
ExplainerParam
ExplainerRegistry
ExplainerResult
OnDemandExplainKey
OnDemandExplainMethod
SurrogateExplainer
- h2o_sonar.lib.api.explanations module
AbcHeatmapExplanation
AutoReportExplanation
CustomArchiveExplanation
DiaExplanation
DurationStatsKey
Explanation
Explanation.DISPLAY_CAT_AUTOREPORT
Explanation.DISPLAY_CAT_COMPLIANCE
Explanation.DISPLAY_CAT_CUSTOM
Explanation.DISPLAY_CAT_DAI_MODEL
Explanation.DISPLAY_CAT_DATA
Explanation.DISPLAY_CAT_EXAMPLE
Explanation.DISPLAY_CAT_LLM
Explanation.DISPLAY_CAT_MOCK
Explanation.DISPLAY_CAT_MODEL
Explanation.DISPLAY_CAT_NLP
Explanation.DISPLAY_CAT_SURROGATES
Explanation.DISPLAY_CAT_SURROGATES_ON_RES
Explanation.DISPLAY_CAT_TEMPLATE
Explanation.add_format()
Explanation.as_class_descriptor()
Explanation.as_descriptor()
Explanation.display_category
Explanation.display_name
Explanation.explainer
Explanation.explanation_scope()
Explanation.explanation_type()
Explanation.format_types
Explanation.get_format()
Explanation.has_local
Explanation.is_global()
Explanation.validate()
ExplanationDescriptor
FlippedPerturbedTestCase
Global3dDataExplanation
GlobalDataFrameExplanation
GlobalDtExplanation
GlobalFeatImpExplanation
GlobalGroupedBarChartExplanation
GlobalHtmlFragmentExplanation
GlobalLinePlotExplanation
GlobalNlpLocoExplanation
GlobalRuleExplanation
GlobalScatterPlotExplanation
GlobalSummaryFeatImpExplanation
IndividualConditionalExplanation
LlmBoolLeaderboardExplanation
LlmBoolLeaderboardExplanation.AdditionalDetails
LlmBoolLeaderboardExplanation.DEFAULT_METRIC_THRESHOLD
LlmBoolLeaderboardExplanation.Failure
LlmBoolLeaderboardExplanation.Failure.actual_output
LlmBoolLeaderboardExplanation.Failure.actual_output_meta
LlmBoolLeaderboardExplanation.Failure.ctx_bytes
LlmBoolLeaderboardExplanation.Failure.ctx_chunks
LlmBoolLeaderboardExplanation.Failure.doc_url
LlmBoolLeaderboardExplanation.Failure.error_message
LlmBoolLeaderboardExplanation.Failure.expected_output
LlmBoolLeaderboardExplanation.Failure.fail_generation
LlmBoolLeaderboardExplanation.Failure.fail_parse
LlmBoolLeaderboardExplanation.Failure.fail_retrieval
LlmBoolLeaderboardExplanation.Failure.input
LlmBoolLeaderboardExplanation.Failure.model_key
LlmBoolLeaderboardExplanation.Failure.output_condition
LlmBoolLeaderboardExplanation.Failure.output_constraints
LlmBoolLeaderboardExplanation.Failure.row_key
LlmBoolLeaderboardExplanation.KEY_INPUT_FAILURES
LlmBoolLeaderboardExplanation.KEY_INPUT_FAILURES_COUNT
LlmBoolLeaderboardExplanation.KEY_INPUT_FAILURES_GENERATION_COUNT
LlmBoolLeaderboardExplanation.KEY_INPUT_FAILURES_PARSE_COUNT
LlmBoolLeaderboardExplanation.KEY_INPUT_FAILURES_RETRIEVAL_COUNT
LlmBoolLeaderboardExplanation.KEY_INPUT_PASSES_COUNT
LlmBoolLeaderboardExplanation.KEY_MODEL_FAILURES
LlmBoolLeaderboardExplanation.KEY_MODEL_FAILURES_COUNT
LlmBoolLeaderboardExplanation.KEY_MODEL_FAILURES_GENERATION_COUNT
LlmBoolLeaderboardExplanation.KEY_MODEL_FAILURES_PARSE_COUNT
LlmBoolLeaderboardExplanation.KEY_MODEL_FAILURES_RETRIEVAL_COUNT
LlmBoolLeaderboardExplanation.KEY_MODEL_PASSES_COUNT
LlmBoolLeaderboardExplanation.KEY_RESULT_CHECK_ERR_MSG
LlmBoolLeaderboardExplanation.KEY_RESULT_CHECK_FAIL
LlmBoolLeaderboardExplanation.KEY_RESULT_CHECK_FAIL_A
LlmBoolLeaderboardExplanation.KEY_RESULT_CHECK_FAIL_P
LlmBoolLeaderboardExplanation.KEY_RESULT_CHECK_FAIL_R
LlmBoolLeaderboardExplanation.KEY_RESULT_CHECK_OK
LlmBoolLeaderboardExplanation.KEY_TOTAL_COST
LlmBoolLeaderboardExplanation.KEY_TOTAL_TIME
LlmBoolLeaderboardExplanation.LEADERBOARD_METRICS_META
LlmBoolLeaderboardExplanation.METRIC_META_MODEL_FAILURES
LlmBoolLeaderboardExplanation.METRIC_META_MODEL_GENERATION_FAILURES
LlmBoolLeaderboardExplanation.METRIC_META_MODEL_PARSE_FAILURES
LlmBoolLeaderboardExplanation.METRIC_META_MODEL_PASSES
LlmBoolLeaderboardExplanation.METRIC_META_MODEL_RETRIEVAL_FAILURES
LlmBoolLeaderboardExplanation.METRIC_MODEL_FAILURES
LlmBoolLeaderboardExplanation.METRIC_MODEL_GENERATION_FAILURES
LlmBoolLeaderboardExplanation.METRIC_MODEL_PARSE_FAILURES
LlmBoolLeaderboardExplanation.METRIC_MODEL_PASSES
LlmBoolLeaderboardExplanation.METRIC_MODEL_RETRIEVAL_FAILURES
LlmBoolLeaderboardExplanation.add_evalstudio_markdown_format()
LlmBoolLeaderboardExplanation.add_failure()
LlmBoolLeaderboardExplanation.add_json_format()
LlmBoolLeaderboardExplanation.add_markdown_format()
LlmBoolLeaderboardExplanation.add_pass()
LlmBoolLeaderboardExplanation.add_total_cost()
LlmBoolLeaderboardExplanation.add_total_time()
LlmBoolLeaderboardExplanation.as_dict()
LlmBoolLeaderboardExplanation.as_evalstudio_markdown()
LlmBoolLeaderboardExplanation.as_html()
LlmBoolLeaderboardExplanation.as_leaderboard_dict()
LlmBoolLeaderboardExplanation.as_markdown()
LlmBoolLeaderboardExplanation.build()
LlmBoolLeaderboardExplanation.check_and_report_negative_cost()
LlmBoolLeaderboardExplanation.evaluation_cost()
LlmBoolLeaderboardExplanation.from_eval_results()
LlmBoolLeaderboardExplanation.get_insights()
LlmBoolLeaderboardExplanation.key_2_rag_type_prefix()
LlmBoolLeaderboardExplanation.sort_models_leaderboard()
LlmBoolLeaderboardExplanation.sort_prompts_by_empty_ctxs()
LlmBoolLeaderboardExplanation.sort_prompts_by_failures()
LlmBoolLeaderboardExplanation.summary_as_markdown()
LlmBoolLeaderboardExplanation.validate()
LlmClassifierLeaderboardExplanation
LlmClassifierLeaderboardExplanation.DEFAULT_METRIC_THRESHOLD
LlmClassifierLeaderboardExplanation.METRIC_ACCURACY
LlmClassifierLeaderboardExplanation.METRIC_F1
LlmClassifierLeaderboardExplanation.METRIC_META_ACCURACY
LlmClassifierLeaderboardExplanation.METRIC_META_F1
LlmClassifierLeaderboardExplanation.METRIC_META_PRECISION
LlmClassifierLeaderboardExplanation.METRIC_META_RECALL
LlmClassifierLeaderboardExplanation.METRIC_PRECISION
LlmClassifierLeaderboardExplanation.METRIC_RECALL
LlmClassifierLeaderboardExplanation.add_evalstudio_markdown_format()
LlmClassifierLeaderboardExplanation.add_json_format()
LlmClassifierLeaderboardExplanation.add_markdown_format()
LlmClassifierLeaderboardExplanation.as_dict()
LlmClassifierLeaderboardExplanation.as_html()
LlmClassifierLeaderboardExplanation.as_markdown()
LlmClassifierLeaderboardExplanation.build()
LlmClassifierLeaderboardExplanation.from_eval_results()
LlmClassifierLeaderboardExplanation.get_insights()
LlmClassifierLeaderboardExplanation.sort_prompts_by_failures()
LlmClassifierLeaderboardExplanation.validate()
LlmEvalResultsExplanation
LlmHeatmapLeaderboardExplanation
LlmHeatmapLeaderboardExplanation.LLM_MODEL_ANONYMOUS
LlmHeatmapLeaderboardExplanation.add_col_value()
LlmHeatmapLeaderboardExplanation.add_evalstudio_markdown_format()
LlmHeatmapLeaderboardExplanation.add_json_format()
LlmHeatmapLeaderboardExplanation.add_markdown_format()
LlmHeatmapLeaderboardExplanation.as_dict()
LlmHeatmapLeaderboardExplanation.as_html()
LlmHeatmapLeaderboardExplanation.as_markdown()
LlmHeatmapLeaderboardExplanation.build()
LlmHeatmapLeaderboardExplanation.from_eval_results()
LlmHeatmapLeaderboardExplanation.get_insights()
LlmHeatmapLeaderboardExplanation.sort_prompts_by_failures()
LlmHeatmapLeaderboardExplanation.truncate()
LlmHeatmapLeaderboardExplanation.validate()
LlmLeaderboardExplanation
LlmProcedureEvalLeaderboardExplanation
LlmProcedureEvalLeaderboardExplanation.KEY_ALIGNMENT_MATRIX
LlmProcedureEvalLeaderboardExplanation.KEY_DYN_PROG_MATRIX
LlmProcedureEvalLeaderboardExplanation.LLM_MODEL_ANONYMOUS
LlmProcedureEvalLeaderboardExplanation.add_col_value()
LlmProcedureEvalLeaderboardExplanation.add_evalstudio_markdown_format()
LlmProcedureEvalLeaderboardExplanation.add_json_format()
LlmProcedureEvalLeaderboardExplanation.add_markdown_format()
LlmProcedureEvalLeaderboardExplanation.as_dict()
LlmProcedureEvalLeaderboardExplanation.as_html()
LlmProcedureEvalLeaderboardExplanation.as_markdown()
LlmProcedureEvalLeaderboardExplanation.build()
LlmProcedureEvalLeaderboardExplanation.from_eval_results()
LlmProcedureEvalLeaderboardExplanation.get_insights()
LlmProcedureEvalLeaderboardExplanation.sort_prompts_by_failures()
LlmProcedureEvalLeaderboardExplanation.truncate()
LlmProcedureEvalLeaderboardExplanation.validate()
LocalDataFrameExplanation
LocalDtExplanation
LocalFeatImpExplanation
LocalHtmlSnippetExplanation
LocalNlpLocoExplanation
LocalRuleExplanation
LocalSummaryFeatImpExplanation
LocalTextSnippetExplanation
LocoExplanation
ModelValidationResultExplanation
NlpTokenizerExplanation
OnDemandExplanation
PartialDependenceExplanation
ProxyExplanation
ReportExplanation
SaExplanation
TextExplanation
TimeSeriesAppExplanation
WorkDirArchiveExplanation
diagnose_perturbation_flips()
- h2o_sonar.lib.api.formats module
CsvFormatCustomExplanationFormat
CustomArchiveZipFormat
CustomCsvFormat
CustomJsonFormat
DatatableCustomExplanationFormat
DiaTextFormat
DocxFormat
EvalStudioMarkdownFormat
ExplanationFormat
ExplanationFormat.DEFAULT_PAGE_SIZE
ExplanationFormat.FEATURE_TYPE_CAT
ExplanationFormat.FEATURE_TYPE_CAT_NUM
ExplanationFormat.FEATURE_TYPE_DATE
ExplanationFormat.FEATURE_TYPE_DATETIME
ExplanationFormat.FEATURE_TYPE_NUM
ExplanationFormat.FEATURE_TYPE_TIME
ExplanationFormat.FILE_PREFIX_EXPLANATION_IDX
ExplanationFormat.KEYWORD_RESIDUALS
ExplanationFormat.KEY_ACTION
ExplanationFormat.KEY_ACTION_TYPE
ExplanationFormat.KEY_ACTUAL
ExplanationFormat.KEY_BIAS
ExplanationFormat.KEY_CATEGORICAL
ExplanationFormat.KEY_DATA
ExplanationFormat.KEY_DATA_HISTOGRAM
ExplanationFormat.KEY_DATA_HISTOGRAM_CAT
ExplanationFormat.KEY_DATA_HISTOGRAM_NUM
ExplanationFormat.KEY_DATE
ExplanationFormat.KEY_DATE_TIME
ExplanationFormat.KEY_DEFAULT_CLASS
ExplanationFormat.KEY_DOC
ExplanationFormat.KEY_EXPLAINER_JOB_KEY
ExplanationFormat.KEY_FEATURES
ExplanationFormat.KEY_FEATURE_TYPE
ExplanationFormat.KEY_FEATURE_VALUE
ExplanationFormat.KEY_FILES
ExplanationFormat.KEY_FILES_DETAILS
ExplanationFormat.KEY_FILES_NUMCAT_ASPECT
ExplanationFormat.KEY_FULLNAME
ExplanationFormat.KEY_ID
ExplanationFormat.KEY_IS_MULTI
ExplanationFormat.KEY_ITEM_ORDER
ExplanationFormat.KEY_KEYWORDS
ExplanationFormat.KEY_LABEL
ExplanationFormat.KEY_METADATA
ExplanationFormat.KEY_METRICS
ExplanationFormat.KEY_MIME
ExplanationFormat.KEY_MLI_KEY
ExplanationFormat.KEY_NAME
ExplanationFormat.KEY_NUMERIC
ExplanationFormat.KEY_ON_DEMAND
ExplanationFormat.KEY_ON_DEMAND_PARAMS
ExplanationFormat.KEY_PAGE_OFFSET
ExplanationFormat.KEY_PAGE_SIZE
ExplanationFormat.KEY_RAW_FEATURES
ExplanationFormat.KEY_ROWS_PER_PAGE
ExplanationFormat.KEY_RUNNING_ACTION
ExplanationFormat.KEY_SCOPE
ExplanationFormat.KEY_SYNC_ON_DEMAND
ExplanationFormat.KEY_TIME
ExplanationFormat.KEY_TOTAL_ROWS
ExplanationFormat.KEY_VALUE
ExplanationFormat.KEY_Y_FILE
ExplanationFormat.LABEL_REGRESSION
ExplanationFormat.SCOPE_GLOBAL
ExplanationFormat.SCOPE_LOCAL
ExplanationFormat.add_data()
ExplanationFormat.add_file()
ExplanationFormat.explanation
ExplanationFormat.file_names
ExplanationFormat.get_data()
ExplanationFormat.get_local_explanation()
ExplanationFormat.get_page()
ExplanationFormat.index_file_name
ExplanationFormat.is_on_demand()
ExplanationFormat.is_paged()
ExplanationFormat.load_meta()
ExplanationFormat.mime
ExplanationFormat.update_data()
ExplanationFormatUtils
Global3dDataJSonCsvFormat
Global3dDataJSonFormat
GlobalDtJSonFormat
GlobalDtJSonFormat.KEY_CHILDREN
GlobalDtJSonFormat.KEY_EDGE_IN
GlobalDtJSonFormat.KEY_EDGE_WEIGHT
GlobalDtJSonFormat.KEY_KEY
GlobalDtJSonFormat.KEY_LEAF_PATH
GlobalDtJSonFormat.KEY_NAME
GlobalDtJSonFormat.KEY_PARENT
GlobalDtJSonFormat.KEY_TOTAL_WEIGHT
GlobalDtJSonFormat.KEY_WEIGHT
GlobalDtJSonFormat.TreeNode
GlobalDtJSonFormat.load_index_file()
GlobalDtJSonFormat.mime
GlobalDtJSonFormat.serialize_data_file()
GlobalDtJSonFormat.serialize_index_file()
GlobalDtJSonFormat.validate_data()
GlobalFeatImpDatatableFormat
GlobalFeatImpJSonCsvFormat
GlobalFeatImpJSonDatatableFormat
GlobalFeatImpJSonDatatableFormat.COL_GLOBAL_SCOPE
GlobalFeatImpJSonDatatableFormat.COL_IMPORTANCE
GlobalFeatImpJSonDatatableFormat.COL_NAME
GlobalFeatImpJSonDatatableFormat.add_data_frame()
GlobalFeatImpJSonDatatableFormat.dict_to_data_frame()
GlobalFeatImpJSonDatatableFormat.from_lists()
GlobalFeatImpJSonDatatableFormat.get_data()
GlobalFeatImpJSonDatatableFormat.get_page()
GlobalFeatImpJSonDatatableFormat.is_paged()
GlobalFeatImpJSonDatatableFormat.mime
GlobalFeatImpJSonDatatableFormat.serialize_index_file()
GlobalFeatImpJSonDatatableFormat.validate_data()
GlobalFeatImpJSonFormat
GlobalFeatImpJSonFormat.KEY_LABEL
GlobalFeatImpJSonFormat.KEY_VALUE
GlobalFeatImpJSonFormat.from_dataframe_to_json()
GlobalFeatImpJSonFormat.from_json_datatable()
GlobalFeatImpJSonFormat.get_global_explanation()
GlobalFeatImpJSonFormat.load_index_file()
GlobalFeatImpJSonFormat.mime
GlobalFeatImpJSonFormat.serialize_data_file()
GlobalFeatImpJSonFormat.serialize_index_file()
GlobalFeatImpJSonFormat.validate_data()
GlobalGroupedBarChartJSonDatatableFormat
GlobalGroupedBarChartJSonDatatableFormat.COL_X
GlobalGroupedBarChartJSonDatatableFormat.COL_Y_GROUP_1
GlobalGroupedBarChartJSonDatatableFormat.COL_Y_GROUP_2
GlobalGroupedBarChartJSonDatatableFormat.add_data_frame()
GlobalGroupedBarChartJSonDatatableFormat.get_data()
GlobalGroupedBarChartJSonDatatableFormat.is_paged()
GlobalGroupedBarChartJSonDatatableFormat.load_index_file()
GlobalGroupedBarChartJSonDatatableFormat.mime
GlobalGroupedBarChartJSonDatatableFormat.serialize_index_file()
GlobalGroupedBarChartJSonDatatableFormat.validate_data()
GlobalLinePlotJSonFormat
GlobalNlpLocoJSonFormat
GlobalNlpLocoJSonFormat.FILTER_TYPE_TEXT_FEATURES
GlobalNlpLocoJSonFormat.KEY_DESCRIPTION
GlobalNlpLocoJSonFormat.KEY_FILTERS
GlobalNlpLocoJSonFormat.KEY_LABEL
GlobalNlpLocoJSonFormat.KEY_NAME
GlobalNlpLocoJSonFormat.KEY_TYPE
GlobalNlpLocoJSonFormat.KEY_VALUE
GlobalNlpLocoJSonFormat.KEY_VALUES
GlobalNlpLocoJSonFormat.from_dataframe_to_json()
GlobalNlpLocoJSonFormat.from_json_datatable()
GlobalNlpLocoJSonFormat.get_global_explanation()
GlobalNlpLocoJSonFormat.get_page()
GlobalNlpLocoJSonFormat.is_paged()
GlobalNlpLocoJSonFormat.load_index_file()
GlobalNlpLocoJSonFormat.mime
GlobalNlpLocoJSonFormat.serialize_data_file()
GlobalNlpLocoJSonFormat.serialize_index_file()
GlobalNlpLocoJSonFormat.validate_data()
GlobalScatterPlotJSonFormat
GlobalSummaryFeatImpJsonDatatableFormat
GlobalSummaryFeatImpJsonDatatableFormat.KEY_FEATURE
GlobalSummaryFeatImpJsonDatatableFormat.KEY_FREQUENCY
GlobalSummaryFeatImpJsonDatatableFormat.KEY_HIGH_VALUE
GlobalSummaryFeatImpJsonDatatableFormat.KEY_ORDER
GlobalSummaryFeatImpJsonDatatableFormat.KEY_SHAPLEY
GlobalSummaryFeatImpJsonDatatableFormat.add_data_frame()
GlobalSummaryFeatImpJsonDatatableFormat.load_index_file()
GlobalSummaryFeatImpJsonDatatableFormat.mime
GlobalSummaryFeatImpJsonDatatableFormat.serialize_index_file()
GlobalSummaryFeatImpJsonDatatableFormat.validate_data()
GlobalSummaryFeatImpJsonFormat
GlobalSummaryFeatImpJsonFormat.DATA_FILE_PREFIX
GlobalSummaryFeatImpJsonFormat.DEFAULT_PAGE_SIZE
GlobalSummaryFeatImpJsonFormat.KEY_FEATURE
GlobalSummaryFeatImpJsonFormat.KEY_FEATURES_PER_PAGE
GlobalSummaryFeatImpJsonFormat.KEY_FREQUENCY
GlobalSummaryFeatImpJsonFormat.KEY_HIGH_VALUE
GlobalSummaryFeatImpJsonFormat.KEY_ORDER
GlobalSummaryFeatImpJsonFormat.KEY_SHAPLEY
GlobalSummaryFeatImpJsonFormat.from_json_datatable()
GlobalSummaryFeatImpJsonFormat.get_page()
GlobalSummaryFeatImpJsonFormat.is_paged()
GlobalSummaryFeatImpJsonFormat.load_index_file()
GlobalSummaryFeatImpJsonFormat.mime
GlobalSummaryFeatImpJsonFormat.serialize_data_file()
GlobalSummaryFeatImpJsonFormat.serialize_index_file()
GlobalSummaryFeatImpJsonFormat.validate_data()
GrammarOfMliFormat
HtmlFormat
IceCsvFormat
IceDatatableFormat
IceJsonDatatableFormat
IceJsonDatatableFormat.FILE_Y_FILE
IceJsonDatatableFormat.KEY_BIN
IceJsonDatatableFormat.KEY_BINS
IceJsonDatatableFormat.KEY_BINS_NUMCAT_ASPECT
IceJsonDatatableFormat.KEY_COL_NAME
IceJsonDatatableFormat.KEY_FEATURE_VALUE
IceJsonDatatableFormat.KEY_ICE
IceJsonDatatableFormat.KEY_PREDICTION
IceJsonDatatableFormat.add_data_frame()
IceJsonDatatableFormat.get_local_explanation()
IceJsonDatatableFormat.is_on_demand()
IceJsonDatatableFormat.load_index_file()
IceJsonDatatableFormat.merge_format()
IceJsonDatatableFormat.mime
IceJsonDatatableFormat.mli_ice_explanation_to_json()
IceJsonDatatableFormat.serialize_index_file()
IceJsonDatatableFormat.serialize_on_demand_index_file()
IceJsonDatatableFormat.validate_data()
LlmHeatmapLeaderboardJSonFormat
LlmLeaderboardJSonFormat
LocalDtJSonFormat
LocalDtJSonFormat.dt_path_to_node_key()
LocalDtJSonFormat.dt_set_tree_path()
LocalDtJSonFormat.get_local_explanation()
LocalDtJSonFormat.is_on_demand()
LocalDtJSonFormat.mime
LocalDtJSonFormat.serialize_index_file()
LocalDtJSonFormat.serialize_on_demand_index_file()
LocalDtJSonFormat.validate_data()
LocalFeatImpDatatableFormat
LocalFeatImpJSonDatatableFormat
LocalFeatImpJSonFormat
LocalFeatImpJSonFormat.KEY_Y
LocalFeatImpJSonFormat.is_on_demand()
LocalFeatImpJSonFormat.load_index_file()
LocalFeatImpJSonFormat.merge_local_and_global_page()
LocalFeatImpJSonFormat.mime
LocalFeatImpJSonFormat.serialize_index_file()
LocalFeatImpJSonFormat.sort_data()
LocalFeatImpJSonFormat.validate_data()
LocalFeatImpWithYhatsJSonDatatableFormat
LocalNlpLocoJSonFormat
LocalOnDemandHtmlFormat
LocalOnDemandTextFormat
LocalSummaryFeatImplJSonFormat
LocalSummaryFeatImplJSonFormat.is_on_demand()
LocalSummaryFeatImplJSonFormat.is_paged()
LocalSummaryFeatImplJSonFormat.load_index_file()
LocalSummaryFeatImplJSonFormat.mime
LocalSummaryFeatImplJSonFormat.serialize_index_file()
LocalSummaryFeatImplJSonFormat.serialize_on_demand_index_file()
LocalSummaryFeatImplJSonFormat.validate_data()
MarkdownFormat
ModelValidationResultArchiveFormat
PartialDependenceCsvFormat
PartialDependenceDatatableFormat
PartialDependenceDatatableFormat.COL_BIN_VALUE
PartialDependenceDatatableFormat.COL_F_LTYPE
PartialDependenceDatatableFormat.COL_F_NAME
PartialDependenceDatatableFormat.COL_IS_OOR
PartialDependenceDatatableFormat.COL_MEAN
PartialDependenceDatatableFormat.COL_SD
PartialDependenceDatatableFormat.COL_SEM
PartialDependenceDatatableFormat.mime
PartialDependenceDatatableFormat.validate_data()
PartialDependenceJSonFormat
PartialDependenceJSonFormat.KEY_BIN
PartialDependenceJSonFormat.KEY_FREQUENCY
PartialDependenceJSonFormat.KEY_OOR
PartialDependenceJSonFormat.KEY_PD
PartialDependenceJSonFormat.KEY_SD
PartialDependenceJSonFormat.KEY_X
PartialDependenceJSonFormat.get_bins()
PartialDependenceJSonFormat.get_numcat_aspects()
PartialDependenceJSonFormat.get_numcat_missing_aspect()
PartialDependenceJSonFormat.load_index_file()
PartialDependenceJSonFormat.merge_format()
PartialDependenceJSonFormat.mime
PartialDependenceJSonFormat.serialize_index_file()
PartialDependenceJSonFormat.set_merge_status()
PartialDependenceJSonFormat.validate_data()
SaTextFormat
TextCustomExplanationFormat
TextCustomExplanationFormat.FILE_IS_ON_DEMAND
TextCustomExplanationFormat.FILTER_CLASS
TextCustomExplanationFormat.FILTER_FEATURE
TextCustomExplanationFormat.FILTER_NUMCAT
TextCustomExplanationFormat.add_data()
TextCustomExplanationFormat.add_file()
TextCustomExplanationFormat.get_data()
TextCustomExplanationFormat.is_on_demand()
TextCustomExplanationFormat.mime
TextCustomExplanationFormat.set_index_commons()
TextCustomExplanationFormat.set_on_demand()
TextCustomExplanationFormat.update_index_file()
TextFormat
WorkDirArchiveZipFormat
get_custom_explanation_formats()
- h2o_sonar.lib.api.interpretations module
ExplainerJob
ExplainerJob.KEY_CHILD_KEYS
ExplainerJob.KEY_CREATED
ExplainerJob.KEY_DURATION
ExplainerJob.KEY_ERROR
ExplainerJob.KEY_EXPLAINER_DESCRIPTOR
ExplainerJob.KEY_JOB_LOCATION
ExplainerJob.KEY_KEY
ExplainerJob.KEY_MESSAGE
ExplainerJob.KEY_PROGRESS
ExplainerJob.KEY_RESULT_DESCRIPTOR
ExplainerJob.KEY_STATUS
ExplainerJob.evaluator_id()
ExplainerJob.explainer_id()
ExplainerJob.from_dict()
ExplainerJob.is_finished()
ExplainerJob.success()
ExplainerJob.tick()
ExplainerJob.to_dict()
HtmlInterpretationFormat
HtmlInterpretationFormat.Context
HtmlInterpretationFormat.KEYWORD_ID_2_NAME
HtmlInterpretationFormat.html_footer()
HtmlInterpretationFormat.html_h2o_sonar_pitch()
HtmlInterpretationFormat.html_head()
HtmlInterpretationFormat.html_safe_str_field()
HtmlInterpretationFormat.html_svg_h2oai_logo()
HtmlInterpretationFormat.to_html()
HtmlInterpretationsFormat
Interpretation
Interpretation.KEY_ALL_EXPLAINERS
Interpretation.KEY_CREATED
Interpretation.KEY_DATASET
Interpretation.KEY_ERROR
Interpretation.KEY_EXECUTED_EXPLAINERS
Interpretation.KEY_EXPLAINERS
Interpretation.KEY_E_PARAMS
Interpretation.KEY_INCOMPATIBLE_EXPLAINERS
Interpretation.KEY_INCOMPATIBLE_EXPLAINERS_DS
Interpretation.KEY_INSIGHTS
Interpretation.KEY_INTERPRETATION_LOCATION
Interpretation.KEY_I_KEY
Interpretation.KEY_I_PARAMS
Interpretation.KEY_MODEL
Interpretation.KEY_MODELS
Interpretation.KEY_OVERALL_RESULT
Interpretation.KEY_PROBLEMS
Interpretation.KEY_PROGRESS
Interpretation.KEY_PROGRESS_MESSAGE
Interpretation.KEY_RESULT
Interpretation.KEY_RESULTS_LOCATION
Interpretation.KEY_SCHEDULED_EXPLAINERS
Interpretation.KEY_STATUS
Interpretation.KEY_TARGET_COL
Interpretation.KEY_TESTSET
Interpretation.KEY_VALIDSET
Interpretation.dict_to_digest()
Interpretation.get_all_explainer_ids()
Interpretation.get_explainer_ids_by_status()
Interpretation.get_explainer_insights()
Interpretation.get_explainer_jobs_by_status()
Interpretation.get_explainer_problems()
Interpretation.get_explainer_result()
Interpretation.get_explainer_result_metadata()
Interpretation.get_explanation_file_path()
Interpretation.get_failed_explainer_ids()
Interpretation.get_finished_explainer_ids()
Interpretation.get_incompatible_explainer_ids()
Interpretation.get_insights()
Interpretation.get_jobs_for_evaluator_id()
Interpretation.get_jobs_for_explainer_id()
Interpretation.get_model_insights()
Interpretation.get_model_problems()
Interpretation.get_problems_by_severity()
Interpretation.get_scheduled_explainer_ids()
Interpretation.get_successful_explainer_ids()
Interpretation.is_evaluator_failed()
Interpretation.is_evaluator_finished()
Interpretation.is_evaluator_scheduled()
Interpretation.is_evaluator_successful()
Interpretation.is_explainer_failed()
Interpretation.is_explainer_finished()
Interpretation.is_explainer_scheduled()
Interpretation.is_explainer_successful()
Interpretation.load()
Interpretation.load_from_json()
Interpretation.register_explainer_result()
Interpretation.set_progress()
Interpretation.to_dict()
Interpretation.to_html()
Interpretation.to_html_4_pdf()
Interpretation.to_json()
Interpretation.to_pdf()
Interpretation.update_overall_result()
Interpretation.validate_and_normalize_params()
InterpretationResult
InterpretationResult.get_evaluator_job()
InterpretationResult.get_evaluator_jobs()
InterpretationResult.get_explainer_job()
InterpretationResult.get_explainer_jobs()
InterpretationResult.get_html_report_location()
InterpretationResult.get_interpretation_dir_location()
InterpretationResult.get_interpretations_html_index_location()
InterpretationResult.get_json_report_location()
InterpretationResult.get_pdf_report_location()
InterpretationResult.get_progress_location()
InterpretationResult.get_results_dir_location()
InterpretationResult.make_zip_archive()
InterpretationResult.remove_duplicate_insights()
InterpretationResult.to_dict()
InterpretationResult.to_json()
Interpretations
OverallResult
PdfInterpretationFormat
- h2o_sonar.lib.api.judges module
- h2o_sonar.lib.api.models module
DriverlessAiModel
DriverlessAiRestServerModel
ExplainableLlmModel
ExplainableLlmModel.KEY_CONNECTION
ExplainableLlmModel.KEY_H2OGPTE_STATS
ExplainableLlmModel.KEY_H2OGPTE_VISION_M
ExplainableLlmModel.KEY_KEY
ExplainableLlmModel.KEY_LLM_MODEL_META
ExplainableLlmModel.KEY_LLM_MODEL_NAME
ExplainableLlmModel.KEY_MODEL_CFG
ExplainableLlmModel.KEY_MODEL_TYPE
ExplainableLlmModel.KEY_NAME
ExplainableLlmModel.KEY_STATS_DURATION
ExplainableLlmModel.KEY_STATS_FAILURE
ExplainableLlmModel.KEY_STATS_RETRY
ExplainableLlmModel.KEY_STATS_SUCCESS
ExplainableLlmModel.KEY_STATS_TIMEOUT
ExplainableLlmModel.clone()
ExplainableLlmModel.from_dict()
ExplainableLlmModel.to_dict()
ExplainableLlmModel.to_json()
ExplainableModel
ExplainableModel.fit()
ExplainableModel.has_transformed_model
ExplainableModel.load()
ExplainableModel.meta
ExplainableModel.predict()
ExplainableModel.predict_datatable()
ExplainableModel.predict_pandas()
ExplainableModel.save()
ExplainableModel.shapley_values()
ExplainableModel.to_dict()
ExplainableModel.to_json()
ExplainableModel.transformed_model
ExplainableModelHandle
ExplainableModelMeta
ExplainableModelMeta.default_feature_importances()
ExplainableModelMeta.feature_importances
ExplainableModelMeta.features_metadata
ExplainableModelMeta.get_model_type()
ExplainableModelMeta.has_shapley_values
ExplainableModelMeta.has_text_transformers
ExplainableModelMeta.is_constant
ExplainableModelMeta.num_labels
ExplainableModelMeta.positive_label_of_interest
ExplainableModelMeta.to_dict()
ExplainableModelMeta.to_json()
ExplainableModelMeta.transformed_features
ExplainableModelMeta.used_features
ExplainableModelType
ExplainableModelType.amazon_bedrock_rag
ExplainableModelType.azure_openai_llm
ExplainableModelType.driverless_ai
ExplainableModelType.driverless_ai_rest
ExplainableModelType.from_connection_type()
ExplainableModelType.h2o3
ExplainableModelType.h2ogpt
ExplainableModelType.h2ogpte
ExplainableModelType.h2ogpte_llm
ExplainableModelType.h2ollmops
ExplainableModelType.is_llm()
ExplainableModelType.is_rag()
ExplainableModelType.mock
ExplainableModelType.ollama
ExplainableModelType.openai_llm
ExplainableModelType.openai_rag
ExplainableModelType.scikit_learn
ExplainableModelType.to_connection_type()
ExplainableModelType.unknown
ExplainableRagModel
ExplainableRagModel.KEY_COLLECTION_ID
ExplainableRagModel.KEY_COLLECTION_NAME
ExplainableRagModel.KEY_CONNECTION
ExplainableRagModel.KEY_DOCUMENTS
ExplainableRagModel.KEY_KEY
ExplainableRagModel.KEY_LLM_MODEL_META
ExplainableRagModel.KEY_LLM_MODEL_NAME
ExplainableRagModel.KEY_MODEL_CFG
ExplainableRagModel.KEY_MODEL_TYPE
ExplainableRagModel.KEY_NAME
ExplainableRagModel.clone()
ExplainableRagModel.from_dict()
ExplainableRagModel.to_dict()
H2o3Model
ModelApi
ModelVendor
OpenAiRagModel
PickleFileModel
ScikitLearnModel
TransformedFeaturesModel
guess_model_labels()
guess_model_used_features()
- h2o_sonar.lib.api.persistences module
ExplainerPersistence
ExplainerPersistence.DIR_EXPLAINER
ExplainerPersistence.DIR_INSIGHTS
ExplainerPersistence.DIR_LOG
ExplainerPersistence.DIR_PROBLEMS
ExplainerPersistence.DIR_WORK
ExplainerPersistence.EXPLAINER_LOG_PREFIX
ExplainerPersistence.EXPLAINER_LOG_SUFFIX_ANON
ExplainerPersistence.FILE_DONE_DONE
ExplainerPersistence.FILE_DONE_FAILED
ExplainerPersistence.FILE_EXPLAINER_PICKLE
ExplainerPersistence.FILE_EXPLANATION
ExplainerPersistence.FILE_INSIGHTS
ExplainerPersistence.FILE_ON_DEMAND_EXPLANATION_SUFFIX
ExplainerPersistence.FILE_PROBLEMS
ExplainerPersistence.FILE_RESULT_DESCRIPTOR
ExplainerPersistence.explainer_id
ExplainerPersistence.explainer_job_key
ExplainerPersistence.get_dirs_for_explainer_id()
ExplainerPersistence.get_evaluator_working_file()
ExplainerPersistence.get_explainer_ann_log_file()
ExplainerPersistence.get_explainer_ann_log_path()
ExplainerPersistence.get_explainer_dir()
ExplainerPersistence.get_explainer_dir_archive()
ExplainerPersistence.get_explainer_insights_dir()
ExplainerPersistence.get_explainer_insights_file()
ExplainerPersistence.get_explainer_log_dir()
ExplainerPersistence.get_explainer_log_file()
ExplainerPersistence.get_explainer_log_path()
ExplainerPersistence.get_explainer_problems_dir()
ExplainerPersistence.get_explainer_problems_file()
ExplainerPersistence.get_explainer_working_dir()
ExplainerPersistence.get_explainer_working_file()
ExplainerPersistence.get_explanation_dir_path()
ExplainerPersistence.get_explanation_file_path()
ExplainerPersistence.get_explanation_meta_path()
ExplainerPersistence.get_key_for_explainer_dir()
ExplainerPersistence.get_locators_for_explainer_id()
ExplainerPersistence.get_relative_path()
ExplainerPersistence.get_result_descriptor_file_path()
ExplainerPersistence.load_insights()
ExplainerPersistence.load_problems()
ExplainerPersistence.load_result_descriptor()
ExplainerPersistence.make_dir()
ExplainerPersistence.make_explainer_dir()
ExplainerPersistence.make_explainer_insights_dir()
ExplainerPersistence.make_explainer_log_dir()
ExplainerPersistence.make_explainer_problems_dir()
ExplainerPersistence.make_explainer_sandbox()
ExplainerPersistence.make_explainer_working_dir()
ExplainerPersistence.makedirs()
ExplainerPersistence.resolve_mli_path()
ExplainerPersistence.rm_explainer_dir()
ExplainerPersistence.save_insights()
ExplainerPersistence.save_json()
ExplainerPersistence.save_problems()
ExplainerPersistence.username
FilesystemPersistence
FilesystemPersistence.copy_file()
FilesystemPersistence.delete()
FilesystemPersistence.delete_dir_contents()
FilesystemPersistence.delete_file()
FilesystemPersistence.delete_tree()
FilesystemPersistence.exists()
FilesystemPersistence.flush_dir_for_file()
FilesystemPersistence.get_default_cwl()
FilesystemPersistence.getcwl()
FilesystemPersistence.is_dir()
FilesystemPersistence.is_file()
FilesystemPersistence.list_dir()
FilesystemPersistence.list_files_by_wildcard()
FilesystemPersistence.load()
FilesystemPersistence.load_json()
FilesystemPersistence.make_dir()
FilesystemPersistence.make_dir_zip_archive()
FilesystemPersistence.save()
FilesystemPersistence.save_json()
FilesystemPersistence.touch()
FilesystemPersistence.type
FilesystemPersistence.update()
InMemoryPersistence
InMemoryPersistence.DIR
InMemoryPersistence.Directory
InMemoryPersistence.copy_file()
InMemoryPersistence.delete()
InMemoryPersistence.delete_dir_contents()
InMemoryPersistence.delete_file()
InMemoryPersistence.delete_tree()
InMemoryPersistence.exists()
InMemoryPersistence.get_default_cwl()
InMemoryPersistence.getcwl()
InMemoryPersistence.is_dir()
InMemoryPersistence.is_file()
InMemoryPersistence.list_dir()
InMemoryPersistence.list_files_by_wildcard()
InMemoryPersistence.load()
InMemoryPersistence.load_json()
InMemoryPersistence.make_dir()
InMemoryPersistence.make_dir_zip_archive()
InMemoryPersistence.save()
InMemoryPersistence.save_json()
InMemoryPersistence.touch()
InMemoryPersistence.type
InterpretationPersistence
InterpretationPersistence.DIR_AD_HOC_EXPLANATION
InterpretationPersistence.DIR_AUTOML_EXPERIMENT
InterpretationPersistence.DIR_MLI_EXPERIMENT
InterpretationPersistence.DIR_MLI_TS_EXPERIMENT
InterpretationPersistence.FILE_COMMON_PARAMS
InterpretationPersistence.FILE_EXPERIMENT_ID_COLS
InterpretationPersistence.FILE_EXPERIMENT_IMAGE
InterpretationPersistence.FILE_EXPERIMENT_TS
InterpretationPersistence.FILE_H2O_SONAR_HTML
InterpretationPersistence.FILE_INTERPRETATION_HTML
InterpretationPersistence.FILE_INTERPRETATION_HTML_4_PDF
InterpretationPersistence.FILE_INTERPRETATION_JSON
InterpretationPersistence.FILE_INTERPRETATION_PDF
InterpretationPersistence.FILE_MLI_EXPERIMENT_LOG
InterpretationPersistence.FILE_PREFIX_DATASET
InterpretationPersistence.FILE_PROGRESS_JSON
InterpretationPersistence.KEY_E_PARAMS
InterpretationPersistence.KEY_RESULT
InterpretationPersistence.ad_hoc_job_key
InterpretationPersistence.base_dir
InterpretationPersistence.create_dataset_path()
InterpretationPersistence.data_dir
InterpretationPersistence.get_ad_hoc_mli_dir_name()
InterpretationPersistence.get_async_log_file_name()
InterpretationPersistence.get_base_dir()
InterpretationPersistence.get_base_dir_file()
InterpretationPersistence.get_experiment_id_cols_path()
InterpretationPersistence.get_html_4_pdf_path()
InterpretationPersistence.get_html_path()
InterpretationPersistence.get_json_path()
InterpretationPersistence.get_mli_dir_name()
InterpretationPersistence.get_pdf_path()
InterpretationPersistence.is_common_params()
InterpretationPersistence.is_safe_name()
InterpretationPersistence.list_interpretations()
InterpretationPersistence.load_common_params()
InterpretationPersistence.load_explainers_params()
InterpretationPersistence.load_is_image_experiment()
InterpretationPersistence.load_is_timeseries_experiment()
InterpretationPersistence.load_message_entity()
InterpretationPersistence.make_base_dir()
InterpretationPersistence.make_dir_zip_archive()
InterpretationPersistence.make_interpretation_sandbox()
InterpretationPersistence.make_tmp_dir()
InterpretationPersistence.mli_key
InterpretationPersistence.resolve_model_path()
InterpretationPersistence.rm_base_dir()
InterpretationPersistence.rm_dir()
InterpretationPersistence.save_as_html()
InterpretationPersistence.save_as_json()
InterpretationPersistence.save_as_pdf()
InterpretationPersistence.save_common_params()
InterpretationPersistence.save_experiment_type_hints()
InterpretationPersistence.save_message_entity()
InterpretationPersistence.tmp_dir
InterpretationPersistence.to_alphanum_name()
InterpretationPersistence.to_server_file_path()
InterpretationPersistence.to_server_path()
InterpretationPersistence.user_dir
JsonPersistableExplanations
NanEncoder
Persistence
Persistence.PREFIX_INTERNAL_STORE
Persistence.check_key()
Persistence.copy_file()
Persistence.delete()
Persistence.delete_dir_contents()
Persistence.delete_file()
Persistence.delete_temp_dir()
Persistence.delete_tree()
Persistence.exists()
Persistence.flush_dir_for_file()
Persistence.getcwl()
Persistence.is_binary_file()
Persistence.is_dir()
Persistence.is_dir_or_file()
Persistence.is_file()
Persistence.key_folder()
Persistence.list_dir()
Persistence.list_files_by_wildcard()
Persistence.load()
Persistence.load_json()
Persistence.make_dir()
Persistence.make_dir_zip_archive()
Persistence.make_key()
Persistence.make_temp_dir()
Persistence.make_temp_file()
Persistence.path_to_internal()
Persistence.safe_name()
Persistence.save()
Persistence.touch()
Persistence.type
Persistence.update()
PersistenceApi
PersistenceDataType
PersistenceType
RobustEncoder
- h2o_sonar.lib.api.plots module
- h2o_sonar.lib.api.problems module
AVIDProblemCode
AVIDProblemCode.E0100_BIAS
AVIDProblemCode.E0200_EXPLAINABILITY
AVIDProblemCode.E0300_TOXICITY
AVIDProblemCode.E0400_MISINFORMATION
AVIDProblemCode.P0100_DATA
AVIDProblemCode.P0200_MODEL
AVIDProblemCode.P0300_PRIVACY
AVIDProblemCode.P0400_SAFETY
AVIDProblemCode.S0400_MODEL_BYPASS
AVIDProblemCode.S0500_EXFILTRATION
AVIDProblemCode.S0600_DATA_POISONING
AVIDProblemCodeType
ProblemAndAction
ProblemCode
ProblemSeverity
problems_for_bool_leaderboard()
problems_for_cls_leaderboard()
problems_for_heat_leaderboard()
- h2o_sonar.lib.api.results module
- Module contents
- h2o_sonar.lib.container package
- Submodules
- h2o_sonar.lib.container.explainer_container module
AsyncLocalContainer
ContainerRegistry
DependencyTreeNode
ExplainerContainer
ExplainerContainer.TYPE_ID
ExplainerContainer.delete_interpretation()
ExplainerContainer.explainers_registry
ExplainerContainer.gc()
ExplainerContainer.get_explainer()
ExplainerContainer.get_explainer_class()
ExplainerContainer.get_explainer_job_keys_by_id()
ExplainerContainer.get_explainer_job_statuses()
ExplainerContainer.get_explainer_local_result()
ExplainerContainer.get_explainer_log_path()
ExplainerContainer.get_explainer_metadata()
ExplainerContainer.get_explainer_result_path()
ExplainerContainer.get_explainer_snapshot_path()
ExplainerContainer.get_interpretation_params()
ExplainerContainer.get_interpretation_status()
ExplainerContainer.hot_deploy_explainer()
ExplainerContainer.list_explainers()
ExplainerContainer.list_interpretations()
ExplainerContainer.load_interpretations()
ExplainerContainer.register_configured_explainers()
ExplainerContainer.register_explainer()
ExplainerContainer.run_interpretation()
ExplainerContainer.setup()
ExplainerContainer.unregister_explainer()
ExplainerContainer.update_explainer_result()
ExplainerExecutor
LocalExplainerContainer
ParallelExplainerExecutor
SequentialExplainerExecutor
register_ootb_explainers()
- Module contents
- h2o_sonar.lib.integrations package
- Submodules
- h2o_sonar.lib.integrations.genai module
AmazonBedrockFoundationModel
AmazonBedrockFoundationModel.customizations_supported
AmazonBedrockFoundationModel.inference_types_supported
AmazonBedrockFoundationModel.input_modalities
AmazonBedrockFoundationModel.model_arn
AmazonBedrockFoundationModel.model_id
AmazonBedrockFoundationModel.model_lifecycle_status
AmazonBedrockFoundationModel.model_name
AmazonBedrockFoundationModel.output_modalities
AmazonBedrockFoundationModel.provider_name
AmazonBedrockFoundationModel.response_streaming_supported
AmazonBedrockKnowledgeBase
AmazonBedrockRagClient
AmazonBedrockRagClient.ES_TEMP_PREFIX
AmazonBedrockRagClient.ask_collection()
AmazonBedrockRagClient.ask_model()
AmazonBedrockRagClient.bedrock
AmazonBedrockRagClient.bedrock_agent
AmazonBedrockRagClient.bedrock_agent_runtime
AmazonBedrockRagClient.bedrock_runtime
AmazonBedrockRagClient.config_factory()
AmazonBedrockRagClient.connection
AmazonBedrockRagClient.create_collection()
AmazonBedrockRagClient.get_rag_conf()
AmazonBedrockRagClient.iam
AmazonBedrockRagClient.is_model_enabled()
AmazonBedrockRagClient.list_collections()
AmazonBedrockRagClient.list_llm_model_names()
AmazonBedrockRagClient.list_llm_models()
AmazonBedrockRagClient.opensearchserverless
AmazonBedrockRagClient.purge_collections()
AmazonBedrockRagClient.purge_uploaded_docs()
AmazonBedrockRagClient.s3
AmazonBedrockRagClient.s3_resource
AmazonBedrockRagClient.sts
AwsClient
AwsResource
H2oGptLlmClient
H2oGpteRagClient
H2oGpteRagClient.CFG_EMBEDDING_MODEL
H2oGpteRagClient.CFG_PROMPT_TEMPLATE_ID
H2oGpteRagClient.DEFAULT_TIMEOUT
H2oGpteRagClient.MODEL_SPEC_AUTO
H2oGpteRagClient.MODEL_SPEC_COL
H2oGpteRagClient.MODEL_SPEC_COL_OPT_E
H2oGpteRagClient.MODEL_SPEC_COL_OPT_N
H2oGpteRagClient.TypedLlmConfigDict
H2oGpteRagClient.TypedLlmConfigDict.chat_conversation
H2oGpteRagClient.TypedLlmConfigDict.llm
H2oGpteRagClient.TypedLlmConfigDict.llm_args
H2oGpteRagClient.TypedLlmConfigDict.pre_prompt_query
H2oGpteRagClient.TypedLlmConfigDict.prompt_query
H2oGpteRagClient.TypedLlmConfigDict.system_prompt
H2oGpteRagClient.TypedLlmConfigDict.text_context_list
H2oGpteRagClient.TypedLlmConfigDict.timeout
H2oGpteRagClient.TypedRagConfigDict
H2oGpteRagClient.TypedRagConfigDict.embedding_model
H2oGpteRagClient.TypedRagConfigDict.llm
H2oGpteRagClient.TypedRagConfigDict.llm_args
H2oGpteRagClient.TypedRagConfigDict.pre_prompt_query
H2oGpteRagClient.TypedRagConfigDict.pre_prompt_summary
H2oGpteRagClient.TypedRagConfigDict.prompt_query
H2oGpteRagClient.TypedRagConfigDict.prompt_summary
H2oGpteRagClient.TypedRagConfigDict.prompt_template_id
H2oGpteRagClient.TypedRagConfigDict.rag_config
H2oGpteRagClient.TypedRagConfigDict.self_reflection_config
H2oGpteRagClient.TypedRagConfigDict.system_prompt
H2oGpteRagClient.TypedRagConfigDict.timeout
H2oGpteRagClient.ask_collection()
H2oGpteRagClient.ask_model()
H2oGpteRagClient.client
H2oGpteRagClient.config_factory()
H2oGpteRagClient.create_collection()
H2oGpteRagClient.humanize_err_msg()
H2oGpteRagClient.list_collections()
H2oGpteRagClient.list_llm_model_names()
H2oGpteRagClient.purge_collections()
H2oGpteRagClient.purge_uploaded_docs()
H2oLlmOpsClient
LlmHostClient
MsAzureOpenAiLlmClient
OllamaClient
OpenAiAssistantsRagClient
OpenAiAssistantsRagClientVersion1
OpenAiAssistantsRagClientVersion1.BASE_LLM_MODELS
OpenAiAssistantsRagClientVersion1.DEFAULT_LLM_MODEL
OpenAiAssistantsRagClientVersion1.HEADERS_VERSION_1
OpenAiAssistantsRagClientVersion1.HEADERS_VERSION_2
OpenAiAssistantsRagClientVersion1.KWARGS_ASSISTANT
OpenAiAssistantsRagClientVersion1.KWARGS_RUN
OpenAiAssistantsRagClientVersion1.KWARGS_THREAD
OpenAiAssistantsRagClientVersion1.ask_collection()
OpenAiAssistantsRagClientVersion1.ask_model()
OpenAiAssistantsRagClientVersion1.client
OpenAiAssistantsRagClientVersion1.config_factory()
OpenAiAssistantsRagClientVersion1.config_normalize()
OpenAiAssistantsRagClientVersion1.config_resolve()
OpenAiAssistantsRagClientVersion1.create_collection()
OpenAiAssistantsRagClientVersion1.list_collections()
OpenAiAssistantsRagClientVersion1.list_llm_model_names()
OpenAiAssistantsRagClientVersion1.purge_collections()
OpenAiAssistantsRagClientVersion1.purge_uploaded_docs()
OpenAiAssistantsRagClientVersion2
OpenAiAssistantsRagClientVersion2.DEFAULT_LLM_MODEL
OpenAiAssistantsRagClientVersion2.HEADERS_VERSION_2
OpenAiAssistantsRagClientVersion2.ask_collection()
OpenAiAssistantsRagClientVersion2.ask_model()
OpenAiAssistantsRagClientVersion2.client
OpenAiAssistantsRagClientVersion2.create_collection()
OpenAiAssistantsRagClientVersion2.list_collections()
OpenAiAssistantsRagClientVersion2.list_llm_model_names()
OpenAiAssistantsRagClientVersion2.purge_collections()
OpenAiAssistantsRagClientVersion2.purge_uploaded_docs()
OpenAiLlmClient
RagChunkRetrievalMethod
RagClient
TimeoutRetryExpBackoffCtx
TypedH2ogptLlmConfigDict
TypedOllamaModelFileDict
TypedOllamaModelFileDict.mirorstat
TypedOllamaModelFileDict.mirorstat_eta
TypedOllamaModelFileDict.mirorstat_tau
TypedOllamaModelFileDict.num_ctx
TypedOllamaModelFileDict.num_predict
TypedOllamaModelFileDict.repeat_last_n
TypedOllamaModelFileDict.repeat_penalty
TypedOllamaModelFileDict.seed
TypedOllamaModelFileDict.stop
TypedOllamaModelFileDict.temperature
TypedOllamaModelFileDict.tfs_z
TypedOllamaModelFileDict.top_k
TypedOllamaModelFileDict.top_p
get_client_for_connection()
log_action()
- h2o_sonar.lib.integrations.mv_adapter module
ExplainerToMvTestAdapter
MvResultJSonEncoder
MvResultPersistence
MvResultPersistence.DIR_MVARTIFACTS
MvResultPersistence.DIR_MVLOG
MvResultPersistence.DIR_MVRESULTS
MvResultPersistence.DIR_MVSETTINGS
MvResultPersistence.DIR_MVTEST
MvResultPersistence.FILE_INDEX
MvResultPersistence.FORMAT_DATETIME
MvResultPersistence.KEY_COLUMNS
MvResultPersistence.KEY_COLUMN_SUMMARIES
MvResultPersistence.KEY_COL_HISTOGRAM_STATS
MvResultPersistence.KEY_DATA
MvResultPersistence.KEY_DIR
MvResultPersistence.KEY_ERROR
MvResultPersistence.KEY_FILENAME
MvResultPersistence.KEY_HASH
MvResultPersistence.KEY_LEVEL
MvResultPersistence.KEY_MSGS
MvResultPersistence.KEY_MV_FPATH
MvResultPersistence.KEY_MV_ID
MvResultPersistence.KEY_MV_NAME
MvResultPersistence.KEY_MV_TYPE
MvResultPersistence.KEY_NAME
MvResultPersistence.KEY_N_COLS
MvResultPersistence.KEY_N_ROWS
MvResultPersistence.KEY_ORIGIN_OBJ_KEY
MvResultPersistence.KEY_PATH
MvResultPersistence.KEY_PLATFORM_MVID
MvResultPersistence.KEY_PLATFORM_OBJ_KEY
MvResultPersistence.KEY_SAMPLE_TABLE
MvResultPersistence.KEY_SHAPE
MvResultPersistence.KEY_SIZE
MvResultPersistence.KEY_SIZE_STR
MvResultPersistence.KEY_SUMMARY
MvResultPersistence.KEY_TEXT
MvResultPersistence.KEY_TS
MvResultPersistence.KEY_TYPE
MvResultPersistence.TYPE_MV_ARTIFACTS
MvResultPersistence.TYPE_MV_ARTIFACT_INFO
MvResultPersistence.TYPE_NONE_STR
MvResultPersistence.TYPE_SHAPE_STR
MvResultPersistence.export_mv_test()
MvResultPersistence.import_mv_test()
MvResultPersistence.set_target_dir()
MvResultPersistence.set_test_dir_name()
MvResultPersistence.split_full_type()
- h2o_sonar.lib.integrations.ragas_adapter module
- Module contents
- h2o_sonar.lib.api package
- Module contents
- Subpackages
- h2o_sonar.methods package
- Subpackages
- h2o_sonar.methods.core package
- Submodules
- h2o_sonar.methods.core.method module
FeatureTypes
FeatureTypes.DEFAULT_DATE_FEATURE_FORMAT
FeatureTypes.KEY_CATEGORICAL_FEATURES
FeatureTypes.KEY_CATNUM_FEATURES
FeatureTypes.KEY_DATE_FEATURES
FeatureTypes.KEY_DATE_FEATURES_FORMAT
FeatureTypes.KEY_DATE_TIME_FEATURES
FeatureTypes.KEY_ID_FEATURES
FeatureTypes.KEY_IMAGE_FEATURES
FeatureTypes.KEY_NUMERIC_FEATURES
FeatureTypes.KEY_QUANTILE_BINS
FeatureTypes.KEY_TEXT_FEATURES
FeatureTypes.KEY_TIME_FEATURES
FeaturesMetadata
FeaturesMetadata.add()
FeaturesMetadata.categorical_features
FeaturesMetadata.categorical_numeric_features
FeaturesMetadata.create_blank_dict()
FeaturesMetadata.date_features
FeaturesMetadata.date_time_features
FeaturesMetadata.empty()
FeaturesMetadata.format_date_features
FeaturesMetadata.get()
FeaturesMetadata.id_features
FeaturesMetadata.image_features
FeaturesMetadata.numeric_features
FeaturesMetadata.qtile_binning_features
FeaturesMetadata.set()
FeaturesMetadata.text_features
FeaturesMetadata.time_features
FeaturesMetadata.to_dict()
FeaturesMetadata.to_json()
Method
- h2o_sonar.methods.core.stats module
- Module contents
- h2o_sonar.methods.utils package
- h2o_sonar.methods.core package
- Module contents
- Subpackages
- h2o_sonar.utils package
- Submodules
- h2o_sonar.utils.binning module
- h2o_sonar.utils.caching module
cache_all_models()
cache_baai_bge_small_en()
cache_baai_bge_small_env15()
cache_bert_base_uncased()
cache_bge_m3()
cache_detoxify_models()
cache_eval_studio_models()
cache_gptscore_evaluator_model()
cache_hkunlp_instructor()
cache_lmppl_perplexity_evaluator_model()
cache_nltk()
cache_nltk_averaged_perceptron_tagger()
cache_nltk_punkt()
cache_nltk_wordnet()
cache_summac_vitc()
cache_tiktoken_blobs()
cache_vectara_hallucination_model()
- h2o_sonar.utils.crypto module
- h2o_sonar.utils.io module
- h2o_sonar.utils.normalization module
- h2o_sonar.utils.preprocessing module
- h2o_sonar.utils.problem_detection module
- h2o_sonar.utils.perturbations module
AbcSynAntPerturbator
AntonymPerturbator
CommaPerturbator
ContextualMisinformationPerturbator
EncodingPerturbator
EncodingPerturbatorBase16
KeywordTyposCharacterPerturbator
Perturbator
PerturbatorDescriptor
PerturbatorRegistry
QwertyPerturbator
RandomCharacterDeletePerturbator
RandomCharacterInsertPerturbator
RandomCharacterReplacementPerturbator
RandomOCRCharacterPerturbator
SynonymPerturbator
WordSwapPerturbator
register_ootb_perturbators()
- h2o_sonar.utils.sampling module
- h2o_sonar.utils.sanitization module
- h2o_sonar.utils.testing module
InMemoryLlmHostPromptCache
InMemoryLlmHostPromptCache.KEY_DATA
InMemoryLlmHostPromptCache.add_test_lab()
InMemoryLlmHostPromptCache.clear()
InMemoryLlmHostPromptCache.evict()
InMemoryLlmHostPromptCache.get()
InMemoryLlmHostPromptCache.get_llm_model_names()
InMemoryLlmHostPromptCache.load_from_json()
InMemoryLlmHostPromptCache.load_from_url()
InMemoryLlmHostPromptCache.put()
InMemoryLlmHostPromptCache.save_to_json()
InMemoryLlmHostPromptCache.to_dict()
LlmHostPromptCache
LlmHostPromptCache.KEY_ACTUAL_OUTPUT
LlmHostPromptCache.KEY_CONTEXT
LlmHostPromptCache.KEY_CORPUS
LlmHostPromptCache.KEY_COST
LlmHostPromptCache.KEY_DURATION
LlmHostPromptCache.KEY_EXTRAS
LlmHostPromptCache.KEY_INPUT
LlmHostPromptCache.KEY_LLM_MODEL_NAME
LlmHostPromptCache.KEY_MODEL_TYPE
LlmHostPromptCache.PREFIX_KEY
LlmHostPromptCache.clear()
LlmHostPromptCache.evict()
LlmHostPromptCache.get()
LlmHostPromptCache.get_key()
LlmHostPromptCache.get_llm_model_names()
LlmHostPromptCache.put()
LlmHostPromptCache.str_key_to_dict()
RagTestCaseConfig
RagTestCaseConfig.KEY_CATEGORIES
RagTestCaseConfig.KEY_CONDITION
RagTestCaseConfig.KEY_CONSTRAINTS
RagTestCaseConfig.KEY_EXPECTED_OUTPUT
RagTestCaseConfig.KEY_KEY
RagTestCaseConfig.KEY_PROMPT
RagTestCaseConfig.KEY_RELS
RagTestCaseConfig.add_relationship()
RagTestCaseConfig.copy()
RagTestCaseConfig.perturb()
RagTestCaseConfig.to_dict()
RagTestConfig
RagTestLab
RagTestLab.bind()
RagTestLab.build()
RagTestLab.complete_dataset()
RagTestLab.complete_from_shards()
RagTestLab.from_eval_results()
RagTestLab.from_llm_test_suite()
RagTestLab.from_rag_test_suite()
RagTestLab.get_evaluated_model_for_key()
RagTestLab.insight_internal_llm_errors()
RagTestLab.integrity_check()
RagTestLab.load_from_json()
RagTestLab.merge()
RagTestLab.purge()
RagTestLab.split_to_shards()
RagTestLab.split_to_shards_by_model()
RagTestLab.stats()
RagTestLab.sync_documents()
RagTestLab.to_dict()
RagTestLab.trim()
RagTestLabPromptCache
RagTestLabPromptCache.ENV_VAR_H2O_SONAR_PROMPT_CACHE
RagTestLabPromptCache.ENV_VAR_H2O_SONAR_PROMPT_CACHE_SIZE
RagTestLabPromptCache.ENV_VAR_H2O_SONAR_PROMPT_CACHE_SRC
RagTestLabPromptCache.ENV_VAR_H2O_SONAR_PROMPT_CACHE_STATIC
RagTestLabPromptCache.MAX_ITEMS
RagTestLabPromptCache.cache()
RagTestLabPromptCache.reinitialize()
RagTestSuiteConfig
RagTestSuiteConfig.KEY_DESCRIPTION
RagTestSuiteConfig.KEY_NAME
RagTestSuiteConfig.KEY_TESTS
RagTestSuiteConfig.KEY_TEST_CASES
RagTestSuiteConfig.add_test_case()
RagTestSuiteConfig.copy()
RagTestSuiteConfig.from_llm_dataset()
RagTestSuiteConfig.load_from_json()
RagTestSuiteConfig.perturb()
RagTestSuiteConfig.save_as_json()
RagTestSuiteConfig.split()
RagTestSuiteConfig.tests
RagTestSuiteConfig.to_dict()
RagTestSuiteConfig.trim_tests()
TestLab
- Module contents
- Submodules
Submodules
h2o_sonar.config module
- class h2o_sonar.config.ConfigItemType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)
Bases:
Enum
Configuration item types.
- CONNECTION = 1
- LICENSE = 2
- class h2o_sonar.config.ConfigKeys
Bases:
object
- KEY_DESCRIPTION = 'description'
- KEY_ENCRYPTED = 'encrypted'
- KEY_NAME = 'name'
- class h2o_sonar.config.ConnectionConfig(connection_type: str, name: str, description: str, server_url: str = '', server_id: str = '', auth_server_url: str = '', environment_url: str = '', client_id: str = '', realm_name: str = '', token: str = '', token_use_type: str = '', username: str = '', password: str = '', extra_params: Dict = None, key: str = '')
Bases:
ConfigKeys
A generic purpose connection configuration which can be used to connect to various H2O.ai and 3rd party products and services.
Overview of the fields which are important for the H2O.ai authentication methods:
Username/password authentication method requires (Keycloak): - authentication server URL: auth_server_url - client ID: client_id - client secret: token + token_use_type==SECRET - username and password (on getting the access token)
Refresh token authentication method requires (Keycloak): - authentication server URL: auth_server_url - client ID: client_id - refresh token: token + token_use_type==REFRESH_TOKEN - realm: realm_name
H2O token provider authentication method requires: - environment URL: environment_url - refresh token: token + token_use_type==REFRESH_TOKEN
- ENCRYPTED_FIELDS = ['token', 'password']
- KEY_AUTH_SERVER_URL = 'auth_server_url'
- KEY_CLIENT_ID = 'client_id'
- KEY_ENV_URL = 'environment_url'
- KEY_EXTRA_PARAMS = 'extra_params'
- KEY_KEY = 'key'
- KEY_PASSWORD = 'password'
- KEY_REALM_NAME = 'realm_name'
- KEY_SERVER_ID = 'server_id'
- KEY_SERVER_URL = 'server_url'
- KEY_TOKEN = 'token'
- KEY_TOKEN_USE_TYPE = 'token_use_type'
- KEY_TYPE = 'connection_type'
- KEY_USERNAME = 'username'
- static from_dict(config_dict: Dict, decrypt: bool = True, encryption_key: str = '') ConnectionConfig
Create the connection configuration from dictionary.
JSon example w/ unencrypted fields:
- {
“key”: “096ca3c2-4715-11ee-9e2f-10828613f8ad”, “connection_type”: “ML_API”, “name”: “My connection name”, “description”: “My connection description.”, “server_url”: “http://localhost:8080”, “server_id”: “my-model-validation-dai”, “auth_server_url”: “http://localhost:8080/auth”, “environment_url”: “https://cloud-qa.h2o.ai/”, “realm_name”: “my_realm”, “client_id”: “my_client_id”, “token”: “”, “token_use_type”: “”, “username”: “sonaruser”, “password”: “s3cr3tpa33word”
}
JSon example w/ encrypted fields:
- {
“key”: “096ca3c2-4715-11ee-9e2f-10828613f8ad”, “connection_type”: “ML_API”, “name”: “My connection name”, “description”: “My connection description.”, “server_url”: “http://localhost:8080”, “server_id”: “my-model-validation-dai”, “auth_server_url”: “http://localhost:8080/auth”, “environment_url”: “https://cloud-qa.h2o.ai/”, “realm_name”: “my_realm”, “client_id”: “my_client_id”, “token”: {
“encrypted”: “gAAAAABkTfy18iis3ya8nitGi…UwGZ0z5XGBvZxu8URMxE14aJJk=”
}, “token_use_type”: “REFRESH_TOKEN”, “username”: “sonaruser”, “password”: {
“encrypted”: “py18iis3ya8nitG=”
}
}
- to_dict(encrypt: bool = True, encryption_key: str = '') Dict
- class h2o_sonar.config.ConnectionConfigType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)
Bases:
Enum
Predefined connection types.
- AMAZON_BEDROCK = 13
- AZURE_OPENAI_CHAT = 11
- DRIVERLESS_AI = 1
- DRIVERLESS_AI_AIEM = 3
- DRIVERLESS_AI_STEAM = 2
- H2O_3 = 4
- H2O_GPT = 5
- H2O_GPT_E = 6
- H2O_LLM_OPS = 7
- HF_SPACES = 12
- OLLAMA = 8
- OPENAI_CHAT = 10
- OPENAI_RAG = 9
- class h2o_sonar.config.EvaluationJudgeConfig(name: str, description: str, judge_type: str, connection: ConnectionConfig, llm_model_name: str = '', collection_id: str = '', key: str = '')
Bases:
object
Evaluation judge configuration.
- KEY_COLLECTION_ID = 'collection_id'
- KEY_CONNECTION = 'connection'
- KEY_DESCRIPTION = 'description'
- KEY_JUDGE_TYPE = 'judge_type'
- KEY_KEY = 'key'
- KEY_LLM_MODEL_NAME = 'llm_model_name'
- KEY_NAME = 'name'
- static from_dict(config_dict: Dict) EvaluationJudgeConfig
- to_dict() Dict
- class h2o_sonar.config.EvaluationJudgeType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)
Bases:
Enum
- azure_openai_llm = 7
- custom = 9
- h2ogpt = 1
- h2ogpte = 2
- h2ogpte_llm = 3
- h2ollmops = 4
- ollama = 8
- openai_llm = 6
- openai_rag = 5
- class h2o_sonar.config.H2o3Config
Bases:
object
H2O-3 configuration keys used by H2O Eval Studio.
- DEFAULT_AUTO_CLEANUP = True
- DEFAULT_AUTO_START = True
- DEFAULT_AUTO_STOP = False
- DEFAULT_HOST = 'localhost'
- DEFAULT_MAX_MEM_SIZE = '4G'
- DEFAULT_MIN_MEM_SIZE = '2G'
- DEFAULT_PORT = 12349
- KEYS = ['h2o_min_mem_size', 'h2o_max_mem_size', 'h2o_host', 'h2o_port', 'h2o_auto_start', 'h2o_auto_cleanup', 'h2o_auto_stop']
- KEY_AUTO_CLEANUP = 'h2o_auto_cleanup'
- KEY_AUTO_START = 'h2o_auto_start'
- KEY_AUTO_STOP = 'h2o_auto_stop'
- KEY_HOST = 'h2o_host'
- KEY_MAX_MEM_SIZE = 'h2o_max_mem_size'
- KEY_MIN_MEM_SIZE = 'h2o_min_mem_size'
- KEY_PORT = 'h2o_port'
- class h2o_sonar.config.H2oSonarConfig(connections: List[ConnectionConfig] | None = None, licenses: List[LicenseConfig] | None = None, evaluation_judges: List[EvaluationJudgeConfig] | None = None, ignore_env: bool | None = False)
Bases:
object
H2O Eval Studio configuration with global configuration items which impact H2O Eval Studio behavior, methods and explainers.
Configuration priority from lowest to highest:
default value
configuration file
environment variable
command line argument
configuration item
- CFG_CREATE_HTML_REPRESENTATIONS = <h2o_sonar.lib.api.commons.Param object>
- CFG_CUSTOM_EXPLAINERS = <h2o_sonar.lib.api.commons.Param object>
- CFG_DEVICE = <h2o_sonar.lib.api.commons.Param object>
- CFG_DO_SAMPLE = <h2o_sonar.lib.api.commons.Param object>
- CFG_ENABLE_DATASET_DOWNLOADING = <h2o_sonar.lib.api.commons.Param object>
- CFG_FORCE_EVAL_JUDGE = <h2o_sonar.lib.api.commons.Param object>
- CFG_H2O_AUTO_CLEANUP = <h2o_sonar.lib.api.commons.Param object>
- CFG_H2O_AUTO_START = <h2o_sonar.lib.api.commons.Param object>
- CFG_H2O_AUTO_STOP = <h2o_sonar.lib.api.commons.Param object>
- CFG_H2O_HOST = <h2o_sonar.lib.api.commons.Param object>
- CFG_H2O_MAX_MEM_SIZE = <h2o_sonar.lib.api.commons.Param object>
- CFG_H2O_MIN_MEM_SIZE = <h2o_sonar.lib.api.commons.Param object>
- CFG_H2O_PORT = <h2o_sonar.lib.api.commons.Param object>
- CFG_HTTP_SSL_CERT_VERIFY = <h2o_sonar.lib.api.commons.Param object>
- CFG_LOOK_AND_FEEL = <h2o_sonar.lib.api.commons.Param object>
- CFG_MODEL_CACHE_DIR = <h2o_sonar.lib.api.commons.Param object>
- CFG_MP_START_METHOD = <h2o_sonar.lib.api.commons.Param object>
- CFG_NUM_QUANTILES = <h2o_sonar.lib.api.commons.Param object>
- CFG_PER_EXPLAINER_LOGGER = <h2o_sonar.lib.api.commons.Param object>
- CFG_SAMPLE_SIZE = <h2o_sonar.lib.api.commons.Param object>
- ENV_VAR_CFG_PREFIX = 'H2O_SONAR_CFG_'
- KEY_CONNECTIONS = 'connections'
- KEY_CREATE_HTML_REPRESENTATIONS = 'create_html_representations'
- KEY_CUSTOM_EXPLAINERS = 'custom_explainers'
- KEY_DEVICE = 'device'
- KEY_EVALUATION_JUDGES = 'evaluation_judges'
- KEY_FORCE_EVAL_JUDGE = 'force_eval_judge'
- KEY_HTTP_SSL_CERT_VERIFY = 'http_ssl_cert_verify'
- KEY_LICENSES = 'licenses'
- KEY_LOOK_AND_FEEL = 'look_and_feel'
- KEY_MODEL_CACHE_DIR = 'model_cache_dir'
- KEY_PER_EXPLAINER_LOGGER = 'per_explainer_logger'
- MP_START_METHOD_FORK = 'fork'
- MP_START_METHOD_FORKSERVER = 'forkserver'
- MP_START_METHOD_SPAWN = 'spawn'
- VALUE_AUTO = ''
- VALUE_CPU = 'cpu'
- VALUE_CUDA = 'cuda'
- VALUE_GPU = 'gpu'
- VALUE_MPS = 'mps'
- VALUE_NPU = 'npu'
- VALUE_STR_FALSE = 'false'
- VALUE_STR_TRUE = 'true'
- add_connection(connection_config: ConnectionConfig)
- add_evaluation_judge(evaluation_judge_config: EvaluationJudgeConfig)
- add_license(license_config: LicenseConfig)
- copy()
- env_and_override()
Get configuration from environment variables following the naming convention and override this instance configuration.
- from_toml()
- get_connection(connection_key: str, connection_type: str = '') ConnectionConfig | None
- get_evaluation_judge(judge_key: str = '') EvaluationJudgeConfig | None
- get_license(license_key: str) LicenseConfig | None
- property http_ssl_cert_verify
- static load(config_path: str, encryption_key: str = '') Dict | None
Load JSon/TOML file with configuration items specified in the configuration file.
- Parameters:
- config_pathstr
Path to the configuration file.
- encryption_keystr
Optional encryption key to decrypt/encrypt sensitive data. If not specified, then shell environment variable
H2O_SONAR_ENCRYPTION_KEY
will be used.
- Returns:
- Dict
Dictionary with the configuration if file can be found and parsed,
None
otherwise.
- load_and_override(config_path: str, encryption_key: str = '')
Load JSon/TOML file and override this instance configuration items with configuration items specified in the configuration file.
- Parameters:
- config_pathstr
Path to the configuration file.
- encryption_keystr
Optional encryption key to decrypt encrypted fields in the configuration file. If not specified, then shell environment variable
H2O_SONAR_ENCRYPTION_KEY
will be used.
- save(config_path: str, config_data: Dict | None = None, encrypt: bool = True, encryption_key: str = '')
Save configuration to JSon file.
- Parameters:
- config_pathstr
Path to the configuration file.
- config_dataDict
Optional dictionary with configuration data
- encryptbool
Whether to encrypt sensitive data.
- encryption_keystr
Optional encryption key to encrypt sensitive data. If not specified, then shell environment variable
H2O_SONAR_ENCRYPTION_KEY
will be used.
- to_dict(encrypt: bool = True, encryption_key: str = '') Dict
- to_toml()
- class h2o_sonar.config.LicenseConfig(product: str, name: str, description: str, license: str = '', license_file: str = '', key: str = '')
Bases:
ConfigKeys
A product license configuration.
- ENCRYPTED_FIELDS = ['license']
- KEY_KEY = 'key'
- KEY_LICENSE = 'license'
- KEY_LICENSE_FILE = 'license_file'
- KEY_PRODUCT = 'product'
- static from_dict(config_dict: Dict, decrypt: bool = True, encryption_key: str = '') LicenseConfig
- to_dict(encrypt: bool = True, encryption_key: str = '') Dict
h2o_sonar.errors module
- exception h2o_sonar.errors.DatasetTooBigError(message, suggestion=None)
Bases:
MliError
The dataset is too big to be processed by H2O Eval Studio.
- exception h2o_sonar.errors.ExplainerCompatibilityError(message, suggestion=None)
Bases:
MliError
Explainer not compatible error.
- exception h2o_sonar.errors.InvalidArgumentError(message, suggestion=None)
Bases:
MliError
Invalid (CLI) argument error.
- exception h2o_sonar.errors.InvalidArgumentValueError(message, suggestion=None)
Bases:
MliError
Invalid (CLI) argument value error.
- exception h2o_sonar.errors.InvalidDataError(message, suggestion=None)
Bases:
MliError
Invalid data error.
- exception h2o_sonar.errors.MliError(message, suggestion=None)
Bases:
Exception
MLI error.
- exception h2o_sonar.errors.MliJsonDeserializationError(message, suggestion=None)
Bases:
MliError
MLI JSon deserialization error.
- exception h2o_sonar.errors.MliJsonSerializationError(message, suggestion=None)
Bases:
MliError
MLI JSon serialization error.
- exception h2o_sonar.errors.MliNotFoundError(message, suggestion=None)
Bases:
MliError
Entity not found error.
- exception h2o_sonar.errors.MliPredictMethodError(message, suggestion=None)
Bases:
MliError
Predict method failure.
- exception h2o_sonar.errors.MliTypeError(message, suggestion=None)
Bases:
MliError
Wrong type error.
- exception h2o_sonar.errors.MliUnsupportedDataFormatError(message, suggestion=None)
Bases:
MliUnsupportedError
Unsupported data type error.
- exception h2o_sonar.errors.MliUnsupportedOperationError(message, suggestion=None)
Bases:
MliUnsupportedError
Unsupported operation.
h2o_sonar.h2o_sonar_cli module
- h2o_sonar.h2o_sonar_cli.main() int
H2O Eval Studio Python library for Responsible AI command line interface (CLI). See
argparse
configuration for more details.
h2o_sonar.interpret module
- h2o_sonar.interpret.add_config_item(config_type: str, config_value: Dict | str, h2o_sonar_config_path: str | Path | None = None, encryption_key: str = '')
Add configuration item.
- Parameters:
- config_typestr
Configuration item type - see config.ConfigItemType for options.
- config_valueUnion[Dict, str]
Configuration item value represented either as dictionary or as string with JSon serialization of the configuration item. It is expected that the config item is NOT encrypted.
- h2o_sonar_config_pathOptional[Union[str, pathlib.Path]]
Path to the H2O Eval Studio configuration file. If
None
, then in-memory (current) configuration singleton is modified.- encryption_keystr
Encryption key to be used for encrypting sensitive data written in the configuration. If
None
, shell environment variableH2O_SONAR_ENCRYPTION_KEY
with the encryption key must be set.
- h2o_sonar.interpret.describe_explainer(explainer: Type[Explainer] | str) Dict
Get explainer description.
- Parameters:
- explainerUnion[Type[e8s.Explainer], str]
Explainer to describe.
- Returns:
- Dict
Dictionary with explainer name and parameters.
- h2o_sonar.interpret.gc()
Free system resources:
shutdowns process pool(s)
runs garbage collector
clears temporary files
- h2o_sonar.interpret.get_config(h2o_sonar_config_path: str | Path | None = None, encryption_key: str = '') Dict
Get configuration item.
- Parameters:
- h2o_sonar_config_pathOptional[Union[str, pathlib.Path]]
Path to the H2O Eval Studio configuration file. If
None
, then in-memory ( current) configuration singleton will be returned.- encryption_keyOptional[str]
Encryption key to be used for decryption of the sensitive data in the configuration. If not set, then the encryption key from shell environment variable
H2O_SONAR_ENCRYPTION_KEY
will be used. If encryption key is not provided, then sensitive data will be returned encrypted.
- h2o_sonar.interpret.list_explainers(experiment_types: List[str] | None = None, explanation_scopes: List[str] | None = None, model_meta: ExplainableModelMeta | None = None, keywords: List[str] | None = None, explainer_filter: List[FilterEntry] | None = None, container: str | ExplainerContainer | None = None, args_as_json_location: str | Path | None = None, extra_params: Dict | None = None) List[ExplainerDescriptor]
List explainers by supported experiment types, scores, keywords and other criteria. Parameters are used with
AND
logical operator.- Parameters:
- experiment_typesOptional[List[str]]
Filter explainers by supported experiment types - regression, binomial or multinomial.
- explanation_scopesOptional[List[str]]
Filter explainers by supported explanation scopes - local or global.
- model_metaOptional[models.ExplainableModelMeta]
Filter explainers by model metadata.
- keywordsOptional[List[str]]
Filter explainers by keywords.
- explainer_filterOptional[List[commons.FilterEntry]]
Filter explainers by generic filter (forward compatible filtering). See
FilterEntry
for supported search types.- containerOptional[Union[str, explainer_container.ExplainerContainer]]
Optional explainer container name or instance to be used to run the interpretation.
- args_as_json_locationOptional[Union[str, pathlib.Path]]
Load all positional arguments and keyword arguments from JSon file. This is useful when input is generated, persisted, repeated and used from CLI (which doesn’t support all the options). IMPORTANT: if this argument is specified, then all other function parameters are ignored.
- extra_paramsOptional[Dict]
Extra parameters.
- Returns:
- List[explainers.ExplainerDescriptor]
Explainers compliant with provided filters.
- h2o_sonar.interpret.list_interpretations(results_location: str | Path | Dict, persistence_type: PersistenceType = PersistenceType.file_system, container: str | ExplainerContainer | None = None, log_level: int = 30, extra_params: Dict | None = None) List[str]
List interpretations in given results location.
- Parameters:
- results_locationUnion[str, pathlib.Path, Dict]
Location used e.g. by
run_interpretation()
to store interpretations results - filesystem (path as string orPath
), memory (dictionary) or DB. IfNone
, then results are loaded from the current directory.- persistence_typepersist.PersistenceType
Optional choice of the persistence type: file-system (default), in-memory or database. This option does not override persistence type in case that container is provided.
- containerOptional[Union[str, explainer_container.ExplainerContainer]]
Optional explainer container name (str) or container instance to be used to run the interpretation.
- log_levelint
Optional container and explainers log level.
- extra_paramsOptional[Dict]
Extra parameters.
- Returns:
- List[str]
Interpretation keys in given results location.
- h2o_sonar.interpret.register_explainer(explainer_class: Type[Explainer], explainer_id: str = '', container: str | ExplainerContainer | None = None, extra_params: Dict | None = None) str
Register explainer.
- Parameters:
- explainer_classType[explainers.Explainer]
Explainer class.
- explainer_idstr
Optional custom explainer ID to be used for explainer identification.
- containerOptional[Union[str, explainer_container.ExplainerContainer]]
Optional explainer container name or instance to be used to run the interpretation.
- extra_paramsOptional[Dict]
Extra parameters.
- Returns:
- str
Explainer ID.
- h2o_sonar.interpret.resolve_container(container: str | ExplainerContainer | None = None, results_location: str | Any = '', persistence_api: PersistenceApi | None = None, persistence_type: PersistenceType | None = PersistenceType.file_system, do_setup: bool = True, logger: SonarLogger | None = None, log_level: int | None = None) ExplainerContainer
Get explainer container instance to configure, register explainers and tune it.
- Parameters:
- containerOptional[Union[str, explainer_container.ExplainerContainer]]
Optional explainer container name (str) or container instance to be used to run the interpretation.
- results_locationOptional[Union[str, pathlib.Path, Dict, Any]]
Where to store interpretation results - filesystem (path as string or
Path
), memory (dictionary) or DB. IfNone
, then results are stored to the current directory.- persistence_apiOptional[persist.PersistenceApi]
Instance of the persistence API allowing to create various persistence types (like file-system or DB)
- persistence_typepersist.PersistenceType
Optional choice of the persistence type: file-system (default), in-memory or database. This option does not override persistence type in case that container is provided.
- do_setupbool
Ensure explainer container set up.
- loggerOptional[loggers.SonarLogger]
Optional custom logger.
- log_levelint
Optional container and explainers log level.
- Returns:
- explainer_container.ExplainerContainer
Explainer container instance.
- h2o_sonar.interpret.run_interpretation(dataset: str | Path | ExplainableDataset | ResourceHandle | Frame | DataFrame | ExplainableDatasetHandle | hmli.H2OFrame, model: str | Path | ExplainableModel | ResourceHandle | ExplainableModelHandle | Any | None = None, models: List[str | Path | ExplainableModel | ResourceHandle | ExplainableModelHandle | Any] | None = None, target_col: str = '', explainers: List[str | ExplainerToRun] | None = None, explainer_keywords: List[str] | None = None, validset: str | Path | ExplainableDataset | ResourceHandle | Frame | DataFrame | ExplainableDatasetHandle | hmli.H2OFrame | None = None, testset: str | Path | ExplainableDataset | ResourceHandle | Frame | DataFrame | ExplainableDatasetHandle | hmli.H2OFrame | None = None, use_raw_features: bool = True, used_features: List | None = None, weight_col: str = '', prediction_col: str = '', drop_cols: List | None = None, sample_num_rows: int | None = 0, sampler: DatasetSampler | None = None, container: str | ExplainerContainer | None = None, results_location: str | Path | Dict | Any | None = None, results_formats: List[str] | None = None, persistence_type: PersistenceType = PersistenceType.file_system, run_asynchronously: bool = False, run_explainers_in_parallel: bool = False, progress_callback: AbstractProgressCallbackContext | None = None, logger: SonarLogger | None = None, log_level: int = 30, args_as_json_location: str | Path | None = None, upload_to: str | ConnectionConfig = None, key: str = '', extra_params: Dict | None = None) Interpretation
Run interpretation.
- Parameters:
- datasetUnion[str, Path, ExplainableDataset, datatable.Frame, hmli.H2OFrame, Any]
Dataset source: explainable dataset instance, datatable frame, string (expect path to CSV, .jay or any other file type supported by datatable), dictionary (used to construct frame).
- modelUnion[str, Path, ExplainableModel, Any]
Path to model (str, Path), explainable model (
ExplainableModel
) or an instance of 3rd party model (like Scikit) to interpret.- modelsList[Union[str, Path, ExplainableModel, Any]]
Paths to models (str, Path), explainable models (
ExplainableModel
) or an instances of 3rd party models (like Scikit) to interpret.- target_colstr
Target column name - must be valid dataset column name.
- explainersOptional[List[Union[str, commons.ExplainerToRun]]]
Explainer IDs to run within the interpretation or
ExplainerToRun
instances with explainer parameters. In case ofNone
or empty list are run all default compatible explainers.- explainer_keywords: Optional[List[str]]
Run compatible explainers which have given keyword (AND). This setting is used only in case that
explainers
parameter is empty list (orNone
).- validsetOptional[Union[src, Path, ExplainableDataset, hmli.H2OFrame, Any]]
Optional path to validation dataset (str, Path) or datatable Frame instance.
- testsetOptional[Union[src, Path, ExplainableDataset, hmli.H2OFrame, Any]]
Optional path to test dataset (str, Path) or datatable Frame instance.
- use_raw_featuresbool
True
to use original features (default),False
to force the use of transformed features in surrogate models.- used_featuresOptional[List]
Optional parameter specifying features (dataset columns) used by the model. This parameter is used in case that an instance of the model (not
ExplainableModel
) is provided by the user - thereforeExplainableModel
’s metadata are not available.- weight_colstr
Name of the weight column to be used by explainers.
- prediction_colstr
Name of the predictions column - in case of 3rd party model (standalone MLI).
- drop_colsOptional[List]
List of the columns to drop from the interpretation i.e. columns names which should not be explained.
- sample_num_rowsOptional[int]
If
None
, then automatically sample based on the dataset and RAM size. If > 0, then do sample thedataset
tosample_num_rows
number of rows. If == 0, then do NOT sample.- samplerOptional[DatasetSampler]
Sampling method (implementation) to be used - see
h2o_sonar.utils.sampling
module (documentation) for available sampling methods. Use a sampler instance to use the specific sampling method.- containerOptional[Union[str, explainer_container.ExplainerContainer]]
Optional explainer container name (str) or container instance to be used to run the interpretation.
- results_locationOptional[Union[str, pathlib.Path, Dict, Any]]
Where to store interpretation results - filesystem (path as string or
Path
), memory (dictionary) or DB. IfNone
, then results are stored to the current directory.- results_formatsOptional[List[str]]
Optional list of the result formats (MIME types) to be generated. If
None
, then report in HTML and JSon is created. Supported formats:MIME_PDF
,MIME_HTML
andMIME_JSON
.- persistence_typepersist.PersistenceType
Optional choice of the persistence type: file-system (default), in-memory or database. This option does not override persistence type in case that container is provided.
- run_asynchronouslybool
True
to run the interpretation asynchronously - the interpretation is run synchronously by default (False
).- run_explainers_in_parallelbool
True
to run explainers in parallel - explainers are run sequentially by default.- progress_callbackOptional[progress.AbstractProgressCallbackContext]
Optional progress callback context which is stacked atop default logging callback in
h2o_sonar.lib.api.interpretations::Interpretation
constructor.- loggerOptional[loggers.SonarLogger]
Optional custom logger which implements
loggers.SonarLogger
interface. Iflogger
is provided, thenlog_level
is ignored.- log_levelint
Optional container and explainers log level. If
logger
is provided, thenlog_level
is ignored.- args_as_json_locationOptional[Union[str, pathlib.Path]]
Load all positional arguments and keyword arguments from JSon file. This is useful when input is generated, persisted, repeated and used from CLI (which doesn’t support all the options). IMPORTANT: if this argument is specified, then all other function parameters are ignored.
- upload_toUnion[str, config.ConnectionConfig]
Upload the interpretation report to the H2O GPT Enterprise in order to talk to the report.
- keystr
Custom interpretation key which must be valid UUID4 string.
- extra_paramsOptional[Dict]
Extra parameters.
- Returns:
- interpretations.Interpretation
Interpretations instance with the explainer results (references).
- h2o_sonar.interpret.unregister_explainer(explainer_id: str, container: str | ExplainerContainer | None = None, extra_params: Dict | None = None) str
Unregister explainer.
- Parameters:
- explainer_idstr
Custom explainer to be unregistered.
- containerOptional[Union[str, explainer_container.ExplainerContainer]]
Optional explainer container name or instance to be used to run the interpretation.
- extra_paramsOptional[Dict]
Extra parameters.
- Returns:
- str
ID of the unregistered explainer or
""
no explainer was unregistered.
- h2o_sonar.interpret.upload_interpretation(interpretation_result: Interpretation | InterpretationResult | str | Path, connection: str | ConnectionConfig, collection_name: str = '', extra_params: Dict | None = None) Tuple[str, str]
Upload interpretation to an LLM like h2oGPTE in order to talk to the interpretation report.
- Returns:
- Tuple[str, str]
h2oGPT Enterprise collection ID and URL.
h2o_sonar.loggers module
- class h2o_sonar.loggers.SonarFileLogger(logger_name: str, log_file: str, log_level=30)
Bases:
SonarLogger
File logger saves log messages to log files.
- FORMATTER = <logging.Formatter object>
- data(data, *args, **kwargs)
- debug(msg, *args, **kwargs)
- error(msg, *args, **kwargs)
- info(msg, *args, **kwargs)
- warning(msg, *args, **kwargs)
- class h2o_sonar.loggers.SonarLogger
Bases:
ABC
Abstract logger base class to be extended by runtime specific logging implementations.
- FILE_NAME_H2O_SONAR_LOG = 'h2o-sonar.log'
- data(data, *args, **kwargs)
- debug(msg, *args, **kwargs)
- error(msg, *args, **kwargs)
- info(msg, *args, **kwargs)
- warning(msg, *args, **kwargs)
- class h2o_sonar.loggers.SonarPrintLogger
Bases:
SonarLogger
Print logger prints log messages using print() method.
- data(data, *args, **kwargs)
- debug(msg, *args, **kwargs)
- error(msg, *args, **kwargs)
- info(msg, *args, **kwargs)
- warning(msg, *args, **kwargs)
- h2o_sonar.loggers.debug(msg, *args, **kwargs)
- h2o_sonar.loggers.error(msg, *args, **kwargs)
- h2o_sonar.loggers.fatal(msg, *args, **kwargs)
- h2o_sonar.loggers.get_level()
- h2o_sonar.loggers.info(msg, *args, **kwargs)
- h2o_sonar.loggers.log(level, msg, *args, **kwargs)
- h2o_sonar.loggers.setLevel(level=0)
- h2o_sonar.loggers.warn(msg, *args, **kwargs)
- h2o_sonar.loggers.warning(msg, *args, **kwargs)