# LLMRegistryApi Method | HTTP request | Description ------------- | ------------- | ------------- [**create_runtime_model**](LLMRegistryApi.md#create-runtime-model) | **POST** /llm-registry/admin/runtime-models | Creates a runtime LLM (admin).| **POST** /llm-registry/admin/runtime-models | Creates a runtime LLM (admin). [**delete_runtime_model**](LLMRegistryApi.md#delete-runtime-model) | **DELETE** /llm-registry/admin/runtime-models/{id} | Deletes a runtime LLM by registry id (admin).| **DELETE** /llm-registry/admin/runtime-models/{id} | Deletes a runtime LLM by registry id (admin). [**get_llm_registry_admin_models**](LLMRegistryApi.md#get-llm-registry-admin-models) | **GET** /llm-registry/admin/models | Lists the full LLM registry (admin).| **GET** /llm-registry/admin/models | Lists the full LLM registry (admin). [**get_role_llm_registry_access**](LLMRegistryApi.md#get-role-llm-registry-access) | **GET** /roles/{role-id}/llm-registry-access | Lists LLM access grants for a role.| **GET** /roles/{role_id}/llm-registry-access | Lists LLM access grants for a role. [**set_role_llm_registry_access**](LLMRegistryApi.md#set-role-llm-registry-access) | **PUT** /roles/{role-id}/llm-registry-access | Replaces LLM access grants for a role.| **PUT** /roles/{role_id}/llm-registry-access | Replaces LLM access grants for a role. [**test_runtime_model**](LLMRegistryApi.md#test-runtime-model) | **POST** /llm-registry/admin/runtime-models/test | Tests a runtime LLM connection (admin).| **POST** /llm-registry/admin/runtime-models/test | Tests a runtime LLM connection (admin). [**test_saved_runtime_model**](LLMRegistryApi.md#test-saved-runtime-model) | **POST** /llm-registry/admin/runtime-models/{id}/test | Tests and updates the readiness of a saved runtime LLM (admin).| **POST** /llm-registry/admin/runtime-models/{id}/test | Tests and updates the readiness of a saved runtime LLM (admin). [**update_runtime_model**](LLMRegistryApi.md#update-runtime-model) | **PUT** /llm-registry/admin/runtime-models/{id} | Not supported -- updating a runtime LLM is disabled (admin).| **PUT** /llm-registry/admin/runtime-models/{id} | Not supported -- updating a runtime LLM is disabled (admin). # **create_runtime_model** > Dict[str, object] create_runtime_model(request_body, skip_test=skip_test) Creates a runtime LLM (admin). Creates a UI/API runtime LLM. The request body is the model configuration (display_name, base_model, inference_server, and any LiteLLM/provider fields) and is stored verbatim. The connection is probed before the model is saved; a failed probe rejects the create. With skip_test=true, the configuration is saved as not_probed and is unavailable for inference until a test of the saved model succeeds. Skipping does not bypass structural validation. A skipped configuration waits for a manual test while it is still not_probed, including after a restart. A FAILED manual test records probe_failed, which does put the configuration back into the startup re-probe population: once its endpoint recovers, a restart promotes it to ok and grants it by the default rules with no further admin action. display_name must be the name the model is served under whenever that name is a LiteLLM proxy routing key: the lite_llm_yaml block's top-level model_name when the config has a block, else base_model when inference_server is \"litellm\". A different display_name is rejected (400), because the model is registered with the proxy under that name. A model reached directly at its provider (inference_server anthropic, google, a vLLM endpoint, ...) with no block is registered under display_name itself, so its display_name is free. A block mux cannot read (hand-written YAML, or JSON declaring no top-level model_name, including model_list) is rejected. Admin only. ### Example * Bearer Authentication (bearerAuth): ```python import h2ogpte.rest_sync from h2ogpte.rest_sync.rest import ApiException from pprint import pprint # Defining the host is optional and defaults to https://h2ogpte.genai.h2o.ai/api/v1 # See configuration.py for a list of all supported configuration parameters. configuration = h2ogpte.rest_sync.Configuration( host = "https://h2ogpte.genai.h2o.ai/api/v1" ) # The client must configure the authentication and authorization parameters # in accordance with the API server security policy. # Examples for each auth method are provided below, use the example that # satisfies your auth use case. # Configure Bearer authorization: bearerAuth configuration = h2ogpte.rest_sync.Configuration( access_token = os.environ["BEARER_TOKEN"] ) # Enter a context with an instance of the API client with h2ogpte.rest_sync.ApiClient(configuration) as api_client: # Create an instance of the API class api_instance = h2ogpte.rest_sync.LLMRegistryApi(api_client) request_body = None # Dict[str, object] | skip_test = False # bool | Save without a connection test; the model remains unavailable until tested successfully. (optional) (default to False) try: # Creates a runtime LLM (admin). api_response = api_instance.create_runtime_model(request_body, skip_test=skip_test) print("The response of LLMRegistryApi->create_runtime_model:\n") pprint(api_response) except Exception as e: print("Exception when calling LLMRegistryApi->create_runtime_model: %s\n" % e) ``` ### Parameters Name | Type | Description | Notes ------------- | ------------- | ------------- | ------------- **request_body** | [**Dict[str, object]**](object.md)| | **skip_test** | **bool**| Save without a connection test; the model remains unavailable until tested successfully. | [optional] [default to False] ### Return type **Dict[str, object]** ### HTTP request headers - **Content-Type**: application/json - **Accept**: application/json ### HTTP response details | Status code | Description | Response headers | |-------------|-------------|------------------| **201** | Successful operation | - | **400** | Bad request | - | **401** | Unauthorized - Invalid or missing API key | - | **403** | Forbidden | - | **409** | Conflict | - | # **delete_runtime_model** > Dict[str, object] delete_runtime_model(id) Deletes a runtime LLM by registry id (admin). Hard-deletes a UI/API runtime LLM by registry UUID (delete is final). Admin only. ### Example * Bearer Authentication (bearerAuth): ```python import h2ogpte.rest_sync from h2ogpte.rest_sync.rest import ApiException from pprint import pprint # Defining the host is optional and defaults to https://h2ogpte.genai.h2o.ai/api/v1 # See configuration.py for a list of all supported configuration parameters. configuration = h2ogpte.rest_sync.Configuration( host = "https://h2ogpte.genai.h2o.ai/api/v1" ) # The client must configure the authentication and authorization parameters # in accordance with the API server security policy. # Examples for each auth method are provided below, use the example that # satisfies your auth use case. # Configure Bearer authorization: bearerAuth configuration = h2ogpte.rest_sync.Configuration( access_token = os.environ["BEARER_TOKEN"] ) # Enter a context with an instance of the API client with h2ogpte.rest_sync.ApiClient(configuration) as api_client: # Create an instance of the API class api_instance = h2ogpte.rest_sync.LLMRegistryApi(api_client) id = 'id_example' # str | The registry UUID of the runtime model. try: # Deletes a runtime LLM by registry id (admin). api_response = api_instance.delete_runtime_model(id) print("The response of LLMRegistryApi->delete_runtime_model:\n") pprint(api_response) except Exception as e: print("Exception when calling LLMRegistryApi->delete_runtime_model: %s\n" % e) ``` ### Parameters Name | Type | Description | Notes ------------- | ------------- | ------------- | ------------- **id** | **str**| The registry UUID of the runtime model. | ### Return type **Dict[str, object]** ### HTTP request headers - **Content-Type**: Not defined - **Accept**: application/json ### HTTP response details | Status code | Description | Response headers | |-------------|-------------|------------------| **200** | Successful operation | - | **401** | Unauthorized - Invalid or missing API key | - | **403** | Forbidden | - | **404** | Not found | - | # **get_llm_registry_admin_models** > List[Dict[str, object]] get_llm_registry_admin_models() Lists the full LLM registry (admin). Returns every effective model with its lower-priority shadow candidates. The full provider configuration is returned under raw_config and MAY include secrets (e.g. api_key, inference_server credentials) -- this matches the legacy admin-only runtime_llms setting admins edited directly. Not yet on the serving path. Admin only. ### Example * Bearer Authentication (bearerAuth): ```python import h2ogpte.rest_sync from h2ogpte.rest_sync.rest import ApiException from pprint import pprint # Defining the host is optional and defaults to https://h2ogpte.genai.h2o.ai/api/v1 # See configuration.py for a list of all supported configuration parameters. configuration = h2ogpte.rest_sync.Configuration( host = "https://h2ogpte.genai.h2o.ai/api/v1" ) # The client must configure the authentication and authorization parameters # in accordance with the API server security policy. # Examples for each auth method are provided below, use the example that # satisfies your auth use case. # Configure Bearer authorization: bearerAuth configuration = h2ogpte.rest_sync.Configuration( access_token = os.environ["BEARER_TOKEN"] ) # Enter a context with an instance of the API client with h2ogpte.rest_sync.ApiClient(configuration) as api_client: # Create an instance of the API class api_instance = h2ogpte.rest_sync.LLMRegistryApi(api_client) try: # Lists the full LLM registry (admin). api_response = api_instance.get_llm_registry_admin_models() print("The response of LLMRegistryApi->get_llm_registry_admin_models:\n") pprint(api_response) except Exception as e: print("Exception when calling LLMRegistryApi->get_llm_registry_admin_models: %s\n" % e) ``` ### Parameters This endpoint does not need any parameter. ### Return type **List[Dict[str, object]]** ### HTTP request headers - **Content-Type**: Not defined - **Accept**: application/json ### HTTP response details | Status code | Description | Response headers | |-------------|-------------|------------------| **200** | Successful operation | - | **401** | Unauthorized - Invalid or missing API key | - | **403** | Forbidden | - | # **get_role_llm_registry_access** > List[Dict[str, object]] get_role_llm_registry_access(role_id) Lists LLM access grants for a role. Returns every active effective model name plus whether the role is granted access to it. Requires the Manage Roles permission. ### Example * Bearer Authentication (bearerAuth): ```python import h2ogpte.rest_sync from h2ogpte.rest_sync.rest import ApiException from pprint import pprint # Defining the host is optional and defaults to https://h2ogpte.genai.h2o.ai/api/v1 # See configuration.py for a list of all supported configuration parameters. configuration = h2ogpte.rest_sync.Configuration( host = "https://h2ogpte.genai.h2o.ai/api/v1" ) # The client must configure the authentication and authorization parameters # in accordance with the API server security policy. # Examples for each auth method are provided below, use the example that # satisfies your auth use case. # Configure Bearer authorization: bearerAuth configuration = h2ogpte.rest_sync.Configuration( access_token = os.environ["BEARER_TOKEN"] ) # Enter a context with an instance of the API client with h2ogpte.rest_sync.ApiClient(configuration) as api_client: # Create an instance of the API class api_instance = h2ogpte.rest_sync.LLMRegistryApi(api_client) role_id = 'role_id_example' # str | The unique identifier of a role. try: # Lists LLM access grants for a role. api_response = api_instance.get_role_llm_registry_access(role_id) print("The response of LLMRegistryApi->get_role_llm_registry_access:\n") pprint(api_response) except Exception as e: print("Exception when calling LLMRegistryApi->get_role_llm_registry_access: %s\n" % e) ``` ### Parameters Name | Type | Description | Notes ------------- | ------------- | ------------- | ------------- **role_id** | **str**| The unique identifier of a role. | ### Return type **List[Dict[str, object]]** ### HTTP request headers - **Content-Type**: Not defined - **Accept**: application/json ### HTTP response details | Status code | Description | Response headers | |-------------|-------------|------------------| **200** | Successful operation | - | **400** | Bad request | - | **401** | Unauthorized - Invalid or missing API key | - | **403** | Forbidden | - | **404** | Not found | - | # **set_role_llm_registry_access** > set_role_llm_registry_access(role_id, set_role_llm_registry_access_request) Replaces LLM access grants for a role. Replaces the role's LLM access grant set. The protected roles viewer and admin are rejected. Requires the Manage Roles permission. ### Example * Bearer Authentication (bearerAuth): ```python import h2ogpte.rest_sync from h2ogpte.rest_sync.models.set_role_llm_registry_access_request import SetRoleLlmRegistryAccessRequest from h2ogpte.rest_sync.rest import ApiException from pprint import pprint # Defining the host is optional and defaults to https://h2ogpte.genai.h2o.ai/api/v1 # See configuration.py for a list of all supported configuration parameters. configuration = h2ogpte.rest_sync.Configuration( host = "https://h2ogpte.genai.h2o.ai/api/v1" ) # The client must configure the authentication and authorization parameters # in accordance with the API server security policy. # Examples for each auth method are provided below, use the example that # satisfies your auth use case. # Configure Bearer authorization: bearerAuth configuration = h2ogpte.rest_sync.Configuration( access_token = os.environ["BEARER_TOKEN"] ) # Enter a context with an instance of the API client with h2ogpte.rest_sync.ApiClient(configuration) as api_client: # Create an instance of the API class api_instance = h2ogpte.rest_sync.LLMRegistryApi(api_client) role_id = 'role_id_example' # str | The unique identifier of a role. set_role_llm_registry_access_request = h2ogpte.rest_sync.SetRoleLlmRegistryAccessRequest() # SetRoleLlmRegistryAccessRequest | try: # Replaces LLM access grants for a role. api_instance.set_role_llm_registry_access(role_id, set_role_llm_registry_access_request) except Exception as e: print("Exception when calling LLMRegistryApi->set_role_llm_registry_access: %s\n" % e) ``` ### Parameters Name | Type | Description | Notes ------------- | ------------- | ------------- | ------------- **role_id** | **str**| The unique identifier of a role. | **set_role_llm_registry_access_request** | [**SetRoleLlmRegistryAccessRequest**](SetRoleLlmRegistryAccessRequest.md)| | ### Return type void (empty response body) ### HTTP request headers - **Content-Type**: application/json - **Accept**: application/json ### HTTP response details | Status code | Description | Response headers | |-------------|-------------|------------------| **204** | Successful operation | - | **400** | Bad request | - | **401** | Unauthorized - Invalid or missing API key | - | **403** | Forbidden | - | **404** | Not found | - | # **test_runtime_model** > Dict[str, object] test_runtime_model(request_body) Tests a runtime LLM connection (admin). Validates a model configuration and asks core to probe it (a non-caching, non-persisting metadata round-trip). No registry write occurs. Applies the same display_name rule as create, so a configuration the create path would refuse never reports a successful test. Admin only. ### Example * Bearer Authentication (bearerAuth): ```python import h2ogpte.rest_sync from h2ogpte.rest_sync.rest import ApiException from pprint import pprint # Defining the host is optional and defaults to https://h2ogpte.genai.h2o.ai/api/v1 # See configuration.py for a list of all supported configuration parameters. configuration = h2ogpte.rest_sync.Configuration( host = "https://h2ogpte.genai.h2o.ai/api/v1" ) # The client must configure the authentication and authorization parameters # in accordance with the API server security policy. # Examples for each auth method are provided below, use the example that # satisfies your auth use case. # Configure Bearer authorization: bearerAuth configuration = h2ogpte.rest_sync.Configuration( access_token = os.environ["BEARER_TOKEN"] ) # Enter a context with an instance of the API client with h2ogpte.rest_sync.ApiClient(configuration) as api_client: # Create an instance of the API class api_instance = h2ogpte.rest_sync.LLMRegistryApi(api_client) request_body = None # Dict[str, object] | try: # Tests a runtime LLM connection (admin). api_response = api_instance.test_runtime_model(request_body) print("The response of LLMRegistryApi->test_runtime_model:\n") pprint(api_response) except Exception as e: print("Exception when calling LLMRegistryApi->test_runtime_model: %s\n" % e) ``` ### Parameters Name | Type | Description | Notes ------------- | ------------- | ------------- | ------------- **request_body** | [**Dict[str, object]**](object.md)| | ### Return type **Dict[str, object]** ### HTTP request headers - **Content-Type**: application/json - **Accept**: application/json ### HTTP response details | Status code | Description | Response headers | |-------------|-------------|------------------| **200** | Successful operation | - | **400** | Bad request | - | **401** | Unauthorized - Invalid or missing API key | - | **403** | Forbidden | - | # **test_saved_runtime_model** > Dict[str, object] test_saved_runtime_model(id) Tests and updates the readiness of a saved runtime LLM (admin). Tests the stored UI/API runtime configuration identified by registry ID. A successful test makes the model serving-eligible. A failed test preserves the configuration and records probe_failed, the error, and the test time; a probe_failed configuration is re-probed automatically at startup, so it can become serving-eligible later without another manual test. Returns 200 with ok=false for a model test failure. A core transport or invalid-response error leaves the prior readiness unchanged. No request body is accepted; the saved configuration is used. Admin only. ### Example * Bearer Authentication (bearerAuth): ```python import h2ogpte.rest_sync from h2ogpte.rest_sync.rest import ApiException from pprint import pprint # Defining the host is optional and defaults to https://h2ogpte.genai.h2o.ai/api/v1 # See configuration.py for a list of all supported configuration parameters. configuration = h2ogpte.rest_sync.Configuration( host = "https://h2ogpte.genai.h2o.ai/api/v1" ) # The client must configure the authentication and authorization parameters # in accordance with the API server security policy. # Examples for each auth method are provided below, use the example that # satisfies your auth use case. # Configure Bearer authorization: bearerAuth configuration = h2ogpte.rest_sync.Configuration( access_token = os.environ["BEARER_TOKEN"] ) # Enter a context with an instance of the API client with h2ogpte.rest_sync.ApiClient(configuration) as api_client: # Create an instance of the API class api_instance = h2ogpte.rest_sync.LLMRegistryApi(api_client) id = 'id_example' # str | try: # Tests and updates the readiness of a saved runtime LLM (admin). api_response = api_instance.test_saved_runtime_model(id) print("The response of LLMRegistryApi->test_saved_runtime_model:\n") pprint(api_response) except Exception as e: print("Exception when calling LLMRegistryApi->test_saved_runtime_model: %s\n" % e) ``` ### Parameters Name | Type | Description | Notes ------------- | ------------- | ------------- | ------------- **id** | **str**| | ### Return type **Dict[str, object]** ### HTTP request headers - **Content-Type**: Not defined - **Accept**: application/json ### HTTP response details | Status code | Description | Response headers | |-------------|-------------|------------------| **200** | Persisted test outcome (ok, and meta or error). | - | **400** | Bad request | - | **401** | Unauthorized - Invalid or missing API key | - | **403** | Forbidden | - | **404** | Not found | - | # **update_runtime_model** > EndpointError update_runtime_model(id) Not supported -- updating a runtime LLM is disabled (admin). Disabled: a runtime LLM can only be created and deleted. Any request returns 400 -- delete the model and create it again to change its configuration. display_name is the routing key at the serving proxy, so an in-place edit can leave a saved, successfully probed model whose every request fails at the proxy. Admin only. ### Example * Bearer Authentication (bearerAuth): ```python import h2ogpte.rest_sync from h2ogpte.rest_sync.models.endpoint_error import EndpointError from h2ogpte.rest_sync.rest import ApiException from pprint import pprint # Defining the host is optional and defaults to https://h2ogpte.genai.h2o.ai/api/v1 # See configuration.py for a list of all supported configuration parameters. configuration = h2ogpte.rest_sync.Configuration( host = "https://h2ogpte.genai.h2o.ai/api/v1" ) # The client must configure the authentication and authorization parameters # in accordance with the API server security policy. # Examples for each auth method are provided below, use the example that # satisfies your auth use case. # Configure Bearer authorization: bearerAuth configuration = h2ogpte.rest_sync.Configuration( access_token = os.environ["BEARER_TOKEN"] ) # Enter a context with an instance of the API client with h2ogpte.rest_sync.ApiClient(configuration) as api_client: # Create an instance of the API class api_instance = h2ogpte.rest_sync.LLMRegistryApi(api_client) id = 'id_example' # str | The registry UUID of the runtime model. try: # Not supported -- updating a runtime LLM is disabled (admin). api_response = api_instance.update_runtime_model(id) print("The response of LLMRegistryApi->update_runtime_model:\n") pprint(api_response) except Exception as e: print("Exception when calling LLMRegistryApi->update_runtime_model: %s\n" % e) ``` ### Parameters Name | Type | Description | Notes ------------- | ------------- | ------------- | ------------- **id** | **str**| The registry UUID of the runtime model. | ### Return type [**EndpointError**](EndpointError.md) ### HTTP request headers - **Content-Type**: Not defined - **Accept**: application/json ### HTTP response details | Status code | Description | Response headers | |-------------|-------------|------------------| **400** | Bad request | - | **401** | Unauthorized - Invalid or missing API key | - | **403** | Forbidden | - |