LLMRegistryApi

Method

HTTP request

Description

create_runtime_model

POST /llm-registry/admin/runtime-models

Creates a runtime LLM (admin).

delete_runtime_model

DELETE /llm-registry/admin/runtime-models/{id}

Deletes a runtime LLM by registry id (admin).

get_llm_registry_admin_models

GET /llm-registry/admin/models

Lists the full LLM registry (admin).

get_role_llm_registry_access

GET /roles/{role-id}/llm-registry-access

Lists LLM access grants for a role.

set_role_llm_registry_access

PUT /roles/{role-id}/llm-registry-access

Replaces LLM access grants for a role.

test_runtime_model

POST /llm-registry/admin/runtime-models/test

Tests a runtime LLM connection (admin).

test_saved_runtime_model

POST /llm-registry/admin/runtime-models/{id}/test

Tests and updates the readiness of a saved runtime LLM (admin).

update_runtime_model

PUT /llm-registry/admin/runtime-models/{id}

Not supported – updating a runtime LLM is disabled (admin).

create_runtime_model

Dict[str, object] create_runtime_model(request_body, skip_test=skip_test)

Creates a runtime LLM (admin).

Creates a UI/API runtime LLM. The request body is the model configuration (display_name, base_model, inference_server, and any LiteLLM/provider fields) and is stored verbatim. The connection is probed before the model is saved; a failed probe rejects the create. With skip_test=true, the configuration is saved as not_probed and is unavailable for inference until a test of the saved model succeeds. Skipping does not bypass structural validation. A skipped configuration waits for a manual test while it is still not_probed, including after a restart. A FAILED manual test records probe_failed, which does put the configuration back into the startup re-probe population: once its endpoint recovers, a restart promotes it to ok and grants it by the default rules with no further admin action. display_name must be the name the model is served under whenever that name is a LiteLLM proxy routing key: the lite_llm_yaml block’s top-level model_name when the config has a block, else base_model when inference_server is “litellm”. A different display_name is rejected (400), because the model is registered with the proxy under that name. A model reached directly at its provider (inference_server anthropic, google, a vLLM endpoint, …) with no block is registered under display_name itself, so its display_name is free. A block mux cannot read (hand-written YAML, or JSON declaring no top-level model_name, including model_list) is rejected. Admin only.

Example

  • Bearer Authentication (bearerAuth):

import h2ogpte.rest_sync
from h2ogpte.rest_sync.rest import ApiException
from pprint import pprint

# Defining the host is optional and defaults to https://h2ogpte.genai.h2o.ai/api/v1
# See configuration.py for a list of all supported configuration parameters.
configuration = h2ogpte.rest_sync.Configuration(
    host = "https://h2ogpte.genai.h2o.ai/api/v1"
)

# The client must configure the authentication and authorization parameters
# in accordance with the API server security policy.
# Examples for each auth method are provided below, use the example that
# satisfies your auth use case.

# Configure Bearer authorization: bearerAuth
configuration = h2ogpte.rest_sync.Configuration(
    access_token = os.environ["BEARER_TOKEN"]
)

# Enter a context with an instance of the API client
with h2ogpte.rest_sync.ApiClient(configuration) as api_client:
    # Create an instance of the API class
    api_instance = h2ogpte.rest_sync.LLMRegistryApi(api_client)
    request_body = None # Dict[str, object] | 
    skip_test = False # bool | Save without a connection test; the model remains unavailable until tested successfully. (optional) (default to False)

    try:
        # Creates a runtime LLM (admin).
        api_response = api_instance.create_runtime_model(request_body, skip_test=skip_test)
        print("The response of LLMRegistryApi->create_runtime_model:\n")
        pprint(api_response)
    except Exception as e:
        print("Exception when calling LLMRegistryApi->create_runtime_model: %s\n" % e)

Parameters

Name

Type

Description

Notes

request_body

Dict[str, object]

skip_test

bool

Save without a connection test; the model remains unavailable until tested successfully.

[optional] [default to False]

Return type

Dict[str, object]

HTTP request headers

  • Content-Type: application/json

  • Accept: application/json

HTTP response details

Status code

Description

Response headers

201

Successful operation

-

400

Bad request

-

401

Unauthorized - Invalid or missing API key

-

403

Forbidden

-

409

Conflict

-

delete_runtime_model

Dict[str, object] delete_runtime_model(id)

Deletes a runtime LLM by registry id (admin).

Hard-deletes a UI/API runtime LLM by registry UUID (delete is final). Admin only.

Example

  • Bearer Authentication (bearerAuth):

import h2ogpte.rest_sync
from h2ogpte.rest_sync.rest import ApiException
from pprint import pprint

# Defining the host is optional and defaults to https://h2ogpte.genai.h2o.ai/api/v1
# See configuration.py for a list of all supported configuration parameters.
configuration = h2ogpte.rest_sync.Configuration(
    host = "https://h2ogpte.genai.h2o.ai/api/v1"
)

# The client must configure the authentication and authorization parameters
# in accordance with the API server security policy.
# Examples for each auth method are provided below, use the example that
# satisfies your auth use case.

# Configure Bearer authorization: bearerAuth
configuration = h2ogpte.rest_sync.Configuration(
    access_token = os.environ["BEARER_TOKEN"]
)

# Enter a context with an instance of the API client
with h2ogpte.rest_sync.ApiClient(configuration) as api_client:
    # Create an instance of the API class
    api_instance = h2ogpte.rest_sync.LLMRegistryApi(api_client)
    id = 'id_example' # str | The registry UUID of the runtime model.

    try:
        # Deletes a runtime LLM by registry id (admin).
        api_response = api_instance.delete_runtime_model(id)
        print("The response of LLMRegistryApi->delete_runtime_model:\n")
        pprint(api_response)
    except Exception as e:
        print("Exception when calling LLMRegistryApi->delete_runtime_model: %s\n" % e)

Parameters

Name

Type

Description

Notes

id

str

The registry UUID of the runtime model.

Return type

Dict[str, object]

HTTP request headers

  • Content-Type: Not defined

  • Accept: application/json

HTTP response details

Status code

Description

Response headers

200

Successful operation

-

401

Unauthorized - Invalid or missing API key

-

403

Forbidden

-

404

Not found

-

get_llm_registry_admin_models

List[Dict[str, object]] get_llm_registry_admin_models()

Lists the full LLM registry (admin).

Returns every effective model with its lower-priority shadow candidates. The full provider configuration is returned under raw_config and MAY include secrets (e.g. api_key, inference_server credentials) – this matches the legacy admin-only runtime_llms setting admins edited directly. Not yet on the serving path. Admin only.

Example

  • Bearer Authentication (bearerAuth):

import h2ogpte.rest_sync
from h2ogpte.rest_sync.rest import ApiException
from pprint import pprint

# Defining the host is optional and defaults to https://h2ogpte.genai.h2o.ai/api/v1
# See configuration.py for a list of all supported configuration parameters.
configuration = h2ogpte.rest_sync.Configuration(
    host = "https://h2ogpte.genai.h2o.ai/api/v1"
)

# The client must configure the authentication and authorization parameters
# in accordance with the API server security policy.
# Examples for each auth method are provided below, use the example that
# satisfies your auth use case.

# Configure Bearer authorization: bearerAuth
configuration = h2ogpte.rest_sync.Configuration(
    access_token = os.environ["BEARER_TOKEN"]
)

# Enter a context with an instance of the API client
with h2ogpte.rest_sync.ApiClient(configuration) as api_client:
    # Create an instance of the API class
    api_instance = h2ogpte.rest_sync.LLMRegistryApi(api_client)

    try:
        # Lists the full LLM registry (admin).
        api_response = api_instance.get_llm_registry_admin_models()
        print("The response of LLMRegistryApi->get_llm_registry_admin_models:\n")
        pprint(api_response)
    except Exception as e:
        print("Exception when calling LLMRegistryApi->get_llm_registry_admin_models: %s\n" % e)

Parameters

This endpoint does not need any parameter.

Return type

List[Dict[str, object]]

HTTP request headers

  • Content-Type: Not defined

  • Accept: application/json

HTTP response details

Status code

Description

Response headers

200

Successful operation

-

401

Unauthorized - Invalid or missing API key

-

403

Forbidden

-

get_role_llm_registry_access

List[Dict[str, object]] get_role_llm_registry_access(role_id)

Lists LLM access grants for a role.

Returns every active effective model name plus whether the role is granted access to it. Requires the Manage Roles permission.

Example

  • Bearer Authentication (bearerAuth):

import h2ogpte.rest_sync
from h2ogpte.rest_sync.rest import ApiException
from pprint import pprint

# Defining the host is optional and defaults to https://h2ogpte.genai.h2o.ai/api/v1
# See configuration.py for a list of all supported configuration parameters.
configuration = h2ogpte.rest_sync.Configuration(
    host = "https://h2ogpte.genai.h2o.ai/api/v1"
)

# The client must configure the authentication and authorization parameters
# in accordance with the API server security policy.
# Examples for each auth method are provided below, use the example that
# satisfies your auth use case.

# Configure Bearer authorization: bearerAuth
configuration = h2ogpte.rest_sync.Configuration(
    access_token = os.environ["BEARER_TOKEN"]
)

# Enter a context with an instance of the API client
with h2ogpte.rest_sync.ApiClient(configuration) as api_client:
    # Create an instance of the API class
    api_instance = h2ogpte.rest_sync.LLMRegistryApi(api_client)
    role_id = 'role_id_example' # str | The unique identifier of a role.

    try:
        # Lists LLM access grants for a role.
        api_response = api_instance.get_role_llm_registry_access(role_id)
        print("The response of LLMRegistryApi->get_role_llm_registry_access:\n")
        pprint(api_response)
    except Exception as e:
        print("Exception when calling LLMRegistryApi->get_role_llm_registry_access: %s\n" % e)

Parameters

Name

Type

Description

Notes

role_id

str

The unique identifier of a role.

Return type

List[Dict[str, object]]

HTTP request headers

  • Content-Type: Not defined

  • Accept: application/json

HTTP response details

Status code

Description

Response headers

200

Successful operation

-

400

Bad request

-

401

Unauthorized - Invalid or missing API key

-

403

Forbidden

-

404

Not found

-

set_role_llm_registry_access

set_role_llm_registry_access(role_id, set_role_llm_registry_access_request)

Replaces LLM access grants for a role.

Replaces the role’s LLM access grant set. The protected roles viewer and admin are rejected. Requires the Manage Roles permission.

Example

  • Bearer Authentication (bearerAuth):

import h2ogpte.rest_sync
from h2ogpte.rest_sync.models.set_role_llm_registry_access_request import SetRoleLlmRegistryAccessRequest
from h2ogpte.rest_sync.rest import ApiException
from pprint import pprint

# Defining the host is optional and defaults to https://h2ogpte.genai.h2o.ai/api/v1
# See configuration.py for a list of all supported configuration parameters.
configuration = h2ogpte.rest_sync.Configuration(
    host = "https://h2ogpte.genai.h2o.ai/api/v1"
)

# The client must configure the authentication and authorization parameters
# in accordance with the API server security policy.
# Examples for each auth method are provided below, use the example that
# satisfies your auth use case.

# Configure Bearer authorization: bearerAuth
configuration = h2ogpte.rest_sync.Configuration(
    access_token = os.environ["BEARER_TOKEN"]
)

# Enter a context with an instance of the API client
with h2ogpte.rest_sync.ApiClient(configuration) as api_client:
    # Create an instance of the API class
    api_instance = h2ogpte.rest_sync.LLMRegistryApi(api_client)
    role_id = 'role_id_example' # str | The unique identifier of a role.
    set_role_llm_registry_access_request = h2ogpte.rest_sync.SetRoleLlmRegistryAccessRequest() # SetRoleLlmRegistryAccessRequest | 

    try:
        # Replaces LLM access grants for a role.
        api_instance.set_role_llm_registry_access(role_id, set_role_llm_registry_access_request)
    except Exception as e:
        print("Exception when calling LLMRegistryApi->set_role_llm_registry_access: %s\n" % e)

Parameters

Name

Type

Description

Notes

role_id

str

The unique identifier of a role.

set_role_llm_registry_access_request

SetRoleLlmRegistryAccessRequest

Return type

void (empty response body)

HTTP request headers

  • Content-Type: application/json

  • Accept: application/json

HTTP response details

Status code

Description

Response headers

204

Successful operation

-

400

Bad request

-

401

Unauthorized - Invalid or missing API key

-

403

Forbidden

-

404

Not found

-

test_runtime_model

Dict[str, object] test_runtime_model(request_body)

Tests a runtime LLM connection (admin).

Validates a model configuration and asks core to probe it (a non-caching, non-persisting metadata round-trip). No registry write occurs. Applies the same display_name rule as create, so a configuration the create path would refuse never reports a successful test. Admin only.

Example

  • Bearer Authentication (bearerAuth):

import h2ogpte.rest_sync
from h2ogpte.rest_sync.rest import ApiException
from pprint import pprint

# Defining the host is optional and defaults to https://h2ogpte.genai.h2o.ai/api/v1
# See configuration.py for a list of all supported configuration parameters.
configuration = h2ogpte.rest_sync.Configuration(
    host = "https://h2ogpte.genai.h2o.ai/api/v1"
)

# The client must configure the authentication and authorization parameters
# in accordance with the API server security policy.
# Examples for each auth method are provided below, use the example that
# satisfies your auth use case.

# Configure Bearer authorization: bearerAuth
configuration = h2ogpte.rest_sync.Configuration(
    access_token = os.environ["BEARER_TOKEN"]
)

# Enter a context with an instance of the API client
with h2ogpte.rest_sync.ApiClient(configuration) as api_client:
    # Create an instance of the API class
    api_instance = h2ogpte.rest_sync.LLMRegistryApi(api_client)
    request_body = None # Dict[str, object] | 

    try:
        # Tests a runtime LLM connection (admin).
        api_response = api_instance.test_runtime_model(request_body)
        print("The response of LLMRegistryApi->test_runtime_model:\n")
        pprint(api_response)
    except Exception as e:
        print("Exception when calling LLMRegistryApi->test_runtime_model: %s\n" % e)

Parameters

Name

Type

Description

Notes

request_body

Dict[str, object]

Return type

Dict[str, object]

HTTP request headers

  • Content-Type: application/json

  • Accept: application/json

HTTP response details

Status code

Description

Response headers

200

Successful operation

-

400

Bad request

-

401

Unauthorized - Invalid or missing API key

-

403

Forbidden

-

test_saved_runtime_model

Dict[str, object] test_saved_runtime_model(id)

Tests and updates the readiness of a saved runtime LLM (admin).

Tests the stored UI/API runtime configuration identified by registry ID. A successful test makes the model serving-eligible. A failed test preserves the configuration and records probe_failed, the error, and the test time; a probe_failed configuration is re-probed automatically at startup, so it can become serving-eligible later without another manual test. Returns 200 with ok=false for a model test failure. A core transport or invalid-response error leaves the prior readiness unchanged. No request body is accepted; the saved configuration is used. Admin only.

Example

  • Bearer Authentication (bearerAuth):

import h2ogpte.rest_sync
from h2ogpte.rest_sync.rest import ApiException
from pprint import pprint

# Defining the host is optional and defaults to https://h2ogpte.genai.h2o.ai/api/v1
# See configuration.py for a list of all supported configuration parameters.
configuration = h2ogpte.rest_sync.Configuration(
    host = "https://h2ogpte.genai.h2o.ai/api/v1"
)

# The client must configure the authentication and authorization parameters
# in accordance with the API server security policy.
# Examples for each auth method are provided below, use the example that
# satisfies your auth use case.

# Configure Bearer authorization: bearerAuth
configuration = h2ogpte.rest_sync.Configuration(
    access_token = os.environ["BEARER_TOKEN"]
)

# Enter a context with an instance of the API client
with h2ogpte.rest_sync.ApiClient(configuration) as api_client:
    # Create an instance of the API class
    api_instance = h2ogpte.rest_sync.LLMRegistryApi(api_client)
    id = 'id_example' # str | 

    try:
        # Tests and updates the readiness of a saved runtime LLM (admin).
        api_response = api_instance.test_saved_runtime_model(id)
        print("The response of LLMRegistryApi->test_saved_runtime_model:\n")
        pprint(api_response)
    except Exception as e:
        print("Exception when calling LLMRegistryApi->test_saved_runtime_model: %s\n" % e)

Parameters

Name

Type

Description

Notes

id

str

Return type

Dict[str, object]

HTTP request headers

  • Content-Type: Not defined

  • Accept: application/json

HTTP response details

Status code

Description

Response headers

200

Persisted test outcome (ok, and meta or error).

-

400

Bad request

-

401

Unauthorized - Invalid or missing API key

-

403

Forbidden

-

404

Not found

-

update_runtime_model

EndpointError update_runtime_model(id)

Not supported – updating a runtime LLM is disabled (admin).

Disabled: a runtime LLM can only be created and deleted. Any request returns 400 – delete the model and create it again to change its configuration. display_name is the routing key at the serving proxy, so an in-place edit can leave a saved, successfully probed model whose every request fails at the proxy. Admin only.

Example

  • Bearer Authentication (bearerAuth):

import h2ogpte.rest_sync
from h2ogpte.rest_sync.models.endpoint_error import EndpointError
from h2ogpte.rest_sync.rest import ApiException
from pprint import pprint

# Defining the host is optional and defaults to https://h2ogpte.genai.h2o.ai/api/v1
# See configuration.py for a list of all supported configuration parameters.
configuration = h2ogpte.rest_sync.Configuration(
    host = "https://h2ogpte.genai.h2o.ai/api/v1"
)

# The client must configure the authentication and authorization parameters
# in accordance with the API server security policy.
# Examples for each auth method are provided below, use the example that
# satisfies your auth use case.

# Configure Bearer authorization: bearerAuth
configuration = h2ogpte.rest_sync.Configuration(
    access_token = os.environ["BEARER_TOKEN"]
)

# Enter a context with an instance of the API client
with h2ogpte.rest_sync.ApiClient(configuration) as api_client:
    # Create an instance of the API class
    api_instance = h2ogpte.rest_sync.LLMRegistryApi(api_client)
    id = 'id_example' # str | The registry UUID of the runtime model.

    try:
        # Not supported -- updating a runtime LLM is disabled (admin).
        api_response = api_instance.update_runtime_model(id)
        print("The response of LLMRegistryApi->update_runtime_model:\n")
        pprint(api_response)
    except Exception as e:
        print("Exception when calling LLMRegistryApi->update_runtime_model: %s\n" % e)

Parameters

Name

Type

Description

Notes

id

str

The registry UUID of the runtime model.

Return type

EndpointError

HTTP request headers

  • Content-Type: Not defined

  • Accept: application/json

HTTP response details

Status code

Description

Response headers

400

Bad request

-

401

Unauthorized - Invalid or missing API key

-

403

Forbidden

-