View usage metrics for each LLM used

Overview

Users can view their usage metrics for each large language model (LLM) they’ve used, regardless of whether the LLM is currently configured or unconfigured.

Example

from h2ogpte import H2OGPTE

client = H2OGPTE(
    address="https://h2ogpte.genai.h2o.ai",
    api_key='sk-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX',
)

# The available units for time intervals are:
# - minute / minutes (for example, 5 minutes)
# - hour / hours (for example, 2 hours)
# - day / days (for example, 3 days)
# - week / weeks (for example, 1 week)
# - year / years (for example, 1 year)
list = client.get_llm_usage_by_llm(interval="1 month")

for llm in list:
    print(
    f"Call count: {llm.call_count}\n"
    f"Input tokens: {llm.input_tokens}\n"
    f"LLM cost: {llm.llm_cost}\n"
    f"LLM name: {llm.llm_name}\n"
    f"Model computed fields: {llm.model_computed_fields}\n"
    f"Model configuration: {llm.model_config}\n"
    f"Model fields: {llm.model_fields}\n"
    f"Output tokens: {llm.output_tokens}"
    )
    print("--" * 50)

Call count: 7
Input tokens: 256952
LLM cost: 0.998826
LLM name: claude-3-5-sonnet-20240620
Model computed fields: {}
Model configuration: {}
Model fields: {'llm_name': FieldInfo(annotation=str, required=True), 'llm_cost': FieldInfo(annotation=float, required=True), 'call_count': FieldInfo(annotation=int, required=True), 'input_tokens': FieldInfo(annotation=int, required=True), 'output_tokens': FieldInfo(annotation=int, required=True)}
Output tokens: 15198
----------------------------------------------------------------------------------------------------
Call count: 9
Input tokens: 76554
LLM cost: 0.02205975
LLM name: claude-3-haiku-20240307
Model computed fields: {}
Model configuration: {}
Model fields: {'llm_name': FieldInfo(annotation=str, required=True), 'llm_cost': FieldInfo(annotation=float, required=True), 'call_count': FieldInfo(annotation=int, required=True), 'input_tokens': FieldInfo(annotation=int, required=True), 'output_tokens': FieldInfo(annotation=int, required=True)}
Output tokens: 2337
....