h2ogpte package

Submodules

h2ogpte.h2ogpte module

class h2ogpte.h2ogpte.H2OGPTE(address: str, api_key: str | None = None, refresh_token: str | None = None, verify: bool | str = True, strict_version_check: bool = False)

Bases: object

Connect to and interact with an h2oGPTe server.

INITIAL_WAIT_INTERVAL = 0.1

MAX_WAIT_INTERVAL = 1.0

TIMEOUT = 3600.0

WAIT_BACKOFF_FACTOR = 1.4

Send a message and get a response from an LLM.

Format of inputs content:

{text_context_list}
"""\n{chat_conversation}{question}

Args:

question:: Text query to send to the LLM.
text_context_list:: List of raw text strings to be included.
system_prompt:: Text sent to models which support system prompts. Gives the model overall context in how to respond. Use auto for the model default, or None for h2oGPTe default. Defaults to ‘’ for no system prompt.
pre_prompt_query:: Text that is prepended before the contextual document chunks in text_context_list. Only used if text_context_list is provided.
prompt_query:: Text that is appended after the contextual document chunks in text_context_list. Only used if text_context_list is provided.
llm:: Name or index of LLM to send the query. Use H2OGPTE.get_llms() to see all available options. Default value is to use the first model (0th index).
llm_args:: Dictionary of kwargs to pass to the llm.
chat_conversation:: List of tuples for (human, bot) conversation that will be pre-appended to an (question, None) case for a query.
timeout:: Timeout in seconds.
kwargs:: Dictionary of kwargs to pass to h2oGPT.

Returns:

Answer: The response text and any errors.

Raises:

TimeoutError: If response isn’t completed in timeout seconds.

cancel_job(job_id: str) → Result

Stops a specific job from running on the server.

Args:

job_id:: String id of the job to cancel.

Returns:

Result: Status of canceling the job.

connect(chat_session_id: str) → Session

Create and participate in a chat session.

This is a live connection to the H2OGPTE server contained to a specific chat session on top of a single collection of documents. Users will find all questions and responses in this session in a single chat history in the UI.

Args:

chat_session_id:: ID of the chat session to connect to.

Returns:

Session: Live chat session connection with an LLM.

count_assets() → ObjectCount

Counts number of objects owned by the user.

Returns:: ObjectCount: The count of chat sessions, collections, and documents.

count_chat_sessions() → int

Counts number of chat sessions owned by the user.

Returns:: int: The count of chat sessions owned by the user.

count_chat_sessions_for_collection(collection_id: str) → int

Counts number of chat sessions in a specific collection.

Args:

collection_id:: String id of the collection to count chat sessions for.

Returns:

int: The count of chat sessions in that collection.

count_collections() → int

Counts number of collections owned by the user.

Returns:: int: The count of collections owned by the user.

count_documents() → int

Counts number of documents accessed by the user.

Returns:: int: The count of documents accessed by the user.

count_documents_in_collection(collection_id: str) → int

Counts the number of documents in a specific collection.

Args:

collection_id:: String id of the collection to count documents for.

Returns:

int: The number of documents in that collection.

count_documents_owned_by_me() → int

Counts number of documents owned by the user.

Returns:: int: The count of documents owned by the user.

count_question_reply_feedback() → int

Fetch user’s questions and answers with feedback count.

Returns:: int: the count of questions and replies that have a user feedback.

create_chat_session(collection_id: str) → str

Creates a new chat session for asking questions of documents.

Args:

collection_id:: String id of the collection to chat with.

Returns:

str: The ID of the newly created chat session.

create_chat_session_on_default_collection() → str

Creates a new chat session for asking questions of documents on the default collection.

Returns:: str: The ID of the newly created chat session.

create_collection(name: str, description: str) → str

Creates a new collection.

Args:

name:: Name of the collection.
description:: Description of the collection

Returns:

str: The ID of the newly created collection.

delete_chat_sessions(chat_session_ids: Iterable[str]) → Result

Deletes chat sessions and related messages.

Args:

chat_session_ids:: List of string ids of chat sessions to delete from the system.

Returns:

Result: Status of the delete job.

delete_collections(collection_ids: Iterable[str], timeout: float | None = None)

Deletes collections from the environment.

Documents in the collection will not be deleted.

Args:

collection_ids:: List of string ids of collections to delete from the system.
timeout:: Timeout in seconds.

delete_document_summaries(summaries_ids: Iterable[str]) → Result

Deletes document summaries.

Args:

summaries_ids:: List of string ids of a document summary to delete from the system.

Returns:

Result: Status of the delete job.

delete_documents(document_ids: Iterable[str], timeout: float | None = None)

Deletes documents from the system.

Args:

document_ids:: List of string ids to delete from the system and all collections.
timeout:: Timeout in seconds.

delete_documents_from_collection(collection_id: str, document_ids: Iterable[str], timeout: float | None = None)

Removes documents from a collection.

See Also: H2OGPTE.delete_documents for completely removing the document from the environment.

Args:

collection_id:: String of the collection to remove documents from.
document_ids:: List of string ids to remove from the collection.
timeout:: Timeout in seconds.

delete_upload(upload_id: str) → str

Delete a file previously uploaded with the “upload” method.

See Also:

upload: Upload the files into the system to then be ingested into a collection. ingest_uploads: Add the uploaded files to a collection.

Args:

upload_id:: ID of a file to remove

Returns:

upload_id: The upload id of the removed.

Raises:

Exception: The delete upload request was unsuccessful.

download_document(destination_directory: str, destination_file_name: str, document_id: str) → Path

Downloads a document to a local system directory.

Args:

destination_directory:: Destination directory to save file into.
destination_file_name:: Destination file name.
document_id:: Document ID.

Returns:

Path: Path of downloaded document

encode_for_retrieval(chunks: List[str]) → List[List[float]]

Encode texts for semantic searching.

See Also: H2OGPTE.match for getting a list of chunks that semantically match each encoded text.

Args:

chunks:: List of strings of texts to be encoded.

Returns:

List of list of floats: Each list in the list is the encoded original text.

Extract information from one or more contexts using an LLM.

pre_prompt_extract and prompt_extract variables must be used together. If these variables are not set, the inputs texts will be summarized into bullet points.

Format of extract content:

"{pre_prompt_extract}"""
{text_context_list}
"""\n{prompt_extract}"

Examples:

extract = h2ogpte.extract_data(
    text_context_list=chunks,
    pre_prompt_extract="Pay attention and look at all people. Your job is to collect their names.\n",
    prompt_extract="List all people's names as JSON.",
)

Args:

text_context_list:: List of raw text strings to extract data from.
system_prompt:: Text sent to models which support system prompts. Gives the model overall context in how to respond. Use auto or None for the model default. Defaults to ‘’ for no system prompt.
pre_prompt_extract:: Text that is prepended before the list of texts. If not set, the inputs will be summarized.
prompt_extract:: Text that is appended after the list of texts. If not set, the inputs will be summarized.
llm:: Name or index of LLM to send the query. Use H2OGPTE.get_llms() to see all available options. Default value is to use the first model (0th index).
llm_args:: Dictionary of kwargs to pass to the llm.
kwargs:: Dictionary of kwargs to pass to h2oGPT.

Returns:

ExtractionAnswer: The list of text responses and any errors.

get_chunks(collection_id: str, chunk_ids: Iterable[int]) → List[Chunk]

Get the text of specific chunks in a collection.

Args:

collection_id:: String id of the collection to search in.
chunk_ids:: List of ints for the chunks to return. Chunks are indexed starting at 1.

Returns:

Chunk: The text of the chunk.

Raises:

Exception: One or more chunks could not be found.

get_collection(collection_id: str) → Collection

Get metadata about a collection.

Args:

collection_id:: String id of the collection to search for.

Returns:

Collection: Metadata about the collection.

Raises:

KeyError: The collection was not found.

get_collection_for_chat_session(chat_session_id: str) → Collection

Get metadata about the collection of a chat session.

Args:

chat_session_id:: String id of the chat session to search for.

Returns:

Collection: Metadata about the collection.

get_collection_questions(collection_id: str, limit: int) → List[SuggestedQuestion]

List suggested questions

Args:

collection_id:: A collection ID of which to return the suggested questions
limit:: How many questions to return.

Returns:

List: A list of questions.

get_default_collection() → CollectionInfo

Get the default collection, to be used for collection API-keys.

Returns:: CollectionInfo: Default collection info.

get_document(document_id: str) → Document

Fetches information about a specific document.

Args:

document_id:: String id of the document.

Returns:

Document: Metadata about the Document.

Raises:

KeyError: The document was not found.

get_job(job_id: str) → Job

Fetches information about a specific job.

Args:

job_id:: String id of the job.

Returns:

Job: Metadata about the Job.

get_llm_names() → List[str]

Lists names of available LLMs in the environment.

Returns:: list of string: Name of each available model.

get_llm_usage_24h() → float

get_llm_usage_24h_by_llm() → List[LLMUsage]

get_llm_usage_24h_with_limits() → LLMUsageLimit

get_llm_usage_6h() → float

get_llm_usage_6h_by_llm() → List[LLMUsage]

get_llm_usage_by_llm(interval: str) → List[LLMUsage]

get_llm_usage_with_limits(interval: str) → LLMUsageLimit

get_llms() → List[dict]

Lists metadata information about available LLMs in the environment.

Returns:: list of dict (string, ANY): Name and details about each available model.

get_meta() → Meta

Returns information about the environment and the user.

Returns:: Meta: Details about the version and license of the environment and the user’s name and email.

get_scheduler_stats() → SchedulerStats

Count the number of global, pending jobs on the server.

Returns:: SchedulerStats: The queue length for number of jobs.

import_document_into_collection(collection_id: str, document_id: str, gen_doc_summaries: bool = False, gen_doc_questions: bool = False, timeout: float | None = None)

Import an already stored document to an existing collection

Args:

collection_id:: Collection ID to add documents to.
document_id:: Document ID to add.
gen_doc_summaries:: Whether to auto-generate document summaries (uses LLM)
gen_doc_questions:: Whether to auto-generate sample questions for each document (uses LLM)
timeout:: Timeout in seconds.

ingest_from_file_system(collection_id: str, root_dir: str, glob: str, gen_doc_summaries: bool = False, gen_doc_questions: bool = False, timeout: float | None = None)

Add files from the local system into a collection.

Args:

collection_id:: String id of the collection to add the ingested documents into.
root_dir:: String path of where to look for files.
glob:: String of the glob pattern used to match files in the root directory.
gen_doc_summaries:: Whether to auto-generate document summaries (uses LLM)
gen_doc_questions:: Whether to auto-generate sample questions for each document (uses LLM)
timeout:: Timeout in seconds

ingest_uploads(collection_id: str, upload_ids: Iterable[str], gen_doc_summaries: bool = False, gen_doc_questions: bool = False, timeout: float | None = None)

Add uploaded documents into a specific collection.

See Also:

upload: Upload the files into the system to then be ingested into a collection. delete_upload: Delete uploaded file

Args:

collection_id:: String id of the collection to add the ingested documents into.
upload_ids:: List of string ids of each uploaded document to add to the collection.
gen_doc_summaries:: Whether to auto-generate document summaries (uses LLM)
gen_doc_questions:: Whether to auto-generate sample questions for each document (uses LLM)
timeout:: Timeout in seconds

ingest_website(collection_id: str, url: str, gen_doc_summaries: bool = False, gen_doc_questions: bool = False, timeout: float | None = None)

Crawl and ingest a website into a collection.

All web pages linked from this URL will be imported. External links will be ignored. Links to other pages on the same domain will be followed as long as they are at the same level or below the URL you specify. Each page will be transformed into a PDF document.

Args:

collection_id:: String id of the collection to add the ingested documents into.
url:: String of the url to crawl.
gen_doc_summaries:: Whether to auto-generate document summaries (uses LLM)
gen_doc_questions:: Whether to auto-generate sample questions for each document (uses LLM)
timeout:: Timeout in seconds

list_chat_message_meta_part(message_id: str, info_type: str) → ChatMessageMeta

Fetch one chat message meta information.

Args:

message_id:: Message id to which the metadata should be pulled.
info_type:: Metadata type to fetch. Valid choices are: “self_reflection”, “usage_stats”, “prompt_raw”, “llm_only”, “rag”, “hyde1”, “hyde2”

Returns:

ChatMessageMeta: Metadata information about the chat message.

list_chat_message_references(message_id: str) → List[ChatMessageReference]

Fetch metadata for references of a chat message.

References are only available for messages sent from an LLM, an empty list will be returned for messages sent by the user.

Args:

message_id:: String id of the message to get references for.

Returns:

list of ChatMessageReference: Metadata including the document name, polygon information, and score.

list_chat_messages(chat_session_id: str, offset: int, limit: int) → List[ChatMessage]

Fetch chat message and metadata for messages in a chat session.

Messages without a reply_to are from the end user, messages with a reply_to are from an LLM and a response to a specific user message.

Args:

chat_session_id:: String id of the chat session to filter by.
offset:: How many chat messages to skip before returning.
limit:: How many chat messages to return.

Returns:

list of ChatMessage: Text and metadata for chat messages.

list_chat_messages_full(chat_session_id: str, offset: int, limit: int) → List[ChatMessageFull]

Fetch chat message and metadata for messages in a chat session.

Messages without a reply_to are from the end user, messages with a reply_to are from an LLM and a response to a specific user message.

Args:

chat_session_id:: String id of the chat session to filter by.
offset:: How many chat messages to skip before returning.
limit:: How many chat messages to return.

Returns:

list of ChatMessageFull: Text and metadata for chat messages.

list_chat_sessions_for_collection(collection_id: str, offset: int, limit: int) → List[ChatSessionForCollection]

Fetch chat session metadata for chat sessions in a collection.

Args:

collection_id:: String id of the collection to filter by.
offset:: How many chat sessions to skip before returning.
limit:: How many chat sessions to return.

Returns:

list of ChatSessionForCollection: Metadata about each chat session including the latest message.

list_collection_permissions(collection_id: str) → List[Permission]

Returns a list of access permissions for a given collection.

The returned list of permissions denotes who has access to the collection and their access level.

Args:

collection_id:: ID of the collection to inspect.

Returns:

list of Permission: Sharing permissions list for the given collection.

list_collections_for_document(document_id: str, offset: int, limit: int) → List[CollectionInfo]

Fetch metadata about each collection the document is a part of.

At this time, each document will only be available in a single collection.

Args:

document_id:: String id of the document to search for.
offset:: How many collections to skip before returning.
limit:: How many collections to return.

Returns:

list of CollectionInfo: Metadata about each collection.

list_documents_in_collection(collection_id: str, offset: int, limit: int) → List[DocumentInfo]

Fetch document metadata for documents in a collection.

Args:

collection_id:: String id of the collection to filter by.
offset:: How many documents to skip before returning.
limit:: How many documents to return.

Returns:

list of DocumentInfo: Metadata about each document.

list_jobs() → List[Job]

List the user’s jobs.

Returns:: list of Job:

list_list_chat_message_meta(message_id: str) → List[ChatMessageMeta]

Fetch chat message meta information.

Args:

message_id:: Message id to which the metadata should be pulled.

Returns:

list of ChatMessageMeta: Metadata about the chat message.

list_question_reply_feedback_data(offset: int, limit: int) → List[QuestionReplyData]

Fetch user’s questions and answers that have a feedback.

Questions and answers with metadata and feedback information.

Args:

offset:: How many conversations to skip before returning.
limit:: How many conversations to return.

Returns:

list of QuestionReplyData: Metadata about questions and answers.

list_recent_chat_sessions(offset: int, limit: int) → List[ChatSessionInfo]

Fetch user’s chat session metadata sorted by last update time.

Chats across all collections will be accessed.

Args:

offset:: How many chat sessions to skip before returning.
limit:: How many chat sessions to return.

Returns:

list of ChatSessionInfo: Metadata about each chat session including the latest message.

list_recent_collections(offset: int, limit: int) → List[CollectionInfo]

Fetch user’s collection metadata sorted by last update time.

Args:

offset:: How many collections to skip before returning.
limit:: How many collections to return.

Returns:

list of CollectionInfo: Metadata about each collection.

list_recent_collections_sort(offset: int, limit: int, sort_column: str, ascending: bool) → List[CollectionInfo]

Fetch user’s collection metadata sorted by last update time.

Args:

offset:: How many collections to skip before returning.
limit:: How many collections to return.
sort_column:: Sort column.
ascending:: When True, return sorted by sort_column in ascending order.

Returns:

list of CollectionInfo: Metadata about each collection.

list_recent_document_summaries(document_id: str, offset: int, limit: int) → List[DocumentSummary]

Fetches recent document summaries

Args:

document_id:: document ID for which to return summaries
offset:: How many summaries to skip before returning summaries.
limit:: How many summaries to return.

list_recent_documents(offset: int, limit: int) → List[DocumentInfo]

Fetch user’s document metadata sorted by last update time.

All documents owned by the user, regardless of collection, are accessed.

Args:

offset:: How many documents to skip before returning.
limit:: How many documents to return.

Returns:

list of DocumentInfo: Metadata about each document.

list_recent_documents_with_summaries(offset: int, limit: int) → List[DocumentInfoSummary]

Fetch user’s document metadata sorted by last update time, including the latest document summary.

All documents owned by the user, regardless of collection, are accessed.

Args:

offset:: How many documents to skip before returning.
limit:: How many documents to return.

Returns:

list of DocumentInfoSummary: Metadata about each document.

list_recent_documents_with_summaries_sort(offset: int, limit: int, sort_column: str, ascending: bool) → List[DocumentInfoSummary]

Fetch user’s document metadata sorted by last update time, including the latest document summary.

All documents owned by the user, regardless of collection, are accessed.

Args:

offset:: How many documents to skip before returning.
limit:: How many documents to return.
sort_column:: Sort column.
ascending:: When True, return sorted by sort_column in ascending order.

Returns:

list of DocumentInfoSummary: Metadata about each document.

list_upload() → List[str]

List pending file uploads to the H2OGPTE backend.

Uploaded files are not yet accessible and need to be ingested into a collection.

See Also:: upload: Upload the files into the system to then be ingested into a collection. ingest_uploads: Add the uploaded files to a collection. delete_upload: Delete uploaded file
Returns:: List[str]: The pending upload ids to be used in ingest jobs.
Raises:: Exception: The upload list request was unsuccessful.

list_users(offset: int, limit: int) → List[User]

List system users.

Returns a list of all registered users fo the system, a registered user, is a users that has logged in at least once.

Args:

offset:: How many users to skip before returning.
limit:: How many users to return.

Returns:

list of User: Metadata about each user.

make_collection_private(collection_id: str)

Make a collection private

Once a collection is private, other users will no longer be able to access chat history or documents related to the collection.

Args:

collection_id:: ID of the collection to make private.

make_collection_public(collection_id: str)

Make a collection public

Once a collection is public, it will be accessible to all authenticated users of the system.

Args:

collection_id:: ID of the collection to make public.

match_chunks(collection_id: str, vectors: List[List[float]], topics: List[str], offset: int, limit: int, cut_off: float = 0, width: int = 0) → List[SearchResult]

Find chunks related to a message using semantic search.

Chunks are sorted by relevance and similarity score to the message.

See Also: H2OGPTE.encode_for_retrieval to create vectors from messages.

Args:

collection_id:: ID of the collection to search within.
vectors:: A list of vectorized message for running semantic search.
topics:: A list of document_ids used to filter which documents in the collection to search.
offset:: How many chunks to skip before returning chunks.
limit:: How many chunks to return.
cut_off:: Exclude matches with distances higher than this cut off.
width:: How many chunks before and after a match to return - not implemented.

Returns:

list of SearchResult: The document, text, score and related information of the chunk.

openid_connect() → str

Tests openid connection.

Returns:: A session ID string.

reset_collection_prompt_settings(collection_id: str) → str

Reset the prompt settings for a given collection.

Args:

collection_id:: ID of the collection to update.

Returns:

str: ID of the updated collection.

search_chunks(collection_id: str, query: str, topics: List[str], offset: int, limit: int) → List[SearchResult]

Find chunks related to a message using lexical search.

Chunks are sorted by relevance and similarity score to the message.

Args:

collection_id:: ID of the collection to search within.
query:: Question or imperative from the end user to search a collection for.
topics:: A list of document_ids used to filter which documents in the collection to search.
offset:: How many chunks to skip before returning chunks.
limit:: How many chunks to return.

Returns:

list of SearchResult: The document, text, score and related information of the chunk.

set_chat_message_votes(chat_message_id: str, votes: int) → Result

Change the vote value of a chat message.

Set the exact value of a vote for a chat message. Any message type can be updated, but only LLM response votes will be visible in the UI. The expectation is 0: unvoted, -1: dislike, 1 like. Values outside of this will not be viewable in the UI.

Args:

chat_message_id:: ID of a chat message, any message can be used but only LLM responses will be visible in the UI.
votes:: Integer value for the message. Only -1 and 1 will be visible in the UI as dislike and like respectively.

Returns:

Result: The status of the update.

Raises:

Exception: The upload request was unsuccessful.

share_collection(collection_id: str, permission: Permission) → ShareResponseStatus

Share a collection to a user.

The permission attribute defined the level of access, and who can access the collection, the collection_id attribute denotes the collection to be shared.

Args:

collection_id:: ID of the collection to share.
permission:: Defines the rule for sharing, i.e. permission level.

Returns:

ShareResponseStatus: Status of share request.

Summarize one or more contexts using an LLM.

Effective prompt created (excluding the system prompt):

"{pre_prompt_summary}
"""
{text_context_list}
"""
{prompt_summary}"

Args:

text_context_list:: List of raw text strings to be summarized.
system_prompt:: Text sent to models which support system prompts. Gives the model overall context in how to respond. Use auto for the model default or None for h2oGPTe defaults. Defaults to ‘’ for no system prompt.
pre_prompt_summary:: Text that is prepended before the list of texts. The default can be customized per environment, but the standard default is "In order to write a concise single-paragraph or bulleted list summary, pay attention to the following text:\n"
prompt_summary:: Text that is appended after the list of texts. The default can be customized per environment, but the standard default is "Using only the text above, write a condensed and concise summary of key results (preferably as bullet points):\n"
llm:: Name or index of LLM to send the query. Use H2OGPTE.get_llms() to see all available options. Default value is to use the first model (0th index).
llm_args:: Dictionary of kwargs to pass to the llm.
kwargs:: Dictionary of kwargs to pass to h2oGPT.

Returns:

Answer: The response text and any errors.

Creates a summary of a document.

Effective prompt created (excluding the system prompt):

"{pre_prompt_summary}
"""
{text from document}
"""
{prompt_summary}"

Args:

document_id:: String id of the document to create a summary from.
system_prompt:: System Prompt
pre_prompt_summary:: Prompt that goes before each large piece of text to summarize
prompt_summary:: Prompt that goes after each large piece of of text to summarize
llm:: LLM to use
llm_args:: Arguments for the LLM
max_num_chunks:: Max limit of chunks to send to the summarizer
sampling_strategy:: How to sample if the document has more chunks than max_num_chunks. Options are “auto”, “uniform”, “first”, “first+last”, default is “auto” (a hybrid of them all).
timeout:: Amount of time in seconds to allow the request to run. The default is 86400 seconds.

Returns:

DocumentSummary: Summary of the document

Raises:

TimeoutError: The request did not complete in time. SessionError: No summary created. Document wasn’t part of a collection, or LLM timed out, etc.

unshare_collection(collection_id: str, permission: Permission) → ShareResponseStatus

Remove sharing of a collection to a user.

The permission attribute defined the level of access, and who can access the collection, the collection_id attribute denotes the collection to be shared. In case of un-sharing, the Permission’s user is sufficient

Args:

collection_id:: ID of the collection to un-share.
permission:: Defines the user for which collection access is revoked.

ShareResponseStatus: Status of share request.

unshare_collection_for_all(collection_id: str) → ShareResponseStatus

Remove sharing of a collection to all other users but the original owner

Args:

collection_id:: ID of the collection to un-share.

ShareResponseStatus: Status of share request.

update_collection(collection_id: str, name: str, description: str) → str

Update the metadata for a given collection.

All variables are required. You can use h2ogpte.get_collection(<id>).name or description to get the existing values if you only want to change one or the other.

Args:

collection_id:: ID of the collection to update.
name:: New name of the collection, this is required.
description:: New description of the collection, this is required.

Returns:

str: ID of the updated collection.

update_collection_prompt_settings(collection_id: str, system_prompt: str, pre_prompt_query: str, prompt_query: str, rag_type: str, hyde_no_rag_llm_prompt_extension: str, auto_gen_description_prompt: str) → str

Update the prompt settings for a given collection.

Updates the prompt settings for a given collection.

Args:

collection_id:: ID of the collection to update.
system_prompt:: Text sent to models which support system prompts. Gives the model overall context in how to respond.
pre_prompt_query:: Text that is prepended before the contextual document chunks.
prompt_query:: Text that is appended to the beginning of the user’s message.
rag_type:: RAG type to use.
hyde_no_rag_llm_prompt_extension:: LLM prompt extension.
auto_gen_description_prompt:: prompt to create a description of the collection.

Returns:

str: ID of the updated collection.

upload(file_name: str, file: Any) → str

Upload a file to the H2OGPTE backend.

Uploaded files are not yet accessible and need to be ingested into a collection.

See Also:

ingest_uploads: Add the uploaded files to a collection. delete_upload: Delete uploaded file

Args:

file_name:: What to name the file on the server, must include file extension.
file:: File object to upload, often an opened file from with open(…) as f.

Returns:

str: The upload id to be used in ingest jobs.

Raises:

Exception: The upload request was unsuccessful.

h2ogpte.h2ogpte.marshal(d)

h2ogpte.h2ogpte.unmarshal(s: str)

h2ogpte.session module

Bases: object

Create and participate in a chat session.

This is a live connection to the h2oGPTe server contained to a specific chat session on top of a single collection of documents. Users will find all questions and responses in this session in a single chat history in the UI.

See Also:

H2OGPTE.connect: To initialize a session on an existing connection.

Args:

address:: Full URL of the h2oGPTe server to connect to.
chat_session_id:: The ID of the chat session the queries should be sent to.
api_key:: API key for authentication to the h2oGPTe server. Users can generate a key by accessing the UI and navigating to the Settings.
refresh_token:: User’s refresh token.

Examples:

# Example 1: Best practice, create a session using the H2OGPTE module
with h2ogpte.connect(chat_session_id) as session:
    answer1 = session.query('How many paper clips were shipped to Scranton?', timeout=10)
    answer2 = session.query('Did David Brent co-sign the contract with Initech?', timeout=10)

# Example 2: Connect and disconnect manually
session = Session(
    address=address,
    api_key=api_key,
    chat_session_id=chat_session_id
)
session.connect()
answer = session.query("Are there any dogs in the documents?")
session.disconnect()

connect()

Connect to an h2oGPTe server.

This is primarily an internal function used when users create a session using with from the H2OGPTE.connection() function.

property connection: ClientConnection

disconnect()

Disconnect from an h2oGPTe server.

This is primarily an internal function used when users create a session using with from the H2OGPTE.connection() function.

Retrieval-augmented generation for a query on a collection.

Finds a collection of chunks relevant to the query using similarity scores. Sends these and any additional instructions to an LLM.

Format of questions or imperatives:

"{pre_prompt_query}
"""
{similar_context_chunks}
"""                {prompt_query}{message}"

Args:

message:

Query or instruction from the end user to the LLM.

system_prompt:

Text sent to models which support system prompts. Gives the model overall context in how to respond. Use auto or None for the model default. Defaults to ‘’ for no system prompt.

pre_prompt_query:

Text that is prepended before the contextual document chunks. The default can be customized per environment, but the standard default is "Pay attention and remember the information below, which will help to answer the question or imperative after the context ends.\n"

prompt_query:

Text that is appended to the beginning of the user’s message. The default can be customized per environment, but the standard default is “According to only the information in the document sources provided within the context above, “

pre_prompt_summary:

Not yet used, use H2OGPTE.summarize_content

prompt_summary:

Not yet used, use H2OGPTE.summarize_content

llm:

Name or index of LLM to send the query. Use H2OGPTE.get_llms() to see all available options. Default value is to use the first model (0th index).

llm_args:

Dictionary of kwargs to pass to the llm.

self_reflection_config: Dictionary of arguments for self-reflection, can contain the following string:string mappings:

llm_reflection: str: "gpt-4-1106-preview" or "" to disable reflection
pre_prompt_reflection: str: "Pay attention to the following context. You will need to evaluate the quality of a response for a given prompt."
prompt_reflection: str: ‘Given the above context, here’s the prompt and the response: """Prompt:\n%s\n"""\n\n"""Response:\n%s\n"""\n\nWhat is the quality of the response for the given prompt? Respond with a score ranging from Score: 0/10 (worst) to Score: 10/10 (best), and give a brief explanation why.'
system_prompt_reflection: str: ""
llm_args_reflection: str: "{}"

rag_config:

Dictionary of arguments to control RAG (retrieval-augmented-generation) types. Can contain the following key/value pairs:

rag_type: str one of

"llm_only" LLM Only (no RAG) - Generates a response to answer the user’s query without any supporting document contexts.

"rag" RAG (Retrieval Augmented Generation) - RAG with neural/lexical hybrid search using the user’s query to find contexts from a collection for generating a response.

"hyde1" HyDE RAG (Hypothetical Document Embedding) - RAG with neural/lexical hybrid search using the user’s query and the LLM response to find contexts from a collection for generating a response. Requires 2 LLM calls.

"hyde2" HyDE RAG+ (Combined HyDE+RAG) - RAG with neural/lexical hybrid search using the user’s query and the HyDE RAG response to find contexts from a collection for generating a response. Requires 3 LLM calls.

no_rag_llm_prompt_extension: str: example: '\nKeep the answer brief, and list the 5 most relevant key words at the end.'

timeout:

Amount of time in seconds to allow the request to run. The default is 1000 seconds.

callback:

Function for processing partial messages, used for streaming responses to an end user.

Returns:

ChatMessage: The response text and details about the response from the LLM. For example:

ChatMessage(
    id='XXX',
    content='The information provided in the context...',
    reply_to='YYY',
    votes=0,
    created_at=datetime.datetime(2023, 10, 24, 20, 12, 34, 875026)
    type_list=[],
)

Raises:

TimeoutError: The request did not complete in time.

h2ogpte.session.deserialize(response: str) → ChatResponse | ChatAcknowledgement

h2ogpte.session.serialize(request: ChatRequest) → str

h2ogpte.types module

class h2ogpte.types.Answer(*, content: str, error: str, prompt_raw: str = '', llm: str, input_tokens: int = 0, output_tokens: int = 0, origin: str = 'N/A')

Bases: BaseModel

content: str

error: str

input_tokens: int

llm: str

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'content': FieldInfo(annotation=str, required=True), 'error': FieldInfo(annotation=str, required=True), 'input_tokens': FieldInfo(annotation=int, required=False, default=0), 'llm': FieldInfo(annotation=str, required=True), 'origin': FieldInfo(annotation=str, required=False, default='N/A'), 'output_tokens': FieldInfo(annotation=int, required=False, default=0), 'prompt_raw': FieldInfo(annotation=str, required=False, default='')}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

origin: str

output_tokens: int

prompt_raw: str

class h2ogpte.types.ChatAcknowledgement(t: str, session_id: str, correlation_id: str, message_id: str)

Bases: object

correlation_id: str

message_id: str

session_id: str

t: str

class h2ogpte.types.ChatMessage(*, id: str, content: str, reply_to: str | None = None, votes: int, created_at: datetime, type_list: List[str] | None = None, error: str | None = None)

Bases: BaseModel

content: str

created_at: datetime

error: str | None

id: str

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'content': FieldInfo(annotation=str, required=True), 'created_at': FieldInfo(annotation=datetime, required=True), 'error': FieldInfo(annotation=Union[str, NoneType], required=False), 'id': FieldInfo(annotation=str, required=True), 'reply_to': FieldInfo(annotation=Union[str, NoneType], required=False), 'type_list': FieldInfo(annotation=Union[List[str], NoneType], required=False), 'votes': FieldInfo(annotation=int, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

reply_to: str | None

type_list: List[str] | None

votes: int

class h2ogpte.types.ChatMessageFull(*, id: str, content: str, reply_to: str | None = None, votes: int, created_at: datetime, type_list: List[ChatMessageMeta] | None = [], error: str | None = None)

Bases: BaseModel

content: str

created_at: datetime

error: str | None

id: str

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'content': FieldInfo(annotation=str, required=True), 'created_at': FieldInfo(annotation=datetime, required=True), 'error': FieldInfo(annotation=Union[str, NoneType], required=False), 'id': FieldInfo(annotation=str, required=True), 'reply_to': FieldInfo(annotation=Union[str, NoneType], required=False), 'type_list': FieldInfo(annotation=Union[List[ChatMessageMeta], NoneType], required=False, default=[]), 'votes': FieldInfo(annotation=int, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

reply_to: str | None

type_list: List[ChatMessageMeta] | None

votes: int

class h2ogpte.types.ChatMessageMeta(*, message_type: str, content: str)

Bases: BaseModel

content: str

message_type: str

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'content': FieldInfo(annotation=str, required=True), 'message_type': FieldInfo(annotation=str, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

class h2ogpte.types.ChatMessageReference(*, document_id: str, document_name: str, chunk_id: int, pages: str, score: float)

Bases: BaseModel

chunk_id: int

document_id: str

document_name: str

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'chunk_id': FieldInfo(annotation=int, required=True), 'document_id': FieldInfo(annotation=str, required=True), 'document_name': FieldInfo(annotation=str, required=True), 'pages': FieldInfo(annotation=str, required=True), 'score': FieldInfo(annotation=float, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

pages: str

score: float

Bases: object

body: str

correlation_id: str

llm: str | int | None

llm_args: str | None

mode: str

pre_prompt_query: str | None

pre_prompt_summary: str | None

prompt_query: str | None

prompt_summary: str | None

rag_config: str | None

self_reflection_config: str | None

session_id: str

system_prompt: str | None

t: str

class h2ogpte.types.ChatResponse(t: str, session_id: str, message_id: str, reply_to_id: str, body: str, error: str)

Bases: object

body: str

error: str

message_id: str

reply_to_id: str

session_id: str

t: str

class h2ogpte.types.ChatSessionCount(*, chat_session_count: int)

Bases: BaseModel

chat_session_count: int

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'chat_session_count': FieldInfo(annotation=int, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

class h2ogpte.types.ChatSessionForCollection(*, id: str, latest_message_content: str | None = None, updated_at: datetime)

Bases: BaseModel

id: str

latest_message_content: str | None

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'id': FieldInfo(annotation=str, required=True), 'latest_message_content': FieldInfo(annotation=Union[str, NoneType], required=False), 'updated_at': FieldInfo(annotation=datetime, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

updated_at: datetime

class h2ogpte.types.ChatSessionInfo(*, id: str, latest_message_content: str | None = None, collection_id: str, collection_name: str, updated_at: datetime)

Bases: BaseModel

collection_id: str

collection_name: str

id: str

latest_message_content: str | None

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'collection_id': FieldInfo(annotation=str, required=True), 'collection_name': FieldInfo(annotation=str, required=True), 'id': FieldInfo(annotation=str, required=True), 'latest_message_content': FieldInfo(annotation=Union[str, NoneType], required=False), 'updated_at': FieldInfo(annotation=datetime, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

updated_at: datetime

class h2ogpte.types.Chunk(*, text: str)

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'text': FieldInfo(annotation=str, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

text: str

class h2ogpte.types.Chunks(*, result: List[Chunk])

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'result': FieldInfo(annotation=List[Chunk], required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

result: List[Chunk]

class h2ogpte.types.Collection(*, id: str, name: str, description: str, document_count: int, document_size: int, created_at: datetime, updated_at: datetime, username: str, system_prompt: str | None = None, pre_prompt_query: str | None = None, prompt_query: str | None = None, rag_type: str | None = None, hyde_no_rag_llm_prompt_extension: str | None = None, auto_gen_description_prompt: str | None = None, is_public: bool)

Bases: BaseModel

auto_gen_description_prompt: str | None

created_at: datetime

description: str

document_count: int

document_size: int

hyde_no_rag_llm_prompt_extension: str | None

id: str

is_public: bool

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'auto_gen_description_prompt': FieldInfo(annotation=Union[str, NoneType], required=False), 'created_at': FieldInfo(annotation=datetime, required=True), 'description': FieldInfo(annotation=str, required=True), 'document_count': FieldInfo(annotation=int, required=True), 'document_size': FieldInfo(annotation=int, required=True), 'hyde_no_rag_llm_prompt_extension': FieldInfo(annotation=Union[str, NoneType], required=False), 'id': FieldInfo(annotation=str, required=True), 'is_public': FieldInfo(annotation=bool, required=True), 'name': FieldInfo(annotation=str, required=True), 'pre_prompt_query': FieldInfo(annotation=Union[str, NoneType], required=False), 'prompt_query': FieldInfo(annotation=Union[str, NoneType], required=False), 'rag_type': FieldInfo(annotation=Union[str, NoneType], required=False), 'system_prompt': FieldInfo(annotation=Union[str, NoneType], required=False), 'updated_at': FieldInfo(annotation=datetime, required=True), 'username': FieldInfo(annotation=str, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

name: str

pre_prompt_query: str | None

prompt_query: str | None

rag_type: str | None

system_prompt: str | None

updated_at: datetime

username: str

class h2ogpte.types.CollectionCount(*, collection_count: int)

Bases: BaseModel

collection_count: int

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'collection_count': FieldInfo(annotation=int, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

class h2ogpte.types.CollectionInfo(*, id: str, name: str, description: str, document_count: int, document_size: int, updated_at: datetime, user_count: int, is_public: bool, username: str, sessions_count: int)

Bases: BaseModel

description: str

document_count: int

document_size: int

id: str

is_public: bool

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'description': FieldInfo(annotation=str, required=True), 'document_count': FieldInfo(annotation=int, required=True), 'document_size': FieldInfo(annotation=int, required=True), 'id': FieldInfo(annotation=str, required=True), 'is_public': FieldInfo(annotation=bool, required=True), 'name': FieldInfo(annotation=str, required=True), 'sessions_count': FieldInfo(annotation=int, required=True), 'updated_at': FieldInfo(annotation=datetime, required=True), 'user_count': FieldInfo(annotation=int, required=True), 'username': FieldInfo(annotation=str, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

name: str

sessions_count: int

updated_at: datetime

user_count: int

username: str

class h2ogpte.types.ConfigItem(*, key_name: str, string_value: str, value_type: str, can_overwrite: bool)

Bases: BaseModel

can_overwrite: bool

key_name: str

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'can_overwrite': FieldInfo(annotation=bool, required=True), 'key_name': FieldInfo(annotation=str, required=True), 'string_value': FieldInfo(annotation=str, required=True), 'value_type': FieldInfo(annotation=str, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

string_value: str

value_type: str

class h2ogpte.types.Document(*, id: str, name: str, type: str, size: int, page_count: int, status: Status, created_at: datetime, updated_at: datetime)

Bases: BaseModel

created_at: datetime

id: str

model_config: ClassVar[ConfigDict] = {'use_enum_values': True}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'created_at': FieldInfo(annotation=datetime, required=True), 'id': FieldInfo(annotation=str, required=True), 'name': FieldInfo(annotation=str, required=True), 'page_count': FieldInfo(annotation=int, required=True), 'size': FieldInfo(annotation=int, required=True), 'status': FieldInfo(annotation=Status, required=True), 'type': FieldInfo(annotation=str, required=True), 'updated_at': FieldInfo(annotation=datetime, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

name: str

page_count: int

size: int

status: Status

type: str

updated_at: datetime

class h2ogpte.types.DocumentCount(*, document_count: int)

Bases: BaseModel

document_count: int

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'document_count': FieldInfo(annotation=int, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

class h2ogpte.types.DocumentInfo(*, id: str, name: str, type: str, size: int, page_count: int, status: Status, updated_at: datetime)

Bases: BaseModel

id: str

model_config: ClassVar[ConfigDict] = {'use_enum_values': True}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'id': FieldInfo(annotation=str, required=True), 'name': FieldInfo(annotation=str, required=True), 'page_count': FieldInfo(annotation=int, required=True), 'size': FieldInfo(annotation=int, required=True), 'status': FieldInfo(annotation=Status, required=True), 'type': FieldInfo(annotation=str, required=True), 'updated_at': FieldInfo(annotation=datetime, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

name: str

page_count: int

size: int

status: Status

type: str

updated_at: datetime

class h2ogpte.types.DocumentInfoSummary(*, id: str, name: str, type: str, size: int, page_count: int, status: Status, updated_at: datetime, summary: str | None, summary_parameters: str | None)

Bases: BaseModel

id: str

model_config: ClassVar[ConfigDict] = {'use_enum_values': True}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'id': FieldInfo(annotation=str, required=True), 'name': FieldInfo(annotation=str, required=True), 'page_count': FieldInfo(annotation=int, required=True), 'size': FieldInfo(annotation=int, required=True), 'status': FieldInfo(annotation=Status, required=True), 'summary': FieldInfo(annotation=Union[str, NoneType], required=True), 'summary_parameters': FieldInfo(annotation=Union[str, NoneType], required=True), 'type': FieldInfo(annotation=str, required=True), 'updated_at': FieldInfo(annotation=datetime, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

name: str

page_count: int

size: int

status: Status

summary: str | None

summary_parameters: str | None

type: str

updated_at: datetime

class h2ogpte.types.DocumentSummary(*, id: str, content: str, error: str, document_id: str, kwargs: str, created_at: datetime)

Bases: BaseModel

content: str

created_at: datetime

document_id: str

error: str

id: str

kwargs: str

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'content': FieldInfo(annotation=str, required=True), 'created_at': FieldInfo(annotation=datetime, required=True), 'document_id': FieldInfo(annotation=str, required=True), 'error': FieldInfo(annotation=str, required=True), 'id': FieldInfo(annotation=str, required=True), 'kwargs': FieldInfo(annotation=str, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

class h2ogpte.types.ExtractionAnswer(*, content: List[str], error: str, llm: str, input_tokens: int = 0, output_tokens: int = 0)

Bases: BaseModel

content: List[str]

error: str

input_tokens: int

llm: str

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'content': FieldInfo(annotation=List[str], required=True), 'error': FieldInfo(annotation=str, required=True), 'input_tokens': FieldInfo(annotation=int, required=False, default=0), 'llm': FieldInfo(annotation=str, required=True), 'output_tokens': FieldInfo(annotation=int, required=False, default=0)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

output_tokens: int

class h2ogpte.types.Identifier(*, id: str, error: str | None = None)

Bases: BaseModel

error: str | None

id: str

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'error': FieldInfo(annotation=Union[str, NoneType], required=False), 'id': FieldInfo(annotation=str, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

exception h2ogpte.types.InvalidArgumentError: Bases: Exception

class h2ogpte.types.Job(*, id: str, passed: float, failed: float, progress: float, completed: bool, canceled: bool, date: datetime, kind: JobKind, statuses: List[JobStatus], errors: List[str], last_update_date: datetime, duration: str)

Bases: BaseModel

canceled: bool

completed: bool

date: datetime

duration: str

errors: List[str]

failed: float

id: str

kind: JobKind

last_update_date: datetime

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'canceled': FieldInfo(annotation=bool, required=True), 'completed': FieldInfo(annotation=bool, required=True), 'date': FieldInfo(annotation=datetime, required=True), 'duration': FieldInfo(annotation=str, required=True), 'errors': FieldInfo(annotation=List[str], required=True), 'failed': FieldInfo(annotation=float, required=True), 'id': FieldInfo(annotation=str, required=True), 'kind': FieldInfo(annotation=JobKind, required=True), 'last_update_date': FieldInfo(annotation=datetime, required=True), 'passed': FieldInfo(annotation=float, required=True), 'progress': FieldInfo(annotation=float, required=True), 'statuses': FieldInfo(annotation=List[JobStatus], required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

passed: float

progress: float

statuses: List[JobStatus]

class h2ogpte.types.JobKind(value)

Bases: str, Enum

An enumeration.

DeleteCollectionsJob = 'DeleteCollectionsJob'

DeleteDocumentsFromCollectionJob = 'DeleteDocumentsFromCollectionJob'

DeleteDocumentsJob = 'DeleteDocumentsJob'

DocumentSummaryJob = 'DocumentSummaryJob'

ImportDocumentIntoCollectionJob = 'ImportDocumentIntoCollectionJob'

IndexFilesJob = 'IndexFilesJob'

IngestFromFileSystemJob = 'IngestFromFileSystemJob'

IngestUploadsJob = 'IngestUploadsJob'

IngestWebsiteJob = 'IngestWebsiteJob'

NoOpJob = 'NoOpJob'

UpdateCollectionStatsJob = 'UpdateCollectionStatsJob'

class h2ogpte.types.JobStatus(*, id: str, status: str)

Bases: BaseModel

id: str

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'id': FieldInfo(annotation=str, required=True), 'status': FieldInfo(annotation=str, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

status: str

class h2ogpte.types.LLMUsage(*, llm_name: str, llm_cost: float)

Bases: BaseModel

llm_cost: float

llm_name: str

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'llm_cost': FieldInfo(annotation=float, required=True), 'llm_name': FieldInfo(annotation=str, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

class h2ogpte.types.LLMUsageLimit(*, current: float, max_allowed_24h: float, cost_unit: str, interval: str | None = None)

Bases: BaseModel

cost_unit: str

current: float

interval: str | None

max_allowed_24h: float

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'cost_unit': FieldInfo(annotation=str, required=True), 'current': FieldInfo(annotation=float, required=True), 'interval': FieldInfo(annotation=Union[str, NoneType], required=False), 'max_allowed_24h': FieldInfo(annotation=float, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

class h2ogpte.types.Meta(*, version: str, build: str, username: str, email: str, license_expired: bool, license_expiry_date: str, global_configs: List[ConfigItem], picture: str | None)

Bases: BaseModel

build: str

email: str

global_configs: List[ConfigItem]

license_expired: bool

license_expiry_date: str

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'build': FieldInfo(annotation=str, required=True), 'email': FieldInfo(annotation=str, required=True), 'global_configs': FieldInfo(annotation=List[ConfigItem], required=True), 'license_expired': FieldInfo(annotation=bool, required=True), 'license_expiry_date': FieldInfo(annotation=str, required=True), 'picture': FieldInfo(annotation=Union[str, NoneType], required=True), 'username': FieldInfo(annotation=str, required=True), 'version': FieldInfo(annotation=str, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

picture: str | None

username: str

version: str

class h2ogpte.types.ObjectCount(*, chat_session_count: int, collection_count: int, document_count: int)

Bases: BaseModel

chat_session_count: int

collection_count: int

document_count: int

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'chat_session_count': FieldInfo(annotation=int, required=True), 'collection_count': FieldInfo(annotation=int, required=True), 'document_count': FieldInfo(annotation=int, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

exception h2ogpte.types.ObjectNotFoundError: Bases: Exception

class h2ogpte.types.PartialChatMessage(*, id: str, content: str, reply_to: str | None = None)

Bases: BaseModel

content: str

id: str

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'content': FieldInfo(annotation=str, required=True), 'id': FieldInfo(annotation=str, required=True), 'reply_to': FieldInfo(annotation=Union[str, NoneType], required=False)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

reply_to: str | None

class h2ogpte.types.Permission(*, username: str)

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'username': FieldInfo(annotation=str, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

username: str

Bases: BaseModel

collection_documents: List[str] | None

collection_id: str | None

collection_name: str | None

expected_answer: str | None

llm: str | None

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'collection_documents': FieldInfo(annotation=Union[List[str], NoneType], required=True), 'collection_id': FieldInfo(annotation=Union[str, NoneType], required=True), 'collection_name': FieldInfo(annotation=Union[str, NoneType], required=True), 'expected_answer': FieldInfo(annotation=Union[str, NoneType], required=True), 'llm': FieldInfo(annotation=Union[str, NoneType], required=True), 'pre_prompt_query': FieldInfo(annotation=Union[str, NoneType], required=True), 'pre_prompt_summary': FieldInfo(annotation=Union[str, NoneType], required=True), 'prompt_query': FieldInfo(annotation=Union[str, NoneType], required=True), 'prompt_summary': FieldInfo(annotation=Union[str, NoneType], required=True), 'question_content': FieldInfo(annotation=str, required=True), 'question_id': FieldInfo(annotation=str, required=True), 'rag_config': FieldInfo(annotation=Union[str, NoneType], required=True), 'reply_content': FieldInfo(annotation=str, required=True), 'reply_id': FieldInfo(annotation=str, required=True), 'response_created_at_time': FieldInfo(annotation=str, required=True), 'system_prompt': FieldInfo(annotation=Union[str, NoneType], required=True), 'user_comment': FieldInfo(annotation=Union[str, NoneType], required=True), 'votes': FieldInfo(annotation=int, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

pre_prompt_query: str | None

pre_prompt_summary: str | None

prompt_query: str | None

prompt_summary: str | None

question_content: str

question_id: str

rag_config: str | None

reply_content: str

reply_id: str

response_created_at_time: str

system_prompt: str | None

user_comment: str | None

votes: int

class h2ogpte.types.QuestionReplyDataCount(*, question_reply_data_count: int)

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'question_reply_data_count': FieldInfo(annotation=int, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

question_reply_data_count: int

class h2ogpte.types.Result(*, status: Status)

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {'use_enum_values': True}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'status': FieldInfo(annotation=Status, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

status: Status

class h2ogpte.types.SchedulerStats(*, queue_length: int)

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'queue_length': FieldInfo(annotation=int, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

queue_length: int

class h2ogpte.types.SearchResult(*, id: int, topic: str, name: str, text: str, size: int, pages: str, score: float)

Bases: BaseModel

id: int

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'id': FieldInfo(annotation=int, required=True), 'name': FieldInfo(annotation=str, required=True), 'pages': FieldInfo(annotation=str, required=True), 'score': FieldInfo(annotation=float, required=True), 'size': FieldInfo(annotation=int, required=True), 'text': FieldInfo(annotation=str, required=True), 'topic': FieldInfo(annotation=str, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

name: str

pages: str

score: float

size: int

text: str

topic: str

class h2ogpte.types.SearchResults(*, result: List[SearchResult])

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'result': FieldInfo(annotation=List[SearchResult], required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

result: List[SearchResult]

exception h2ogpte.types.SessionError: Bases: Exception

class h2ogpte.types.ShareResponseStatus(*, status: str)

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'status': FieldInfo(annotation=str, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

status: str

class h2ogpte.types.Status(value)

Bases: str, Enum

An enumeration.

Canceled = 'canceled'

Completed = 'completed'

Failed = 'failed'

Queued = 'queued'

Running = 'running'

Scheduled = 'scheduled'

Unknown = 'unknown'

class h2ogpte.types.SuggestedQuestion(*, question: str)

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'question': FieldInfo(annotation=str, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

question: str

exception h2ogpte.types.UnauthorizedError: Bases: Exception

class h2ogpte.types.User(*, username: str)

Bases: BaseModel

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'username': FieldInfo(annotation=str, required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

username: str

Module contents

h2oGPTe - AI for documents and more

h2ogpte is the Python client library for H2O.ai’s Enterprise h2oGPTe, a RAG (Retrieval-Augmented Generation) based platform built on top of many open-source software components such as h2oGPT, hnswlib, Torch, Transformers, Golang, Python, k8s, Docker, PyMuPDF, DocTR, and many more.

h2oGPTe is designed to help organizations improve their business using generative AI. It focuses on scaling as your organization expands the number of use cases, users, and documents and has the goal of being your one stop for integrating any model or LLM functionality into your business.

Main Features

Contextualize chat with your own data using RAG (Retrieval-Augmented Generation)
Scalable backend and frontend, multi-user, high throughput
Fully containerized with Kubernetes
Multi-modal support for text, images, and audio
Highly customizable prompting for:
- talk to LLM
- talk to document
- talk to collection of documents
- talk to every page of a collection (Map/Reduce), summary, extraction
LLM agnostic, choose the model you need for your use case

class h2ogpte.H2OGPTE(address: str, api_key: str | None = None, refresh_token: str | None = None, verify: bool | str = True, strict_version_check: bool = False)

Bases: object

Connect to and interact with an h2oGPTe server.

INITIAL_WAIT_INTERVAL = 0.1

MAX_WAIT_INTERVAL = 1.0

TIMEOUT = 3600.0

WAIT_BACKOFF_FACTOR = 1.4

Send a message and get a response from an LLM.

Format of inputs content:

{text_context_list}
"""\n{chat_conversation}{question}

Args:

question:: Text query to send to the LLM.
text_context_list:: List of raw text strings to be included.
system_prompt:: Text sent to models which support system prompts. Gives the model overall context in how to respond. Use auto for the model default, or None for h2oGPTe default. Defaults to ‘’ for no system prompt.
pre_prompt_query:: Text that is prepended before the contextual document chunks in text_context_list. Only used if text_context_list is provided.
prompt_query:: Text that is appended after the contextual document chunks in text_context_list. Only used if text_context_list is provided.
llm:: Name or index of LLM to send the query. Use H2OGPTE.get_llms() to see all available options. Default value is to use the first model (0th index).
llm_args:: Dictionary of kwargs to pass to the llm.
chat_conversation:: List of tuples for (human, bot) conversation that will be pre-appended to an (question, None) case for a query.
timeout:: Timeout in seconds.
kwargs:: Dictionary of kwargs to pass to h2oGPT.

Returns:

Answer: The response text and any errors.

Raises:

TimeoutError: If response isn’t completed in timeout seconds.

cancel_job(job_id: str) → Result

Stops a specific job from running on the server.

Args:

job_id:: String id of the job to cancel.

Returns:

Result: Status of canceling the job.

connect(chat_session_id: str) → Session

Create and participate in a chat session.

This is a live connection to the H2OGPTE server contained to a specific chat session on top of a single collection of documents. Users will find all questions and responses in this session in a single chat history in the UI.

Args:

chat_session_id:: ID of the chat session to connect to.

Returns:

Session: Live chat session connection with an LLM.

count_assets() → ObjectCount

Counts number of objects owned by the user.

Returns:: ObjectCount: The count of chat sessions, collections, and documents.

count_chat_sessions() → int

Counts number of chat sessions owned by the user.

Returns:: int: The count of chat sessions owned by the user.

count_chat_sessions_for_collection(collection_id: str) → int

Counts number of chat sessions in a specific collection.

Args:

collection_id:: String id of the collection to count chat sessions for.

Returns:

int: The count of chat sessions in that collection.

count_collections() → int

Counts number of collections owned by the user.

Returns:: int: The count of collections owned by the user.

count_documents() → int

Counts number of documents accessed by the user.

Returns:: int: The count of documents accessed by the user.

count_documents_in_collection(collection_id: str) → int

Counts the number of documents in a specific collection.

Args:

collection_id:: String id of the collection to count documents for.

Returns:

int: The number of documents in that collection.

count_documents_owned_by_me() → int

Counts number of documents owned by the user.

Returns:: int: The count of documents owned by the user.

count_question_reply_feedback() → int

Fetch user’s questions and answers with feedback count.

Returns:: int: the count of questions and replies that have a user feedback.

create_chat_session(collection_id: str) → str

Creates a new chat session for asking questions of documents.

Args:

collection_id:: String id of the collection to chat with.

Returns:

str: The ID of the newly created chat session.

create_chat_session_on_default_collection() → str

Creates a new chat session for asking questions of documents on the default collection.

Returns:: str: The ID of the newly created chat session.

create_collection(name: str, description: str) → str

Creates a new collection.

Args:

name:: Name of the collection.
description:: Description of the collection

Returns:

str: The ID of the newly created collection.

delete_chat_sessions(chat_session_ids: Iterable[str]) → Result

Deletes chat sessions and related messages.

Args:

chat_session_ids:: List of string ids of chat sessions to delete from the system.

Returns:

Result: Status of the delete job.

delete_collections(collection_ids: Iterable[str], timeout: float | None = None)

Deletes collections from the environment.

Documents in the collection will not be deleted.

Args:

collection_ids:: List of string ids of collections to delete from the system.
timeout:: Timeout in seconds.

delete_document_summaries(summaries_ids: Iterable[str]) → Result

Deletes document summaries.

Args:

summaries_ids:: List of string ids of a document summary to delete from the system.

Returns:

Result: Status of the delete job.

delete_documents(document_ids: Iterable[str], timeout: float | None = None)

Deletes documents from the system.

Args:

document_ids:: List of string ids to delete from the system and all collections.
timeout:: Timeout in seconds.

delete_documents_from_collection(collection_id: str, document_ids: Iterable[str], timeout: float | None = None)

Removes documents from a collection.

See Also: H2OGPTE.delete_documents for completely removing the document from the environment.

Args:

collection_id:: String of the collection to remove documents from.
document_ids:: List of string ids to remove from the collection.
timeout:: Timeout in seconds.

delete_upload(upload_id: str) → str

Delete a file previously uploaded with the “upload” method.

See Also:

upload: Upload the files into the system to then be ingested into a collection. ingest_uploads: Add the uploaded files to a collection.

Args:

upload_id:: ID of a file to remove

Returns:

upload_id: The upload id of the removed.

Raises:

Exception: The delete upload request was unsuccessful.

download_document(destination_directory: str, destination_file_name: str, document_id: str) → Path

Downloads a document to a local system directory.

Args:

destination_directory:: Destination directory to save file into.
destination_file_name:: Destination file name.
document_id:: Document ID.

Returns:

Path: Path of downloaded document

encode_for_retrieval(chunks: List[str]) → List[List[float]]

Encode texts for semantic searching.

See Also: H2OGPTE.match for getting a list of chunks that semantically match each encoded text.

Args:

chunks:: List of strings of texts to be encoded.

Returns:

List of list of floats: Each list in the list is the encoded original text.

Extract information from one or more contexts using an LLM.

pre_prompt_extract and prompt_extract variables must be used together. If these variables are not set, the inputs texts will be summarized into bullet points.

Format of extract content:

"{pre_prompt_extract}"""
{text_context_list}
"""\n{prompt_extract}"

Examples:

extract = h2ogpte.extract_data(
    text_context_list=chunks,
    pre_prompt_extract="Pay attention and look at all people. Your job is to collect their names.\n",
    prompt_extract="List all people's names as JSON.",
)

Args:

text_context_list:: List of raw text strings to extract data from.
system_prompt:: Text sent to models which support system prompts. Gives the model overall context in how to respond. Use auto or None for the model default. Defaults to ‘’ for no system prompt.
pre_prompt_extract:: Text that is prepended before the list of texts. If not set, the inputs will be summarized.
prompt_extract:: Text that is appended after the list of texts. If not set, the inputs will be summarized.
llm:: Name or index of LLM to send the query. Use H2OGPTE.get_llms() to see all available options. Default value is to use the first model (0th index).
llm_args:: Dictionary of kwargs to pass to the llm.
kwargs:: Dictionary of kwargs to pass to h2oGPT.

Returns:

ExtractionAnswer: The list of text responses and any errors.

get_chunks(collection_id: str, chunk_ids: Iterable[int]) → List[Chunk]

Get the text of specific chunks in a collection.

Args:

collection_id:: String id of the collection to search in.
chunk_ids:: List of ints for the chunks to return. Chunks are indexed starting at 1.

Returns:

Chunk: The text of the chunk.

Raises:

Exception: One or more chunks could not be found.

get_collection(collection_id: str) → Collection

Get metadata about a collection.

Args:

collection_id:: String id of the collection to search for.

Returns:

Collection: Metadata about the collection.

Raises:

KeyError: The collection was not found.

get_collection_for_chat_session(chat_session_id: str) → Collection

Get metadata about the collection of a chat session.

Args:

chat_session_id:: String id of the chat session to search for.

Returns:

Collection: Metadata about the collection.

get_collection_questions(collection_id: str, limit: int) → List[SuggestedQuestion]

List suggested questions

Args:

collection_id:: A collection ID of which to return the suggested questions
limit:: How many questions to return.

Returns:

List: A list of questions.

get_default_collection() → CollectionInfo

Get the default collection, to be used for collection API-keys.

Returns:: CollectionInfo: Default collection info.

get_document(document_id: str) → Document

Fetches information about a specific document.

Args:

document_id:: String id of the document.

Returns:

Document: Metadata about the Document.

Raises:

KeyError: The document was not found.

get_job(job_id: str) → Job

Fetches information about a specific job.

Args:

job_id:: String id of the job.

Returns:

Job: Metadata about the Job.

get_llm_names() → List[str]

Lists names of available LLMs in the environment.

Returns:: list of string: Name of each available model.

get_llm_usage_24h() → float

get_llm_usage_24h_by_llm() → List[LLMUsage]

get_llm_usage_24h_with_limits() → LLMUsageLimit

get_llm_usage_6h() → float

get_llm_usage_6h_by_llm() → List[LLMUsage]

get_llm_usage_by_llm(interval: str) → List[LLMUsage]

get_llm_usage_with_limits(interval: str) → LLMUsageLimit

get_llms() → List[dict]

Lists metadata information about available LLMs in the environment.

Returns:: list of dict (string, ANY): Name and details about each available model.

get_meta() → Meta

Returns information about the environment and the user.

Returns:: Meta: Details about the version and license of the environment and the user’s name and email.

get_scheduler_stats() → SchedulerStats

Count the number of global, pending jobs on the server.

Returns:: SchedulerStats: The queue length for number of jobs.

import_document_into_collection(collection_id: str, document_id: str, gen_doc_summaries: bool = False, gen_doc_questions: bool = False, timeout: float | None = None)

Import an already stored document to an existing collection

Args:

collection_id:: Collection ID to add documents to.
document_id:: Document ID to add.
gen_doc_summaries:: Whether to auto-generate document summaries (uses LLM)
gen_doc_questions:: Whether to auto-generate sample questions for each document (uses LLM)
timeout:: Timeout in seconds.

ingest_from_file_system(collection_id: str, root_dir: str, glob: str, gen_doc_summaries: bool = False, gen_doc_questions: bool = False, timeout: float | None = None)

Add files from the local system into a collection.

Args:

collection_id:: String id of the collection to add the ingested documents into.
root_dir:: String path of where to look for files.
glob:: String of the glob pattern used to match files in the root directory.
gen_doc_summaries:: Whether to auto-generate document summaries (uses LLM)
gen_doc_questions:: Whether to auto-generate sample questions for each document (uses LLM)
timeout:: Timeout in seconds

ingest_uploads(collection_id: str, upload_ids: Iterable[str], gen_doc_summaries: bool = False, gen_doc_questions: bool = False, timeout: float | None = None)

Add uploaded documents into a specific collection.

See Also:

upload: Upload the files into the system to then be ingested into a collection. delete_upload: Delete uploaded file

Args:

collection_id:: String id of the collection to add the ingested documents into.
upload_ids:: List of string ids of each uploaded document to add to the collection.
gen_doc_summaries:: Whether to auto-generate document summaries (uses LLM)
gen_doc_questions:: Whether to auto-generate sample questions for each document (uses LLM)
timeout:: Timeout in seconds

ingest_website(collection_id: str, url: str, gen_doc_summaries: bool = False, gen_doc_questions: bool = False, timeout: float | None = None)

Crawl and ingest a website into a collection.

All web pages linked from this URL will be imported. External links will be ignored. Links to other pages on the same domain will be followed as long as they are at the same level or below the URL you specify. Each page will be transformed into a PDF document.

Args:

collection_id:: String id of the collection to add the ingested documents into.
url:: String of the url to crawl.
gen_doc_summaries:: Whether to auto-generate document summaries (uses LLM)
gen_doc_questions:: Whether to auto-generate sample questions for each document (uses LLM)
timeout:: Timeout in seconds

list_chat_message_meta_part(message_id: str, info_type: str) → ChatMessageMeta

Fetch one chat message meta information.

Args:

message_id:: Message id to which the metadata should be pulled.
info_type:: Metadata type to fetch. Valid choices are: “self_reflection”, “usage_stats”, “prompt_raw”, “llm_only”, “rag”, “hyde1”, “hyde2”

Returns:

ChatMessageMeta: Metadata information about the chat message.

list_chat_message_references(message_id: str) → List[ChatMessageReference]

Fetch metadata for references of a chat message.

References are only available for messages sent from an LLM, an empty list will be returned for messages sent by the user.

Args:

message_id:: String id of the message to get references for.

Returns:

list of ChatMessageReference: Metadata including the document name, polygon information, and score.

list_chat_messages(chat_session_id: str, offset: int, limit: int) → List[ChatMessage]

Fetch chat message and metadata for messages in a chat session.

Messages without a reply_to are from the end user, messages with a reply_to are from an LLM and a response to a specific user message.

Args:

chat_session_id:: String id of the chat session to filter by.
offset:: How many chat messages to skip before returning.
limit:: How many chat messages to return.

Returns:

list of ChatMessage: Text and metadata for chat messages.

list_chat_messages_full(chat_session_id: str, offset: int, limit: int) → List[ChatMessageFull]

Fetch chat message and metadata for messages in a chat session.

Messages without a reply_to are from the end user, messages with a reply_to are from an LLM and a response to a specific user message.

Args:

chat_session_id:: String id of the chat session to filter by.
offset:: How many chat messages to skip before returning.
limit:: How many chat messages to return.

Returns:

list of ChatMessageFull: Text and metadata for chat messages.

list_chat_sessions_for_collection(collection_id: str, offset: int, limit: int) → List[ChatSessionForCollection]

Fetch chat session metadata for chat sessions in a collection.

Args:

collection_id:: String id of the collection to filter by.
offset:: How many chat sessions to skip before returning.
limit:: How many chat sessions to return.

Returns:

list of ChatSessionForCollection: Metadata about each chat session including the latest message.

list_collection_permissions(collection_id: str) → List[Permission]

Returns a list of access permissions for a given collection.

The returned list of permissions denotes who has access to the collection and their access level.

Args:

collection_id:: ID of the collection to inspect.

Returns:

list of Permission: Sharing permissions list for the given collection.

list_collections_for_document(document_id: str, offset: int, limit: int) → List[CollectionInfo]

Fetch metadata about each collection the document is a part of.

At this time, each document will only be available in a single collection.

Args:

document_id:: String id of the document to search for.
offset:: How many collections to skip before returning.
limit:: How many collections to return.

Returns:

list of CollectionInfo: Metadata about each collection.

list_documents_in_collection(collection_id: str, offset: int, limit: int) → List[DocumentInfo]

Fetch document metadata for documents in a collection.

Args:

collection_id:: String id of the collection to filter by.
offset:: How many documents to skip before returning.
limit:: How many documents to return.

Returns:

list of DocumentInfo: Metadata about each document.

list_jobs() → List[Job]

List the user’s jobs.

Returns:: list of Job:

list_list_chat_message_meta(message_id: str) → List[ChatMessageMeta]

Fetch chat message meta information.

Args:

message_id:: Message id to which the metadata should be pulled.

Returns:

list of ChatMessageMeta: Metadata about the chat message.

list_question_reply_feedback_data(offset: int, limit: int) → List[QuestionReplyData]

Fetch user’s questions and answers that have a feedback.

Questions and answers with metadata and feedback information.

Args:

offset:: How many conversations to skip before returning.
limit:: How many conversations to return.

Returns:

list of QuestionReplyData: Metadata about questions and answers.

list_recent_chat_sessions(offset: int, limit: int) → List[ChatSessionInfo]

Fetch user’s chat session metadata sorted by last update time.

Chats across all collections will be accessed.

Args:

offset:: How many chat sessions to skip before returning.
limit:: How many chat sessions to return.

Returns:

list of ChatSessionInfo: Metadata about each chat session including the latest message.

list_recent_collections(offset: int, limit: int) → List[CollectionInfo]

Fetch user’s collection metadata sorted by last update time.

Args:

offset:: How many collections to skip before returning.
limit:: How many collections to return.

Returns:

list of CollectionInfo: Metadata about each collection.

list_recent_collections_sort(offset: int, limit: int, sort_column: str, ascending: bool) → List[CollectionInfo]

Fetch user’s collection metadata sorted by last update time.

Args:

offset:: How many collections to skip before returning.
limit:: How many collections to return.
sort_column:: Sort column.
ascending:: When True, return sorted by sort_column in ascending order.

Returns:

list of CollectionInfo: Metadata about each collection.

list_recent_document_summaries(document_id: str, offset: int, limit: int) → List[DocumentSummary]

Fetches recent document summaries

Args:

document_id:: document ID for which to return summaries
offset:: How many summaries to skip before returning summaries.
limit:: How many summaries to return.

list_recent_documents(offset: int, limit: int) → List[DocumentInfo]

Fetch user’s document metadata sorted by last update time.

All documents owned by the user, regardless of collection, are accessed.

Args:

offset:: How many documents to skip before returning.
limit:: How many documents to return.

Returns:

list of DocumentInfo: Metadata about each document.

list_recent_documents_with_summaries(offset: int, limit: int) → List[DocumentInfoSummary]

Fetch user’s document metadata sorted by last update time, including the latest document summary.

All documents owned by the user, regardless of collection, are accessed.

Args:

offset:: How many documents to skip before returning.
limit:: How many documents to return.

Returns:

list of DocumentInfoSummary: Metadata about each document.

list_recent_documents_with_summaries_sort(offset: int, limit: int, sort_column: str, ascending: bool) → List[DocumentInfoSummary]

Fetch user’s document metadata sorted by last update time, including the latest document summary.

All documents owned by the user, regardless of collection, are accessed.

Args:

offset:: How many documents to skip before returning.
limit:: How many documents to return.
sort_column:: Sort column.
ascending:: When True, return sorted by sort_column in ascending order.

Returns:

list of DocumentInfoSummary: Metadata about each document.

list_upload() → List[str]

List pending file uploads to the H2OGPTE backend.

Uploaded files are not yet accessible and need to be ingested into a collection.

See Also:: upload: Upload the files into the system to then be ingested into a collection. ingest_uploads: Add the uploaded files to a collection. delete_upload: Delete uploaded file
Returns:: List[str]: The pending upload ids to be used in ingest jobs.
Raises:: Exception: The upload list request was unsuccessful.

list_users(offset: int, limit: int) → List[User]

List system users.

Returns a list of all registered users fo the system, a registered user, is a users that has logged in at least once.

Args:

offset:: How many users to skip before returning.
limit:: How many users to return.

Returns:

list of User: Metadata about each user.

make_collection_private(collection_id: str)

Make a collection private

Once a collection is private, other users will no longer be able to access chat history or documents related to the collection.

Args:

collection_id:: ID of the collection to make private.

make_collection_public(collection_id: str)

Make a collection public

Once a collection is public, it will be accessible to all authenticated users of the system.

Args:

collection_id:: ID of the collection to make public.

match_chunks(collection_id: str, vectors: List[List[float]], topics: List[str], offset: int, limit: int, cut_off: float = 0, width: int = 0) → List[SearchResult]

Find chunks related to a message using semantic search.

Chunks are sorted by relevance and similarity score to the message.

See Also: H2OGPTE.encode_for_retrieval to create vectors from messages.

Args:

collection_id:: ID of the collection to search within.
vectors:: A list of vectorized message for running semantic search.
topics:: A list of document_ids used to filter which documents in the collection to search.
offset:: How many chunks to skip before returning chunks.
limit:: How many chunks to return.
cut_off:: Exclude matches with distances higher than this cut off.
width:: How many chunks before and after a match to return - not implemented.

Returns:

list of SearchResult: The document, text, score and related information of the chunk.

openid_connect() → str

Tests openid connection.

Returns:: A session ID string.

reset_collection_prompt_settings(collection_id: str) → str

Reset the prompt settings for a given collection.

Args:

collection_id:: ID of the collection to update.

Returns:

str: ID of the updated collection.

search_chunks(collection_id: str, query: str, topics: List[str], offset: int, limit: int) → List[SearchResult]

Find chunks related to a message using lexical search.

Chunks are sorted by relevance and similarity score to the message.

Args:

collection_id:: ID of the collection to search within.
query:: Question or imperative from the end user to search a collection for.
topics:: A list of document_ids used to filter which documents in the collection to search.
offset:: How many chunks to skip before returning chunks.
limit:: How many chunks to return.

Returns:

list of SearchResult: The document, text, score and related information of the chunk.

set_chat_message_votes(chat_message_id: str, votes: int) → Result

Change the vote value of a chat message.

Set the exact value of a vote for a chat message. Any message type can be updated, but only LLM response votes will be visible in the UI. The expectation is 0: unvoted, -1: dislike, 1 like. Values outside of this will not be viewable in the UI.

Args:

chat_message_id:: ID of a chat message, any message can be used but only LLM responses will be visible in the UI.
votes:: Integer value for the message. Only -1 and 1 will be visible in the UI as dislike and like respectively.

Returns:

Result: The status of the update.

Raises:

Exception: The upload request was unsuccessful.

share_collection(collection_id: str, permission: Permission) → ShareResponseStatus

Share a collection to a user.

The permission attribute defined the level of access, and who can access the collection, the collection_id attribute denotes the collection to be shared.

Args:

collection_id:: ID of the collection to share.
permission:: Defines the rule for sharing, i.e. permission level.

Returns:

ShareResponseStatus: Status of share request.

Summarize one or more contexts using an LLM.

Effective prompt created (excluding the system prompt):

"{pre_prompt_summary}
"""
{text_context_list}
"""
{prompt_summary}"

Args:

text_context_list:: List of raw text strings to be summarized.
system_prompt:: Text sent to models which support system prompts. Gives the model overall context in how to respond. Use auto for the model default or None for h2oGPTe defaults. Defaults to ‘’ for no system prompt.
pre_prompt_summary:: Text that is prepended before the list of texts. The default can be customized per environment, but the standard default is "In order to write a concise single-paragraph or bulleted list summary, pay attention to the following text:\n"
prompt_summary:: Text that is appended after the list of texts. The default can be customized per environment, but the standard default is "Using only the text above, write a condensed and concise summary of key results (preferably as bullet points):\n"
llm:: Name or index of LLM to send the query. Use H2OGPTE.get_llms() to see all available options. Default value is to use the first model (0th index).
llm_args:: Dictionary of kwargs to pass to the llm.
kwargs:: Dictionary of kwargs to pass to h2oGPT.

Returns:

Answer: The response text and any errors.

Creates a summary of a document.

Effective prompt created (excluding the system prompt):

"{pre_prompt_summary}
"""
{text from document}
"""
{prompt_summary}"

Args:

document_id:: String id of the document to create a summary from.
system_prompt:: System Prompt
pre_prompt_summary:: Prompt that goes before each large piece of text to summarize
prompt_summary:: Prompt that goes after each large piece of of text to summarize
llm:: LLM to use
llm_args:: Arguments for the LLM
max_num_chunks:: Max limit of chunks to send to the summarizer
sampling_strategy:: How to sample if the document has more chunks than max_num_chunks. Options are “auto”, “uniform”, “first”, “first+last”, default is “auto” (a hybrid of them all).
timeout:: Amount of time in seconds to allow the request to run. The default is 86400 seconds.

Returns:

DocumentSummary: Summary of the document

Raises:

TimeoutError: The request did not complete in time. SessionError: No summary created. Document wasn’t part of a collection, or LLM timed out, etc.

unshare_collection(collection_id: str, permission: Permission) → ShareResponseStatus

Remove sharing of a collection to a user.

The permission attribute defined the level of access, and who can access the collection, the collection_id attribute denotes the collection to be shared. In case of un-sharing, the Permission’s user is sufficient

Args:

collection_id:: ID of the collection to un-share.
permission:: Defines the user for which collection access is revoked.

ShareResponseStatus: Status of share request.

unshare_collection_for_all(collection_id: str) → ShareResponseStatus

Remove sharing of a collection to all other users but the original owner

Args:

collection_id:: ID of the collection to un-share.

ShareResponseStatus: Status of share request.

update_collection(collection_id: str, name: str, description: str) → str

Update the metadata for a given collection.

All variables are required. You can use h2ogpte.get_collection(<id>).name or description to get the existing values if you only want to change one or the other.

Args:

collection_id:: ID of the collection to update.
name:: New name of the collection, this is required.
description:: New description of the collection, this is required.

Returns:

str: ID of the updated collection.

update_collection_prompt_settings(collection_id: str, system_prompt: str, pre_prompt_query: str, prompt_query: str, rag_type: str, hyde_no_rag_llm_prompt_extension: str, auto_gen_description_prompt: str) → str

Update the prompt settings for a given collection.

Updates the prompt settings for a given collection.

Args:

collection_id:: ID of the collection to update.
system_prompt:: Text sent to models which support system prompts. Gives the model overall context in how to respond.
pre_prompt_query:: Text that is prepended before the contextual document chunks.
prompt_query:: Text that is appended to the beginning of the user’s message.
rag_type:: RAG type to use.
hyde_no_rag_llm_prompt_extension:: LLM prompt extension.
auto_gen_description_prompt:: prompt to create a description of the collection.

Returns:

str: ID of the updated collection.

upload(file_name: str, file: Any) → str

Upload a file to the H2OGPTE backend.

Uploaded files are not yet accessible and need to be ingested into a collection.

See Also:

ingest_uploads: Add the uploaded files to a collection. delete_upload: Delete uploaded file

Args:

file_name:: What to name the file on the server, must include file extension.
file:: File object to upload, often an opened file from with open(…) as f.

Returns:

str: The upload id to be used in ingest jobs.

Raises:

Exception: The upload request was unsuccessful.

class h2ogpte.H2OGPTEAsync(address: str, api_key: str | None = None, refresh_token: str | None = None, verify: bool | str = True, strict_version_check: bool = False)

Bases: object

Connect to and interact with an h2oGPTe server, via an async interface.

INITIAL_WAIT_INTERVAL = 0.1

MAX_WAIT_INTERVAL = 1.0

TIMEOUT = 3600.0

WAIT_BACKOFF_FACTOR = 1.4

Send a message and get a response from an LLM.

Format of input content:

{text_context_list}
"""\n{chat_conversation}{question}

Args:

question:: Text query to send to the LLM.
text_context_list:: List of raw text strings to be included.
system_prompt:: Text sent to models which support system prompts. Gives the model overall context in how to respond. Use auto for the model default, or None for h2oGPTe default. Defaults to ‘’ for no system prompt.
pre_prompt_query:: Text that is prepended before the contextual document chunks in text_context_list. Only used if text_context_list is provided.
prompt_query:: Text that is appended after the contextual document chunks in text_context_list. Only used if text_context_list is provided.
llm:: Name or index of LLM to send the query. Use H2OGPTE.get_llms() to see all available options. Default value is to use the first model (0th index).
llm_args:: Dictionary of kwargs to pass to the llm.
chat_conversation:: List of tuples for (human, bot) conversation that will be pre-appended to an (question, None) case for a query.
timeout:: Timeout in seconds.
kwargs:: Dictionary of kwargs to pass to h2oGPT.

Returns:

Answer: The response text and any errors.

Raises:

TimeoutError: If response isn’t completed in timeout seconds.

async cancel_job(job_id: str) → Result

Stops a specific job from running on the server.

Args:

job_id:: String id of the job to cancel.

Returns:

Result: Status of canceling the job.

connect(chat_session_id: str) → SessionAsync

Create and participate in a chat session. This is a live connection to the H2OGPTE server contained to a specific chat session on top of a single collection of documents. Users will find all questions and responses in this session in a single chat history in the UI.

Args:

chat_session_id:: ID of the chat session to connect to.

Returns:

Session: Live chat session connection with an LLM.

async count_assets() → ObjectCount

Counts number of objects owned by the user.

Returns:: ObjectCount: The count of chat sessions, collections, and documents.

async count_chat_sessions() → int: Returns the count of chat sessions owned by the user.

async count_chat_sessions_for_collection(collection_id: str) → int

Counts number of chat sessions in a specific collection.

Args:

collection_id:: String id of the collection to count chat sessions for.

Returns:

int: The count of chat sessions in that collection.

async count_collections() → int

Counts number of collections owned by the user.

Returns:: int: The count of collections owned by the user.

async count_documents() → int: Returns the count of documents accessed by the user.

async count_documents_in_collection(collection_id: str) → int

Counts the number of documents in a specific collection.

Args:

collection_id:: String id of the collection to count documents for.

Returns:

int: The number of documents in that collection.

async count_documents_owned_by_me() → int: Returns the counts of documents owned by the user.

async count_question_reply_feedback() → int

Fetch user’s questions and answers count.

Returns:: int: the count of questions and replies.

async create_chat_session(collection_id: str) → str

Creates a new chat session for asking questions of documents.

Args:

collection_id:: String id of the collection to chat with.

Returns:

str: The ID of the newly created chat session.

async create_chat_session_on_default_collection() → str

Creates a new chat session for asking questions of documents on the default collection.

Returns:: str: The ID of the newly created chat session.

async create_collection(name: str, description: str) → str

Creates a new collection.

Args:

name:: Name of the collection.
description:: Description of the collection

Returns:

str: The ID of the newly created collection.

async delete_chat_sessions(chat_session_ids: Iterable[str]) → Result

Deletes chat sessions and related messages.

Args:

chat_session_ids:: List of string ids of chat sessions to delete from the system.

Returns:

Result: Status of the delete job.

async delete_collections(collection_ids: Iterable[str], timeout: float | None = None) → Job

Deletes collections from the environment. Documents in the collection will not be deleted.

Args:

collection_ids:: List of string ids of collections to delete from the system.
timeout:: Timeout in seconds.

async delete_document_summaries(summaries_ids: Iterable[str]) → Result

Deletes document summaries.

Args:

summaries_ids:: List of string ids of a document summary to delete from the system.

Returns:

Result: Status of the delete job.

async delete_documents(document_ids: Iterable[str], timeout: float | None = None) → Job

Deletes documents from the system.

Args:

document_ids:: List of string ids to delete from the system and all collections.
timeout:: Timeout in seconds.

async delete_documents_from_collection(collection_id: str, document_ids: Iterable[str], timeout: float | None = None) → Job

Removes documents from a collection.

See Also: H2OGPTE.delete_documents for completely removing the document from the environment.

Args:

collection_id:: String of the collection to remove documents from.
document_ids:: List of string ids to remove from the collection.
timeout:: Timeout in seconds.

async delete_upload(upload_id: str) → str

Delete a file previously uploaded with the “upload” method.

See Also:

upload: Upload the files into the system to then be ingested into a collection. ingest_uploads: Add the uploaded files to a collection.

Args:

upload_id:: ID of a file to remove

Returns:

upload_id: The upload id of the removed.

Raises:

Exception: The delete upload request was unsuccessful.

async download_document(destination_directory: str | Path, destination_file_name: str, document_id: str) → Path

Downloads a document to a local system directory.

Args:

destination_directory:: Destination directory to save file into.
destination_file_name:: Destination file name.
document_id:: Document ID.

Returns:

The Path to the file written to disk.

async encode_for_retrieval(chunks: Iterable[str]) → List[List[float]]

Encode texts for semantic searching.

See Also: H2OGPTE.match for getting a list of chunks that semantically match each encoded text.

Args:

chunks:: List of strings of texts to be encoded.

Returns:

List of list of floats: Each list in the list is the encoded original text.

Extract information from one or more contexts using an LLM. pre_prompt_extract and prompt_extract variables must be used together. If these variables are not set, the inputs texts will be summarized into bullet points. Format of extract content:

"{pre_prompt_extract}"""
{text_context_list}
"""\n{prompt_extract}"

Examples:

extract = h2ogpte.extract_data(
    text_context_list=chunks,
    pre_prompt_extract="Pay attention and look at all people. Your job
                        is to collect their names.\n",
    prompt_extract="List all people's names as JSON.",
)

Args:

text_context_list:: List of raw text strings to extract data from.
system_prompt:: Text sent to models which support system prompts. Gives the model overall context in how to respond. Use auto or None for the model default. Defaults to ‘’ for no system prompt.
pre_prompt_extract:: Text that is prepended before the list of texts. If not set, the inputs will be summarized.
prompt_extract:: Text that is appended after the list of texts. If not set, the inputs will be summarized.
llm:: Name or index of LLM to send the query. Use H2OGPTE.get_llms() to see all available options. Default value is to use the first model (0th index).
llm_args:: Dictionary of kwargs to pass to the llm.
kwargs:: Dictionary of kwargs to pass to h2oGPT.

Returns:

ExtractionAnswer: The list of text responses and any errors.

async get_chunks(collection_id: str, chunk_ids: Iterable[int]) → List[Chunk]

Get the text of specific chunks in a collection.

Args:

collection_id:: String id of the collection to search in.
chunk_ids:: List of ints for the chunks to return. Chunks are indexed starting at 1.

Returns:

Chunk: The text of the chunk.

Raises:

Exception: One or more chunks could not be found.

async get_collection(collection_id: str) → Collection

Get metadata about a collection.

Args:

collection_id:: String id of the collection to search for.

Returns:

Collection: Metadata about the collection.

Raises:

KeyError: The collection was not found.

async get_collection_for_chat_session(chat_session_id: str) → Collection

Get metadata about the collection of a chat session.

Args:

chat_session_id:: String id of the chat session to search for.

Returns:

Collection: Metadata about the collection.

async get_collection_questions(collection_id: str, limit: int) → List[SuggestedQuestion]

List suggested questions

Args:

collection_id:: A collection ID of which to return the suggested questions
limit:: How many questions to return.

Returns:

List: A list of questions.

async get_default_collection() → CollectionInfo

Get the default collection, to be used for collection API-keys.

Returns:: CollectionInfo: Default collection info.

async get_document(document_id: str) → Document

Fetches information about a specific document.

Args:

document_id:: String id of the document.

Returns:

Document: Metadata about the Document.

Raises:

KeyError: The document was not found.

async get_job(job_id: str) → Job

Fetches information about a specific job.

Args:

job_id:: String id of the job.

Returns:

Job: Metadata about the Job.

async get_llm_names() → List[str]

Lists names of available LLMs in the environment.

Returns:: list of string: Name of each available model.

async get_llm_usage_24h() → float

async get_llm_usage_24h_by_llm() → List[LLMUsage]

async get_llm_usage_24h_with_limits() → LLMUsageLimit

async get_llm_usage_6h() → float

async get_llm_usage_6h_by_llm() → List[LLMUsage]

async get_llm_usage_by_llm(interval: str) → List[LLMUsage]

async get_llm_usage_with_limits(interval: str) → LLMUsageLimit

async get_llms() → List[Dict[str, Any]]

Lists metadata information about available LLMs in the environment.

Returns:: list of dict (string, ANY): Name and details about each available model.

async get_meta() → Meta: Returns various information about the server environment, including the current build version, license information, the user, etc.

async get_scheduler_stats() → SchedulerStats

Count the number of global, pending jobs on the server.

Returns:: SchedulerStats: The queue length for number of jobs.

async import_document_into_collection(collection_id: str, document_id: str, gen_doc_summaries: bool = False, gen_doc_questions: bool = False, timeout: float | None = None)

Import an already stored document to an existing collection

Args:

collection_id:: Collection ID to add documents to.
document_id:: Document ID to add.
gen_doc_summaries:: Whether to auto-generate document summaries (uses LLM)
gen_doc_questions:: Whether to auto-generate sample questions for each document (uses LLM)
timeout:: Timeout in seconds.

async ingest_from_file_system(collection_id: str, root_dir: str, glob: str, gen_doc_summaries: bool = False, gen_doc_questions: bool = False, timeout: float | None = None) → Job

Add files from the local system into a collection.

Args:

collection_id:: String id of the collection to add the ingested documents into.
root_dir:: String path of where to look for files.
glob:: String of the glob pattern used to match files in the root directory.
gen_doc_summaries:: Whether to auto-generate document summaries (uses LLM)
gen_doc_questions:: Whether to auto-generate sample questions for each document (uses LLM)
timeout:: Timeout in seconds

async ingest_uploads(collection_id: str, upload_ids: Iterable[str], gen_doc_summaries: bool = False, gen_doc_questions: bool = False, timeout: float | None = None) → Job

Add uploaded documents into a specific collection.

See Also:

upload: Upload the files into the system to then be ingested into a: collection.

delete_upload: Delete uploaded file.

Args:

collection_id:: String id of the collection to add the ingested documents into.
upload_ids:: List of string ids of each uploaded document to add to the collection.
gen_doc_summaries:: Whether to auto-generate document summaries (uses LLM)
gen_doc_questions:: Whether to auto-generate sample questions for each document (uses LLM)
timeout:: Timeout in seconds

async ingest_website(collection_id: str, url: str, gen_doc_summaries: bool = False, gen_doc_questions: bool = False, timeout: float | None = None) → Job

Crawl and ingest a website into a collection.

All web pages linked from this URL will be imported. External links will be ignored. Links to other pages on the same domain will be followed as long as they are at the same level or below the URL you specify. Each page will be transformed into a PDF document.

Args:

collection_id:: String id of the collection to add the ingested documents into.
url:: String of the url to crawl.
gen_doc_summaries:: Whether to auto-generate document summaries (uses LLM)
gen_doc_questions:: Whether to auto-generate sample questions for each document (uses LLM)
timeout:: Timeout in seconds

async list_chat_message_meta_part(message_id: str, info_type: str) → ChatMessageMeta

Fetch one chat message meta information.

Args:

message_id:: Message id to which the metadata should be pulled.
info_type:: Metadata type to fetch. Valid choices are: “self_reflection”, “usage_stats”, “prompt_raw”, “llm_only”, “rag”, “hyde1”, “hyde2”

Returns:

ChatMessageMeta: Metadata information about the chat message.

async list_chat_message_references(message_id: str) → List[ChatMessageReference]

Fetch metadata for references of a chat message.

References are only available for messages sent from an LLM, an empty list will be returned for messages sent by the user.

Args:

message_id:: String id of the message to get references for.

Returns:

list of ChatMessageReference: Metadata including the document name, polygon information, and score.

async list_chat_messages(chat_session_id: str, offset: int, limit: int) → List[ChatMessage]

Fetch chat message and metadata for messages in a chat session.

Messages without a reply_to are from the end user, messages with a reply_to are from an LLM and a response to a specific user message.

Args:

chat_session_id:: String id of the chat session to filter by.
offset:: How many chat messages to skip before returning.
limit:: How many chat messages to return.

Returns:

list of ChatMessage: Text and metadata for chat messages.

async list_chat_messages_full(chat_session_id: str, offset: int, limit: int) → List[ChatMessageFull]

Fetch chat message and metadata for messages in a chat session.

Messages without a reply_to are from the end user, messages with a reply_to are from an LLM and a response to a specific user message.

Args:

chat_session_id:: String id of the chat session to filter by.
offset:: How many chat messages to skip before returning.
limit:: How many chat messages to return.

Returns:

list of ChatMessageFull: Text and metadata for chat messages.

async list_chat_sessions_for_collection(collection_id: str, offset: int, limit: int) → List[ChatSessionForCollection]

Fetch chat session metadata for chat sessions in a collection.

Args:

collection_id:: String id of the collection to filter by.
offset:: How many chat sessions to skip before returning.
limit:: How many chat sessions to return.

Returns:

list of ChatSessionForCollection: Metadata about each chat session including the latest message.

async list_collection_permissions(collection_id: str) → List[Permission]

Returns a list of access permissions for a given collection.

The returned list of permissions denotes who has access to the collection and their access level.

Args:

collection_id:: ID of the collection to inspect.

Returns:

list of Permission: Sharing permissions list for the given collection.

async list_collections_for_document(document_id: str, offset: int, limit: int) → List[CollectionInfo]

Fetch metadata about each collection the document is a part of. At this time, each document will only be available in a single collection.

Args:

document_id:: String id of the document to search for.
offset:: How many collections to skip before returning.
limit:: How many collections to return.

Returns:

list of CollectionInfo: Metadata about each collection.

async list_documents_in_collection(collection_id: str, offset: int, limit: int) → List[DocumentInfo]

Fetch document metadata for documents in a collection.

Args:

collection_id:: String id of the collection to filter by.
offset:: How many documents to skip before returning.
limit:: How many documents to return.

Returns:

list of DocumentInfo: Metadata about each document.

async list_jobs() → List[Job]

List the user’s jobs.

Returns:: list of Job:

async list_list_chat_message_meta(message_id: str) → List[ChatMessageMeta]

Fetch chat message meta information.

Args:

message_id:: Message id to which the metadata should be pulled.

Returns:

list of ChatMessageMeta: Metadata about the chat message.

async list_question_reply_feedback_data(offset: int, limit: int) → List[QuestionReplyData]

Fetch user’s questions and answers.

Questions and answers with metadata.

Args:

offset:: How many conversations to skip before returning.
limit:: How many conversations to return.

Returns:

list of QuestionReplyData: Metadata about questions and answers.

async list_recent_chat_sessions(offset: int, limit: int) → List[ChatSessionInfo]

Fetch user’s chat session metadata sorted by last update time. Chats across all collections will be accessed.

Args:

offset:: How many chat sessions to skip before returning.
limit:: How many chat sessions to return.

Returns:

list of ChatSessionInfo: Metadata about each chat session including the latest message.

async list_recent_collections(offset: int, limit: int) → List[CollectionInfo]

Fetch user’s collection metadata sorted by last update time.

Args:

offset:: How many collections to skip before returning.
limit:: How many collections to return.

Returns:

list of CollectionInfo: Metadata about each collection.

async list_recent_collections_sort(offset: int, limit: int, sort_column: str, ascending: bool) → List[CollectionInfo]

Fetch user’s collection metadata sorted by last update time.

Args:

offset:: How many collections to skip before returning.
limit:: How many collections to return.
sort_column:: Sort column.
ascending:: When True, return sorted by sort_column in ascending order.

Returns:

list of CollectionInfo: Metadata about each collection.

async list_recent_document_summaries(document_id: str, offset: int, limit: int) → List[DocumentSummary]

Fetches recent document summaries

Args:

document_id:: document ID for which to return summaries
offset:: How many summaries to skip before returning summaries.
limit:: How many summaries to return.

async list_recent_documents(offset: int, limit: int) → List[DocumentInfo]

Fetch user’s document metadata sorted by last update time. All documents owned by the user, regardless of collection, are accessed.

Args:

offset:: How many documents to skip before returning.
limit:: How many documents to return.

Returns:

list of DocumentInfo: Metadata about each document.

async list_recent_documents_with_summaries(offset: int, limit: int) → List[DocumentInfoSummary]

Fetch user’s document metadata sorted by last update time, including the latest document summary. All documents owned by the user, regardless of collection, are accessed.

Args:

offset:: How many documents to skip before returning.
limit:: How many documents to return.

Returns:

list of DocumentInfoSummary: Metadata about each document.

async list_recent_documents_with_summaries_sort(offset: int, limit: int, sort_column: str, ascending: bool) → List[DocumentInfoSummary]

Fetch user’s document metadata sorted by last update time, including the latest document summary. All documents owned by the user, regardless of collection, are accessed.

Args:

offset:: How many documents to skip before returning.
limit:: How many documents to return.
sort_column:: Sort column.
ascending:: When True, return sorted by sort_column in ascending order.

Returns:

list of DocumentInfoSummary: Metadata about each document.

async list_upload() → List[str]

List pending file uploads to the H2OGPTE backend. Uploaded files are not yet accessible and need to be ingested into a collection.

See Also:: upload: Upload the files into the system to then be ingested into a collection. ingest_uploads: Add the uploaded files to a collection. delete_upload: Delete uploaded file
Returns:: List[str]: The pending upload ids to be used in ingest jobs.
Raises:: Exception: The upload list request was unsuccessful.

async list_users(offset: int, limit: int) → List[User]

List system users.

Returns a list of all registered users fo the system, a registered user, is a users that has logged in at least once.

Args:

offset:: How many users to skip before returning.
limit:: How many users to return.

Returns:

list of User: Metadata about each user.

async make_collection_private(collection_id: str)

Make a collection private Once a collection is private, other users will no longer be able to access chat history or documents related to the collection.

Args:

collection_id:: ID of the collection to make private.

async make_collection_public(collection_id: str) → None

Make a collection public Once a collection is public, it will be accessible to all authenticated users of the system.

Args:

collection_id:: ID of the collection to make public.

async match_chunks(collection_id: str, vectors: List[List[float]], topics: List[str], offset: int, limit: int, cut_off: float = 0, width: int = 0) → List[SearchResult]

Find chunks related to a message using semantic search. Chunks are sorted by relevance and similarity score to the message. See Also: H2OGPTE.encode_for_retrieval to create vectors from messages.

Args:

collection_id:: ID of the collection to search within.
vectors:: A list of vectorized message for running semantic search.
topics:: A list of document_ids used to filter which documents in the collection to search.
offset:: How many chunks to skip before returning chunks.
limit:: How many chunks to return.
cut_off:: Exclude matches with distances higher than this cut off.
width:: How many chunks before and after a match to return - not implemented.

Returns:

list of SearchResult: The document, text, score and related information of the chunk.

async openid_connect() → str

Tests openid connection.

Returns:: A session ID string.

async reset_collection_prompt_settings(collection_id: str) → str

Reset the prompt settings for a given collection.

Args:

collection_id:: ID of the collection to update.

Returns:

str: ID of the updated collection.

async search_chunks(collection_id: str, query: str, topics: List[str], offset: int, limit: int) → List[SearchResult]

Find chunks related to a message using lexical search. Chunks are sorted by relevance and similarity score to the message.

Args:

collection_id:: ID of the collection to search within.
query:: Question or imperative from the end user to search a collection for.
topics:: A list of document_ids used to filter which documents in the collection to search.
offset:: How many chunks to skip before returning chunks.
limit:: How many chunks to return.

Returns:

list of SearchResult: The document, text, score and related information of the chunk.

async set_chat_message_votes(chat_message_id: str, votes: int) → Result

Change the vote value of a chat message. Set the exact value of a vote for a chat message. Any message type can be updated, but only LLM response votes will be visible in the UI. The expectation is 0: unvoted, -1: dislike, 1 like. Values outside of this will not be viewable in the UI.

Args:

chat_message_id:: ID of a chat message, any message can be used but only LLM responses will be visible in the UI.
votes:: Integer value for the message. Only -1 and 1 will be visible in the UI as dislike and like respectively.

Returns:

Result: The status of the update.

Raises:

Exception: The upload request was unsuccessful.

async share_collection(collection_id: str, permission: Permission) → ShareResponseStatus

Share a collection to a user.

The permission attribute defined the level of access, and who can access the collection, the collection_id attribute denotes the collection to be shared.

Args:

collection_id:: ID of the collection to share.
permission:: Defines the rule for sharing, i.e. permission level.

Returns:

ShareResponseStatus: Status of share request.

Summarize one or more contexts using an LLM.

Format of summary content:

"{pre_prompt_summary}"""
{text_context_list}
"""\n{prompt_summary}"

Args:

text_context_list:: List of raw text strings to be summarized.
system_prompt:: Text sent to models which support system prompts. Gives the model overall context in how to respond. Use auto for the model default or None for h2oGPTe defaults. Defaults to ‘’ for no system prompt.
pre_prompt_summary:: Text that is prepended before the list of texts. The default can be customized per environment, but the standard default is "In order to write a concise single-paragraph or bulleted list summary, pay attention to the following text:\n"
prompt_summary:: Text that is appended after the list of texts. The default can be customized per environment, but the standard default is "Using only the text above, write a condensed and concise summary of key results (preferably as bullet points):\n"
llm:: Name or index of LLM to send the query. Use H2OGPTE.get_llms() to see all available options. Default value is to use the first model (0th index).
llm_args:: Dictionary of kwargs to pass to the llm.
kwargs:: Dictionary of kwargs to pass to h2oGPT.

Returns:

Answer: The response text and any errors.

Creates a summary of a document.

Args:

document_id:: String id of the document to create a summary from.
system_prompt:: System Prompt
pre_prompt_summary:: Prompt that goes before each large piece of text to summarize
prompt_summary:: Prompt that goes after each large piece of of text to summarize
llm:: LLM to use
llm_args:: Arguments for the LLM
max_num_chunks:: Max limit of chunks to send to the summarizer
sampling_strategy:: How to sample if the document has more chunks than max_num_chunks. Options are “auto”, “uniform”, “first”, “first+last”, default is “auto” (a hybrid of them all).
timeout:: Amount of time in seconds to allow the request to run. The default is 86400 seconds.

Returns:

DocumentSummary: Summary of the document

Raises:

TimeoutError: The request did not complete in time. SessionError: No summary created. Document wasn’t part of a collection, or LLM timed out, etc.

async unshare_collection(collection_id: str, permission: Permission) → ShareResponseStatus

Remove sharing of a collection to a user. The permission attribute defined the level of access, and who can access the collection, the collection_id attribute denotes the collection to be shared. In case of un-sharing, the Permission’s user is sufficient.

Args:

collection_id:: ID of the collection to un-share.
permission:: Defines the user for which collection access is revoked.

ShareResponseStatus: Status of share request.

async unshare_collection_for_all(collection_id: str) → ShareResponseStatus

Remove sharing of a collection to all other users but the original owner.

Args:

collection_id:: ID of the collection to un-share.

ShareResponseStatus: Status of share request.

async update_collection(collection_id: str, name: str, description: str) → str

Update the metadata for a given collection. All variables are required. You can use h2ogpte.get_collection(<id>).name or description to get the existing values if you only want to change one or the other.

Args:

collection_id:: ID of the collection to update.
name:: New name of the collection, this is required.
description:: New description of the collection, this is required.

Returns:

str: ID of the updated collection.

async update_collection_prompt_settings(collection_id: str, system_prompt: str, pre_prompt_query: str, prompt_query: str, rag_type: str, hyde_no_rag_llm_prompt_extension: str, auto_gen_description_prompt: str) → str

Update the prompt settings for a given collection. Updates the prompt settings for a given collection.

Args:

collection_id:: ID of the collection to update.
system_prompt:: Text sent to models which support system prompts. Gives the model overall context in how to respond.
pre_prompt_query:: Text that is prepended before the contextual document chunks.
prompt_query:: Text that is appended to the beginning of the user’s message.
rag_type:: RAG type to use.
hyde_no_rag_llm_prompt_extension:: LLM prompt extension.
auto_gen_description_prompt:: prompt to create a description of the collection.

Returns:

str: ID of the updated collection.

async upload(file_name: str, file: IO[bytes] | bytes) → str

Upload a file to the H2OGPTE backend. Uploaded files are not yet accessible and need to be ingested into a collection.

See Also:

ingest_uploads: Add the uploaded files to a collection. delete_upload: Delete uploaded file

Args:

file_name:: What to name the file on the server, must include file extension.
file:: File object to upload, often an opened file from with open(…) as f.

Returns:

str: The upload id to be used in ingest jobs.

Raises:

Exception: The upload request was unsuccessful.

Bases: object

Create and participate in a chat session.

This is a live connection to the h2oGPTe server contained to a specific chat session on top of a single collection of documents. Users will find all questions and responses in this session in a single chat history in the UI.

See Also:

H2OGPTE.connect: To initialize a session on an existing connection.

Args:

address:: Full URL of the h2oGPTe server to connect to.
chat_session_id:: The ID of the chat session the queries should be sent to.
api_key:: API key for authentication to the h2oGPTe server. Users can generate a key by accessing the UI and navigating to the Settings.
refresh_token:: User’s refresh token.

Examples:

# Example 1: Best practice, create a session using the H2OGPTE module
with h2ogpte.connect(chat_session_id) as session:
    answer1 = session.query('How many paper clips were shipped to Scranton?', timeout=10)
    answer2 = session.query('Did David Brent co-sign the contract with Initech?', timeout=10)

# Example 2: Connect and disconnect manually
session = Session(
    address=address,
    api_key=api_key,
    chat_session_id=chat_session_id
)
session.connect()
answer = session.query("Are there any dogs in the documents?")
session.disconnect()

connect()

Connect to an h2oGPTe server.

This is primarily an internal function used when users create a session using with from the H2OGPTE.connection() function.

property connection: ClientConnection

disconnect()

Disconnect from an h2oGPTe server.

This is primarily an internal function used when users create a session using with from the H2OGPTE.connection() function.

Retrieval-augmented generation for a query on a collection.

Finds a collection of chunks relevant to the query using similarity scores. Sends these and any additional instructions to an LLM.

Format of questions or imperatives:

"{pre_prompt_query}
"""
{similar_context_chunks}
"""                {prompt_query}{message}"

Args:

message:

Query or instruction from the end user to the LLM.

system_prompt:

Text sent to models which support system prompts. Gives the model overall context in how to respond. Use auto or None for the model default. Defaults to ‘’ for no system prompt.

pre_prompt_query:

Text that is prepended before the contextual document chunks. The default can be customized per environment, but the standard default is "Pay attention and remember the information below, which will help to answer the question or imperative after the context ends.\n"

prompt_query:

Text that is appended to the beginning of the user’s message. The default can be customized per environment, but the standard default is “According to only the information in the document sources provided within the context above, “

pre_prompt_summary:

Not yet used, use H2OGPTE.summarize_content

prompt_summary:

Not yet used, use H2OGPTE.summarize_content

llm:

Name or index of LLM to send the query. Use H2OGPTE.get_llms() to see all available options. Default value is to use the first model (0th index).

llm_args:

Dictionary of kwargs to pass to the llm.

self_reflection_config: Dictionary of arguments for self-reflection, can contain the following string:string mappings:

llm_reflection: str: "gpt-4-1106-preview" or "" to disable reflection
pre_prompt_reflection: str: "Pay attention to the following context. You will need to evaluate the quality of a response for a given prompt."
prompt_reflection: str: ‘Given the above context, here’s the prompt and the response: """Prompt:\n%s\n"""\n\n"""Response:\n%s\n"""\n\nWhat is the quality of the response for the given prompt? Respond with a score ranging from Score: 0/10 (worst) to Score: 10/10 (best), and give a brief explanation why.'
system_prompt_reflection: str: ""
llm_args_reflection: str: "{}"

rag_config:

Dictionary of arguments to control RAG (retrieval-augmented-generation) types. Can contain the following key/value pairs:

rag_type: str one of

"llm_only" LLM Only (no RAG) - Generates a response to answer the user’s query without any supporting document contexts.

"rag" RAG (Retrieval Augmented Generation) - RAG with neural/lexical hybrid search using the user’s query to find contexts from a collection for generating a response.

"hyde1" HyDE RAG (Hypothetical Document Embedding) - RAG with neural/lexical hybrid search using the user’s query and the LLM response to find contexts from a collection for generating a response. Requires 2 LLM calls.

"hyde2" HyDE RAG+ (Combined HyDE+RAG) - RAG with neural/lexical hybrid search using the user’s query and the HyDE RAG response to find contexts from a collection for generating a response. Requires 3 LLM calls.

no_rag_llm_prompt_extension: str: example: '\nKeep the answer brief, and list the 5 most relevant key words at the end.'

timeout:

Amount of time in seconds to allow the request to run. The default is 1000 seconds.

callback:

Function for processing partial messages, used for streaming responses to an end user.

Returns:

ChatMessage: The response text and details about the response from the LLM. For example:

ChatMessage(
    id='XXX',
    content='The information provided in the context...',
    reply_to='YYY',
    votes=0,
    created_at=datetime.datetime(2023, 10, 24, 20, 12, 34, 875026)
    type_list=[],
)

Raises:

TimeoutError: The request did not complete in time.

class h2ogpte.SessionAsync(chat_session_id: str, client: H2OGPTEAsync)

Bases: object

Create and participate in a chat session. This is a live connection to the h2oGPTe server contained to a specific chat session on top of a single collection of documents. Users will find all questions and responses in this session in a single chat history in the UI. See Also:

H2OGPTE.connect: To initialize a session on an existing connection.

Args:

address:: Full URL of the h2oGPTe server to connect to.
api_key:: API key for authentication to the h2oGPTe server. Users can generate a key by accessing the UI and navigating to the Settings.
chat_session_id:: The ID of the chat session the queries should be sent to.
verify:: Whether to verify the server’s TLS/SSL certificate. Can be a boolean or a path to a CA bundle. Defaults to True.

Examples::

async with h2ogpte.connect(chat_session_id) as session:

answer1 = await session.query(: ‘How many paper clips were shipped to Scranton?’

) answer2 = await session.query(

‘Did David Brent co-sign the contract with Initech?’

)

Retrieval-augmented generation for a query on a collection. Finds a collection of chunks relevant to the query using similarity scores. Sends these and any additional instructions to an LLM. Format of questions or imperatives:

"{pre_prompt_query}
"""
{similar_context_chunks}
"""            {prompt_query}{message}"

Args:

message:

Query or instruction from the end user to the LLM.

system_prompt:

Text sent to models which support system prompts. Gives the model overall context in how to respond. Use auto or None for the model default. Defaults to ‘’ for no system prompt.

pre_prompt_query:

Text that is prepended before the contextual document chunks. The default can be customized per environment, but the standard default is "Pay attention and remember the information below, which will help to answer the question or imperative after the context ends.\n"

prompt_query:

Text that is appended to the beginning of the user’s message. The default can be customized per environment, but the standard default is “According to only the information in the document sources provided within the context above, “

pre_prompt_summary:

Not yet used, use H2OGPTE.summarize_content

prompt_summary:

Not yet used, use H2OGPTE.summarize_content

llm:

Name or index of LLM to send the query. Use H2OGPTE.get_llms() to see all available options. Default value is to use the first model (0th index).

llm_args:

Dictionary of kwargs to pass to the llm.

self_reflection_config:

Dictionary of arguments for self-reflection, can contain the following string:string mappings:

llm_reflection: str
"gpt-4-0613" or "" to disable reflection

pre_prompt_reflection: str
"Pay attention to the following context. You will need to evaluate the quality of a response for a given prompt."

prompt_reflection: str
‘Given the above context, here’s the prompt and the response: """Prompt:\n%s\n"""\n\n""" Response:\n%s\n"""\n\nWhat is the quality of the response for the given prompt? Respond with a score ranging from Score: 0/10 (worst) to Score: 10/10 (best), and give a brief explanation why.'

system_prompt_reflection: str
""

llm_args_reflection: str
"{}"

rag_config:

Dictionary of arguments to control RAG (retrieval-augmented-generation) types. Can contain the following key/value pairs: rag_type: str one of

"llm_only" LLM Only (no RAG) - Generates a response to
answer the user’s query without any supporting document contexts.

"rag" RAG (Retrieval Augmented Generation) - RAG with
neural/lexical hybrid search using the user’s query to find contexts from a collection for generating a response.

"hyde1" HyDE RAG (Hypothetical Document Embedding) - RAG
with neural/lexical hybrid search using the user’s query and the LLM response to find contexts from a collection for generating a response. Requires 2 LLM calls.

"hyde2" HyDE RAG+ (Combined HyDE+RAG) - RAG with
neural/lexical hybrid search using the user’s query and the HyDE RAG response to find contexts from a collection for generating a response. Requires 3 LLM calls.

no_rag_llm_prompt_extension: str: example: '\nKeep the answer brief, and list the 5 most relevant key words at the end.'

timeout:

Amount of time in seconds to allow the request to run. The default is 1000 seconds.

callback:

Function for processing partial messages, used for streaming responses to an end user.

Returns:

ChatMessage: The response text and details about the response from the LLM. For example:

ChatMessage(
    id='XXX',
    content='The information provided in the context...',
    reply_to='YYY',
    votes=0,
    created_at=datetime.datetime(2023, 10, 24, 20, 12, 34, 875026)
    type_list=[],
)

Raises:

TimeoutError: The request did not complete in time.

property websocket: WebSocketClientProtocol