Skip to main content

Overview

H2O AI Hybrid Cloud (H2OAIC) App Store is a scalable, light-weight system for managing and scheduling of Wave apps in Kubernetes.

Architecture

The H2OAIC App Store Server is the heart of the H2O AI Hybrid Cloud. It is responsible for managing applications and their instances. It consists of the following major components:

  • App Store Frontend - Provides a web interface for interacting with the App Store
  • CLI - Provides a command line interface for interacting with the App Store
  • App Repository - Manages app bundles and metadata, relying on Blob storage and PostgreSQL
  • Scheduler - Interfaces with Kubernetes to start and manage running applications and secrets
  • API - Provides access to App Repository and Scheduler for web UI and CLI
  • Router - Handles authorization and routing of incoming traffic to app instances

The H2OAIC Launcher manages the runtime environment and lifecycle of a single app instance.

App Store Server​

The App Store runs as a single deployment within Kubernetes and provides the App Store frontend, App Repository, Scheduler, API Server, and Router services. It can run with multiple replicas in a highly-available manner to ensure zero downtime updates and fault tolerance as well as performance at scale.

App Store Frontend​

The App Store frontend is primary user interface for users of the H2O AI Hybrid Cloud, providing users an easy-to-use interface for managing apps and app instances. With built in support for visitor access, administrators can tailor which apps specific users can see/launch.

CLI​

The H2OAIC provides a CLI that users can utilize to manage their workloads and serves as the primary tool that app developers will use to publish apps into the App Store. Users can list apps, launch and manage instances, as well as get access to instance diagnostics like logs and instance status. See the CLI documentation

App Repository​

Management of app bundles (app source packages uploaded by app developers) and metadata is handled by the App Repository. It is a straightforward web application which uses PostgreSQL to store and query app metadata extracted from app.toml (see Developer guide), including app tags. It utilizes a Blob/Object store to persist the bundles and other files, such as icons and screenshots.

It provides two notable HTTP handlers: one for importing a new bundle and one for downloading a bundle for execution by the Launcher.

Router​

Requests to app instances pass through the App Store server, enforcing their authorization settings; see Instance access controls

The App Store uses the hostname of the incoming request to properly route requests (e.g., <instance>.wave.h2o.ai), so it's important to provide a setup where a wildcard DNS record and TLS certificate may be used; see Deployment for details.

When the App Store receives a request for an app instance, it first consults Scheduler to locate the instance within Kubernetes, determines and enforces access restrictions, before proxying the request to the destination Kubernetes service. All requests, even websocket requests, are authenticated before they are passed to the running app instance. The visibility level of an instance may be updated at any time using the CLI.

Scheduler​

The App Store utilizes Helm to launch and manage running Wave app instances without requiring an external database to maintain app instance state, i.e., the Kubernetes API is the only place storing the app instance state. Administrators can manage a list of eligible docker images that Wave apps can be launched in using the RuntimeVersions configuration. The scheduler can optionally mount Kubernetes secrets, attach Persistent Volumes or ensure GPU allocation for apps that require it. It can also read instance status and logs.

The scheduler can be configured to limit the number of instances per user, the number of published app versions per user, and more.

The configuration options may be either set within the [Scheduler] section of the server configuration ConfigMap TOML, or set via environment variables.

The environment variable names in the table below need to be prefixed with H2O_CLOUD_SCHEDULER_:

TOML Key
Environment Variable		Description
KubeConfig
KUBE_CONFIG		Optional; specifies a path to a Kubernetes configuration file for cluster access; empty means in-cluster access(empty)
Namespace
NAMESPACE		Kubernetes namespace to schedule apps withindefault
HelmAssetDir
HELM_ASSET_DIR		Directory to extract Helm assets to for scheduling appsvar/lib/h2oq8s/helm
ServiceType
SERVICE_TYPE		Kubernetes service type to create when scheduling appsClusterIP
StorageClass
STORAGE_CLASS		Kubernetes PersistentVolume StorageClass to attach to apps requiring persistent storage(empty)
WriteTimeoutSeconds
WRITE_TIMEOUT_SECONDS		Default timeout for running, terminating and updating instances300
ResourcePath
RESOURCE_PATH		Path within app containers to mount a PersistentVolume at if ResourceVolumeSize is specified in the app.toml/resources
InstanceLimit
INSTANCE_LIMIT		Maximum number of instances a full access user may have running10
VisitorInstanceLimit
VISITOR_INSTANCE_LIMIT		Maximum number of instances users without full access may have running5
AppVersionLimit
APP_VERSION_LIMIT		Maximum number of apps a full-access user may import10
AppServiceAccount
APP_SERVICE_ACCOUNT		Kubernetes ServiceAccount to be used by apps(empty)
AllowedGPUTypes
ALLOWED_GPU_TYPES		Names of allowed GPU types; empty means GPU support is disabled; should match existing values of the hac.h2o.ai/accelerator K8s node label(empty)

The default app resources may be either set within the [Scheduler.DefaultRuntimeLimit] section of the server configuration ConfigMap TOML, or set via environment variables.

The environment variable names in the table below need to be prefixed with H2O_CLOUD_SCHEDULER_DEFAULT_RUNTIME_LIMIT_:

TOML Key /
Environment Variable		DescriptionDefault
MemoryLimit
MEMORY_LIMIT		Default memory reservation for apps if unspecified by the app; needs to conform to the K8s resource model2Gi
MemoryReservation
MEMORY_RESERVATION		Default memory limit for apps if unspecified by the app; needs to conform to the K8s resource model512Mi
CPULimit
CPU_LIMIT		Default CPU reservation for apps if unspecified by the app;needs to conform to the K8s resource model(empty)
CPUReservation
CPU_RESERVATION		Default CPU limit for apps if unspecified by the app; needs to conform to the K8s resource model(empty)
GPUType
GPU_TYPE		GPU type that should be assigned by default; empty means any, delegating to the K8s scheduler; should match existing values of the hac.h2o.ai/accelerator K8s node label(empty)

Launcher​

The H2OAIC Launcher is the runtime environment responsible for installing dependencies, starting the waved server, and managing the lifecycle of the app.

When an app instance is launched, the launcher will download the .wave bundle from the App Store server. This will show the user of the instance a Sourcing The Wave Application message in the browser.

Next it will install any python, and system packages that the app requires. This will show the user of the instance a Installing Application Dependencies message in the browser.

After the necessary packages have been installed in the python virtual environment, the waved server will start, and the lifecycle of the wave python code will begin. Both the stdout of the waved server and wave application with be piped to stdout. If either of the waved or the application code fail to start the HAC launcher will exit.

In the server configuration you can specify multiple RuntimeVersions, effectively docker images, that can be used to launch apps.

[[RuntimeVersion]]
Name = 'deb10_py37_wlatest'
Image = "container-registry/wave-launcher:latest"
Default = true
Deprecated = false

Attributes:

  • RuntimeVersion (struct) - A base image to be used to launch apps repeated;
    • Name (string) - name .
    • Image (string) - container image name.
    • Default (bool) - pick this image as the default runtime to use
    • Deprecated (bool) - prevent new apps from using this image

See Developer guide for more information on using runtime versions.

Storage​

The App Store uses multiple different data stores for various purposes:

  1. PostgreSQL - Stores metadata about apps and tags
  2. Kubernetes/Helm - Stores state and configuration for running instances
  3. Blob Storage - Published Wave app bundles and static app resources like icons & screenshots

Authentication & Authorization​

H2OAI leverages OpenID Connect (OIDC) to authenticate and authorize users within the H2O AI Hybrid Cloud. This means it's easy to federate logins with a host of different IdP providers using Keycloak. See Authorization for more.

Feedback