Driverless AI Deployment Template for AWS Sagemaker Hosted Scorer¶
This template scores realtime data using Driverless AI Mojo pipeline with Java Runtime and plugs into the AWS SageMaker workflow documented here
It is a REST API which accepts one data point at a time for prediction in real-time in the hosted SageMaker environment. The user needs to provide Driverless AI license key and model's pipeline.mojo file for scoring, see deploy section for details. The versions of software used to create the template like mojo runtime are listed here.
Overview¶
Build Image¶
Generation of the Docker image is plugged into the build process of this project.
Run the following command in the root project directory to run the build
process.
./gradlew :aws-sagemaker-hosted-scorer:jibDockerBuild
Verify that the Docker image was created, and take note of the version created.
docker images --format "{{.Repository}} \t {{.Tag}}" | grep "h2oai/sagemaker-hosted-scorer"
Optional: Test the build¶
After building, run to test the produced Docker container locally like this:
Step 1: Put a pipeline.mojo and valid license.sig into this directory (aws-sagemaker-hosted-scorer).
Step 2: Start the docker instance.
docker run \
--rm \
--init \
-ti \
-v `pwd`:/opt/ml/model \
-p 8080:8080 \
harbor.h2o.ai/opsh2oai/h2oai/sagemaker-hosted-scorer \
serve
Step 3: Use curl to send a JSON-formatted row to the scorer as shown in the details below.
Deploy to SageMaker¶
Create h2oai/sagemaker-hosted-scorer
repository in Sagemaker for the scorer service image.
Use the output of the command below to docker login
:
aws ecr get-login --region <region> --no-include-email
Tag the scorer service image for loading into the h2oai/sagemaker-hosted-scorer
repository.
docker tag <IMAGE ID> <aws_account_id>.dkr.ecr.<region>.amazonaws.com/h2oai/sagemaker-hosted-scorer
Then push the scorer service image to AWS ECR (Elastic Container Registry):
docker push <aws_account_id>.dkr.ecr.<region>.amazonaws.com/h2oai/sagemaker-hosted-scorer
Then create a model package with the pipeline file and the license key, and copy it to S3:
tar cvf mojo.tar pipeline.mojo license.sig
gzip mojo.tar
aws s3 cp mojo.tar.gz s3://<your-bucket>/
Next create the appropriate model and endpoint on Sagemaker.
Check that the endpoint is available with aws sagemaker list-endpoints
.
Examples¶
There is the examples directory, which houses a sample python notebook, which utilizes the AWS Sagemaker SDK to deploy a Sagemaker model pythonically. See the notebook here
For an example of an endpoint query being made via Python go here
Details¶
AWS Model Creation API¶
https://docs.aws.amazon.com/sagemaker/latest/dg/API_CreateModel.html
-
CreateEndpoint
- Environment
- DRIVERLESS_AI_LICENSE_KEY=base64key
- ModelDataURL=s3://blah/blah/model.tar.gz
- Environment
-
DRIVERLESS_AI_LICENSE_KEY
environment variable must contain the base64-encoded key (optional if a license.sig isns't included in mojo.tar.gz) ModelDataURL
must point to an S3 URL with a .tar.gz file of the MOJO artifact
Docker container¶
The docker container produced in this directory conforms to the specification described here:
- https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-inference-code.html#your-algorithms-inference-code-run-image
Sagemaker starts the container with the following command:
docker run image serve
Our container consists of the following entrypoint:
ENTRYPOINT ["java", "-jar", "serve.jar"]
Example Predictions¶
You can test the container locally with the following curl command:
curl \
-X POST \
-H "Content-Type: application/json" \
-d @test.json http://localhost:8080/invocations
test.json:
{
"fields": [
"field1", "field2"
],
"includeFieldsInOutput": [
"field2"
],
"rows": [
[
"value1", "value2"
],
[
"value1", "value2"
]
]
}