Models

A model is generally used to identify and solve a high-level business problem via identification or prediction.

Examples include “identify fraudulent transactions”, “predict when equipment will next need maintenance”, and “forecast data center costs next quarter”.

Model design is often an iterative process. As such, many variants (also called a version) of a given model might be evaluated before arriving at the best solution to the problem given business constraints.

Each variant is a fully specified artifact that contains hyperparameters and the trained state of the model, along with a complete operational definition of how to execute the model. Every variant of the model must be given a unique and descriptive name.

Once defined, each variant can then be deployed independently to generate fully reproducible inferences. This can be used, for example, to run comparisons necessary to evaluate different variants. These deployments can have their resources independently tuned as necessary.

Example

Let’s take a quick look at how we can define and deploy a model.

import wallaroo

First, let’s initialize a reference to the Wallaroo system.

wl = wallaroo.Client()

Next, we can create a deployment for our model using default resource allocations. The upload_model function uploads the file and deploy() creates a single-model pipeline to make predictions, in a running state and ready to perform inference.

pipeline = wl.upload_model('ccfraud' , "./keras_ccfraud.onnx").deploy('ccfraud-deployment')

Once deployed, we can run inference:

filename = './dev_smoke_test.json'
pipeline.infer_from_file(filename)

[InferenceResult({'check_failures': None,
  'elapsed': 163619,
  'model_id': 'ccfraud',
  'model_version': '2021-05-01',
  'original_data': {'tensor': [[1.0678324729342086,
                                0.21778102664937624,
                                -1.7115145261843976,
                                0.6822857209662413,
                                1.0138553066742804,
                                -0.43350000129006655,
                                0.7395859436561657,
                                -0.28828395953577357,
                                -0.44726268795990787,
                                0.5146124987725894,
                                0.3791316964287545,
                                0.5190619748123175,
                                -0.4904593221655364,
                                1.1656456468728569,
                                -0.9776307444180006,
                                -0.6322198962519854,
                                -0.6891477694494687,
                                0.17833178574255615,
                                0.1397992467197424,
                                -0.35542206494183326,
                                0.4394217876939808,
                                1.4588397511627804,
                                -0.3886829614721505,
                                0.4353492889350186,
                                1.7420053483337177,
                                -0.4434654615252943,
                                -0.15157478906219238,
                                -0.26684517248765616,
                                -1.454961775612449]]},
  'outputs': [{'Float': {'data': [0.001497417688369751],
                         'dim': [1, 1],
                         'v': 1}}],
  'pipeline_name': 'ccfraud-deployment',
  'time': 1630599510450})]

And then shut down the deployment:

pipeline.undeploy()

{'name': 'ccfraud-deployment', 'create_time': datetime.datetime(2021, 5, 21, 4, 11, 11, 816565, tzinfo=tzutc()), 'definition': "[{'ModelInference': {'models': [{'name': 'ccfraud', 'version': '76cacac7-1dc7-41fc-8b7a-1e5dc17a093c', 'sha': '4dc88d159249ccce83942ada69b919cb91455d5fd0e4bfc287de3f21d1aafb1b'}]}}]"}

API Reference

See the Model section.