Quick Start (Predictor Method)

:::warning The traditional predictor.py deployment method is no longer the best practice. It is recommended to use the custom deployment method. Please refer to the Quick Start Documentation for the recommended deployment approach. :::

This guide uses PyTorch ResNet50 as an example to introduce the model deployment process. All code can be obtained from openbayes-serving-examples/pytorch/image-classifier-resnet50/.

Prepare the Model

import torch
from torchvision import models

# Load pretrained model
model = models.resnet50(pretrained=True)

# Save to local
torch.save(model.state_dict(), "resnet50.pt")

Write Model Deployment Script `predict.py`

The predictor.py file needs to contain a class named Predictor with the following structure:

import openbayes_serving as serv


class Predictor:
    def __init__(self):
        """
        Responsible for loading the corresponding model and initializing metadata
        """
        ...

    def predict(self, json):
        """
        Receives HTTP request content, performs necessary processing before prediction, and finally returns the result to the caller
        """
        ...


if __name__ == '__main__':
    serv.run(Predictor)

Specific parameters and descriptions can be found in Serving Predictor.

For PyTorch ResNet50, the content of predictor.py is as follows:

import torch
import cv2
import numpy as np
import json
import requests
from torchvision import models, transforms

import openbayes_serving as serv


def get_url_image(url_image):
    resp = requests.get(url_image, stream=True).raw
    image = np.asarray(bytearray(resp.read()), dtype="uint8")
    image = cv2.imdecode(image, cv2.IMREAD_COLOR)
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    return image

```python
class Predictor:
    def __init__(self):
        # Load classification metadata
        classes = json.load(open('classes.json'))
        self.idx2label = [classes[str(k)][1] for k in range(len(classes))]

        # Specify model file name
        model_name = 'resnet50.pt'

        # Load model
        self.device = "cuda" if torch.cuda.is_available() else "cpu"
        self.model = models.resnet50()
        self.model.load_state_dict(torch.load(model_name))
        self.model.eval()
        self.model = self.model.to(self.device)

        # Image preprocessing, including normalization and resize
        normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
        self.transform = transforms.Compose(
            [
                transforms.ToPILImage(),
                transforms.Resize([224, 224]),
                transforms.ToTensor(),
                normalize,
            ]
        )

    def predict(self, json):
        # Get image URL from json.url, requiring the HTTP request content to be {"url": "xxx"}
        imageurl = json["url"]
        # Get image content
        image = get_url_image(imageurl)
        # Image preprocessing
        image = self.transform(image)
        image = torch.tensor(image.numpy()[np.newaxis, ...])

        # Inference
        results = self.model(image.to(self.device))

        # Get top 5 results
        top5_idx = results[0].sort()[1][-5:]

        # Get names of top 5 classifications
        top5_labels = [self.idx2label[idx] for idx in top5_idx]
        top5_labels = top5_labels[::-1]

        return top5_labels


if __name__ == '__main__':
    serv.run(Predictor)

Upload to Data Repository

Place the prepared .pt file, predictor.py file, and other required files in the same directory. The files that must be included in the directory are shown below:

.
├── classes.json
├── predictor.py
└── resnet50.pt

Where:

resnet50.pt is the model file.
predictor.py currently HyperAI model deployment requires its filename to be fixed as predictor.py. Using other filenames will cause deployment failure.
classes.json contains the metadata for ResNet classification, used when displaying classification names.

Create a model repository:

Upload the above three files:

View the upload results:

Create Serving

In the left navigation bar "Computing Containers" -> "Model Deployment", create a new Serving:

Bind the model directory and select the model version directory just uploaded:

Click deploy and wait for the deployment status to show "Running":

Click on the current latest deployment version (which is also the only deployment version at present) to view the logs:

Testing

After the model shows "Running" status, the "Overview" page will display the HTTP access address for the current deployment.

You can test the currently deployed service using the curl tool:

curl -X POST \
    <service address> \
    -H "Content-Type: application/json" \
    --data-raw '{ "url": "http://openbayes-public.cn-bj.ufileos.com/cat.jpg" }'

"Model Deployment" uses JSON format requests by default, where the url field corresponds to the url retrieved from json in our predictor.py mentioned above.

You can obtain the corresponding result:

["tabby", "Egyptian_cat", "tiger_cat", "lynx", "tiger"]