Use AI Hub to optimize a model - Qualcomm Dragonwing Documentation

For quick prototyping of models on Qualcomm AI hardware, AI Hub provides a way to optimize, validate, and deploy machine learning models on-device for vision, audio, and speech use cases.

Set up your environment

Set up your Python environment

Install miniconda on your host machine.Set up a Python virtual environment for AI Hub:

conda activate

conda create python=3.10 -n qai_hub

conda activate qai_hub

Install git

sudo apt-get install git

Install the AI Hub Python client

pip3 install qai-hub

pip3 install "qai-hub[torch]"

Go to AI Hub and sign in with your Qualcomm ID to view information about jobs you create.Once signed in, go to Account > Settings > API Token to obtain the API token used to configure your client.

Configure the client with your API token

qai-hub configure --api_token <INSERT_API_TOKEN>

Choose an AI Hub workflow

Try a preoptimized model

Browse the AI Hub Model Zoo

Go to AI Hub Model Zoo to access preoptimized models available for Qualcomm evaluation kits.

Filter models for your EVK

Select the matching chipset in the left pane. For example, select Qualcomm QCS6490 for the Qualcomm Dragonwing™ RB3 Gen 2.

Select a model

Select a model from the filtered view to go to the model page.

Choose runtime and precision

On the model page, select the runtime and precision.

Download the model

Select Download to download the model. The downloaded model is preoptimized and ready for deployment. See Run inference for more information.

Bring your own model

Select a pretrained model

Select a pretrained model in PyTorch or ONNX format.

Submit the model for compilation or optimization

Submit a model for compilation or optimization to AI Hub using Python APIs.When submitting a compilation job, select a device or chipset for your EVK and the target runtime. For Qualcomm Dragonwing™ RB3 Gen 2, the LiteRT runtime is supported.

Chipset	Runtime	CPU	GPU	HTP
Qualcomm Dragonwing™ RB3 Gen 2	LiteRT	INT8,FP16, FP32	FP16,FP32	INT8,INT16

On submission, AI Hub generates a unique ID for the job. You can use this job ID to view job details.

AI Hub optimizes the model

AI Hub optimizes the model based on your device and runtime selections.

Optionally, you can submit a job to profile or run inference on the optimized model (using Python APIs) on a real device provisioned from a device farm.
- Profiling: Benchmarks the model on a provisioned device and provides statistics, including average inference times at the layer level, runtime configuration, etc.
- Inference: Performs inference using an optimized model on data submitted as part of the inference job by running the model on a provisioned device.

Review the job and download the optimized model

Each submitted job is available for review in the AI Hub portal. A completed compilation job provides a downloadable link to the optimized model, which can then be deployed on a local development device such as Qualcomm Dragonwing™ RB3 Gen 2.

The following example, taken from the AI Hub documentation, uploads a pretrained MobileNet V2 model from PyTorch to AI Hub and compiles it to an optimized LiteRT model for Qualcomm Dragonwing™ RB3 Gen 2.

import torch
import torchvision

import qai_hub as hub

client = hub.Client()

# Using pre-trained MobileNet
torch_model = torchvision.models.mobilenet_v2(pretrained=True)
torch_model.eval()

# Trace model
input_shape: tuple[int, ...] = (1, 3, 224, 224)
example_input = torch.rand(input_shape)
with torch.no_grad():
    pt2_model = torch.export.export(torch_model, (example_input,))

# Compile model on a specific device
compile_job = client.submit_compile_job(
    pt2_model,
    name="MobileNet_V2",
    device=hub.Device("Dragonwing RB3 Gen 2 Vision Kit"),
    input_specs=dict(image=input_shape),
)

# Download the optimized compiled model
compile_job.download_target_model("MobileNet_V2.tflite")

To deactivate a previously activated qai_hub environment, use the following command.

conda deactivate

Once the model is downloaded, it is ready for deployment. See Run inference for next steps. For more details about the AI Hub workflow and APIs, see the AI Hub documentation, explore the AI Hub tutorial videos, or watch the following video about how to profile models in AI Hub.

​Set up your environment

​Choose an AI Hub workflow

​Try a preoptimized model

​Bring your own model

Set up your environment

Choose an AI Hub workflow

Try a preoptimized model

Bring your own model