Model porting best practices

​

Export a custom YOLOv8 model using the QAIRT SDK

​

Prerequisites

Install the Qualcomm AI Runtime SDK on a host computer with Python >= 3.10 and PyTorch >=1.8. For more details, follow Install Qualcomm AI Runtime SDK. Run the following commands on the host computer.

1. Activate your virtual environment.
   source <venv_path>/bin/activate
   
2. Install the Ultralytics package and export the ONNX model.
   pip install ultralytics
   
   yolo export model=yolov8s.pt imgsz=320 format=onnx opset=11 optimize=True simplify=true
   

​

Procedure

1. Convert the ONNX model to DLC.
   snpe-onnx-to-dlc -i yolov8s.onnx
   
2. Generate quantized DLC.
   1. Prepare the calibration data set.
   2. Gather 5-10 images that used during training and save these images in the input directory.
   3. Use the preprocess.py script to convert .jpg images into the RAW files required for quantization.
      In this example, the model uses an input dimension of 320x320.
      1. Download the script as follows:
         wget https://raw.githubusercontent.com/qualcomm/sample-apps-for-qualcomm-linux/refs/heads/main/qualcomm-linux/scripts/preprocess.py
         
      2. Run the script with the following options:
         python preprocess.py <INPUT PATH> <OUTPUT PATH> 1 0
         
         • <INPUT PATH>: Folder containing the original images
         • <OUTPUT PATH>: Folder where the RAW files will be generated
   4. Create an input.txt file containing the paths to all generated RAW files. The quantization process needs this file.
3. Quantize the model, using snpe-dlc-quantize to convert the model to quantized DLC.
   snpe-dlc-quantize --input_dlc yolov8s.dlc --input_list input.txt
   

​

Run the demo

1. Download the labels file. See Download model and label files.
2. On the host computer, set the user environment variable:
   export USER=root
   
3. Push the test video file /etc/media on the device.
   scp <video file> $USER@<IP address of target device>:/etc/media/
   
4. Push the quantized YoloV8 model to the device.
   scp <model file> $USER@<IP address of target device>:/etc/models/
   
5. Retrieve the output tensor of the model, for example, output0.
6. Sign in to the device using SSH:
   ssh $USER@<IP_ADDRESS_OF_TARGET_DEVICE>
   
7. In the new shell, run the following commands:
   gst-launch-1.0 -e filesrc location=/etc/media/video.mp4 ! qtdemux ! queue ! h264parse ! v4l2h264dec capture-io-mode=4 output-io-mode=4 ! video/x-raw,format=NV12 ! queue ! tee name=split split. ! queue ! qtivcomposer name=mixer ! queue ! waylandsink fullscreen=true sync=true split. ! queue ! qtimlvconverter ! queue ! qtimlsnpe delegate=dsp model=/etc/models/yolov8s_quantized.dlc tensors="<output0>" ! queue ! qtimlpostprocess settings="{\"confidence\": 51.0}" results=10 module=yolov8 labels=/etc/labels/yolov8.json ! video/x-raw,width=640,height=360 ! queue ! mixer.