Detect objects with sample_object_detection - Qualcomm Dragonwing Documentation

Ultralytics YOLOv8 is a machine learning model that predicts bounding boxes, segmentation masks and classes of objects in an image.

`sample_object_detection` pipeline flow

The following figure shows the pipeline flow for sample_object_detection.

Object detection pipeline from camera capture through YOLO preprocessing, inference, postprocessing, and overlay to the detection result topic. — Pipeline flow for sample_object_detection.

ROS nodes used in the `sample_object_detection` pipeline

The following table lists the ROS nodes used in the sample_object_detection pipeline.

Node name	Description
qrb ros camera	Qualcomm ROS 2 package that captures images with parameters and publishes them to ROS topics.
yolo preprocess	Subscribes to image data, reshapes and resizes it, and republishes it to a downstream topic.
qrb ros nn interface	Loads a trained AI model, receives preprocessed images, performs inference, and publishes results.
yolo postprocess	Matches inference output with yolo label files.
yolo overlay	Subscribes to the yolo postprocess and image data, shows the object detect results with a ROS topic.

ROS topics used in the `sample_object_detection` pipeline

The following table lists the ROS topics used in the sample_object_detection pipeline.

ROS topic	Type	Published by
`/camera/color/image_raw`	`<sensor_msgs.msg.Image>`	`qrb_ros_camera`
`/qrb_inference_input_tensor`	`<qrb_ros_tensor_list_msgs.msg.TensorList>`	`yolo_preprocess_node`
`/yolo_detect_result`	`<vision_msgs.msg.Detection2DArray>`	`nn_inference_node`
`/yolo_detect_tensor_output`	`<qrb_ros_tensor_list_msgs.msg.TensorList>`	`yolo_detection_postprocess_node`
`/yolo_detect_overlay`	`<sensor_msgs.msg.Image>`	`yolo_detection_overlay_node`

Prerequisites

You have set up the device, installed ROS2 Jazzy and Qualcomm Intelligent Robotics (QIR) SDK on the device according to Install the QIR SDK.
On the host computer, you have downloaded and built the yolo model according to steps in the README of qrb_ros_tensor_process.
When downloading the yolo model, use target-runtime and device as follows:

Generate YOLOv8 model

Set up the QAI-Hub on the host computer.

a. Sign in to QAI-Hub and get your API token.b. Install QAI-Hub on the host computer.

 ## It is recommended to use python venv to avoid impacting the host environment.
 python3 -m venv venv_qaihub
 source venv_qaihub/bin/activate

 ## Install QAI-Hub related python packages.
 pip3 install qai-hub

 ## Configure QAI-Hub token, replacing xxx with your own token got from QAI-Hub page.
 qai-hub configure --api_token xxx

 pip3 install "qai-hub-models[yolov8-det]"

Export the model on the host computer.

After executing the following commands, the model file and coco8.yaml are generated in the host computer.

python -m qai_hub_models.models.yolov8_det.export --target-runtime "qnn_context_binary" --chipset "qualcomm-qcs9075" --skip-profiling --skip-inferencing

# Push model files to the device's default path according to the preceding export log.

# On the device, create the path if it doesn't exist.
mkdir /opt/model


# On the host computer, scp all model files to the device.
scp <Host_Export_model_Path>/yolov8_det.bin ubuntu@<device_ip>:/opt/model

sudo find / -name "coco8.yaml" -print -quit
scp <label-file-path>/coco8.yaml ubuntu@<device_ip>:/opt/

python -m qai_hub_models.models.yolov8_det.export --target-runtime "qnn_context_binary" --chipset "qualcomm-qcs8275" --skip-profiling --skip-inferencing

# Push model files to the device's default path according to the preceding export log.

# On the device, create the path if it doesn't exist.
mkdir /opt/model

# On the host computer, scp all model files to the device.
scp <Host_Export_model_Path>/yolov8_det.bin ubuntu@<device_ip>:/opt/model

sudo find / -name "coco8.yaml" -print -quit
scp <label-file-path>/coco8.yaml ubuntu@<device_ip>:/opt/        

Run out-of-the-box `sample_object_detection`

On the device, run the sample application:

Run the sample application

source /opt/ros/jazzy/setup.bash
ros2 launch sample_object_detection launch_with_qrb_ros_camera.py model:=/opt/model/yolov8_det.bin

Then, you can check the ROS topics with the name /yolo_detect_overlay in rviz2.

Build from the source of `sample_object_detection`

Ensure that you run the Prerequisites first before performing the following steps.

On the device, run the sample application:

Download the source code from the qrb_ros_samples repository

Download the source code

mkdir -p ~/qrb_ros_ws/src && cd ~/qrb_ros_ws/src
git clone https://github.com/qualcomm-qrb-ros/qrb_ros_samples.git

Build the sample_object_detection sample application from the source code

Build the sample application

cd ~/qrb_ros_ws/src/qrb_ros_samples/ai_vision/sample_object_detection
colcon build
source install/setup.bash

Run and test the sample application

Run and test according to steps 2–3 of Run out-of-the-box sample_object_detection.

​sample_object_detection pipeline flow

​ROS nodes used in the sample_object_detection pipeline

​ROS topics used in the sample_object_detection pipeline

​Prerequisites

​Generate YOLOv8 model

​Run out-of-the-box sample_object_detection

​Build from the source of sample_object_detection

`sample_object_detection` pipeline flow

ROS nodes used in the `sample_object_detection` pipeline

ROS topics used in the `sample_object_detection` pipeline

Prerequisites

Generate YOLOv8 model

Run out-of-the-box `sample_object_detection`

Build from the source of `sample_object_detection`