Execute AI model inference with qrb_ros_nn_inference - Qualcomm Dragonwing Documentation

qrb_ros_nn_inference is a ROS2 package that runs neural network models on the device, so your robotics application gets AI-based perception without leaving the ROS graph. It provides the following functionalities:

Model inference API which supports three model formats:
- .tflite
- .so
- .bin
Model inference acceleration based on Qualcomm platforms.

Pipeline flow for `qrb_ros_nn_inference`

The following figure shows how a preprocess node feeds an input tensor to qrb_ros_nn_inference, which publishes the output tensor to a postprocess node.

Preprocess node publishing an input tensor to the nn inference node, which publishes an output tensor to a postprocess node.

Figure : qrb_ros_nn_inference pipeline

ROS topics used in the `qrb_ros_nn_inference` pipeline

The following table lists the ROS topics in the pipeline, their message types, and the node that publishes or subscribes to each one.

ROS topic	Message type	Published/Subscribed by	Description
`/qrb_inference_input_tensor`	`qrb_ros_tensor_list_msgs::msg::TensorList`	Subscribed by `qrb_ros_nn_inference`	Subscribes to the input tensor for model inference.
`/qrb_inference_output_tensor`	`qrb_ros_tensor_list_msgs::msg::TensorList`	Published by `qrb_ros_nn_inference`	Publishes the output tensor of model.

ROS node parameters used in the `qrb_ros_nn_inference` pipeline

The following table lists the node parameters, their types, and their default values.

Node parameter	Type	Default value	Description
`backend_option`	`string`	`""`	Hardware acceleration option for model inference. For valid values, see qrb_ros_nn_inference documentation.
`model_path`	`string`	`""`	Indicates the path of the model file.

Prerequisites

You have Set up the device according to Set up the environment for running sample applications.
On a qcom-robotics-image device (Overlay Config #1), ensure that you have installed the Qualcomm AI Engine Direct SDK (qairt-sdk) and the qrb-ros-nn-inference package. For instructions, see Enable and evaluate Qualcomm Linux overlays on QIR SDK. On a qcom-robotics-proprietary-image device (Overlay Config #2), the package is already installed.
You have downloaded the model from Qualcomm AI hub.
You have prepared the preprocess node and postprocess node according to the qrb_ros_nn_inference example.

Run out-of-the-box `qrb_ros_nn_inference`

Try me

Set up QIR SDK and ROS2 environment on the device

Start one terminal, and run the following commands.Value range of ROS_DOMAIN_ID: [0, 232]

ssh root@[ip-addr]

SSH Session

source /usr/share/qirp-setup.sh
export ROS_DOMAIN_ID=xx

Launch the nodes

Launch three nodes: preprocess node, qrb_ros_nn_inference node and postprocess node.

SSH Session

ros2 launch model_inference.launch.py

Build and run `qrb_ros_nn_inference`

Try me

Build qrb_ros_nn_inference on the host

cd <qirp_decompressed_workspace>
source setup.sh
cd qirp-samples/platform/qrb_ros_nn_inference
colcon build --merge-install --cmake-args ${CMAKE_ARGS}

Push qrb_ros_nn_inference to the device

NoteEnsure that the QIR SDK is installed on the device.

cd qirp-samples/platform/qrb_ros_nn_inference/install
tar czvf qrb_ros_nn_inference.tar.gz include lib share
scp qrb_ros_nn_inference.tar.gz root@[ip-addr]:/opt/
ssh root@[ip-addr]

SSH Session

tar --no-overwrite-dir --no-same-owner -zxf /opt/qrb_ros_nn_inference.tar.gz -C /usr/ros/jazzy/

Set up QIR SDK and ROS2 environment on the device

Start one terminal, and run the following commands.Value range of ROS_DOMAIN_ID: [0, 232]

ssh root@[ip-addr]

SSH Session

source /usr/share/qirp-setup.sh
export ROS_DOMAIN_ID=xx

Launch the nodes

Launch three nodes: preprocess node, qrb_ros_nn_inference node and postprocess node.

SSH Session

ros2 launch model_inference.launch.py

Limitations

For the latest limitations, see qrb_ros_nn_inference on GitHub.

​Pipeline flow for qrb_ros_nn_inference

​ROS topics used in the qrb_ros_nn_inference pipeline

​ROS node parameters used in the qrb_ros_nn_inference pipeline

​Prerequisites

​Run out-of-the-box qrb_ros_nn_inference

​Build and run qrb_ros_nn_inference

​Limitations

Pipeline flow for `qrb_ros_nn_inference`

ROS topics used in the `qrb_ros_nn_inference` pipeline

ROS node parameters used in the `qrb_ros_nn_inference` pipeline

Prerequisites

Run out-of-the-box `qrb_ros_nn_inference`

Build and run `qrb_ros_nn_inference`

Limitations