Execute AI model inference with qrb_ros_nn_inference

qrb_ros_nn_inference is a ROS2 package for performing neural network model, providing AI-based perception for robotics applications. It provides the following functionalities:

• Model inference API which supports three model formats:
  • .tflite
  • .so
  • .bin
• Model inference acceleration based on Qualcomm platforms

​

Pipeline flow for qrb_ros_nn_inference

Figure : The pipeline of qrb_ros_nn_inference

​

ROS topics used in the qrb_ros_nn_inference pipeline

ROS topic	Message type	Published/Subscribed by	Description
/qrb_inference_input_tensor	qrb_ros_tensor_list_msgs::msg::TensorList	Subscribed by qrb_ros_nn_inference	Subscribes to the input tensor for model inference.
/qrb_inference_output_tensor	qrb_ros_tensor_list_msgs::msg::TensorList	Published by qrb_ros_nn_inference	Publishes the output tensor of model.

​

ROS node parameters used in the qrb_ros_nn_inference pipeline

Node parameter	Type	Default value	Description
backend_option	string	""	Hardware acceleration option for model inference. For valid values, see qrb_ros_nn_inference documentation.
model_path	string	""	Indicates the path of the model file.

​

Prerequisites

• You have Set up the device according to Set up the environment for running sample applications.
• You have downloaded the model from Qualcomm AI hub.
• You have prepared the preprocess node and postprocess node according to the qrb_ros_nn_inference example.

​

Run out-of-the-box qrb_ros_nn_inference

1. Start one terminal, and run the following commands to set up QIR SDK and ROS2 environment on the device. Value range of ROS_DOMAIN_ID: [0, 232]
   ssh root@[ip-addr]
   (ssh) source /usr/share/qirp-setup.sh
   (ssh) export ROS_DOMAIN_ID=xx
   
2. Launch three nodes: preprocess node, qrb_ros_nn_inference node and postprocess node.
   (ssh) ros2 launch model_inference.launch.py
   

​

Build and run qrb_ros_nn_inference

1. Build qrb_ros_nn_inference provided by the QIR SDK on the host.
   cd <qirp_decompressed_workspace>
   source setup.sh
   cd qirp-samples/platform/qrb_ros_nn_inference
   colcon build --merge-install --cmake-args ${CMAKE_ARGS}
   
2. Push qrb_ros_nn_inference to the device.
   NoteEnsure that the QIR SDK is installed on the device.
   cd qirp-samples/platform/qrb_ros_nn_inference/install
   tar czvf qrb_ros_nn_inference.tar.gz include lib share
   scp qrb_ros_nn_inference.tar.gz root@[ip-addr]:/opt/
   ssh root@[ip-addr]
   (ssh) tar --no-overwrite-dir --no-same-owner -zxf /opt/qrb_ros_nn_inference.tar.gz -C /usr/ros/jazzy/
   
3. Start one terminal, and run the following commands to set up QIR SDK and ROS2 environment on the device. Value range of ROS_DOMAIN_ID: [0, 232]
   ssh root@[ip-addr]
   (ssh) source /usr/share/qirp-setup.sh
   (ssh) export ROS_DOMAIN_ID=xx
   
4. Launch three nodes: preprocess node, qrb_ros_nn_inference node and postprocess node.
   (ssh) ros2 launch model_inference.launch.py
   

​

Limitation

For the latest limitations, see qrb_ros_nn_inference on GitHub.