> ## Documentation Index
> Fetch the complete documentation index at: https://dragonwingdocs.qualcomm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Execute AI model inference with qrb_ros_nn_inference

[qrb\_ros\_nn\_inference](https://github.com/qualcomm-qrb-ros/qrb_ros_nn_inference) is a ROS2 package for performing neural network model, providing AI-based perception for robotics applications.

**It provides the following functionalities:**

* Model inference API which supports three model formats:
  * `.tflite`
  * `.so`
  * `.bin`
* Model inference acceleration based on Qualcomm platforms

## Pipeline flow for  `qrb_ros_nn_inference`

<img src="https://mintcdn.com/qualcomm-prod/IP0o0G5fE3MFmCJ8/SDKs/QIR-SDK-2.0/media/80-65220-2-qirp-sdk-qsg/nn-inference-pipeline.svg?fit=max&auto=format&n=IP0o0G5fE3MFmCJ8&q=85&s=a380f719946ebb2e68450dc8862846de" alt="pipeline" style={{ display: 'block', margin: '0 auto' }} width="982" height="243" data-path="SDKs/QIR-SDK-2.0/media/80-65220-2-qirp-sdk-qsg/nn-inference-pipeline.svg" />

<p style={{ textAlign: 'center', fontWeight: 'bold' }}>
  Figure : The pipeline of qrb\_ros\_nn\_inference
</p>

## ROS topics used in the `qrb_ros_nn_inference `pipeline

<table>
  <thead>
    <tr>
      <th style={{ width: '20%' }}>ROS topic</th>
      <th style={{ width: '25%' }}>Message type</th>
      <th style={{ width: '25%' }}>Published/Subscribed by</th>
      <th style={{ width: '30%' }}>Description</th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td>`/qrb_inference_input_tensor`</td>
      <td>`qrb_ros_tensor_list_msgs::msg::TensorList`</td>
      <td>Subscribed by `qrb_ros_nn_inference`</td>
      <td>Subscribes to the input tensor for model inference.</td>
    </tr>

    <tr>
      <td>`/qrb_inference_output_tensor`</td>
      <td>`qrb_ros_tensor_list_msgs::msg::TensorList`</td>
      <td>Published by `qrb_ros_nn_inference`</td>
      <td>Publishes the output tensor of model.</td>
    </tr>
  </tbody>
</table>

## ROS node parameters used in the `qrb_ros_nn_inference `pipeline

<table>
  <thead>
    <tr>
      <th style={{ width: '20%' }}>Node parameter</th>
      <th style={{ width: '25%' }}>Type</th>
      <th style={{ width: '25%' }}>Default value</th>
      <th style={{ width: '30%' }}>Description</th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td>`backend_option`</td>
      <td style={{textAlign: 'center'}}>`string`</td>
      <td style={{textAlign: 'center'}}>`""`</td>
      <td>Hardware acceleration option for model inference. For valid values, see [qrb\_ros\_nn\_inference documentation](https://github.com/qualcomm-qrb-ros/qrb_ros_nn_inference/blob/main/qrb_inference_manager/Documentation.md#2.1).</td>
    </tr>

    <tr>
      <td>`model_path`</td>
      <td style={{textAlign: 'center'}}>`string`</td>
      <td style={{textAlign: 'center'}}>`""`</td>
      <td>Indicates the path of the model file.</td>
    </tr>
  </tbody>
</table>

## Prerequisites

* You have **Set up the device** according to [Set up the environment for running sample applications](./set-up-env-for-sample-app).
* You have downloaded the model from [Qualcomm AI hub](https://aihub.qualcomm.com/).
* You have prepared the preprocess node and postprocess node according to the [qrb\_ros\_nn\_inference example](https://github.com/qualcomm-qrb-ros/qrb_ros_nn_inference/tree/main/test).

## Run out-of-the-box `qrb_ros_nn_inference`

1. Start one terminal, and run the following commands to set up QIR SDK and ROS2 environment on the device.

   Value range of `ROS_DOMAIN_ID`: \[0, 232]

   ```cpp theme={null}
   ssh root@[ip-addr]
   (ssh) source /usr/share/qirp-setup.sh
   (ssh) export ROS_DOMAIN_ID=xx
   ```
2. Launch three nodes: preprocess node, `qrb_ros_nn_inference` node and postprocess node.
   ```undefined theme={null}
   (ssh) ros2 launch model_inference.launch.py
   ```

## Build and run `qrb_ros_nn_inference`

1. Build `qrb_ros_nn_inference` provided by the QIR SDK on the host.
   ```powershell theme={null}
   cd <qirp_decompressed_workspace>
   source setup.sh
   cd qirp-samples/platform/qrb_ros_nn_inference
   colcon build --merge-install --cmake-args ${CMAKE_ARGS}
   ```
2. Push `qrb_ros_nn_inference` to the device.

   <Note>
     **Note**

     Ensure that the QIR SDK is installed on the device.
   </Note>

   ```powershell theme={null}
   cd qirp-samples/platform/qrb_ros_nn_inference/install
   tar czvf qrb_ros_nn_inference.tar.gz include lib share
   scp qrb_ros_nn_inference.tar.gz root@[ip-addr]:/opt/
   ssh root@[ip-addr]
   (ssh) tar --no-overwrite-dir --no-same-owner -zxf /opt/qrb_ros_nn_inference.tar.gz -C /usr/ros/jazzy/
   ```
3. Start one terminal, and run the following commands to set up QIR SDK and ROS2 environment on the device.
   Value range of `ROS_DOMAIN_ID`: \[0, 232]
   ```cpp theme={null}
   ssh root@[ip-addr]
   (ssh) source /usr/share/qirp-setup.sh
   (ssh) export ROS_DOMAIN_ID=xx
   ```
4. Launch three nodes: preprocess node, `qrb_ros_nn_inference` node and postprocess node.
   ```undefined theme={null}
   (ssh) ros2 launch model_inference.launch.py
   ```

## Limitation

For the latest limitations, see [qrb\_ros\_nn\_inference](https://github.com/qualcomm-qrb-ros/qrb_ros_nn_inference) on GitHub.