> ## Documentation Index
> Fetch the complete documentation index at: https://dragonwingdocs.qualcomm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Classify images with sample_resnet101

The `sample_resnet101` sample application classifies images using a `ResNet-101` model with ROS 2 nodes. It uses Qualcomm® AI Engine Direct SDK (QNN) for model inference.

`resnet101` is a machine learning model that can classify images from the ImageNet dataset. It also serves as the basis for building more advanced models tailored to specific use cases.

<Note>
  **Note**

  For more information, see [sample\_resnet101](https://github.com/qualcomm-qrb-ros/qrb_ros_samples/tree/main/ai_vision/sample_resnet101) on GitHub.
</Note>

<video
  autoPlay
  muted
  loop
  playsInline
  width={500}
  height={521}
  src="https://mintcdn.com/qualcomm-prod/AeBa_PbSuUY_ELHw/SDKs/QIR-SDK-2.0/media/80-65220-2-qirp-sdk-qsg/resnet101-output.mp4?fit=max&auto=format&n=AeBa_PbSuUY_ELHw&q=85&s=c77a0b321362f25554e54c68689be208"
  style={{
width: "70%",
display: "block",
margin: "0 auto"}}
  data-path="SDKs/QIR-SDK-2.0/media/80-65220-2-qirp-sdk-qsg/resnet101-output.mp4"
/>

<p style={{ textAlign: 'center', fontWeight: 'bold' }}>
  Figure: Sample effects of `sample_resnet101`
</p>

## Pipeline flow for `sample_resnet101`

<img src="https://mintcdn.com/qualcomm-prod/AeBa_PbSuUY_ELHw/SDKs/QIR-SDK-2.0/media/80-65220-2-qirp-sdk-qsg/resnet101-pipeline-regenerate.svg?fit=max&auto=format&n=AeBa_PbSuUY_ELHw&q=85&s=7c69d7203a6a5129eb5c602b63338e15" style={{width: "100%"}} width="901" height="323" data-path="SDKs/QIR-SDK-2.0/media/80-65220-2-qirp-sdk-qsg/resnet101-pipeline-regenerate.svg" />

<p style={{ textAlign: 'center', fontWeight: 'bold' }}>
  Figure: Pipeline flow for `sample_resnet101`
</p>

## ROS nodes used in `sample_resnet101`

| **ROS node**                                                                        | **Description**                                                                                                                                                                                       |
| :---------------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [qrb\_ros\_camera](https://github.com/qualcomm-qrb-ros/qrb_ros_camera)              | Qualcomm ROS 2 package that captures images with parameters and publishes them to ROS topics. Runs as node `camera_node`.                                                                             |
| `qrb_ros_resnet101`                                                                 | Preprocess node (`resnet101_node`) from the `sample_resnet101` package. Subscribes to the image topic, converts and resizes it to 224x224x3, and publishes a tensor to `/qrb_inference_input_tensor`. |
| [qrb\_ros\_nn\_inference](https://github.com/qualcomm-qrb-ros/qrb_ros_nn_inference) | Loads the `ResNet101_w8a8.bin` model, receives the preprocessed tensor, runs inference on the QNN HTP backend, and publishes the output tensor. Runs as node `nn_inference_node`.                     |
| `qrb_ros_resnet101_posprocess`                                                      | Postprocess node (`resnet101_postprocess_node`) from the `sample_resnet101` package. Maps the output tensor to an ImageNet label and publishes the classification result.                             |
| [image\_publisher](https://github.com/ros-perception/image_pipeline)                | ROS 2 Jazzy package that publishes the image ROS topic with a local path. Used instead of `qrb_ros_camera` when the sample is launched with `launch_with_image_publisher.py`.                         |

## ROS topics used in `sample_resnet101`

| **ROS topic**                  |                  **Type**                 |       **Published by**       |
| :----------------------------- | :---------------------------------------: | :--------------------------: |
| `/cam0_stream1`                |          `sensor_msgs.msg.Image`          |         `camera_node`        |
| `/image_raw`                   |          `sensor_msgs.msg.Image`          |       `image_publisher`      |
| `/qrb_inference_input_tensor`  | `qrb_ros_tensor_list_msgs.msg.TensorList` |       `resnet101_node`       |
| `/qrb_inference_output_tensor` | `qrb_ros_tensor_list_msgs.msg.TensorList` |      `nn_inference_node`     |
| `/resnet101_output`            |           `std_msgs.msg.String`           | `resnet101_postprocess_node` |

<Note>
  **Note**

  The preprocess node subscribes to `image_raw`. With `launch_with_qrb_ros_camera.py` this subscription is remapped to `/cam0_stream1`; with `launch_with_image_publisher.py` it keeps the default `/image_raw`.
</Note>

## Prerequisites

You have completed the settings in [Set up the environment for running sample applications](./set-up-env-for-sample-app).

## Run out-of-the-box `sample_resnet101`

<Accordion title="Try me">
  <Steps>
    <Step title="Run the sample on the development kit">
      ```bash title="SSH Session" theme={null}
      # Set up the runtime environment
      source /usr/share/qirp-setup.sh -m
      # Set the ROS_DOMAIN_ID
      export ROS_DOMAIN_ID=123
      ros2 launch sample_resnet101 launch_with_qrb_ros_camera.py
      ```
    </Step>

    <Step title="Check the classification result on the host computer">
      ```bash title="SSH Session" theme={null}
      export ROS_DOMAIN_ID=123
      ros2 topic echo /resnet101_output
      ```
    </Step>
  </Steps>
</Accordion>

## Build and run `sample_resnet101`

The following steps build the `sample_resnet101` package using Dragonwing IQ-9075 Evaluation Kit as an example.

<Accordion title="Try me">
  <Steps>
    <Step title="Build and package on the host computer">
      1. Build the sample application project.
         ```bash theme={null}
         cd <decompressed_workspace>/images/iq-9075-evk/qirpsdk_artifacts/iq-9075-evk/
         tar -zxvf qirp-sdk_<qirp_version>.tar.gz
         cd qirp-sdk
         source setup.sh
         # build sample
         cd <qirp_decompressed_path>/qirp-samples/ai_vision/sample_resnet101
         colcon build
         ```
      2. Package and push the sample application to the device.
         ```bash theme={null}
         # package and push build result of sample
         cd <qirp_decompressed_path>/qirp-samples/ai_vision/sample_resnet101/install/sample_resnet101
         tar -czvf sample_resnet101.tar.gz lib share
         scp sample_resnet101.tar.gz root@[ip-addr]:/opt/
         ```
    </Step>

    <Step title="Install and run on the development kit">
      1. Install the sample application.
         ```bash title="SSH Session" theme={null}
         # Install sample package
         tar --no-overwrite-dir --no-same-owner -zxf /opt/sample_resnet101.tar.gz -C /usr/ros/jazzy/
         ```
      2. Run the sample application with the steps in [Run out-of-the-box sample\_resnet101](#run-out-of-the-box-sample_resnet101).
    </Step>
  </Steps>
</Accordion>