> ## Documentation Index
> Fetch the complete documentation index at: https://dragonwingdocs.qualcomm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Detect objects with sample_object_detection

The `sample_object_detection` sample application detects and classifies objects in images using ROS 2 nodes. It uses Qualcomm® AI Engine Direct SDK (QNN) for model inference.

The sample uses the Ultralytics YOLOv8 model to predict bounding boxes and classes of objects in an image.

<Note>
  **Note**

  For more information, see [sample\_object\_detection](https://github.com/qualcomm-qrb-ros/qrb_ros_samples/tree/main/ai_vision/sample_object_detection) on GitHub.
</Note>

<video
  autoPlay
  muted
  loop
  playsInline
  width={320}
  height={320}
  src="https://mintcdn.com/qualcomm-prod/EDJV-hu6qJhZi4pm/SDKs/QIR-SDK-2.0/media/80-65220-2-qirp-sdk-qsg/sample_object_detection_result.mp4?fit=max&auto=format&n=EDJV-hu6qJhZi4pm&q=85&s=338a87f812e7f11c3ac4f8dc07ed95c2"
  style={{
width: "50%",
display: "block",
margin: "0 auto" }}
  data-path="SDKs/QIR-SDK-2.0/media/80-65220-2-qirp-sdk-qsg/sample_object_detection_result.mp4"
/>

<p style={{ textAlign: 'center', fontWeight: 'bold' }}>
  Figure: Sample effects of `sample_object_detection`
</p>

## Pipeline flow for `sample_object_detection`

<img src="https://mintcdn.com/qualcomm-prod/AeBa_PbSuUY_ELHw/SDKs/QIR-SDK-2.0/media/80-65220-2-qirp-sdk-qsg/object_detection_pipeline-regenerate.svg?fit=max&auto=format&n=AeBa_PbSuUY_ELHw&q=85&s=cf91b4a825f26f1055b22a708f2e3600" style={{width: "100%"}} width="1216" height="322" data-path="SDKs/QIR-SDK-2.0/media/80-65220-2-qirp-sdk-qsg/object_detection_pipeline-regenerate.svg" />

<p style={{ textAlign: 'center', fontWeight: 'bold' }}>
  Figure: Pipeline flow for `sample_object_detection`
</p>

## ROS nodes used in `sample_object_detection`

| **ROS node**                                                                        | **Description**                                                                                           |
| :---------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------- |
| [qrb\_ros\_camera](https://github.com/qualcomm-qrb-ros/qrb_ros_camera)              | Qualcomm ROS 2 package that captures images with parameters and publishes them to ROS topics.             |
| [yolo\_preprocess](https://github.com/qualcomm-qrb-ros/qrb_ros_tensor_process)      | Subscribes to image data, reshapes and resizes it, and republishes it to a downstream topic.              |
| [qrb\_ros\_nn\_inference](https://github.com/qualcomm-qrb-ros/qrb_ros_nn_inference) | Loads a trained AI model, receives preprocessed images, performs inference, and publishes results.        |
| [yolo\_postprocess](https://github.com/qualcomm-qrb-ros/qrb_ros_tensor_process)     | Matches inference output with YOLO label files.                                                           |
| [yolo\_overlay](https://github.com/qualcomm-qrb-ros/qrb_ros_tensor_process)         | Subscribes to the YOLO postprocess and image data, and shows the object detection results as a ROS topic. |

## ROS topics used in `sample_object_detection`

| **ROS Topic**                 | **Type**                                  | **Published By**                  |
| :---------------------------- | :---------------------------------------- | :-------------------------------- |
| `/cam0_stream1`               | `sensor_msgs.msg.Image`                   | `camera_node`                     |
| `/qrb_inference_input_tensor` | `qrb_ros_tensor_list_msgs.msg.TensorList` | `yolo_preprocess_node`            |
| `/resized_image`              | `sensor_msgs.msg.Image`                   | `yolo_preprocess_node`            |
| `/yolo_detect_tensor_output`  | `qrb_ros_tensor_list_msgs.msg.TensorList` | `nn_inference_node`               |
| `/yolo_detect_result`         | `vision_msgs.msg.Detection2DArray`        | `yolo_detection_postprocess_node` |
| `/yolo_detect_overlay`        | `sensor_msgs.msg.Image`                   | `yolo_detection_overlay_node`     |

<Note>
  **Note**

  `yolo_detection_overlay_node` synchronizes two inputs, `/yolo_detect_result` and `/resized_image`, using an exact-time policy. Both must be publishing for `/yolo_detect_overlay` to be produced.
</Note>

## Prerequisites

* You have completed the settings in [Set up the environment for running sample applications](./set-up-env-for-sample-app).
* You have generated the YOLOv8 model using QAI-Hub on the host computer with the following steps.

## Generate YOLOv8 model

1. Set up QAI-Hub on the host computer.
   1. Sign in to [QAI-Hub](https://app.aihub.qualcomm.com/docs/hub/getting_started.html) and get your API token.
   2. Install QAI-Hub.
      ```python theme={null}
      ## It is recommended to use python venv to avoid impacting the host environment.
      python3 -m venv venv_qaihub
      source venv_qaihub/bin/activate
      ## Install QAI-Hub related python packages.
      pip3 install qai-hub==0.41.0
      ## Configure QAI-Hub token, replacing xxx with your own token got from QAI-Hub page.
      qai-hub configure --api_token xxx
      pip3 install "qai-hub-models[yolov8-det]==0.42.0"
      ```
2. Export the model on the host computer.
   ```python theme={null}
   # For IQ-9075, after executing the following commands, the model file and coco8.yaml are generated in the host computer.
   python -m qai_hub_models.models.yolov8_det.export --target-runtime "qnn_context_binary" --chipset "qualcomm-qcs9075-proxy" --skip-profiling --skip-inferencing --compile-options='--qairt_version=2.41'
   ```
   ```bash title="SSH Session" theme={null}
   # Push model files to the device's default path according to the preceding export log.
   # On the device, create the path if it doesn't exist.
   mkdir /opt/model
   ```
   ```bash theme={null}
   # On the host computer, scp all model files to the device.
   scp <Host_Export_model_Path>/yolov8_det_float/yolov8_det.bin root@<device_ip>:/opt/model
   scp <Host_Export_model_Path>/coco8.yaml root@<device_ip>:/opt/
   ```

For more information, see [qrb\_ros\_tensor\_process](https://github.com/qualcomm-qrb-ros/qrb_ros_tensor_process/tree/main/cv_tensor_process/yolo_v8_process) on GitHub.

## Run out-of-the-box `sample_object_detection`

<Accordion title="Try me">
  <Steps>
    <Step title="Run the sample on the development kit">
      ```bash title="SSH Session" theme={null}
      # Set up the runtime environment
      source /usr/share/qirp-setup.sh -m
      # Set the ROS_DOMAIN_ID
      export ROS_DOMAIN_ID=123
      # Launch the sample object detection
      ros2 launch sample_object_detection launch_with_qrb_ros_camera.py model:=/opt/model/yolov8_det.bin
      ```
    </Step>

    <Step title="Check the results on the host computer">
      In a terminal of the host computer, check the ROS topics with the name `/yolo_detect_overlay` in `rqt`.
    </Step>
  </Steps>
</Accordion>

## Build and run `sample_object_detection`

The following steps build the `sample_object_detection` package using Dragonwing IQ-9075 Evaluation Kit as an example.

<Accordion title="Try me">
  <Steps>
    <Step title="Build and package on the host computer">
      1. Build the sample application project.
         ```bash theme={null}
         cd <decompressed_workspace>/images/iq-9075-evk/qirpsdk_artifacts/iq-9075-evk/
         tar -zxvf qirp-sdk_<qirp_version>.tar.gz
         cd qirp-sdk
         source setup.sh
         # build sample
         cd <qirp_decompressed_path>/qirp-samples/ai_vision/sample_object_detection
         colcon build
         ```
      2. Package and push the sample application to the device.
         ```bash theme={null}
         # package and push build result of sample
         cd <qirp_decompressed_path>/qirp-samples/ai_vision/sample_object_detection/install/sample_object_detection
         tar -czvf sample_object_detection.tar.gz lib share
         scp sample_object_detection.tar.gz root@[ip-addr]:/opt/
         ```
    </Step>

    <Step title="Install and run on the development kit">
      1. Install the sample application.
         ```bash title="SSH Session" theme={null}
         # Install sample package
         tar --no-overwrite-dir --no-same-owner -zxf /opt/sample_object_detection.tar.gz -C /usr/ros/jazzy/
         ```
      2. Run the sample application with the steps in [Run out-of-the-box sample\_object\_detection](#run-out-of-the-box-sample_object_detection).
    </Step>
  </Steps>
</Accordion>