> ## Documentation Index
> Fetch the complete documentation index at: https://dragonwingdocs.qualcomm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Detect objects with sample_object_detection

> The sample_object_detection is a Python launch file utilizing QNN for model inference. It demonstrates camera data streaming, AI-based inference, and real-time visualization of object detection results.

Ultralytics YOLOv8 is a machine learning model that predicts bounding boxes, segmentation masks and classes of objects in an image.

## `sample_object_detection` pipeline flow

The following figure shows the pipeline flow for `sample_object_detection`.

<Frame caption="Pipeline flow for sample_object_detection.">
  <img src="https://mintcdn.com/qualcomm-prod/eHVY1H1lLEVhgq58/SDKs/QIR-SDK-Ubuntu/images/image50.svg?fit=max&auto=format&n=eHVY1H1lLEVhgq58&q=85&s=5c6ed800f501683a4e0a4ec6287023ab" alt="Object detection pipeline from camera capture through YOLO preprocessing, inference, postprocessing, and overlay to the detection result topic." width="1062" height="302" data-path="SDKs/QIR-SDK-Ubuntu/images/image50.svg" />
</Frame>

## ROS nodes used in the `sample_object_detection` pipeline

The following table lists the ROS nodes used in the `sample_object_detection` pipeline.

| Node name                                                                        | Description                                                                                          |
| -------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
| [qrb ros camera](https://github.com/qualcomm-qrb-ros/qrb_ros_camera)             | Qualcomm ROS 2 package that captures images with parameters and publishes them to ROS topics.        |
| [yolo preprocess](https://github.com/qualcomm-qrb-ros/qrb_ros_tensor_process)    | Subscribes to image data, reshapes and resizes it, and republishes it to a downstream topic.         |
| [qrb ros nn interface](https://github.com/qualcomm-qrb-ros/qrb_ros_nn_inference) | Loads a trained AI model, receives preprocessed images, performs inference, and publishes results.   |
| [yolo postprocess](https://github.com/qualcomm-qrb-ros/qrb_ros_tensor_process)   | Matches inference output with yolo label files.                                                      |
| [yolo overlay](https://github.com/qualcomm-qrb-ros/qrb_ros_tensor_process)       | Subscribes to the yolo postprocess and image data, shows the object detect results with a ROS topic. |

## ROS topics used in the `sample_object_detection` pipeline

The following table lists the ROS topics used in the `sample_object_detection` pipeline.

| ROS topic                     | Type                                        | Published by                      |
| ----------------------------- | ------------------------------------------- | --------------------------------- |
| `/camera/color/image_raw`     | `<sensor_msgs.msg.Image>`                   | `qrb_ros_camera`                  |
| `/qrb_inference_input_tensor` | `<qrb_ros_tensor_list_msgs.msg.TensorList>` | `yolo_preprocess_node`            |
| `/yolo_detect_result`         | `<vision_msgs.msg.Detection2DArray>`        | `nn_inference_node`               |
| `/yolo_detect_tensor_output`  | `<qrb_ros_tensor_list_msgs.msg.TensorList>` | `yolo_detection_postprocess_node` |
| `/yolo_detect_overlay`        | `<sensor_msgs.msg.Image>`                   | `yolo_detection_overlay_node`     |

## Prerequisites

* You have set up the device, installed ROS2 Jazzy and Qualcomm Intelligent Robotics (QIR) SDK on the device according to [Install the QIR SDK](./install-the-qir-sdk).

* On the host computer, you have downloaded and built the yolo model according to steps in the README of [qrb\_ros\_tensor\_process](https://github.com/qualcomm-qrb-ros/qrb_ros_tensor_process).

  <Note>
    When downloading the yolo model, use `target-runtime` and `device` as follows:
  </Note>

## Generate YOLOv8 model

<Steps>
  <Step title="Set up the QAI-Hub on the host computer.">
    a. Sign in to [QAI-Hub](https://app.aihub.qualcomm.com/docs/hub/getting_started.html) and get your API token.

    b. Install QAI-Hub on the host computer.

    ```bash theme={null}
     ## It is recommended to use python venv to avoid impacting the host environment.
     python3 -m venv venv_qaihub
     source venv_qaihub/bin/activate

     ## Install QAI-Hub related python packages.
     pip3 install qai-hub

     ## Configure QAI-Hub token, replacing xxx with your own token got from QAI-Hub page.
     qai-hub configure --api_token xxx

     pip3 install "qai-hub-models[yolov8-det]"
    ```
  </Step>

  <Step title="Export the model on the host computer.">
    After executing the following commands, the model file and coco8.yaml are generated in the host computer.

    <CodeGroup>
      ```bash IQ-9075     theme={null}
      python -m qai_hub_models.models.yolov8_det.export --target-runtime "qnn_context_binary" --chipset "qualcomm-qcs9075" --skip-profiling --skip-inferencing

      # Push model files to the device's default path according to the preceding export log.

      # On the device, create the path if it doesn't exist.
      mkdir /opt/model


      # On the host computer, scp all model files to the device.
      scp <Host_Export_model_Path>/yolov8_det.bin ubuntu@<device_ip>:/opt/model

      sudo find / -name "coco8.yaml" -print -quit
      scp <label-file-path>/coco8.yaml ubuntu@<device_ip>:/opt/
      ```

      ```bash VENTUO Q (QCS8275)    theme={null}
      python -m qai_hub_models.models.yolov8_det.export --target-runtime "qnn_context_binary" --chipset "qualcomm-qcs8275" --skip-profiling --skip-inferencing

      # Push model files to the device's default path according to the preceding export log.

      # On the device, create the path if it doesn't exist.
      mkdir /opt/model

      # On the host computer, scp all model files to the device.
      scp <Host_Export_model_Path>/yolov8_det.bin ubuntu@<device_ip>:/opt/model

      sudo find / -name "coco8.yaml" -print -quit
      scp <label-file-path>/coco8.yaml ubuntu@<device_ip>:/opt/        
      ```
    </CodeGroup>
  </Step>
</Steps>

## Run out-of-the-box `sample_object_detection`

On the device, run the sample application:

```bash Run the sample application theme={null}
source /opt/ros/jazzy/setup.bash
ros2 launch sample_object_detection launch_with_qrb_ros_camera.py model:=/opt/model/yolov8_det.bin
```

Then, you can check the ROS topics with the name `/yolo_detect_overlay` in `rviz2`.

## Build from the source of `sample_object_detection`

<Note>
  Ensure that you run the [Prerequisites](./detect-objects-with-sample_object_detection#prerequisites) first before performing the following steps.
</Note>

On the device, run the sample application:

<Steps>
  <Step title="Download the source code from the qrb_ros_samples repository">
    ```bash Download the source code theme={null}
    mkdir -p ~/qrb_ros_ws/src && cd ~/qrb_ros_ws/src
    git clone https://github.com/qualcomm-qrb-ros/qrb_ros_samples.git
    ```
  </Step>

  <Step title="Build the sample_object_detection sample application from the source code">
    ```bash Build the sample application theme={null}
    cd ~/qrb_ros_ws/src/qrb_ros_samples/ai_vision/sample_object_detection
    colcon build
    source install/setup.bash
    ```
  </Step>

  <Step title="Run and test the sample application">
    Run and test according to steps 2–3 of [Run out-of-the-box `sample_object_detection`](./detect-objects-with-sample_object_detection#run-out-of-the-box-sample_object_detection).
  </Step>
</Steps>