> ## Documentation Index
> Fetch the complete documentation index at: https://dragonwingdocs.qualcomm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Detect hands with sample_hand_detection

The `sample_hand_detection` sample application detects hands and hand landmarks from images or camera streams using Python-based ROS 2 nodes. It uses Qualcomm® AI Engine Direct SDK (QNN) for model inference.

The model is sourced from the MediaPipe Hand Landmark Detector, which is a machine learning pipeline that predicts bounding boxes and pose skeletons of hands in an image. This sample application allows you to input either a static image named `input_image.jpg` or a live camera stream, and then it publishes the result as ROS topic `/handlandmark_result`.

<Note>
  **Note**

  For more information, see [sample\_hand\_detection](https://github.com/qualcomm-qrb-ros/qrb_ros_samples/tree/main/ai_vision/sample_hand_detection) on GitHub.
</Note>

<video
  autoPlay
  muted
  loop
  playsInline
  width={770}
  height={582}
  src="https://mintcdn.com/qualcomm-prod/AeBa_PbSuUY_ELHw/SDKs/QIR-SDK-2.0/media/80-65220-2-qirp-sdk-qsg/hand_result.mp4?fit=max&auto=format&n=AeBa_PbSuUY_ELHw&q=85&s=3999c39db6e342a05b4a330130bf0a87"
  style={{
width: "70%",
display: "block",
margin: "0 auto"
}}
  data-path="SDKs/QIR-SDK-2.0/media/80-65220-2-qirp-sdk-qsg/hand_result.mp4"
/>

<p style={{ textAlign: 'center', fontWeight: 'bold' }}>
  Figure: Sample effects of `sample_hand_detection`
</p>

## Pipeline flow for `sample_hand_detection`

<img src="https://mintcdn.com/qualcomm-prod/IP0o0G5fE3MFmCJ8/SDKs/QIR-SDK-2.0/media/80-65220-2-qirp-sdk-qsg/ga1.6-hand-detection.svg?fit=max&auto=format&n=IP0o0G5fE3MFmCJ8&q=85&s=ebfccef6879247128840b9b62a286f34" style={{width: "100%", display: "block", margin: "0 auto"}} width="1246" height="322" data-path="SDKs/QIR-SDK-2.0/media/80-65220-2-qirp-sdk-qsg/ga1.6-hand-detection.svg" />

<p style={{ textAlign: 'center', fontWeight: 'bold' }}>
  Figure: Pipeline flow for `sample_hand_detection`
</p>

## ROS nodes used in `sample_hand_detection`

<table>
  <thead>
    <tr>
      <th style={{ width: '35%' }}>ROS node</th>
      <th style={{ width: '65%' }}>Description</th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td>[qrb\_ros\_camera](https://github.com/qualcomm-qrb-ros/qrb_ros_camera)</td>
      <td>Qualcomm ROS 2 package that captures images with parameters and publishes them to ROS topics.</td>
    </tr>

    <tr>
      <td>[qrb\_ros\_nn\_inference](https://github.com/qualcomm-qrb-ros/qrb_ros_nn_inference)</td>
      <td>Loads a trained AI model, receives preprocessed images, performs inference, and publishes results.</td>
    </tr>

    <tr>
      <td>`qrb_ros_hand_detector`</td>
      <td>Subscribes to an image topic (from static image or camera), performs preprocessing and postprocessing, and publishes detection results.</td>
    </tr>

    <tr>
      <td>`palm_preprocessor`</td>
      <td>Prepares input image data for palm detection by resizing, normalizing images, and converting image formats to match model requirements.</td>
    </tr>

    <tr>
      <td>`palm_postprocessor`</td>
      <td>Interprets raw output from palm detection model, extracts bounding boxes and confidence scores, applies filtering and optional NMS, and maps results to original image coordinates.</td>
    </tr>

    <tr>
      <td>`landmark_preprocessor`</td>
      <td>Processes the cropped hand regions based on palm detection results. Resizes and normalizes the image, and may apply geometric transformations to align the hand orientation.</td>
    </tr>

    <tr>
      <td>`landmark_postprocessor`</td>
      <td>Decodes the output of the landmark detection model, extracts hand keypoints and maps them back to the original image space.</td>
    </tr>
  </tbody>
</table>

## ROS topics used in `sample_hand_detection`

<table>
  <thead>
    <tr>
      <th style={{ width: '35%' }} align="center">ROS Topic</th>
      <th style={{ width: '30%' }} align="center">Type</th>
      <th style={{ width: '30%' }} align="center">Published By</th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td align="left"><code>/handlandmark\_result</code></td>
      <td align="left"><code>sensor\_msgs.msg.Image</code></td>
      <td align="left"><code>qrb\_ros\_hand\_detector</code></td>
    </tr>

    <tr>
      <td align="left"><code>/palm\_detector\_input\_tensor</code></td>
      <td align="left"><code>qrb\_ros\_tensor\_list\_msgs.msg.TensorList</code></td>
      <td align="left"><code>qrb\_ros\_hand\_detector</code></td>
    </tr>

    <tr>
      <td align="left"><code>/palm\_detector\_output\_tensor</code></td>
      <td align="left"><code>qrb\_ros\_tensor\_list\_msgs.msg.TensorList</code></td>
      <td align="left"><code>qrb\_ros\_nn\_inference</code></td>
    </tr>

    <tr>
      <td align="left"><code>/landmark\_detector\_input\_tensor</code></td>
      <td align="left"><code>qrb\_ros\_tensor\_list\_msgs.msg.TensorList</code></td>
      <td align="left"><code>qrb\_ros\_hand\_detector</code></td>
    </tr>

    <tr>
      <td align="left"><code>/landmark\_detector\_output\_tensor</code></td>
      <td align="left"><code>qrb\_ros\_tensor\_list\_msgs.msg.TensorList</code></td>
      <td align="left"><code>qrb\_ros\_nn\_inference</code></td>
    </tr>

    <tr>
      <td align="left"><code>/image\_raw</code></td>
      <td align="left"><code>sensor\_msgs.msg.Image</code></td>
      <td align="left"><code>qrb\_ros\_camera</code></td>
    </tr>
  </tbody>
</table>

## Prerequisites

You have completed the settings in [Set up the environment for running sample applications](./set-up-env-for-sample-app).

## Run out-of-the-box `sample_hand_detection`

<Accordion title="Try me">
  <Steps>
    <Step title="Run the sample on the development kit">
      ```bash title="SSH Session" theme={null}
      # Set up the runtime environment
      source /usr/share/qirp-setup.sh -m
      # Set the ROS_DOMAIN_ID
      export ROS_DOMAIN_ID=123
      export PYTHONPATH=$PYTHONPATH:/opt/ros/jazzy/lib/python3.14/site-packages/sample_hand_detection
      # Launch the sample hand detection node with qrb_ros_camera ros node.
      ros2 launch sample_hand_detection launch_with_qrb_ros_camera.py model_path:=/opt/model/
      ```
    </Step>

    <Step title="View the results on the host computer">
      1. Start the `rqt`.
         ```bash title="SSH Session" theme={null}
         # YOUR_HOST_IP is the IP address of the Host where you want to view the sample output.
         export DISPLAY=YOUR_HOST_IP:0
         export ROS_DOMAIN_ID=123
         rqt
         ```
      2. Select the following buttons in sequence:
         **Plugins** --> **Visualization** --> **Image View**
      3. Select `handlandmark_result` to see the results.
    </Step>
  </Steps>
</Accordion>

## Build and run `sample_hand_detection`

The following steps build the `sample_hand_detection` package using Dragonwing IQ-9075 Evaluation Kit as an example.

<Accordion title="Try me">
  <Steps>
    <Step title="Build and package on the host computer">
      1. Build the sample application project.
         ```bash theme={null}
         cd <decompressed_workspace>/images/iq-9075-evk/qirpsdk_artifacts/iq-9075-evk/
         tar -zxvf qirp-sdk_<qirp_version>.tar.gz
         cd qirp-sdk
         source setup.sh
         # build sample
         cd <qirp_decompressed_path>/qirp-samples/ai_vision/sample_hand_detection
         colcon build
         ```
      2. Package and push the sample application to the device.
         ```bash theme={null}
         # package and push build result of sample
         cd <qirp_decompressed_path>/qirp-samples/ai_vision/sample_hand_detection/install/sample_hand_detection
         tar -czvf sample_hand_detection.tar.gz lib share
         scp sample_hand_detection.tar.gz root@[ip-addr]:/opt/
         ```
    </Step>

    <Step title="Install and run on the development kit">
      1. Install the sample application.
         ```bash title="SSH Session" theme={null}
         # Install sample package
         tar --no-overwrite-dir --no-same-owner -zxf /opt/sample_hand_detection.tar.gz -C /usr/ros/jazzy/
         ```
      2. Run the sample application with the steps in [Run out-of-the-box sample\_hand\_detection](#run-out-of-the-box-sample_hand_detection).
    </Step>
  </Steps>
</Accordion>