> ## Documentation Index
> Fetch the complete documentation index at: https://dragonwingdocs.qualcomm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# 使用 sample_hand_detection 检测手部

`sample_hand_detection` 示例应用使用基于 Python 的 ROS 2 节点从图像或摄像头流中检测手部和手部关键点。它使用 Qualcomm® AI Engine Direct SDK (QNN) 进行模型推理。

该模型来自 MediaPipe Hand Landmark Detector，这是一个机器学习流水线，可预测图像中手部的边界框和姿势骨架。该示例应用允许你输入名为 `input_image.jpg` 的静态图像或实时摄像头流，然后将结果作为 ROS topic `/handlandmark_result` 发布。

<Note>
  **注意**

  有关更多信息，请参见 GitHub 上的 [sample\_hand\_detection](https://github.com/qualcomm-qrb-ros/qrb_ros_samples/tree/main/ai_vision/sample_hand_detection)。
</Note>

<video
  autoPlay
  muted
  loop
  playsInline
  width={770}
  height={582}
  src="https://mintlify.s3.us-west-1.amazonaws.com/qualcomm-prod/zh/SDKs/QIR-SDK-2.0/media/80-65220-2-qirp-sdk-qsg/hand_result.mp4"
  style={{
width: "70%",
display: "block",
margin: "0 auto"
}}
/>

<p style={{ textAlign: 'center', fontWeight: 'bold' }}>
  图：`sample_hand_detection` 的示例效果
</p>

## `sample_hand_detection` 的流水线流程

<img src="https://mintlify.s3.us-west-1.amazonaws.com/qualcomm-prod/zh/SDKs/QIR-SDK-2.0/media/80-65220-2-qirp-sdk-qsg/ga1.6-hand-detection.svg" style={{width: "100%", display: "block", margin: "0 auto"}} />

<p style={{ textAlign: 'center', fontWeight: 'bold' }}>
  图：`sample_hand_detection` 的流水线流程
</p>

## `sample_hand_detection` 中使用的 ROS 节点

<table>
  <thead>
    <tr>
      <th style={{ width: '35%' }}>ROS 节点</th>
      <th style={{ width: '65%' }}>说明</th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td>[qrb\_ros\_camera](https://github.com/qualcomm-qrb-ros/qrb_ros_camera)</td>
      <td>Qualcomm ROS 2 包，使用参数捕获图像并将其发布到 ROS topic。</td>
    </tr>

    <tr>
      <td>[qrb\_ros\_nn\_inference](https://github.com/qualcomm-qrb-ros/qrb_ros_nn_inference)</td>
      <td>加载训练好的 AI 模型，接收预处理后的图像，执行推理并发布结果。</td>
    </tr>

    <tr>
      <td>`qrb_ros_hand_detector`</td>
      <td>订阅一个图像 topic（来自静态图像或摄像头），执行预处理和后处理，并发布检测结果。</td>
    </tr>

    <tr>
      <td>`palm_preprocessor`</td>
      <td>通过调整大小、归一化图像和转换图像格式来准备手掌检测的输入图像数据，以满足模型的要求。</td>
    </tr>

    <tr>
      <td>`palm_postprocessor`</td>
      <td>解释手掌检测模型的原始输出，提取边界框和置信度分数，应用过滤和可选的 NMS，并将结果映射到原始图像坐标。</td>
    </tr>

    <tr>
      <td>`landmark_preprocessor`</td>
      <td>基于手掌检测结果处理裁剪出的手部区域。调整并归一化图像，并可能应用几何变换以对齐手部方向。</td>
    </tr>

    <tr>
      <td>`landmark_postprocessor`</td>
      <td>解码关键点检测模型的输出，提取手部关键点并将它们映射回原始图像空间。</td>
    </tr>
  </tbody>
</table>

## `sample_hand_detection` 中使用的 ROS topic

<table>
  <thead>
    <tr>
      <th style={{ width: '35%' }} align="center">ROS topic</th>
      <th style={{ width: '30%' }} align="center">类型</th>
      <th style={{ width: '30%' }} align="center">发布者</th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td align="left"><code>/handlandmark\_result</code></td>
      <td align="left"><code>sensor\_msgs.msg.Image</code></td>
      <td align="left"><code>qrb\_ros\_hand\_detector</code></td>
    </tr>

    <tr>
      <td align="left"><code>/palm\_detector\_input\_tensor</code></td>
      <td align="left"><code>qrb\_ros\_tensor\_list\_msgs.msg.TensorList</code></td>
      <td align="left"><code>qrb\_ros\_hand\_detector</code></td>
    </tr>

    <tr>
      <td align="left"><code>/palm\_detector\_output\_tensor</code></td>
      <td align="left"><code>qrb\_ros\_tensor\_list\_msgs.msg.TensorList</code></td>
      <td align="left"><code>qrb\_ros\_nn\_inference</code></td>
    </tr>

    <tr>
      <td align="left"><code>/landmark\_detector\_input\_tensor</code></td>
      <td align="left"><code>qrb\_ros\_tensor\_list\_msgs.msg.TensorList</code></td>
      <td align="left"><code>qrb\_ros\_hand\_detector</code></td>
    </tr>

    <tr>
      <td align="left"><code>/landmark\_detector\_output\_tensor</code></td>
      <td align="left"><code>qrb\_ros\_tensor\_list\_msgs.msg.TensorList</code></td>
      <td align="left"><code>qrb\_ros\_nn\_inference</code></td>
    </tr>

    <tr>
      <td align="left"><code>/image\_raw</code></td>
      <td align="left"><code>sensor\_msgs.msg.Image</code></td>
      <td align="left"><code>qrb\_ros\_camera</code></td>
    </tr>
  </tbody>
</table>

## 前提条件

你已完成 [设置运行示例应用的环境](./set-up-env-for-sample-app) 中的设置。

## 开箱即用运行 `sample_hand_detection`

<Accordion title="试一试">
  <Steps>
    <Step title="在开发套件上运行示例">
      ```bash title="SSH Session" theme={null}
      # Set up the runtime environment
      source /usr/share/qirp-setup.sh -m
      # Set the ROS_DOMAIN_ID
      export ROS_DOMAIN_ID=123
      export PYTHONPATH=$PYTHONPATH:/opt/ros/jazzy/lib/python3.14/site-packages/sample_hand_detection
      # Launch the sample hand detection node with qrb_ros_camera ros node.
      ros2 launch sample_hand_detection launch_with_qrb_ros_camera.py model_path:=/opt/model/
      ```
    </Step>

    <Step title="在主机计算机上查看结果">
      1. 启动 `rqt`。
         ```bash title="SSH Session" theme={null}
         # YOUR_HOST_IP is the IP address of the Host where you want to view the sample output.
         export DISPLAY=YOUR_HOST_IP:0
         export ROS_DOMAIN_ID=123
         rqt
         ```
      2. 依次选择以下按钮：
         **Plugins** --> **Visualization** --> **Image View**
      3. 选择 `handlandmark_result` 以查看结果。
    </Step>
  </Steps>
</Accordion>

## 构建并运行 `sample_hand_detection`

以下步骤以 Dragonwing IQ-9075 Evaluation Kit 为例构建 `sample_hand_detection` 包。

<Accordion title="试一试">
  <Steps>
    <Step title="在主机计算机上构建并打包">
      1. 构建示例应用项目。
         ```bash theme={null}
         cd <decompressed_workspace>/images/iq-9075-evk/qirpsdk_artifacts/iq-9075-evk/
         tar -zxvf qirp-sdk_<qirp_version>.tar.gz
         cd qirp-sdk
         source setup.sh
         # build sample
         cd <qirp_decompressed_path>/qirp-samples/ai_vision/sample_hand_detection
         colcon build
         ```
      2. 打包并将示例应用推送到设备。
         ```bash theme={null}
         # package and push build result of sample
         cd <qirp_decompressed_path>/qirp-samples/ai_vision/sample_hand_detection/install/sample_hand_detection
         tar -czvf sample_hand_detection.tar.gz lib share
         scp sample_hand_detection.tar.gz root@[ip-addr]:/opt/
         ```
    </Step>

    <Step title="在开发套件上安装并运行">
      1. 安装示例应用。
         ```bash title="SSH Session" theme={null}
         # Install sample package
         tar --no-overwrite-dir --no-same-owner -zxf /opt/sample_hand_detection.tar.gz -C /usr/ros/jazzy/
         ```
      2. 按照 [开箱即用运行 sample\_hand\_detection](#run-out-of-the-box-sample_hand_detection) 中的步骤运行示例应用。
    </Step>
  </Steps>
</Accordion>