Enable people tracking with follow_me - Qualcomm Dragonwing Documentation

Robot running the follow_me application as it detects a person and drives to keep pace behind them.

Pipeline flow for `follow_me`

The following figure shows the pipeline flow for follow_me.

The follow_me pipeline from Orbbec camera color and depth streams through detection to velocity commands on the cmd_vel topic. — The follow_me pipeline.

ROS nodes used in the `follow_me` pipeline

The following table lists the ROS nodes used in the follow_me pipeline and describes what each node does.

ROS Node	Description
`follow_me`	`follow_me` is a custom ROS2 node that supports human detection and posture updates. It can also calculate the necessary control commands for the robot and send them in the standard ROS2 format.
`orbbec_camera`	`orbbec_camera` is the `OrbbecSDK` ROS2 wrapper, which provides integration of Orbbec cameras within the ROS2 environment.

ROS topics used in the `follow_me` pipeline

The following table lists the ROS topics used in the follow_me pipeline, with the message type and the publisher of each topic.

ROS Topic	Type	Published By
`/camera/color/image_raw`	`sensor_msgs::msg::Image`	`orbbec-camera`
`/camera/depth/image_raw`	`sensor_msgs::msg::Image`	`orbbec-camera`
`/camera/color/camera_info`	`sensor_msgs::msg::CameraInfo`	`orbbec-camera`
`/cmd_vel`	`geometry_msgs::msg::Twist`	Controller

Prerequisites

You have set up the device, installed ROS2 Jazzy and the Qualcomm Intelligent Robotics (QIR) SDK on the device according to Install the QIR SDK.

Set up the device
Set up the host computer

Run out-of-the-box `follow_me`

Install dependencies

Add the package repositories

sudo add-apt-repository ppa:ubuntu-qcom-iot/qcom-ppa
sudo add-apt-repository ppa:ubuntu-qcom-iot/qirp
sudo apt update

Install the sample_apriltag packages

Install the packages

sudo apt install ros-jazzy-orbbec-camera
sudo apt install ros-jazzy-ncnn
sudo apt install ros-jazzy-follow-me

Download model files

Download and install the model files

# Download label file
wget https://raw.githubusercontent.com/amikelive/coco-labels/master/coco-labels-2014_2017.txt -O labels.txt
# Download model weights
wget https://raw.githubusercontent.com/dog-qiuqiu/FastestDet/main/example/ncnn/FastestDet.bin -O FastestDet.bin
wget https://raw.githubusercontent.com/dog-qiuqiu/FastestDet/main/example/ncnn/FastestDet.param -O FastestDet.param
# copy model files
sudo mkdir -p /usr/share/follow-me/model/
sudo cp FastestDet.bin FastestDet.param labels.txt /usr/share/follow-me/model/

Set up the Robot base

If you have a Qualcomm’s robot base, set it up.
If you are using a custom or third-party robot base, apply configurations appropriate to your own robot base. Ensure the robot base can publish odometry data of the type nav_msgs.msg.Odometry2 on the /odom topic.

Set up the Orbbec camera in a new terminal

Launch the Orbbec camera

# color & depth image
export ROS_DOMAIN_ID=100
source /opt/ros/jazzy/setup.sh
ros2 launch orbbec_camera gemini_330_series.launch.py depth_registration:=true enable_depth:=true

Run the follow_me binary

Run follow_me

#run follow me
export ROS_DOMAIN_ID=100
source /opt/ros/jazzy/setup.sh
follow_me

Build and run `follow_me`

This sample application is for authorized users. If you want to get the source code, see Working with Qualcomm.

Install the dependencies

sudo add-apt-repository ppa:ubuntu-qcom-iot/qcom-ppa
sudo add-apt-repository ppa:ubuntu-qcom-iot/qirp
sudo apt update
sudo apt install ncnn-dev

Download source code from qrb-ros-sample repository

Clone the sample repository

mkdir -p ~/qrb_ros_sample_ws/src && cd ~/qrb_ros_sample_ws/src
source /opt/ros/jazzy/setup.bash
git clone https://github.com/qualcomm-qrb-ros/qrb_ros_samples.git

Build sample from source code

Build the sample

cd ~/qrb_ros_sample_ws/src/qrb_ros_samples/robotics/sample_followme/
colcon build
source install/setup.bash

Run and test

Run and test according to steps 4-6 of Run out-of-the-box follow_me.

​Pipeline flow for follow_me

​ROS nodes used in the follow_me pipeline

​ROS topics used in the follow_me pipeline

​Prerequisites

​Run out-of-the-box follow_me

​Build and run follow_me

Pipeline flow for `follow_me`

ROS nodes used in the `follow_me` pipeline

ROS topics used in the `follow_me` pipeline

Prerequisites

Run out-of-the-box `follow_me`

Build and run `follow_me`