Estimate image depth values with sample-depth-estimation

The sample_depth_estimation sample application estimates per-pixel depth maps from RGB images using ROS 2 nodes. It uses the Qualcomm® AI Engine Direct SDK (QNN) for on-device model inference. sample_depth_estimation accepts an RGB image named input_image.jpg as input or subscribes to the /sample_container/cam0_stream1 topic from the qrb_ros_camera node. It publishes the depth estimation results as a per-pixel depth map to the /sample_container/depth_map topic.

Note

For more information, see the sample_depth_estimation GitHub repository.
This model comes from Depth Anything V2, in which depth refers to the distance from the camera to each point in the image, estimated using a deep convolutional neural network.

Figure: Sample effects of sample_depth_estimat

Pipeline flow for `sample_depth_estimation`

Figure: Pipeline flow for sample_depth_estimation

ROS nodes used in the `sample_depth_estimation` pipeline

ROS Node	Description
`sample_depth_estimation`	A Python-based ROS 2 Jazzy package that processes per-pixel depth values. This ROS node subscribes to an image topic, and publishes depth estimation result topic after preprocessing and postprocessing.
`qrb_ros_camera`	Qualcomm ROS 2 package that captures images with parameters and publishes them to ROS topics.
`qrb_ros_nn_inference`	This ROS node loads a trained AI model, receives preprocessed images, performs inference, and publishes results.
`image_publisher`	A ROS 2 Jazzy package that publishes the image ROS topic with the local path. For more information, see image_publisher.

ROS topics used in the `sample_depth_estimation` pipeline

ROS Topic	Type	Description
`/image_raw`	`sensor_msgs.msg.Image`	Published image information, from `image_publisher`. Subscribed with an absolute name, so it is not namespaced.
`/sample_container/cam0_stream1`	`sensor_msgs.msg.Image`	Camera image stream, from `qrb_ros_camera`.
`/sample_container/qrb_inference_input_tensor`	`qrb_ros_tensor_list_msgs.msg.TensorList`	Preprocess message.
`/sample_container/qrb_inference_output_tensor`	`qrb_ros_tensor_list_msgs.msg.TensorList`	Postprocess message.
`/sample_container/depth_map`	`sensor_msgs.msg.Image`	Contains per-pixel depth values as a color map.

NoteThe launch files put every node in the sample_container namespace, so the sample’s topics are prefixed with /sample_container/. Only /image_raw is subscribed with an absolute name and therefore stays global.

Prerequisites

You have completed the following settings in Set up the environment for running sample applications.

Set up the device
Set up the host docker

Run out-of-the-box `sample_depth_estimation`

Try me

Run the sample on the development kit

SSH Session

source /usr/share/qirp-setup.sh -m
export ROS_DOMAIN_ID=55

# Launch the sample depth estimation with image publisher.
# You can replace 'image_path' with your desired image.
ros2 launch sample_depth_estimation launch_with_image_publisher.py \
    image_path:=/usr/ros/jazzy/share/sample_depth_estimation/resource/input_image.jpg

# Launch the sample depth estimation node with qrb_ros_camera.
ros2 launch sample_depth_estimation launch_with_qrb_ros_camera.py

View the depth estimation result on the host computer

Start a terminal and run the following command to check the depth estimation result.
SSH Session
export ROS_DOMAIN_ID=55 ros2 topic echo /sample_container/depth_map

Start the rqt to view the depth estimation result on the host docker, for more information, see rqt.

SSH Session

# YOUR_HOST_IP is the IP address of the Host where you want to view the sample output.
export DISPLAY=YOUR_HOST_IP:0
export ROS_DOMAIN_ID=55
rqt

Select the following buttons in sequence. Plugins —> Visualization —> Image View
Select topic /sample_container/depth_map by manual on rqt gui, then picture show on rqt successfully.

Build and run `sample_depth_estimation`

The following steps build the sample_depth_estimation package using Dragonwing IQ-9075 Evaluation Kit as an example.

Try me

Build and package on the host computer

Build the sample application project.

SSH Session

cd <decompressed_workspace>/images/iq-9075-evk/qirpsdk_artifacts/iq-9075-evk/
tar -zxvf qirp-sdk_<qirp_version>.tar.gz
cd <qir_decompressed_path>/qirp-sdk
source setup.sh
# build sample
cd <qirp_decompressed_path>/qirp-samples/ai_vision/sample_depth_estimation
colcon build

Package and push the sample application to the device.

SSH Session

# package and push build result of sample
cd ./install/sample_depth_estimation
tar -czvf sample_depth_estimation.tar.gz lib share
scp sample_depth_estimation.tar.gz root@[ip-addr]:/opt/

Install and run on the development kit

Install the sample application.

SSH Session

# Install sample package
tar --no-overwrite-dir --no-same-owner -zxf /opt/sample_depth_estimation.tar.gz -C /usr/ros/jazzy/

Run the sample application with the steps in Run out-of-the-box sample_depth_estimation.

​Pipeline flow for sample_depth_estimation

​ROS nodes used in the sample_depth_estimation pipeline

​ROS topics used in the sample_depth_estimation pipeline

​Prerequisites

​Run out-of-the-box sample_depth_estimation

​Build and run sample_depth_estimation

Pipeline flow for `sample_depth_estimation`

ROS nodes used in the `sample_depth_estimation` pipeline

ROS topics used in the `sample_depth_estimation` pipeline

Prerequisites

Run out-of-the-box `sample_depth_estimation`

Build and run `sample_depth_estimation`