Estimate image depth values with sample_depth_estimation

The sample_depth_estimation sample application takes an RGB image named input_image.jpg or the /cam0_stream1 ROS topic from qrb_ros_camera. It runs model inference with the Qualcomm AI Engine Direct SDK (QNN) and publishes the result as the /depth_map ROS topic containing per-pixel depth values. The model has its source from Depth Anything V2, a deep convolutional neural network model for depth estimation. The following figure shows an example depth estimation result.

Office scene on the left beside its colorized depth map on the right, where warmer colors mark surfaces closer to the camera.

`sample_depth_estimation` pipeline flow

The following figure shows the pipeline flow for sample_depth_estimation.

Depth estimation pipeline from camera or image publisher input through preprocessing, inference, and postprocessing to the depth map topic. — sample_depth_estimation pipeline.

ROS nodes used in `sample_depth_estimation`

The following table lists the ROS nodes used in sample_depth_estimation.

Node name	Description
qrb ros camera	Qualcomm ROS 2 package that captures images with parameters and publishes them to ROS topics.
Image publisher	Publishes image data to a ROS topic—can be camera frames, local files, or processed outputs.
Sample depth estimation	Subscribes to input images for preprocessing, then performs postprocessing on the output tensor published by the qrb ros nn interface node.
qrb ros nn interface	Loads a trained AI model, receives preprocessed images, performs inference, and publishes results.

ROS topics used in `sample_depth_estimation`

The following table lists the ROS topics used in sample_depth_estimation.

ROS topic	Type	Description
`/image_raw`	`<sensor_msgs.msg.Image>`	Published image information.
`/qrb_inference_input_tensor`	`<qrb_ros_tensor_list_msgs.msg.TensorList>`	Preprocessed message.
`/qrb_inference_output_tensor`	`<qrb_ros_tensor_list_msgs.msg.TensorList>`	Neural network interface result with model.
`/depth_map`	`<sensor_msgs.msg.Image>`	Depth map result.

Prerequisites

You have set up the device, installed ROS2 Jazzy and Qualcomm Intelligent Robotics (QIR) SDK on the device according to Install the QIR SDK.

Build from the source of `sample_depth_estimation`

Build sample_depth_estimation from source when you need to change the sample code. The device steps install the dependencies, clone the source repository, and build the package.

Device steps

Download the Depth-Anything-V2 model

Download the model

sudo mkdir -p /opt/model && cd /opt/model
sudo wget https://huggingface.co/qualcomm/Depth-Anything-V2/resolve/19ce3645e11de17eed7e869eebcc07dd352834f3/Depth-Anything-V2.bin?download=true -O Depth-Anything-V2.bin

Install dependencies

Install the dependencies

sudo apt install -y ros-dev-tools
sudo apt install -y ros-jazzy-qrb-ros-camera

Download the source code from the qrb_ros_samples repository

Download the source code

mkdir -p ~/qrb_ros_ws/src && cd ~/qrb_ros_ws/src
git clone -b jazzy-rel https://github.com/qualcomm-qrb-ros/qrb_ros_samples.git

Build the sample application from source code

Build the sample application

cd ~/qrb_ros_ws/src/qrb_ros_samples/ai_vision/sample_depth_estimation
rosdep install -i --from-path ./ --rosdistro jazzy -y
colcon build

Set up the environment with the source command and launch the sample application

Launch the sample application

source install/setup.bash
ros2 launch sample_depth_estimation launch_with_image_publisher.py

You can replace this with a custom image file or model path

Launch with a custom image and model

source install/setup.bash
ros2 launch sample_depth_estimation launch_with_image_publisher.py image_path:=<your local image path> model_path:=<your local model path>

You can also launch with qrb_ros_camera if you connect to the GMSL camera

Launch with qrb_ros_camera

source install/setup.bash
ros2 launch sample_depth_estimation launch_with_qrb_ros_camera.py

When using this launch script, it uses the default parameters, which send the local input_image.jpg file with a publishing rate of 10 Hz

image_path_arg = DeclareLaunchArgument(
         'image_path',
         default_value=os.path.join(package_path, "resource", "input_image.jpg"),
         description='Path to the input image file'
    )
    # Node for image_publisher
    image_publisher_node = Node(
        package='image_publisher',
        executable='image_publisher_node',
        namespace=namespace,
        name='image_publisher_node',
        output='screen',
        parameters=[
            {'filename': image_path},
            {'rate': 10.0},  # Set the publishing rate to 10 Hz
        ]
    )

You can then check ROS topics with the topic name /depth_map in rqt

To install rqt, see the ROS 2 Jazzy documentation.a. Run rqt and select the following buttons in sequence: Plugins --> Visualization --> Image View.b. Select /sample_container/depth_map to see the results.

​sample_depth_estimation pipeline flow

​ROS nodes used in sample_depth_estimation

​ROS topics used in sample_depth_estimation

​Prerequisites

​Build from the source of sample_depth_estimation

​Device steps