Convert between NV12 and RGB888 with qrb_ros_colorspace_convert

Qualcomm’s smart devices, such as the Qualcomm® Dragonwing™ IQ-9075 development kits, use NV12 as the default image color space conversion format. However, the more common color space format is RGB888. The qrb_ros_colorspace_convert sample application implements the following functionalities:

Provides ROS nodes
- API to convert NV12 to RGB888
- API to convert RGB888 to NV12
Supports dmabuf fd as input and output
Input and output image receive and send with QRB ROS transport
Hardware acceleration with GPU by OpenGL ES

Pipeline flow for `qrb_ros_colorspace_convert`

The following figure shows the pipeline flow of qrb_ros_colorspace_convert.

Color space conversion pipeline where a custom ROS node exchanges dmabuf fd image buffers with the colorspace_convert node over ROS topics. — qrb_ros_color_space_convert pipeline.

The following table describes the ROS nodes used in the qrb_ros_colorspace_convert software architecture.

ROS Node	Description
`custom_ros_node`	`custom_ros_node` is the ROS package that publishes the input image information to the `colorspace_convert_node`. You should implement this ROS2 node.
`colorspace_convert_node`	`colorspace_convert_node` is a ROS package to convert color space between NV12 and RGB888.

The following table describes the ROS topics used in the qrb_ros_colorspace_convert software architecture.

ROS topic	Message type	Published by/Subscription	Description
`/image_raw`	`sensor_msgs::msg::Image`	Subscribes to images from other nodes.	The input NV12/RGB888 encoded image.
`/image`	`sensor_msgs::msg::Image`	Published by `colorspace_convert_node`.	Publishes the converted NV12/RGB888 encoded image.

Prerequisites

You have set up the device, installed ROS2 Jazzy and Qualcomm Intelligent Robotics (QIR) SDK on the device according to Install the QIR SDK.
The custom_ros_node is ready.
You have connected the device to the display monitor.

Run out-of-the-box `qrb_ros_colorspace_convert`

Install the color space conversion package according to whether you have installed the desktop or server QIR SDK image

See Install the QIR SDK.

qirp-sdk is the QIR SDK server image package.
qirp-sdk-desktop is the QIR SDK desktop image package.
For Ventuno Q, use the package listed under Desktop image.

Server image

sudo apt install ros-jazzy-qrb-ros-colorspace-convert

Desktop image

sudo apt install ros-jazzy-qrb-ros-colorspace-convert-opencv

Launch the qrb_ros_color_space_convert node

Launch the node

source /opt/ros/jazzy/setup.bash
# convert color space from NV12 to RGB888
ros2 launch qrb_ros_colorspace_convert colorspace_convert.launch.py 'conversion_type:=nv12_to_rgb8' 'latency_fps_test:=False'
# convert color space from RGB888 to NV12
ros2 launch qrb_ros_colorspace_convert colorspace_convert.launch.py 'conversion_type:=rgb8_to_nv12' 'latency_fps_test:=False'

Confirm the color space conversion result with the following commands:

ros2 topic echo /image_raw | grep "encoding"
ros2 topic echo /image | grep "encoding"

Build from source

Install dependencies according to whether you have installed the desktop or server QIR SDK image

See Install the QIR SDK.

qirp-sdk is the QIR SDK server image package.
qirp-sdk-desktop is the QIR SDK desktop image package.
For Ventuno Q, use the package listed under Desktop image.

Server image

sudo apt install ros-jazzy-qrb-ros-transport-image-type
sudo apt install ros-jazzy-qrb-ros-transport-point-cloud2-type
sudo apt install ros-jazzy-qrb-ros-transport-imu-type
sudo apt install ros-jazzy-pcl-conversions
sudo apt install qcom-adreno-dev
sudo apt install weston
sudo apt install weston-autostart
# reboot is required for Weston to start automatically

Desktop image

sudo apt install ros-jazzy-qrb-ros-transport-image-type
sudo apt install ros-jazzy-qrb-ros-transport-point-cloud2-type
sudo apt install ros-jazzy-qrb-ros-transport-imu-type
sudo apt install ros-jazzy-pcl-conversions
sudo apt install libopencv-dev libopencv-contrib-dev

Download the source code and build with colcon

Download and build

source /opt/ros/jazzy/setup.bash
git clone https://github.com/qualcomm-qrb-ros/qrb_ros_color_space_convert.git

# Server image
colcon build --cmake-args -DUSE_OPENCV_BACKEND=OFF

# Desktop image
colcon build --cmake-args -DUSE_OPENCV_BACKEND=ON

Run and debug

Run the node

source install/setup.bash
# convert color space from NV12 to RGB888
ros2 launch qrb_ros_colorspace_convert colorspace_convert.launch.py 'conversion_type:=nv12_to_rgb8' 'latency_fps_test:=False'
# convert color space from RGB888 to NV12
ros2 launch qrb_ros_colorspace_convert colorspace_convert.launch.py 'conversion_type:=rgb8_to_nv12' 'latency_fps_test:=False'

Confirm the color space conversion result with the following commands:

ros2 topic echo /image_raw | grep "encoding"
ros2 topic echo /image | grep "encoding"

​Pipeline flow for qrb_ros_colorspace_convert

​Prerequisites

​Run out-of-the-box qrb_ros_colorspace_convert

​Build from source

Pipeline flow for `qrb_ros_colorspace_convert`

Prerequisites

Run out-of-the-box `qrb_ros_colorspace_convert`

Build from source