Encode and decode video with qrb_ros_video - Qualcomm Dragonwing Documentation

The qrb_ros_video sample application is a ROS package that provides hardware-accelerated video processing on Qualcomm robotics platforms. It encodes raw YUV frames into H.264 or H.265 streams and decodes those streams back into raw frames, using the Qualcomm Video Processing Unit (VPU). Buffers move through qrb_ros_transport as DMA buffers, so frames are passed without a copy, and the encoder accepts input directly from qrb_ros_camera.

NoteFor more information, see the qrb_ros_video GitHub repo.

Pipeline flow for `qrb_ros_video`

The encoder_launch.py and decoder_launch.py launch files start two independent composable-node containers, each in its own namespace. The encoder container reads raw frames with ImageReader, encodes them, and writes the compressed stream to a file with CompressedWriter. The decoder container reads a compressed stream with CompressedReader, decodes it, and writes the frames with ImageWriter. The encoder also accepts frames directly from qrb_ros_camera if you remap its input.

Figure: qrb_ros_video pipeline

ROS nodes used in the `qrb_ros_video` pipeline

ROS node	Description
`qrb_ros::video::Encoder`	Converts raw images to a compressed H.264 or H.265 stream using the Qualcomm VPU.
`qrb_ros::video::Decoder`	Decodes a compressed H.264 or H.265 stream back into raw image frames.
`qrb_ros::video::ImageReader`	Reads raw YUV frames from a file and publishes them to the encoder.
`qrb_ros::video::ImageWriter`	Subscribes to the decoded frames and writes them to a file.
`qrb_ros::video::CompressedReader`	Reads a compressed stream from a file and publishes it to the decoder.
`qrb_ros::video::CompressedWriter`	Subscribes to the compressed stream and writes it to a file.

ROS topics used in the `qrb_ros_video` pipeline

ROS topic	Message type	Published/Subscribed by	Description
`/recording_ns/encoder_node/input`	`qrb_ros::transport::type::Image`	Published by `reader_node` (`ImageReader`), subscribed by `encoder_node`.	Uncompressed YUV frames.
`/recording_ns/writer_node/compressed_image`	`sensor_msgs::msg::CompressedImage`	Published by `encoder_node`, subscribed by `writer_node` (`CompressedWriter`).	Compressed H.264 or H.265 video stream.
`/playback_ns/decoder_node/input`	`sensor_msgs::msg::CompressedImage`	Published by `reader_node` (`CompressedReader`), subscribed by `decoder_node`.	Compressed H.264 or H.265 video stream.
`/playback_ns/writer_node/raw_image`	`qrb_ros::transport::type::Image`	Published by `decoder_node`, subscribed by `writer_node` (`ImageWriter`).	Decoded YUV frames.

NoteEach node declares its endpoints as the relative names input and output. The launch files remap them to the names above and place every node in the recording_ns or playback_ns namespace, so /input and /output do not exist at runtime. Run ros2 topic list to see the effective names.

ROS parameters for `qrb_ros_video`

Table: Encoder parameters

Parameter	Type	Default	Description
`format`	string	`h264`	Video codec format: `h264` or `h265`.
`pixel-format`	string	`nv12`	Input pixel format: `nv12` or `p010`.
`width`	int	`1920`	Video width.
`height`	int	`1080`	Video height.
`framerate`	int	`30`	Frames per second.
`bitrate`	int	`5000000`	Target bitrate, in bits per second. `encoder_launch.py` overrides this to `20000000`.
`rate-control`	string	`variable`	Rate control mode: `variable` or `cbr`.
`profile`	string	`main`	Codec profile: `baseline`, `main`, or `high`.
`level`	string	`4.1`	Codec level.

Table: Decoder parameters

Parameter	Type	Default	Description
`format`	string	`h264`	Input codec format.
`pixel-format`	string	`nv12`	Output pixel format.

Prerequisites

You have Set up the device according to Set up the environment for running sample applications.

Run out-of-the-box `qrb_ros_video`

Try me

Set up the environment

Start a commandline terminal on your device and run the following commands.

ssh root@[ip-addr]

SSH Session

source /usr/share/qirp-setup.sh
export ROS_DOMAIN_ID=xx

NoteValue range of ROS_DOMAIN_ID: [0, 232]

Encode a local YUV file

Copy the raw YUV sample file to the /data/ directory on the device.
scp 1920_1080_nv12.yuv root@[ip-addr]:/data/
Run the encoder.
SSH Session
ros2 launch qrb_ros_video encoder_launch.py
reader_node reads /data/1920_1080_nv12.yuv and writer_node writes the encoded stream to /data/1920_1080_nv12.mp4. Override either path with the url parameter of the corresponding node.

Decode a local video file

Copy the video sample file to the /data/ directory on the device.
scp 1920_1080.mp4 root@[ip-addr]:/data/
Run the decoder.
SSH Session
ros2 launch qrb_ros_video decoder_launch.py
reader_node reads /data/1920_1080.mp4. writer_node writes the decoded frames to /dev/null by default, so no output file is produced; set its url parameter to keep the decoded YUV.

Verify the ROS topics

In a second SSH terminal, list the topics and check the stream that the encoder or the decoder publishes.

SSH Session

source /usr/share/qirp-setup.sh
export ROS_DOMAIN_ID=xx
ros2 topic list
ros2 topic hz /recording_ns/writer_node/compressed_image
ros2 topic hz /playback_ns/writer_node/raw_image

Build and run `qrb_ros_video`

Try me

Clone qrb_ros_video on the host

cd <qirp_decompressed_workspace>
source setup.sh
git clone -b stable/0.1.7 https://github.com/qualcomm-qrb-ros/qrb_ros_video.git

Build qrb_ros_video on the host

cd qrb_ros_video
colcon build --merge-install --cmake-args ${CMAKE_ARGS}

Push qrb_ros_video to the device

cd qrb_ros_video/install
tar czvf qrb_ros_video.tar.gz lib share include
scp qrb_ros_video.tar.gz root@[ip-addr]:/opt/
ssh root@[ip-addr]

SSH Session

tar --no-overwrite-dir --no-same-owner -zxf /opt/qrb_ros_video.tar.gz -C /usr/ros/jazzy/

Run qrb_ros_video on the device

Run qrb_ros_video on the device by following Run out-of-the-box qrb_ros_video.

Limitations

Hardware-accelerated encoding and decoding require the Qualcomm VPU, so the video overlay must be enabled on the device.
The supported codec formats are H.264 and H.265. The supported input pixel formats are NV12 and P010.

​Pipeline flow for qrb_ros_video

​ROS nodes used in the qrb_ros_video pipeline

​ROS topics used in the qrb_ros_video pipeline

​ROS parameters for qrb_ros_video

​Prerequisites

​Run out-of-the-box qrb_ros_video

​Build and run qrb_ros_video

​Limitations

Pipeline flow for `qrb_ros_video`

ROS nodes used in the `qrb_ros_video` pipeline

ROS topics used in the `qrb_ros_video` pipeline

ROS parameters for `qrb_ros_video`

Prerequisites

Run out-of-the-box `qrb_ros_video`

Build and run `qrb_ros_video`

Limitations