Run a zero-copy camera with qrb_ros_camera - Qualcomm Dragonwing Documentation

The qrb_ros_camera is a ROS package that publishes the images from Qualcomm CSI and GMSL cameras. It provides the following:

Support for concurrent multiple streams output.
Support for composable ROS node.
Zero-copy transport powered by QRB ROS Transport.
Support for NV12 only as output format, limited by the Camera Service.

Pipeline flow for `qrb_ros_camera`

The following figure shows the pipeline flow for qrb_ros_camera.

Camera hardware and a camera info YAML file feeding the qrb_ros_camera ROS node through the camera server, which publishes two output topics. — qrb_ros_camera pipeline.

The qrb_ros_camera is a ROS 2 package. It creates an image publisher with qrb_ros_transport for zero-copy transport. It supports node composition, making it possible to improve performance using ROS intra-process communication. The qrb_camera is a C++ library. It provides APIs to qrb_ros_camera for querying images from the lower layer Camera Service and CamX libraries. It includes 2 modules:

The camera_manager module manages the camera stream, which enables the multi-stream support.
The camera_client module calls Camera Service APIs to manage camera streams.

The qrb_ros_transport is a ROS 2 package that supports zero-copy image transport with Linux DMA buffer and implements ROS type adaption. It’s compatible with both intraprocessing and interprocessing communication. The Camera Service is a Qualcomm multimedia framework. It exports APIs for accessing Qualcomm multimedia hardware. The CamX provides the foundation for image capture, processing, and management on Qualcomm-powered devices.

`qrb_ros_camera` ROS interfaces and parameters

The qrb_ros_camera node exposes ROS interfaces and ROS parameters. Use them to subscribe to the published image topics and to configure the camera streams.

ROS interfaces

The following table describes the ROS interfaces provided by qrb_ros_camera.

Interface	Name	Type	Description
`Publisher`	`/cam${camera_id}_${stream_name}`	`sensor_msgs/msg/Image`	Outputs images.
`Publisher`	`/cam${camera_id}_camera_info`	`sensor_msgs/msg/CameraInfo`	Provides camera information.

ROS parameters

The following table describes the ROS parameters for qrb_ros_camera.

Name	Type	Description	Default value
`camera_id`	`int64`	The camera device ID	`0`
`stream_size`	`uint64`	Count of camera stream	`1`
`stream_name`	`string[]`	camera stream names	`["stream1"]`
`${stream_name}.width`	`uint32`	image width	`1920`
`${stream_name}.height`	`uint32`	image height	`1080`
`${stream_name}.fps`	`uint32`	output image frequency(Hz)	`30`
`camera_info_path`	`string`	Camera metadata file path	`config/camera_info_imx577.yaml`

`qrb_ros_camera` library APIs

The following table describes the qrb_camera C++ library functions that qrb_ros_camera calls.

Function	Parameter	Description
`int create_camera(CameraType type, uint32_t camera_id)`	• `type`: camera type • `camera_id`: camera id	Creates a camera and returns the camera index on success.
`bool set_camera_parameter(int index, CameraConfigure & param)`	• `index`: camera index • `param`: camera parameters	Returns `true` when the camera parameters are set successfully.
`bool start_camera(int index)`	`index`: camera index	Starts the camera. Returns `true` when started successfully.
`void stop_camera(int index)`	`index`: camera index	Stops the camera.
`bool register_callback(int index, ImageCallback image_cb, PointCloudCallback point_cloud_cb)`	• `index`: camera index • `Image_cb`: image callback • `Point_cloud_cb`: point cloud msg callback	Registers the callback for image and point cloud messages.

Prerequisites

You have set up the device, installed ROS2 Jazzy and Qualcomm Intelligent Robotics (QIR) SDK on the device according to Install the QIR SDK.

Run out-of-the-box `qrb_ros_camera`

Install the qrb_ros_camera packages

Install the packages

sudo apt install ros-jazzy-qrb-ros-camera

Use the camera node

Launch the camera node

source /opt/ros/jazzy/setup.bash
ros2 launch qrb_ros_camera qrb_ros_camera_launch.py

Check ROS topics or view the image with the topic /cam${camera_id}_${stream_name} in RVIZ or RQT

source /opt/ros/jazzy/setup.bash
ros2 topic echo /cam0_stream1
ros2 topic echo /cam0_camera_info

Enable multiple streams

By using the stream_size and stream_name parameters, you can configure multiple streams for one camera in your launch file, such as /opt/ros/jazzy/share/qrb_ros_camera/launch/qrb_ros_camera_launch.py.

qrb_ros_camera_launch.py

parameters=[{
    'camera_id': 0,
    'stream_size': 2,
    'stream_name': ["stream1", "stream2"],
    'stream1':{
        'width':1920,
        'height':1080,
        'fps':30,
    },
    'stream2':{
        'width':1080,
        'height':720,
        'fps':60,
    },
    'camera_info_path': os.path.join(
        get_package_share_directory('qrb_ros_camera'),
        'config', 'camera_info_imx577.yaml'),
}]

Enable the zero-copy transport

The qrb_ros_camera supports directly sharing image dmabuf_fd between nodes, which can avoid image data memory copy with DDS.

For details about this feature, see ROS documentation.

Following is an example that uses launch (recommended) to compose multiple nodes.

def generate_launch_description():
    container = ComposableNodeContainer(
        name='my_container',
        namespace='',
        package='rclcpp_components',
        executable='component_container',
        composable_node_descriptions=[
            ComposableNode(
                package='qrb_ros_camera',
                plugin='qrb_ros::camera::CameraNode',
                name='camera_node',
                parameters=[{
                    # ...
                }]
            ),
            ComposableNode(
                package='qrb_ros_camera',
                plugin='qrb_ros::camera::TestNode',
                name='sub_node',
            )
        ],
        output='screen',
    )
    return launch.LaunchDescription([container])

Build and run `qrb_ros_camera`

Install dependencies

Install the dependencies

sudo apt install ros-jazzy-qrb-ros-transport-image-type \
ros-dev-tools \
  qcom-camera-server \
  qcom-syslog-plumber-dev \
  libqmmf-dev \
  qcom-camxapi-dev

Clone the repository to the device

Download the source code

mkdir -p ~/ros2_ws/src
cd ~/ros2_ws/src
git clone https://github.com/qualcomm-qrb-ros/qrb_ros_camera.git

Build and run qrb_ros_camera

Build and launch

cd ~/ros2_ws
colcon build
source install/setup.bash
ros2 launch qrb_ros_camera qrb_ros_camera_launch.py

​Pipeline flow for qrb_ros_camera

​qrb_ros_camera ROS interfaces and parameters

​ROS interfaces

​ROS parameters

​qrb_ros_camera library APIs

​Prerequisites

​Run out-of-the-box qrb_ros_camera

​Build and run qrb_ros_camera

Pipeline flow for `qrb_ros_camera`

`qrb_ros_camera` ROS interfaces and parameters

ROS interfaces

ROS parameters

`qrb_ros_camera` library APIs

Prerequisites

Run out-of-the-box `qrb_ros_camera`

Build and run `qrb_ros_camera`