> ## Documentation Index
> Fetch the complete documentation index at: https://dragonwingdocs.qualcomm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Encode and decode video with qrb_ros_video

The `qrb_ros_video` sample application is a ROS package that provides hardware-accelerated video processing on Qualcomm robotics platforms. It encodes raw YUV frames into H.264 or H.265 streams and decodes those streams back into raw frames, using the Qualcomm Video Processing Unit (VPU). Buffers move through `qrb_ros_transport` as DMA buffers, so frames are passed without a copy, and the encoder accepts input directly from `qrb_ros_camera`.

<Note>
  **Note**

  For more information, see the [qrb\_ros\_video](https://github.com/qualcomm-qrb-ros/qrb_ros_video) GitHub repo.
</Note>

## Pipeline flow for `qrb_ros_video`

The `encoder_launch.py` and `decoder_launch.py` launch files start two independent composable-node containers, each in its own namespace. The encoder container reads raw frames with `ImageReader`, encodes them, and writes the compressed stream to a file with `CompressedWriter`. The decoder container reads a compressed stream with `CompressedReader`, decodes it, and writes the frames with `ImageWriter`. The encoder also accepts frames directly from `qrb_ros_camera` if you remap its input.

<img
  src="https://mintcdn.com/qualcomm-prod/AeBa_PbSuUY_ELHw/SDKs/QIR-SDK-2.0/media/80-65220-2-qirp-sdk-qsg/qrb-ros-video-pipeline-regenerate.svg?fit=max&auto=format&n=AeBa_PbSuUY_ELHw&q=85&s=0cedacf11bec5e2bc777faa9c547f718"
  style={{
width: "100%",
display: "block",
margin: "0 auto" }}
  width="1028"
  height="350"
  data-path="SDKs/QIR-SDK-2.0/media/80-65220-2-qirp-sdk-qsg/qrb-ros-video-pipeline-regenerate.svg"
/>

<p style={{ textAlign: 'center', fontWeight: 'bold' }}>
  Figure: qrb\_ros\_video pipeline
</p>

## ROS nodes used in the `qrb_ros_video` pipeline

|            **ROS node**            |                                  **Description**                                  |
| :--------------------------------: | :-------------------------------------------------------------------------------: |
|      `qrb_ros::video::Encoder`     | Converts raw images to a compressed H.264 or H.265 stream using the Qualcomm VPU. |
|      `qrb_ros::video::Decoder`     |       Decodes a compressed H.264 or H.265 stream back into raw image frames.      |
|    `qrb_ros::video::ImageReader`   |        Reads raw YUV frames from a file and publishes them to the encoder.        |
|    `qrb_ros::video::ImageWriter`   |            Subscribes to the decoded frames and writes them to a file.            |
| `qrb_ros::video::CompressedReader` |       Reads a compressed stream from a file and publishes it to the decoder.      |
| `qrb_ros::video::CompressedWriter` |            Subscribes to the compressed stream and writes it to a file.           |

## ROS topics used in the `qrb_ros_video` pipeline

|                 **ROS topic**                |           **Message type**          |                           **Published/Subscribed by**                          |             **Description**             |
| :------------------------------------------: | :---------------------------------: | :----------------------------------------------------------------------------: | :-------------------------------------: |
|      `/recording_ns/encoder_node/input`      |  `qrb_ros::transport::type::Image`  |    Published by `reader_node` (`ImageReader`), subscribed by `encoder_node`.   |         Uncompressed YUV frames.        |
| `/recording_ns/writer_node/compressed_image` | `sensor_msgs::msg::CompressedImage` | Published by `encoder_node`, subscribed by `writer_node` (`CompressedWriter`). | Compressed H.264 or H.265 video stream. |
|       `/playback_ns/decoder_node/input`      | `sensor_msgs::msg::CompressedImage` | Published by `reader_node` (`CompressedReader`), subscribed by `decoder_node`. | Compressed H.264 or H.265 video stream. |
|     `/playback_ns/writer_node/raw_image`     |  `qrb_ros::transport::type::Image`  |    Published by `decoder_node`, subscribed by `writer_node` (`ImageWriter`).   |           Decoded YUV frames.           |

<Note>
  **Note**

  Each node declares its endpoints as the relative names `input` and `output`. The launch files remap them to the names above and place every node in the `recording_ns` or `playback_ns` namespace, so `/input` and `/output` do not exist at runtime. Run `ros2 topic list` to see the effective names.
</Note>

## ROS parameters for `qrb_ros_video`

**Table: Encoder parameters**

| **Parameter**  | **Type** | **Default** | **Description**                                                                       |
| :------------- | :------- | :---------- | :------------------------------------------------------------------------------------ |
| `format`       | string   | `h264`      | Video codec format: `h264` or `h265`.                                                 |
| `pixel-format` | string   | `nv12`      | Input pixel format: `nv12` or `p010`.                                                 |
| `width`        | int      | `1920`      | Video width.                                                                          |
| `height`       | int      | `1080`      | Video height.                                                                         |
| `framerate`    | int      | `30`        | Frames per second.                                                                    |
| `bitrate`      | int      | `5000000`   | Target bitrate, in bits per second. `encoder_launch.py` overrides this to `20000000`. |
| `rate-control` | string   | `variable`  | Rate control mode: `variable` or `cbr`.                                               |
| `profile`      | string   | `main`      | Codec profile: `baseline`, `main`, or `high`.                                         |
| `level`        | string   | `4.1`       | Codec level.                                                                          |

**Table: Decoder parameters**

| **Parameter**  | **Type** | **Default** | **Description**      |
| :------------- | :------- | :---------- | :------------------- |
| `format`       | string   | `h264`      | Input codec format.  |
| `pixel-format` | string   | `nv12`      | Output pixel format. |

## Prerequisites

* You have **Set up the device** according to [Set up the environment for running sample applications](./set-up-env-for-sample-app).

## Run out-of-the-box `qrb_ros_video`

<Accordion title="Try me">
  <Steps>
    <Step title="Set up the environment">
      Start a commandline terminal on your device and run the following commands.

      ```bash theme={null}
      ssh root@[ip-addr]
      ```

      ```bash title="SSH Session" theme={null}
      source /usr/share/qirp-setup.sh
      export ROS_DOMAIN_ID=xx
      ```

      <Note>
        **Note**

        Value range of `ROS_DOMAIN_ID`: \[0, 232]
      </Note>
    </Step>

    <Step title="Encode a local YUV file">
      1. Copy the raw YUV sample file to the `/data/` directory on the device.
         ```bash theme={null}
         scp 1920_1080_nv12.yuv root@[ip-addr]:/data/
         ```
      2. Run the encoder.
         ```bash title="SSH Session" theme={null}
         ros2 launch qrb_ros_video encoder_launch.py
         ```
         `reader_node` reads `/data/1920_1080_nv12.yuv` and `writer_node` writes the encoded stream to `/data/1920_1080_nv12.mp4`. Override either path with the `url` parameter of the corresponding node.
    </Step>

    <Step title="Decode a local video file">
      1. Copy the video sample file to the `/data/` directory on the device.
         ```bash theme={null}
         scp 1920_1080.mp4 root@[ip-addr]:/data/
         ```
      2. Run the decoder.
         ```bash title="SSH Session" theme={null}
         ros2 launch qrb_ros_video decoder_launch.py
         ```
         `reader_node` reads `/data/1920_1080.mp4`. `writer_node` writes the decoded frames to `/dev/null` by default, so no output file is produced; set its `url` parameter to keep the decoded YUV.
    </Step>

    <Step title="Verify the ROS topics">
      In a second SSH terminal, list the topics and check the stream that the encoder or the decoder publishes.

      ```bash title="SSH Session" theme={null}
      source /usr/share/qirp-setup.sh
      export ROS_DOMAIN_ID=xx
      ros2 topic list
      ros2 topic hz /recording_ns/writer_node/compressed_image
      ros2 topic hz /playback_ns/writer_node/raw_image
      ```
    </Step>
  </Steps>
</Accordion>

## Build and run `qrb_ros_video`

<Accordion title="Try me">
  <Steps>
    <Step title="Clone qrb_ros_video on the host">
      ```bash theme={null}
      cd <qirp_decompressed_workspace>
      source setup.sh
      git clone -b stable/0.1.7 https://github.com/qualcomm-qrb-ros/qrb_ros_video.git
      ```
    </Step>

    <Step title="Build qrb_ros_video on the host">
      ```bash theme={null}
      cd qrb_ros_video
      colcon build --merge-install --cmake-args ${CMAKE_ARGS}
      ```
    </Step>

    <Step title="Push qrb_ros_video to the device">
      ```bash theme={null}
      cd qrb_ros_video/install
      tar czvf qrb_ros_video.tar.gz lib share include
      scp qrb_ros_video.tar.gz root@[ip-addr]:/opt/
      ssh root@[ip-addr]
      ```

      ```bash title="SSH Session" theme={null}
      tar --no-overwrite-dir --no-same-owner -zxf /opt/qrb_ros_video.tar.gz -C /usr/ros/jazzy/
      ```
    </Step>

    <Step title="Run qrb_ros_video on the device">
      Run `qrb_ros_video` on the device by following [Run out-of-the-box qrb\_ros\_video](#run-out-of-the-box-qrb_ros_video).
    </Step>
  </Steps>
</Accordion>

## Limitations

* Hardware-accelerated encoding and decoding require the Qualcomm VPU, so the video overlay must be enabled on the device.
* The supported codec formats are H.264 and H.265. The supported input pixel formats are NV12 and P010.