> ## Documentation Index
> Fetch the complete documentation index at: https://dragonwingdocs.qualcomm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# qtirtspbin

> A GstBin-derived sink element that encapsulates RTSP streaming functionality by integrating GStreamer RTSP Server APIs.

# Overview

`qtirtspbin` is a GStreamer bin element that simplifies the construction and management of **RTSP streaming pipelines** for applications that need to publish media over a network. It operates as an **RTSP server**, exposing a local streaming endpoint that RTSP-compatible clients, such as VLC, can connect to in order to receive live or processed media streams.

Rather than functioning as a client that consumes an external RTSP source, `qtirtspbin` provides a server-side streaming interface that allows an application to distribute its output to one or more downstream consumers. This makes it well suited for use cases where media, analytics results, or other generated data must be shared across the network in real time.

The element supports a variety of payload types, including:

* **Encoded video streams** such as H.264 and H.265
* **Audio streams**
* **Machine learning metadata**
* **Serialized custom data**

`qtirtspbin`  can handle multiple input types and automatically assembles the internal pipeline required for stream packetization, transport, and RTSP session handling. By managing these lower-level details internally, it reduces the amount of application-side logic needed to build a complete streaming solution.

In addition to basic RTSP transmission, `qtirtspbin` includes built-in mechanisms for:

* **Session Management**: supports multiple simultaneous client connections and manages each RTSP session
* **Stream synchronization**: preserves consistent timing across video, audio, and metadata streams
* **Custom payload support**: enables transmission of non-standard data types by encapsulating them in RTP payloads

Overall, `qtirtspbin` provides a flexible and developer-friendly abstraction for building RTSP-based streaming applications that need to deliver synchronized media and auxiliary data to network clients.

<img src="https://mintcdn.com/qualcomm-prod/Q99GUO8_W1Zdqsi7/SDKs/IMSDK/plugin-reference/images/rtspbin_internal_architecture.png?fit=max&auto=format&n=Q99GUO8_W1Zdqsi7&q=85&s=e220e6e50d6322eb21c9ef8059ae64e2" alt="" width="1221" height="681" data-path="SDKs/IMSDK/plugin-reference/images/rtspbin_internal_architecture.png" />

## Example Pipeline

<img src="https://mintcdn.com/qualcomm-prod/Q99GUO8_W1Zdqsi7/SDKs/IMSDK/plugin-reference/images/rtspbin_example_pipeline.png?fit=max&auto=format&n=Q99GUO8_W1Zdqsi7&q=85&s=890bc29243fc0f972f17f6b1357c442d" alt="" width="713" height="71" data-path="SDKs/IMSDK/plugin-reference/images/rtspbin_example_pipeline.png" />

<Steps>
  <Step title="Download Required Files">
    | File             | Download                                                                                                                            | Save as                     |
    | ---------------- | ----------------------------------------------------------------------------------------------------------------------------------- | --------------------------- |
    | Sample video     | <a href="https://github.com/qualcomm/sample-apps-for-qualcomm-linux/raw/refs/heads/main/artifacts/videos/video.mp4">Input video</a> | `Draw_1080p_180s_30FPS.mp4` |
    | YOLOX W8A8 model | [Qualcomm AI Hub — YOLOX](https://aihub.qualcomm.com/iot/models/yolox)                                                              | `yolo_x_w8a8.tflite`        |
    | Detection labels | <a href="../labels/yolov8.json" download="yolov8.json">yolov8.json</a>                                                              | `yolov8.json`               |
  </Step>

  <Step title="Copy files to device">
    <CodeGroup>
      ```bash SCP (SSH) theme={null}
      # Replace $HOME to the appropriate device path before running the commands.
      # For QLI:    /root
      # For Ubuntu: /home/ubuntu
      # Modify this based on your platform and ensure files are copied to the correct location on the device.
      # Run from your host machine — replace <user> and <device-ip>

      ssh <user>@<device-ip> "mkdir -p $HOME/{models,labels,media,media/output}"
      scp Draw_1080p_180s_30FPS.mp4   <user>@<device-ip>:$HOME/media/
      scp yolo_x_w8a8.tflite          <user>@<device-ip>:$HOME/models/
      scp yolov8.json                  <user>@<device-ip>:$HOME/labels/
      ```
    </CodeGroup>
  </Step>

  <Step title="Connect to device">
    <CodeGroup>
      ```bash SCP (SSH) theme={null}
      # Run from your host machine — replace <user> and <device-ip>
      ssh <user>@<device-ip>
      ```
    </CodeGroup>
  </Step>

  <Step title="Set environment variables">
    Run below command on your device

    ```bash theme={null}
    export SRC_VIDEO_NAME=Draw_1080p_180s_30FPS.mp4
    export MODEL_NAME=yolo_x_w8a8.tflite
    export LABELS_NAME=yolov8.json
    ```
  </Step>

  <Step title="Run the pipeline">
    ```bash theme={null}
    gst-launch-1.0 -e --gst-debug=2 \
    qticamsrc ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! \
    v4l2h264enc capture-io-mode=4 output-io-mode=4 ! queue ! \
    h264parse config-interval=1 ! queue ! qtirtspbin address=<ip addr of device>

    Connect to the stream with VLC: rtsp://<ip addr of device>:8900/live
    ```
  </Step>
</Steps>

<Note>
  Ensure that the device has to be connected to network to try out these example pipelines.
</Note>

# Hierarchy

[GObject](https://docs.gtk.org/gobject/)<br />
   <Icon icon="arrow-turn-down-right" iconType="solid" />[GstObject](https://gstreamer.freedesktop.org/documentation/gstreamer/gstobject.html?gi-language=c)<br />
      <Icon icon="arrow-turn-down-right" iconType="solid" />[GstElement](https://gstreamer.freedesktop.org/documentation/gstreamer/gstelement.html?gi-language=c)<br />
         <Icon icon="arrow-turn-down-right" iconType="solid" />[GstBin](https://gstreamer.freedesktop.org/documentation/base/gstbasetransform.html?gi-language=c)<br />
            <Icon icon="arrow-turn-down-right" iconType="solid" />qtirtspbin

# Pad Templates

### sink

| Capabilities               |              |
| -------------------------- | ------------ |
| `video/x-h264`             | `format: NA` |
| `video/x-h265`             | `format: NA` |
| `audio/mpeg`               | `format: NA` |
| `text/x-raw`               | `format: NA` |
| Availability: *On request* |              |
| Direction: *sink*          |              |
| Pad Name: `sink_%u`        |              |

# Element Properties

| Property  | Description                                                                                                                                                                                                                                                                               |
| --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `address` | IP address of the server.<br /><br />`Type: String`<br />`Default: "127.0.0.1"`<br />`Flags: readable/writable`                                                                                                                                                                           |
| `mode`    | Operational mode.<br /><br />`Type: Enum `<br />`Default: 0, "async"`<br />`Range:`<br />    `(0): async - Return buffers immediately if no client is connected`<br />    `(1): sync - Restricts the number of input buffers`<br />`Flags: readable/writable (changeable in NULL, READY)` |
| `mpoint`  | Mounting point.<br /><br />`Type: String`<br />`Default: "/live"`<br />`Flags: readable/writable`                                                                                                                                                                                         |
| `port`    | Port to listen on.<br /><br />`Type: String`<br />`Default: "8900"`<br />`Flags: readable/writable`                                                                                                                                                                                       |

# Internal Architecture and Streaming model:

`qtirtspbin` is a GStreamer bin element that internally manages the components required to transmit media streams over RTSP. Unlike an RTSP client, it operates as an RTSP server and exposes a mount point that external clients can connect to for receiving the stream

## Core components

Internally, `qtirtspbin` dynamically constructs a pipeline that includes the following elements:

### Input pads

The plugin accepts multiple types of input through its pads, including:

* **Encoded video**, such as H.264 and H.265
* **Audio Streams**
* **ML metadata**, such as JSON
* **Custom serialized data**

### RTP Packetizer

Incoming media and data are converted into RTP packets using the appropriate payloaders, such as:

* `rtph264pay` and `rtph265pay` for video
* `rtpmp4apay` for audio
* `rtpgstpay` for metadata or other structured custom data

### RTSP Server Interface

`qtirtspbin` launches an internal RTSP server, typically on `127.0.0.1:8900`, and registers one or more mount points such as `/live` or `/stream`. Through this interface, it manages:

* Client sessions
* Stream negotiation
* Reconnection behavior

## Processing flow

1. **Media Input**
   * External elements push encoded media or data into `qtirtspbin` through request pads

2. **Stream Identification and Routing**
   * Each incoming stream is identified
   * The stream is routed to the appropriate RTP payloader.

3. **Packetization**
   * Media is packetized into RTP format.
   * Synchronization across streams is handled internally.

4. **Session Management**
   * The plugin registers the stream with the RTSP server.
   * Clients connect using RTSP and receive the live stream.

5. **Client Handling**
   * Multiple clients can connect at the same time.
   * The plugin manages buffering, latency, and session state.

## Custom payload support

In addition to standard media types such as video and audio, `qtirtspbin` can also transport **custom data streams** over RTSP by encapsulating them in RTP payloads.

This is useful for applications that need to deliver non-standard or structured information, such as:

* **Machine learning inference results** like bounding boxes, labels, and scores
* **Sensor data** such as telemetry or depth information
* **Application-specific metadata** such as JSON or other serialized formats

For custom payloads, the flow is typically:

**Input**

Custom data is pushed into `qtirtspbin` through a request pad.

**Payloading**

The plugin uses `rtpgstpay` to encapsulate the incoming buffer into RTP packets.

**Transmission**

The RTP stream is published through the internal RTSP server on the configured mount point.

**Client Consumption**

Clients such as VLC, FFmpeg, or custom RTSP receivers can consume the stream and deserialize the payload based on the agreed format.

# Usage

<Note>
  Ensure you have followed the [prerequisites](qtirtspbin#example-pipeline) before continuing
</Note>

### Single stream from camera to the RTSP

Demonstrates real-time RTSP stream reception from live camera source.

<img src="https://mintcdn.com/qualcomm-prod/Q99GUO8_W1Zdqsi7/SDKs/IMSDK/plugin-reference/images/rtspbin_ex1.png?fit=max&auto=format&n=Q99GUO8_W1Zdqsi7&q=85&s=aec6818e11b9a99a6c35b71bbfc1ef91" alt="" width="713" height="71" data-path="SDKs/IMSDK/plugin-reference/images/rtspbin_ex1.png" />

```bash theme={null}
gst-launch-1.0 -e --gst-debug=2 \
qticamsrc ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! \
v4l2h264enc capture-io-mode=4 output-io-mode=4 ! queue ! \
h264parse config-interval=1 ! queue ! qtirtspbin address=<ip addr of device>

Connect to the stream with VLC: rtsp://<ip addr of device>:8900/live
```

### Two streams from camera to RTSP

Demonstrates usage of rtspbin to stream two different live feeds. Each `qtirtspbin` has internal RTSP server and the client should connect to it. When creating an unique port should be set to each instance of the plugin.

<img src="https://mintcdn.com/qualcomm-prod/Q99GUO8_W1Zdqsi7/SDKs/IMSDK/plugin-reference/images/rtspbin_ex2.png?fit=max&auto=format&n=Q99GUO8_W1Zdqsi7&q=85&s=134b2fe4f204ea6df0e579610d512e27" alt="" width="868" height="192" data-path="SDKs/IMSDK/plugin-reference/images/rtspbin_ex2.png" />

```bash theme={null}
gst-launch-1.0 -e qticamsrc ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! tee name=camsrc ! v4l2h264enc capture-io-mode=4 output-io-mode=4 ! queue ! h264parse config-interval=1 ! queue ! qtirtspbin address=<ip addr of device> port=8900 \
camsrc. ! v4l2h264enc capture-io-mode=4 output-io-mode=4 ! queue ! h264parse config-interval=1 ! queue ! qtirtspbin address=<ip addr of device> port=8901

Connect to the stream with VLC: rtsp://<ip addr of device>:8900/live
Connect to the stream with VLC: rtsp://<ip addr of device>:8901/live
```

### Single stream from camera to the RTSP with ML detection

Demonstrates streaming of live camera stream and ML detection metadata on two request pads of the `qtirtspbin`. The video and ML metadata are received on the other pipeline and muxed together to overlay the objects detected and display the bounding boxes.

<img src="https://mintcdn.com/qualcomm-prod/Q99GUO8_W1Zdqsi7/SDKs/IMSDK/plugin-reference/images/rtspbin_ex3.png?fit=max&auto=format&n=Q99GUO8_W1Zdqsi7&q=85&s=42f847a77bcd4c8396b1ea62d683a2c6" alt="" width="1151" height="191" data-path="SDKs/IMSDK/plugin-reference/images/rtspbin_ex3.png" />

```bash theme={null}
#Console 1:
gst-launch-1.0 -e qticamsrc ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! \
queue ! tee name=split ! \
  v4l2h264enc capture-io-mode=4 output-io-mode=4 ! queue ! \
  h264parse config-interval=-1 ! queue ! qtirtspbin name=rtsp address=<ip addr of device> \
split. ! queue ! qtimlvconverter ! queue ! \
  qtimltflite delegate=external external-delegate-path=libQnnTFLiteDelegate.so external-delegate-options="QNNExternalDelegate,backend_type=htp;" model=$HOME/models/$MODEL_NAME ! queue ! \
  qtimlpostprocess results=5 module=yolov8 labels=$HOME/labels/$LABELS_NAME settings="{\"confidence\": 75.0}" ! text/x-raw ! queue ! rtsp.

#Console 2:
gst-launch-1.0 -e rtspsrc location=rtsp://<ip addr of device>:8900/live ! rtph264depay ! video/x-h264,colorimetry=bt709 ! h264parse ! v4l2h264dec capture-io-mode=4 output-io-mode=4 ! video/x-raw,format=NV12 ! queue ! qtimetamux name=metamux ! queue ! qtivoverlay ! waylandsink fullscreen=true async=true sync=false rtspsrc location=rtsp://<ip addr of device>:8900/live ! rtpgstdepay ! text/x-raw ! queue ! metamux.
```