Skip to main content

​
Overview

qtibatch is a GStreamer element that aggregates multiple video or audio buffers into a single batched buffer for downstream processing. It is intended for multimedia and AI pipelines where downstream elements operate more efficiently on grouped input than on individual buffers. The element supports two batching modes:
  • Multi-stream batching — combines buffers arriving concurrently from multiple parallel streams
  • Sequential batching — combines successive buffers from a single stream
This makes qtibatch suitable for both parallel multi-stream workloads and temporal workflows that require multiple consecutive frames or samples as a single input unit. Batching is commonly used to:
  • Improve hardware utilization
  • Increase throughput for downstream inference or analytics stages
  • Enable models that require batched input
  • Construct temporal or depth-oriented input tensors from consecutive frames
  • Create overlapping batches for sliding-window style processing
Typical use cases include:
  • Multi-stream video batching — aggregating frames from multiple video sources into a single batch for parallel processing
  • Multi-stream audio batching — aggregating audio buffers from multiple sources into a single batch for parallel processing
  • Depth batching — grouping consecutive frames from the same stream to create tensors with an additional temporal or depth dimension
  • Overlapping batch construction — building batches with shared frames between adjacent batches for temporal models or window-based analysis
qtibatch performs buffer aggregation only. It does not modify video pixels or audio samples, and it does not perform inference, preprocessing, or post-processing. Its sole responsibility is to package multiple input buffers into a single logical batch for efficient downstream consumption.
qtibatch pipeline

​
Hierarchy

GObject
   GstObject
      GstElement
         qtibatch

​
Pad Templates

​
sink

Capabilities
video/x-rawformat: { ANY }
audio/x-rawformat: { ANY }
Availability: On request
Direction: sink
Pad Name: sink_%u

​
src

Capabilities
video/x-rawformat: { ANY }
note: batched buffers
audio/x-rawformat: { ANY }
note: batched buffers
Availability: Always
Direction: source

​
Element Properties

PropertyDescription
moving-window-sizeNumber of new buffers that will be used for output frames.

Type: Unsigned Integer
Default: 1
Range: 1 - 16
Flags: readable/writable (changeable only in NULL, READY)

​
Internal Architecture Details

qtibatch is organized around three functional components:

​
Sink Pads

qtibatch supports either a single sink pad or multiple sink pads, depending on the batching mode:
  • A single sink pad is used for sequential batching of successive buffers from the same stream
  • Multiple sink pads are used for parallel batching of buffers from multiple input streams
Each sink pad receives individual video or audio buffers from upstream elements.

​
Aggregation Logic

The aggregation logic collects incoming buffers, tracks their arrival status, and determines when a batch is ready to be produced. Batch formation is driven by the negotiated batching configuration and the active batching mode.

​
Source Pad

A single source pad emits the batched output buffer. The output buffer carries references to the aggregated input buffers together with batch metadata that describes the batch contents.

​
Processing Flow

​
Caps Negotiation

During caps negotiation, qtibatch verifies that all sink pads use compatible media (audio or video) types and formats. It also derives the effective batching configuration, including the batch size and the expected input/output tensor or media characteristics required for downstream processing.

​
Buffer Reception

Incoming video or audio buffers are received asynchronously on the sink pads and queued internally on a per-pad basis.

​
Synchronization

qtibatch synchronizes buffers according to the selected batching mode:
  • For parallel streams, the element waits for buffers that correspond to the same time interval across all active sink pads
  • For single stream, the element waits for the required number of successive buffers from the same input stream
For parallel batching, if a buffer does not arrive on a given pad within the expected time window, that pad is treated as missing for the current batch. For single-stream batching, timeout-based skipping is not applied.

​
Batch Formation

Buffers are grouped into a batch according to the selected batching strategy:
  • Successive buffers from a single stream
  • Time-aligned buffers from multiple streams
The resulting batch preserves buffer ordering and, where applicable, temporal alignment across inputs.

​
Metadata Attachment

qtibatch attaches batch metadata to the output buffer. This metadata describes:
  • The effective batch size
  • The ordering of aggregated buffers
  • Any missing inputs in parallel-stream mode

​
Output Delivery

Once the batch is complete, the batched buffer is pushed downstream for further processing.

​
Batching Behavior

Batch formation in qtibatch is determined by the negotiated batch size together with the active batching mode. Incoming buffers are always queued internally on a per-pad basis. The batch completion logic depends on whether the element is aggregating buffers from a single stream or from multiple parallel streams.

​
Sequential Batching (Single Stream)

In sequential mode, qtibatch aggregates successive buffers from a single input stream until the configured batch size is reached. Batch formation is count-based. This mode supports a moving-window mechanism controlled by the moving-window-size property:
  • A value of 0 produces non-overlapping batches
  • A value greater than 0 produces overlapping batches, reusing the specified number of buffers from the previous batch in the next batch
This behavior is useful for temporal inference workloads, such as action recognition or sequence-based models, where continuity across adjacent batches is important. Sequential batching is strictly driven by buffer count and does not use timeout-based GAP insertion.

​
Parallel Batching (Multiple Streams)

In parallel mode, qtibatch aggregates buffers arriving from multiple sink pads. The goal is to form a batch from buffers that belong to the same time interval across all input streams. If a buffer is not received on one of the sink pads within the expected synchronization window, that input is treated as missing for the current batch. In this case, the batch metadata records the missing input so downstream elements can interpret the batch correctly — this is done through inserting a GAP buffer. The synchronization window and timeout behavior are derived from buffer duration, which is calculated from the negotiated input framerate or audio timing information.

​
Memory and Buffer Management

qtibatch is designed to integrate into high-performance and zero-copy multimedia pipelines.

​
Buffer Handling

qtibatch does not copy media payloads during batch construction. Instead, the batched output buffer holds references to the original input buffers.

​
Allocation and Pool Behavior

qtibatch does not require a custom allocator, a specific memory type, or a dedicated buffer pool. Buffer allocation and memory layout remain under the control of upstream and downstream elements. This design allows qtibatch to fit into a wide range of pipeline topologies without imposing additional memory management constraints.

​
Usage

​
Four-Source Batched Object Detection with Compositor

Demo pipeline with batching of 4 sources. A detection inference is run and the results are overlaid via composer on the screen. use
1

Download Required Files

FileDownloadSave as
Yolov8 Detection W8A8 Batch 4 modelExport from Qualcomm AI Hubyolov8_det_w8a8_batch_4.tflite
Detection labelsyolov8.jsonyolov8.json
Sample videoInput videoDraw_1080p_180s_30FPS.mp4
2

Copy files to device

# Replace $HOME to the appropriate device path before running the commands.
# For QLI:    /root
# For Ubuntu: /home/ubuntu
# Modify this based on your platform and ensure files are copied to the correct location on the device.
# Run from your host machine — replace <user> and <device-ip>

ssh <user>@<device-ip> "mkdir -p $HOME/{models,labels,media,media/output}"
scp yolov8_det_w8a8_batch_4.tflite        <user>@<device-ip>:$HOME/models/
scp yolov8.json                           <user>@<device-ip>:$HOME/labels/
scp Draw_1080p_180s_30FPS.mp4             <user>@<device-ip>:$HOME/media/
3

Connect to device

# Run from your host machine — replace <user> and <device-ip>
ssh <user>@<device-ip>
4

Set environment variables

Run below command on your device
export MODEL_NAME=yolov8_det_w8a8_batch_4.tflite
export LABELS_NAME=yolov8.json
export SRC_VIDEO_NAME_1=Draw_1080p_180s_30FPS.mp4
export SRC_VIDEO_NAME_2=Draw_1080p_180s_30FPS.mp4
export SRC_VIDEO_NAME_3=Draw_1080p_180s_30FPS.mp4
export SRC_VIDEO_NAME_4=Draw_1080p_180s_30FPS.mp4
5

Run the pipeline

Run the pipeline
gst-launch-1.0 -e --gst-debug=2 \
qtimltflite name=inference delegate=external external-delegate-path=libQnnTFLiteDelegate.so external-delegate-options="QNNExternalDelegate,backend_type=htp,htp_performance_mode=(string)2;" model=$HOME/models/$MODEL_NAME \
filesrc location=$HOME/media/$SRC_VIDEO_NAME_1 ! qtdemux ! queue ! h264parse ! v4l2h264dec capture-io-mode=4 output-io-mode=4 ! video/x-raw ! qtivtransform ! video/x-raw,format=NV12,width=640,height=360 ! queue ! tee name=tee_0 \
filesrc location=$HOME/media/$SRC_VIDEO_NAME_2 ! qtdemux ! queue ! h264parse ! v4l2h264dec capture-io-mode=4 output-io-mode=4 ! video/x-raw ! qtivtransform ! video/x-raw,format=NV12,width=640,height=360 ! queue ! tee name=tee_1 \
filesrc location=$HOME/media/$SRC_VIDEO_NAME_3 ! qtdemux ! queue ! h264parse ! v4l2h264dec capture-io-mode=4 output-io-mode=4 ! video/x-raw ! qtivtransform ! video/x-raw,format=NV12,width=640,height=360 ! queue ! tee name=tee_2 \
filesrc location=$HOME/media/$SRC_VIDEO_NAME_4 ! qtdemux ! queue ! h264parse ! v4l2h264dec capture-io-mode=4 output-io-mode=4 ! video/x-raw ! qtivtransform ! video/x-raw,format=NV12,width=640,height=360 ! queue ! tee name=tee_3 \
tee_0. ! video/x-raw,format=NV12 ! mixer. \
tee_0. ! video/x-raw,format=NV12 ! batch. \
tee_1. ! video/x-raw,format=NV12 ! mixer. \
tee_1. ! video/x-raw,format=NV12 ! batch. \
tee_2. ! video/x-raw,format=NV12 ! mixer. \
tee_2. ! video/x-raw,format=NV12 ! batch. \
tee_3. ! video/x-raw,format=NV12 ! mixer. \
tee_3. ! video/x-raw,format=NV12 ! batch. \
qtibatch name=batch ! queue ! qtimlvconverter ! queue ! inference. inference. ! queue ! qtimldemux name=mldemux \
mldemux. ! queue ! qtimlpostprocess results=10 module=yolov8 labels=$HOME/labels/$LABELS_NAME settings="{\"confidence\": 70.0}" ! video/x-raw,width=640,height=360 ! queue ! mixer. \
mldemux. ! queue ! qtimlpostprocess results=10 module=yolov8 labels=$HOME/labels/$LABELS_NAME settings="{\"confidence\": 70.0}" ! video/x-raw,width=640,height=360 ! queue ! mixer. \
mldemux. ! queue ! qtimlpostprocess results=10 module=yolov8 labels=$HOME/labels/$LABELS_NAME settings="{\"confidence\": 70.0}" ! video/x-raw,width=640,height=360 ! queue ! mixer. \
mldemux. ! queue ! qtimlpostprocess results=10 module=yolov8 labels=$HOME/labels/$LABELS_NAME settings="{\"confidence\": 70.0}" ! video/x-raw,width=640,height=360 ! queue ! mixer. \
qtivcomposer name=mixer \
sink_0::position="<0, 0>" sink_0::dimensions="<960, 540>" \
sink_1::position="<960,  0>" sink_1::dimensions="<960, 540>" \
sink_2::position="<0, 540>" sink_2::dimensions="<960, 540>" \
sink_3::position="<960, 540>" sink_3::dimensions="<960, 540>" \
sink_4::position="<0, 0>" sink_4::dimensions="<960, 540>" \
sink_5::position="<960, 0>" sink_5::dimensions="<960, 540>" \
sink_6::position="<0, 540>" sink_6::dimensions="<960, 540>" \
sink_7::position="<960, 540>" sink_7::dimensions="<960, 540>" \
mixer. ! video/x-raw,format=NV12 ! queue ! waylandsink sync=false fullscreen=true