Overview

v4l2vp9dec is a hardware-accelerated video decoder that uses the Video4Linux2 (V4L2) stateful decoder API to offload VP9 video decoding to the Qualcomm Video Processing Unit (VPU). This plugin is provided and maintained by the GStreamer community. This document focuses on its usage in conjunction with Qualcomm-specific QIM SDK GStreamer plugins, along with relevant use cases and internal architectural considerations. v4l2vp9dec is typically used to decode VP9 streams provided by:

File sources (WebM, Matroska/MKV containers)
RTSP sources
HLS/HTTP streaming sources (e.g., YouTube, web video)

The element is responsible only for decoding video frames. It does not demux or parse streams on its own — those operations must be handled by peer GStreamer elements upstream (e.g., matroskademux).

VP9 frames are self-contained and self-framing. Unlike H.264/H.265, no separate parse element (e.g., vp9parse) is required between the demuxer and v4l2vp9dec.

Example Pipeline

Download Required Files

vp9_1080p.webm file is used here as an example.

Copy files to device

# Replace $HOME to the appropriate device path before running the commands.
# For QLI:    /root
# For Ubuntu: /home/ubuntu
# Modify this based on your platform and ensure files are copied to the correct location on the device.
# Run from your host machine — replace <user> and <device-ip>

ssh <user>@<device-ip> "mkdir -p $HOME/{media}"
scp vp9_1080p.webm <user>@<device-ip>:$HOME/media/

Connect to device

# Run from your host machine — replace <user> and <device-ip>
ssh <user>@<device-ip>

Set environment variables

Run below command on your device

export SRC_VIDEO_NAME=vp9_1080p.webm

Run the pipeline

gst-launch-1.0 -e \
  filesrc location=$HOME/media/$SRC_VIDEO_NAME ! \
  matroskademux ! queue ! \
  v4l2vp9dec capture-io-mode=4 output-io-mode=4 ! \
  video/x-raw,format=NV12 ! queue ! \
  waylandsink fullscreen=true sync=false

Key Responsibilities

v4l2vp9dec is responsible for:

Hardware acceleration — offloads VP9 decoding to the dedicated VPU
V4L2 state management — manages the V4L2 stateful decoder state machine (device open, buffer allocation, stream on/off)
Buffer I/O management — handles buffer exchange between GStreamer and the V4L2 driver using DMABuf, MMAP, or UserPtr modes
Format negotiation — negotiates raw output formats supported by the hardware, including UBWC-compressed formats (NV12_Q08C, NV12_Q10LE32C) for reduced memory bandwidth
Multi-stream support — supports multiple concurrent decoder instances (subject to hardware resource limits)
Error handling — supports decoder error controls such as max-errors and corrupted-frame discard behavior

Hierarchy

GObject
   GstObject
      GstElement
         GstVideoDecoder
            GstV4l2VideoDec
               v4l2vp9dec

Pad Templates

sink

Capabilities
`video/x-vp9`	`format: NA`
Availability: Always
Direction: sink

src

Capabilities
`video/x-raw`	`format: { NV12_Q08C, NV12_Q10LE32C, NV12, NV21 }` `width: [1, 32768]` `height: [1, 32768]`
Availability: Always
Direction: source

The exact set of profiles reported depends on what the underlying V4L2 driver enumerates via V4L2_CID_MPEG_VIDEO_VP9_PROFILE (or V4L2_CID_MPEG_VIDEO_VPX_PROFILE). VP9 does not define decoder levels in the V4L2 API; therefore, no level probing is performed.

Element Properties

Property	Description
`automatic-request-sync-point-flags`	Flags used when automatically requesting sync points. `Type: Flags` `Default: 0x3 (corrupt-output+discard-input)` `Flags: readable/writable`
`automatic-request-sync-points`	Automatically requests sync points (keyframes) when useful, such as after decoding errors. `Type: Boolean` `Default: false` `Flags: readable/writable`
`capture-io-mode`	I/O mode for the capture queue (src pad). Controls how decoded frame buffers are allocated and transferred. `Type: Enum` `Default: 0, "auto"` `Flags: readable/writable`
`device`	The V4L2 device node path. Set automatically at registration time. Read-only after element creation. `Type: String` `Default: "/dev/video32"` `Flags: readable`
`device-fd`	File descriptor of the opened V4L2 device. A value of `-1` indicates that the device is not open. `Type: Integer` `Default: -1` `Flags: readable`
`device-name`	Human-readable name of the V4L2 device as reported by the driver via VIDIOC_QUERYCAP. `Type: String` `Default: NULL` `Flags: readable`
`discard-corrupted-frames`	When enabled, frames marked as corrupted by the driver are dropped instead of being passed downstream. `Type: Boolean` `Default: false` `Flags: readable/writable`
`extra-controls`	Extra V4L2 controls (CIDs) to set on the device, specified as a GstStructure. Applied via VIDIOC_S_EXT_CTRLS. `Type: GstStructure` `Default: NULL` `Flags: readable/writable`
`max-errors`	Maximum number of consecutive decoder errors before the element returns a flow error. A value of `-1` means unlimited. `Type: Integer` `Default: -1` `Flags: readable/writable`
`min-force-key-unit-interval`	Minimum interval in nanoseconds between force-keyunit requests sent upstream. A value of `0` means no minimum interval. `Type: Unsigned Integer64` `Default: 0` `Flags: readable/writable`
`output-io-mode`	I/O mode for the output queue (sink pad). Controls how compressed input buffers are submitted to the driver. `Type: Enum` `Default: 0, "auto"` `Flags: readable/writable`
`qos`	Handles Quality-of-Service events from downstream. When enabled, the decoder may drop frames to maintain real-time playback. `Type: Boolean` `Default: true` `Flags: readable/writable`

I/O Mode Values

Both capture-io-mode and output-io-mode accept the same GstV4l2IOMode enumeration:

Value	Integer	Description
`auto`	0	Automatically select the best I/O mode. Prefers DMABuf export for capture, MMAP for output.
`rw`	1	Use read/write system calls. Rarely used for M2M devices.
`mmap`	2	Use kernel memory-mapped buffers (`V4L2_MEMORY_MMAP`).
`userptr`	3	Use user-space pointer buffers (`V4L2_MEMORY_USERPTR`).
`dmabuf`	4	Export buffers as DMABuf file descriptors (`V4L2_MEMORY_DMABUF`). Enables zero-copy sharing with downstream GPU/display elements.
`dmabuf-import`	5	Import DMABuf file descriptors from an external allocator into the V4L2 queue.

Internal Architecture

v4l2vp9dec operates using two V4L2 queue objects internally:

Output queue (V4L2_BUF_TYPE_VIDEO_OUTPUT) — receives compressed VP9 frame buffers from upstream
Capture queue (V4L2_BUF_TYPE_VIDEO_CAPTURE) — produces decoded raw video frames for downstream

State Transitions

Transition	Decoder Behavior	Notes
`NULL → READY`	Element becomes ready. Static configuration available.	Configure `capture-io-mode` and `output-io-mode` before active decoding.
`READY → PAUSED`	Decoder activates. V4L2 session setup begins.	Caps negotiation: sink is `video/x-vp9`, src is `video/x-raw` (NV12/NV12_Q08C). VP9 frames from WebM/MKV are self-framing; no separate parse element is required.
`PAUSED → PLAYING`	Continuous decode starts.	With `capture-io-mode=4` and `output-io-mode=4`, pipeline is configured for DMABuf zero-copy.
`PLAYING → PAUSED`	Decoding paused at GStreamer scheduling level.	No full decoder teardown. Resume continues from current position.
`PLAYING/PAUSED → READY`	Decoder stops. V4L2 streaming stopped, buffers released.	Key cleanup transition.
`READY → NULL`	Element fully shut down. All resources released.
EOS while PLAYING	Decoder drains pending input, outputs remaining frames, then forwards EOS.
FLUSH	Current decode interrupted. Queued buffers discarded.	Common during seek or pipeline reset. Decoder waits for new VP9 keyframe after flush.

Dynamic Resolution Change

v4l2vp9dec supports mid-stream dynamic resolution changes without requiring a pipeline restart, handled through the V4L2 source change event mechanism:

At initialization, the decoder subscribes to V4L2_EVENT_SOURCE_CHANGE events on the capture queue.
When the driver detects a resolution change in the bitstream, it signals the event.
The decoder stops the capture queue, discards the existing buffer pool, re-negotiates format via VIDIOC_G_FMT, sets a new output state with updated dimensions, and reallocates the capture buffer pool.

This allows seamless decoding of adaptive bitrate VP9 streams and content with embedded resolution changes.

Memory and Buffer Management

DMABuf Usage

Setting capture-io-mode=4 exports decoded frame buffers as DMABuf file descriptors. These can be imported directly by waylandsink or qtivcomposer without CPU memory copies, enabling zero-copy pipelines.

Alignment Requirements

Decoded buffers follow Qualcomm hardware alignment requirements (e.g., 128-byte stride alignment).

Format Support

Standard NV12 is common. UBWC (Universal Bandwidth Compression) is supported through formats like NV12_Q08C to reduce memory bandwidth.

Codec Header Initialization

VP9 is a frame-based codec — each VP9 frame is self-contained and does not require a separate out-of-band codec header. The first input buffer is used directly as the initialization data. If the input caps contain codec_data (e.g., VP9 codec private data from a WebM container), that data is sent first.

Latency

The decoder computes and reports pipeline latency based on the minimum number of capture buffers required by the driver and the frame duration:

latency = min_buffers × frame_duration

Drain and Flush

Drain — sends V4L2_DEC_CMD_STOP to signal end-of-stream; waits for all remaining frames to be produced
Flush — stops both queues, resets buffer pools, and restarts streaming; used during seek operations

Usage

Ensure you have followed the prerequisites before continuing

Decode VP9 from a WebM File and Display

Demonstrates hardware-accelerated VP9 decoding from a WebM file with DMABuf zero-copy for Wayland preview.

gst-launch-1.0 -e \
  filesrc location=$HOME/media/$SRC_VIDEO_NAME ! \
  matroskademux ! queue ! \
  v4l2vp9dec output-io-mode=dmabuf capture-io-mode=dmabuf ! \
  video/x-raw,format=NV12 ! queue ! \
  waylandsink fullscreen=true

Four-Stream Side-by-Side Composition

Demonstrates decoding four VP9 input streams using four hardware decoder instances, arranging them in a 2×2 grid using qtivcomposer, and displaying the composed frame.

gst-launch-1.0 -e --gst-debug=2 \
  qtivcomposer name=mixer \
    sink_0::position="<0, 0>"     sink_0::dimensions="<960, 540>" \
    sink_1::position="<960, 0>"   sink_1::dimensions="<960, 540>" \
    sink_2::position="<0, 540>"   sink_2::dimensions="<960, 540>" \
    sink_3::position="<960, 540>" sink_3::dimensions="<960, 540>" \
  ! queue ! waylandsink fullscreen=true sync=false enable-last-sample=false \
  filesrc location=$HOME/media/$SRC_VIDEO_NAME ! \
    matroskademux ! queue ! v4l2vp9dec output-io-mode=4 capture-io-mode=4 ! queue ! mixer. \
  filesrc location=$HOME/media/$SRC_VIDEO_NAME ! \
    matroskademux ! queue ! v4l2vp9dec output-io-mode=4 capture-io-mode=4 ! queue ! mixer. \
  filesrc location=$HOME/media/$SRC_VIDEO_NAME ! \
    matroskademux ! queue ! v4l2vp9dec output-io-mode=4 capture-io-mode=4 ! queue ! mixer. \
  filesrc location=$HOME/media/$SRC_VIDEO_NAME ! \
    matroskademux ! queue ! v4l2vp9dec output-io-mode=4 capture-io-mode=4 ! queue ! mixer.

​Overview

​Example Pipeline

​Key Responsibilities

​Hierarchy

​Pad Templates

​sink

​src

​Element Properties

​I/O Mode Values

​Internal Architecture

​State Transitions

​Dynamic Resolution Change

​Memory and Buffer Management

​DMABuf Usage

​Alignment Requirements

​Format Support

​Codec Header Initialization

​Latency

​Drain and Flush

​Usage

​Decode VP9 from a WebM File and Display

​Four-Stream Side-by-Side Composition

Overview

Example Pipeline

Key Responsibilities

Hierarchy

Pad Templates

sink

src

Element Properties

I/O Mode Values

Internal Architecture

State Transitions

Dynamic Resolution Change

Memory and Buffer Management

DMABuf Usage

Alignment Requirements

Format Support

Codec Header Initialization

Latency

Drain and Flush

Usage

Decode VP9 from a WebM File and Display

Four-Stream Side-by-Side Composition