Overview

v4l2h264enc is a hardware-accelerated video encoder that uses the Video4Linux2 (V4L2) stateful encoder API to offload H.264 (AVC) video encoding to the Qualcomm Video Processing Unit (VPU). This plugin is provided and maintained by the GStreamer community. This document focuses on its usage in conjunction with Qualcomm-specific QIM SDK GStreamer plugins, along with relevant use cases and internal architectural considerations. v4l2h264enc is typically used in encoding pipelines for:

Camera capture and local recording
Video transcoding (e.g., H.265 → H.264)
Live streaming (RTSP, HLS, UDP)
Multi-stream simultaneous encoding

The element is responsible only for encoding raw video frames. It does not mux or parse streams on its own — those operations must be handled by peer GStreamer elements downstream (e.g., h264parse, mp4mux).

Example Pipeline

Set environment variables

Run below command on your device

mkdir -p $HOME/{media,media/output}

Connect to device

# Run from your host machine — replace <user> and <device-ip>
ssh <user>@<device-ip>

Run the pipeline

gst-launch-1.0 -e \
  qticamsrc ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! \
  queue ! v4l2h264enc output-io-mode=dmabuf-import capture-io-mode=dmabuf ! \
  queue ! h264parse ! mp4mux ! \
  filesink location=$HOME/media/output/output.mp4

Key Responsibilities

v4l2h264enc is responsible for:

Hardware acceleration — offloads H.264/AVC encoding to the dedicated VPU
V4L2 state management — manages the V4L2 stateful encoder state machine (device open, buffer allocation, stream on/off)
Buffer I/O management — handles buffer exchange between GStreamer and the V4L2 driver using DMABuf, MMAP, or UserPtr modes
Format negotiation — negotiates raw input formats supported by the hardware, including UBWC-compressed formats for reduced memory bandwidth
Profile and level selection — negotiates H.264 profile and level with downstream elements based on hardware capabilities probed at registration time
Encoder parameter control — supports runtime configuration of encoding parameters (bitrate, GOP size, QP values, entropy mode, etc.) via the extra-controls property
Multi-stream support — supports multiple concurrent encoder instances (subject to hardware resource limits)

Hierarchy

GObject
   GstObject
      GstElement
         GstVideoEncoder
            GstV4l2VideoEnc
               v4l2h264enc

Pad Templates

sink

Capabilities
`video/x-raw`	`format: { NV12_Q08C, NV12_Q10LE32C, NV12, NV21 }` `width: [1, 32768]` `height: [1, 32768]`
Availability: Always
Direction: sink

src

Capabilities
`video/x-h264`	`stream-format: byte-stream` `alignment: au` `profiles: baseline, main, high, high-10, high-4:2:2, and more` `levels: 1–6.2`
Availability: Always
Direction: source

The exact set of profiles and levels reported depends on what the underlying V4L2 driver enumerates via V4L2_CID_MPEG_VIDEO_H264_PROFILE and V4L2_CID_MPEG_VIDEO_H264_LEVEL.

Element Properties

Property	Description
`capture-io-mode`	I/O mode for the capture queue (src pad). Controls how compressed output buffers are received from the driver. `Type: Enum` `Default: 0, "auto"` `Flags: readable/writable`
`device`	The V4L2 device node path. Set automatically at registration time and read-only after element creation. `Type: String` `Default: "/dev/video32"` `Flags: readable`
`device-fd`	File descriptor of the opened V4L2 device. A value of `-1` indicates that the device is not open. `Type: Integer` `Default: -1` `Flags: readable`
`device-name`	Human-readable name of the V4L2 device as reported by the driver via VIDIOC_QUERYCAP. `Type: String` `Default: NULL` `Flags: readable`
`extra-controls`	Extra V4L2 controls (CIDs) to set on the device, specified as a GstStructure. Applied via VIDIOC_S_EXT_CTRLS and used to configure encoding parameters at runtime. `Type: GstStructure` `Default: NULL` `Flags: readable/writable`
`output-io-mode`	I/O mode for the output queue (sink pad). Controls how raw input buffers are submitted to the driver. `Type: Enum` `Default: 0, "auto"` `Flags: readable/writable`

I/O Mode Values

Both capture-io-mode and output-io-mode accept the same GstV4l2IOMode enumeration:

Value	Integer	Description
`auto`	0	Automatically select the best I/O mode.
`rw`	1	Use read/write system calls. Rarely used for M2M devices.
`mmap`	2	Use kernel memory-mapped buffers (`V4L2_MEMORY_MMAP`).
`userptr`	3	Use user-space pointer buffers (`V4L2_MEMORY_USERPTR`).
`dmabuf`	4	Export buffers as DMABuf file descriptors (`V4L2_MEMORY_DMABUF`). Use for `capture-io-mode` to export encoded output.
`dmabuf-import`	5	Import DMABuf file descriptors from an external allocator. Use for `output-io-mode` when upstream (camera, decoder) provides DMABuf handles for zero-copy input.

Internal Architecture

v4l2h264enc operates using two V4L2 queue objects internally:

Output queue (V4L2_BUF_TYPE_VIDEO_OUTPUT) — receives raw video frame buffers from upstream
Capture queue (V4L2_BUF_TYPE_VIDEO_CAPTURE) — produces encoded H.264 bitstream buffers for downstream

State Transitions

Transition	Encoder Behavior	Notes
`NULL → READY`	Element becomes ready. Static configuration available.	Configure `capture-io-mode`, `output-io-mode`, and `extra-controls` before active encoding.
`READY → PAUSED`	Encoder activates. V4L2 session setup begins.	Caps negotiation: sink is `video/x-raw`, src is `video/x-h264`. Profile and level are negotiated with downstream.
`PAUSED → PLAYING`	Continuous encoding starts. Raw frames consumed, H.264 AUs pushed downstream.	Normal running state.
`PLAYING → PAUSED`	Encoding paused at GStreamer scheduling level.	No full encoder teardown. Resume continues from current position.
`PLAYING/PAUSED → READY`	Encoder stops. V4L2 streaming stopped, buffers released.	Key cleanup transition for stopping recording or preparing for reuse.
`READY → NULL`	Element fully shut down. All resources released.
EOS while PLAYING	Encoder drains pending raw input, outputs remaining encoded frames, then forwards EOS.	Sends `V4L2_ENC_CMD_STOP` via `VIDIOC_ENCODER_CMD`.
FLUSH	Current encode interrupted. Queued buffers discarded.	Common during seek or pipeline reset. Encoder waits for new valid input after flush.

Memory and Buffer Management

DMABuf Usage

For zero-copy pipelines where raw frames originate from a hardware source (camera, GPU, or hardware decoder):

Set output-io-mode=dmabuf-import (value 5) to import DMABuf handles from upstream directly into the V4L2 output queue without CPU copies
Set capture-io-mode=dmabuf (value 4) to export encoded output as DMABuf file descriptors for downstream elements

Alignment Requirements

Input buffers follow Qualcomm hardware alignment requirements (e.g., 128-byte stride alignment). UBWC-compressed input formats (NV12_Q08C) are supported for reduced memory bandwidth when the upstream source produces UBWC output.

Encoder Parameter Control

Encoding parameters are configured via the extra-controls property using V4L2 control IDs. Common parameters include:

Control	Description
`video_bitrate`	Target bitrate in bits per second
`video_gop_size`	GOP (Group of Pictures) size — interval between keyframes
`video_h264_i_frame_qp`	QP value for I-frames
`video_h264_p_frame_qp`	QP value for P-frames
`video_h264_b_frame_qp`	QP value for B-frames
`video_h264_entropy_mode`	Entropy coding mode (CAVLC=0, CABAC=1)

Example:

extra-controls="controls,video_bitrate=4000000,video_gop_size=30"

Drain and Flush

Drain — sends V4L2_ENC_CMD_STOP via VIDIOC_ENCODER_CMD; waits for all remaining encoded frames to be produced
Flush — stops both queues, resets buffer pools, and restarts streaming; used during seek operations

Usage

Ensure you have followed the prerequisites before continuing

Record Camera to H.264 MP4 File

Captures raw video from a camera source, encodes to H.264 with DMABuf zero-copy, and muxes into an MP4 container.

gst-launch-1.0 -e \
  qticamsrc ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! \
  queue ! v4l2h264enc output-io-mode=dmabuf-import capture-io-mode=dmabuf ! \
  h264parse ! mp4mux ! \
  filesink location=$HOME/media/output/output.mp4

Transcode H.265 to H.264

Decodes an H.265 stream from an MP4 file using the hardware decoder and re-encodes to H.264 using the hardware encoder. DMABuf is used for zero-copy transfer between decoder and encoder.

gst-launch-1.0 -e \
  filesrc location=<h265_enc_mp4_file_path> ! qtdemux ! h265parse ! \
  v4l2h265dec output-io-mode=4 capture-io-mode=4 ! \
  queue ! v4l2h264enc output-io-mode=5 capture-io-mode=4 ! \
  h264parse ! mp4mux ! \
  filesink location=$HOME/media/output/output_h264.mp4

Four-Stream Simultaneous Encoding

Encodes four concurrent H.264 streams from four sources using four independent hardware encoder instances.

gst-launch-1.0 -e \
  videotestsrc num-buffers=200 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! \
    queue ! v4l2h264enc output-io-mode=4 capture-io-mode=4 ! \
    h264parse ! mp4mux ! filesink location=$HOME/media/output/output0.mp4 \
  videotestsrc num-buffers=200 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! \
    queue ! v4l2h264enc output-io-mode=4 capture-io-mode=4 ! \
    h264parse ! mp4mux ! filesink location=$HOME/media/output/output1.mp4 \
  videotestsrc num-buffers=200 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! \
    queue ! v4l2h264enc output-io-mode=4 capture-io-mode=4 ! \
    h264parse ! mp4mux ! filesink location=$HOME/media/output/output2.mp4 \
  videotestsrc num-buffers=200 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! \
    queue ! v4l2h264enc output-io-mode=4 capture-io-mode=4 ! \
    h264parse ! mp4mux ! filesink location=$HOME/media/output/output3.mp4

​Overview

​Example Pipeline

​Key Responsibilities

​Hierarchy

​Pad Templates

​sink

​src

​Element Properties

​I/O Mode Values

​Internal Architecture

​State Transitions

​Memory and Buffer Management

​DMABuf Usage

​Alignment Requirements

​Encoder Parameter Control

​Drain and Flush

​Usage

​Record Camera to H.264 MP4 File

​Transcode H.265 to H.264

​Four-Stream Simultaneous Encoding

Overview

Example Pipeline

Key Responsibilities

Hierarchy

Pad Templates

sink

src

Element Properties

I/O Mode Values

Internal Architecture

State Transitions

Memory and Buffer Management

DMABuf Usage

Alignment Requirements

Encoder Parameter Control

Drain and Flush

Usage

Record Camera to H.264 MP4 File

Transcode H.265 to H.264

Four-Stream Simultaneous Encoding