Multimedia - Qualcomm Dragonwing Documentation

The multimedia sample applications provide custom use cases for camera, video, audio, graphics, and display capabilities of Qualcomm Linux. The following tables list all available GStreamer C/C++ multimedia applications and their platform support. Select the appropriate configuration tab for your setup.

Config #1
Config #2

Application	Source code	Description	QCS6490	IQ-8275	IQ-9075	IQ-615
Camera
Single camera streaming	`gst-camera-single-stream-example`	Single camera: preview, encode, YUV dump, or RTSP.	✓	✓	✓	✓
Multi-camera streaming	`gst-multi-camera-example`	Concurrent streaming from two cameras.	×	×	×	TBD
Multistream	`gst-multi-stream-example`	Single camera duplicated into two streams via tee.	×	×	×	×
Live snapshot	`gst-snapshot-stream-example`	Preview + JPEG snapshot capture simultaneously.	×	×	×	×
RTMP camera streaming	`gst-rtmp-stream-example`	ISP / RTSP camera feed converted to RTMP stream.	×	×	×	×
Activate/deactivate streams at runtime	`gst-activate-deactivate-streams-runtime-example`	Change resolution on-the-fly without sensor restart.	×	×	×	×
Add/remove streams at runtime	`gst-add-remove-streams-runtime`	Dynamically add and remove streams.	×	×	×	×
Camera frame processing	`gst-appsink-example`	Apply transforms to camera frames before encoding.	×	×	×	×
Camera metadata	`gst-camera-metadata-example`	Capture scene mode, white balance, and bit rate metadata.	×	×	×	×
Camera stream configuration	`gst-add-streams-as-bundle-example`	Configure camera streams as a bundle.	×	×	×	×
Camera burst capture	`gst-camera-burst-capture-example`	Burst-mode snapshot capture.	×	×	×	×
Camera switch	`gst-camera-switch-example`	Switch between two cameras without stopping pipeline.	×	×	×	×
SHDR / LDC / EIS	`gst-camera-shdr-ldc-eis-example`	Super HDR, lens distortion correction, and image stabilization.	×	×	×	×
Decode JPEG images	`gst-jpg-decode-example`	Decode JPEG images and preview on screen.	✓	✓	✓	✓
Video
Video Wall	`gst-concurrent-videoplay-composition`	Concurrent video playback composed on display.	✓	✓	✓	✓
Smart codec	`gst-smartcodec-example`	Reduce bandwidth / storage for camera input.	×	×	×	×
Multiformat encode/decode	`gst-videocodec-concurrent-playback`	Concurrent decode and playback of multiple formats.	✓	✓	✓	✓
Audio-video playback	`gst-audio-video-playback`	Simultaneous audio and video decode and playback.	✓	✓	✓	✓
Video transcoding	`gst-video-transcode-example`	Transcode AVC↔HEVC.	✓	✓	✓	✓
Video playback	`gst-video-playback-example`	Decode and play back a video file.	✓	✓	✓	✓
Record and preview transformed video	`gst-transform-example`	Rotate, flip, and scale a video stream.	×	×	×	×
Video composition using Weston	`gst-weston-composition-example`	Compose live camera and offline file sources.	×	×	×	×
Encode audio and video	`gst-audio-video-encode`	Encode audio and video to a file.	×	×	×	×
Audio
Audio playback	`gst-audio-decode-example`	Decode and play back WAV, MP3, or FLAC.	✓	✓	✓	✓
Audio recording	`gst-audio-encode-example`	Record and encode audio to FLAC or WAV.	✓	✓	✓	✓

Application	Source code	Description	QCS6490	IQ-8275	IQ-9075	IQ-615
Camera
Single camera streaming	`gst-camera-single-stream-example`	Single camera: preview, encode, YUV dump, or RTSP.	✓	✓	✓	✓
Multi-camera streaming	`gst-multi-camera-example`	Concurrent streaming from two cameras.	✓	✓	✓	×
Multistream	`gst-multi-stream-example`	Single camera duplicated into two streams via tee.	✓	✓	✓	×
Live snapshot	`gst-snapshot-stream-example`	Preview + JPEG snapshot capture simultaneously.	✓	×	×	×
RTMP camera streaming	`gst-rtmp-stream-example`	ISP / RTSP camera feed converted to RTMP stream.	✓	✓	✓	×
Activate/deactivate streams at runtime	`gst-activate-deactivate-streams-runtime-example`	Change resolution on-the-fly without sensor restart.	✓	✓	✓	×
Add/remove streams at runtime	`gst-add-remove-streams-runtime`	Dynamically add and remove streams.	✓	✓	✓	×
Camera frame processing	`gst-appsink-example`	Apply transforms to camera frames before encoding.	✓	✓	✓	✓
Camera metadata	`gst-camera-metadata-example`	Capture scene mode, white balance, and bit rate metadata.	✓	✓	✓	×
Camera stream configuration	`gst-add-streams-as-bundle-example`	Configure camera streams as a bundle.	✓	✓	✓	×
Camera burst capture	`gst-camera-burst-capture-example`	Burst-mode snapshot capture.	✓	×	×	×
Camera switch	`gst-camera-switch-example`	Switch between two cameras without stopping pipeline.	✓	✓	✓	×
SHDR / LDC / EIS	`gst-camera-shdr-ldc-eis-example`	Super HDR, lens distortion correction, and image stabilization.	✓	×	×	×
Decode JPEG images	`gst-jpg-decode-example`	Decode JPEG images and preview on screen.	✓	✓	✓	✓
Video
Video Wall	`gst-concurrent-videoplay-composition`	Concurrent video playback composed on display.	✓	✓	✓	✓
Smart codec	`gst-smartcodec-example`	Reduce bandwidth / storage for camera input.	✓	✓	✓	×
Multiformat encode/decode	`gst-videocodec-concurrent-playback`	Concurrent decode and playback of multiple formats.	✓	✓	✓	✓
Audio-video playback	`gst-audio-video-playback`	Simultaneous audio and video decode and playback.	✓	✓	✓	✓
Video transcoding	`gst-video-transcode-example`	Transcode AVC↔HEVC.	✓	✓	✓	✓
Video playback	`gst-video-playback-example`	Decode and play back a video file.	✓	✓	✓	✓
Record and preview transformed video	`gst-transform-example`	Rotate, flip, and scale a video stream.	✓	✓	✓	✓
Video composition using Weston	`gst-weston-composition-example`	Compose live camera and offline file sources.	✓	✓	✓	✓
Encode audio and video	`gst-audio-video-encode`	Encode audio and video to a file.	✓	✓	✓	✓
Audio
Audio playback	`gst-audio-decode-example`	Decode and play back WAV, MP3, or FLAC.	✓	✓	✓	✓
Audio recording	`gst-audio-encode-example`	Record and encode audio to FLAC or WAV.	✓	✓	✓	✓

For camera and video sample applications that require a YUV player, see the Sourceforge YUV player project.

Prerequisites

Try me

Note: Some of the steps in the pre-requisties will be removed from future releases once the necessary fixes are mainlined.

Set up Wi-Fi

Connect to the Wireless Access Point (Wi-Fi Router):

nmcli dev wifi connect <WiFi-SSID> password <WiFi-password>

Check the connection and device status:

nmcli -p device

Login to the target deviceLocate the IP address of the device according to the type of network connection, using the UART console on the Linux host:For Ethernet:

ip address show eth2

For Wi-Fi:

ip address show wlp1s0

Use the IP address from the Linux host to establish an SSH connection to the device:

ssh root@<ip-address>

Example:

ssh root@192.168.0.222

Connect to the SSH shell using the following password:

oelinux123

Set up Wi-Fi

Connect to the Wireless Access Point (Wi-Fi Router):

nmcli dev wifi connect <WiFi-SSID> password <WiFi-password>

Check the connection and device status:

nmcli -p device

Login to the target deviceLocate the IP address of the device according to the type of network connection, using the UART console on the Linux host:For Ethernet:

ip address show eth2

For Wi-Fi:

ip address show wlp1s0

Use the IP address from the Linux host to establish an SSH connection to the device:

ssh root@<ip-address>

Example:

ssh root@192.168.0.222

Connect to the SSH shell using the following password:

oelinux123

Download Models and Artifacts

On the target device, obtain the download_artifacts.sh script, set executable permissions, and run it to download the model, media, and label files:

cd /tmp/
curl -L -O https://raw.githubusercontent.com/quic/sample-apps-for-qualcomm-linux/refs/heads/main/qualcomm-linux/scripts/download_artifacts.sh
chmod +x download_artifacts.sh
./download_artifacts.sh

Enable qticamsrc

In the terminal of the target device, run the following command to enable the qticamsrc on Config #2:

echo -n "camx" > /var/data
efivar -n 882f8c2b-9646-435f-8de5-f208ff80c1bd-VendorDtbOverlays -w -f /var/data
efivar -n 882f8c2b-9646-435f-8de5-f208ff80c1bd-VendorDtbOverlays -p
sync
reboot

Enable libcamera

For Dragonwing RB3 Gen 2 Development Kit, enable the libcamera using the following instructions:

Note: The libcamera plugin supports only the IMX577 camera sensor. Connect the IMX577 sensor before enabling libcamera.

In the terminal of the target device, enable the bootloader mode using the following command:

reboot bootloader

Once the device enters bootloader mode, flash the Vision Kit CDT file from the extracted folder:

fastboot flash cdt cdt_vision_kit.bin

Note: You can obtain the cdt_vision_kit.bin file from Qualcomm multimedia proprietary image at images/rb3gen2-core-kit/qcom-multimedia-proprietary-image-rb3gen2-core-kit:

Reboot the device:

fastboot reboot

Enable Audio and GPU Delegate

In the terminal of the target device, run the following command to enable audio:

systemctl stop pipewire wireplumber pipewire.socket pipewire-manager.socket
chmod 777 /dev/dma_heap/system
adsprpcd audiopd &
systemctl start pipewire wireplumber
wpctl status

To set the default devices for sink and source, get the device numbers from wpctl status and run the following command:

wpctl set-default <device ID>

In the terminal of the target device, run the following command to enable the GPU delegate and backend:

ln -sf /usr/lib/libOpenCL.so.1 /usr/lib/libOpenCL.so
export OCL_ICD_FILENAMES=/usr/lib/libOpenCL_adreno.so.1

Camera Applications

Single Camera Streaming

Description: The gst-camera-single-stream-example application demonstrates single camera use cases with possible outputs: live camera preview on display, store video encoder output, dump the camera YUV, and live RTSP streaming.

Live camera preview on display
Encode MP4 saved to file
Encode YUV saved to file
RTSP use case
Usage Example: For example, you can use this application in a security system where the goal is to capture a video from one camera and stream it across a network. You can either view it or archive it for future review. The following figure shows the pipeline flow for various single camera stream use cases.For more information, see pipeline flow. Figure: gst-camera-single-stream-example pipeline

Note: The camera-source-bin uses the libcamerasrc and qtivtransform plugins in Config #1 and the qticamsrc plugin in the Config #2 to generate the output stream.

Application: gst-camera-single-stream-example

Try me

Run the application

Complete prerequisites

Ensure that you complete the Prerequisites.

In the terminal of the target device, select a use case and run the command

Live camera use case: View the live camera on Waylandsink:

gst-camera-single-stream-example -o 0 -w 1920 -h 1080

MP4 use case: Dump the MP4 to the /etc/media/ folder:

gst-camera-single-stream-example -o 1 -w 1920 -h 1080

YUV use case: Dump the YUV data to the /etc/media/ folder:

gst-camera-single-stream-example -o 2 -w 1920 -h 1080

RTSP use case: View the RTSP stream (run the RTSP server) on the Linux host computer:

Run the RTSP server in a separate console on the target device with udpsrc (you can run it in the background as a service):

gst-rtsp-server  -p 8900 -a <ip_address> -m /live "( udpsrc name=pay0 port=<port_num> caps=\"application/x-rtp,media=video,clock-rate=90000,encoding-name=H264,payload=96\" )"

For Example:

gst-rtsp-server -p 8900 -a 10.92.182.47 -m /live "( udpsrc name=pay0 port=8554 caps=\"application/x-rtp,media=video,clock-rate=90000,encoding-name=H264,payload=96\" )"

Then start single camera streaming:

gst-camera-single-stream-example -o 3 -w 1280 -h 720 -i <ip_address> -p <port_num>

In the terminal of the host computer, run one of the following commands to view the stream

For VLC:

vlc -vvv rtsp://<ip_address>:8900/live

For FFplay:

ffplay rtsp://<ip_address>:8900/live

To Stop the running use case

Press Ctrl + C.

In the terminal of the target device, run the following command to display the available help options:

gst-camera-single-stream-example --help

In the terminal of the host computer, run the following command to pull the files from the target device:

scp root@<IP address of target device>:/etc/media/<filename to pull> <destination directory>

Expected output

The following table lists the expected output based on the selected output type.

Figure2: Expected output for gst-camera-single-stream-exampleTable:Expected output for gst-camera-single-stream-example

Output	Description
Display	Preview the stream.
Dump the MP4 and YUV data	Save the output to a file at `/etc/media/`.
RTSP	View the output on a remote display.

Pipeline flow

The following table lists the plugins used in the single camera streaming pipeline:

Pipeline	Description
Live camera preview on display	`qticamsrc` captures the camera live stream. `Capsfilter` is applied to enforce constraints on the raw video data. The data is sent to the Wayland display sink (`waylandsink`) for live preview.
Video encoding	`qticamsrc` captures the camera live stream. `Capsfilter` is applied to enforce constraints on the raw video data. The data is sent to the Wayland display sink (`waylandsink`) for live preview. `v4l2h264enc` is used to encode the video using the H.264 format. `H264parse` is used to parse the video. `Mp4mux` is used to multiplex the video into an MP4 container. `Filesink` is used to write the video to a file.
Dump the camera YUV to a filesink	`qticamsrc` captures the camera live stream. `Capsfilter` is applied to enforce constraints on the raw video data. `Filesink` is used to dump the YUV data into a file.
RTSP streaming	`qticamsrc` captures the camera live stream. `Capsfilter` is applied to enforce constraints on the raw video data. `v4l2h264enc` is used to encode the video using the H.264 format. `H264parse` is used to parse the video, rtph264pay is used to load the stream to RTSP, and then the stream is sent to a UDP sink for RTSP streaming.

Multi-Camera

Description: The gst-multi-camera-example application allows you to stream from two camera sensors simultaneously. The application composes the camera feeds side by side to display on a screen or encodes and stores the video streams to files. A few use cases that need many camera inputs are dash camera or stereo camera. You can use this application as a reference and build your use case. For example, a security system where the goal is to capture a video from several cameras. You can either view it or archive it for a future review. The following figure shows the pipeline for several camera streams. For more information about the elements used in this pipeline, see pipeline flow.

Figure: gst-multi-camera-stream-example pipeline Application: gst-multi-camera-example

Try me

Run the application

Note: In the following commands, provide the necessary parameters, such as width, height, and output type. The width and height changes are applicable to the primary camera only.

Complete prerequisites

Ensure that you complete the Prerequisites.

In the terminal of the target device, select any of the following use cases and run the respective command:

Waylandsink use case: View the Waylandsink output:

gst-multi-camera-example -o 0 -w 1920 -h 1080

Encoder use case: View the encoder output:

gst-multi-camera-example -o 1 -w 1920 -h 1080

To Stop the running use case

Press Ctrl + C.

In the terminal of the target device, run the following command to display the available help options:

gst-multi-camera-example --help

In the terminal of the host computer, run the following command to pull the files from the target device:

scp root@<IP address of target device>:/etc/media/<filename to pull>

Expected Output

The output is displayed on the screen and saved to a file.

If the output type is display, you can preview the stream on the display.
If the output type is video encoding, then the encoded files are saved at /etc/media/cam1_vid.mp4 and /etc/media/cam2_vid.mp4 for camera 1 and camera 2 respectively.

Figure: Expected output for gst-multi-camera-example application–preview

Pipeline flow

The following table lists the plugins used in the multi camera streaming pipeline:

Pipeline	Description
Preview on display	`qticamsrc` captures video from both camera0 and camera1. `Capsfilter` is applied to enforce constraints on the raw video data. `qtivcomposer` composites the video streams and sends the composited video data to Wayland display sink. `waylandsink` shows the live preview.
Encoder dump on the device	`qticamsrc` captures video from both camera0 and camera1. `Capsfilter` is applied to enforce constraints on the raw video data. `v4l2h264enc` is used to encode the video using the H.264 format. `H264parse` is used to parse the video. `Mp4mux` is used to multiplex the video into an MP4 container. `Filesink` is used to write the video to a file.

Camera Switch

Description: The gst-camera-switch-example application allows you to switch between two cameras. The following figure shows the pipeline, which demonstrates the camera switch between the two cameras of the device without changing the state of the pipeline. The switching is done in the Playing state after every 5 seconds. For more information about the plugins used in this pipeline, see Pipeline flow.

Figure: gst-camera-switch-example pipeline Application: gst-camera-switch-example

Try me

Run the application

Complete prerequisites

Ensure that you complete the Prerequisites.

In the terminal of the target device, run the respective command for any of the following use cases:

Camera switch:

gst-camera-switch-example <OPTION>

Display the stream after switching (default option):

gst-camera-switch-example -d 1

Encode the stream after switching:

gst-camera-switch-example -d 0

To Stop the running use case

Press Ctrl + C.

In the terminal of the target device, run the following command to display the available help options:

gst-camera-switch-example --help

In the terminal of the host computer, run the following command to pull the files from the target device:

  scp root@<IP address of target device>:/etc/media/<filename to pull>

Expected output

The application generates the output as a preview or an encoded MP4 file.

Figure: Expected output for gst-camera-switch-example application

Pipeline flow

The following table lists the plugins used in the camera switch pipeline:

Pipeline	Description
Display preview	`qticamsrc` captures the camera live stream and uses capsfilter to create a branch for preview. The data is sent to the Wayland display sink (`waylandsink`) for preview.
Encode as MP4 file	`qticamsrc` captures the camera live stream and uses capsfilter to create a branch for encoding. The stream data is encoded. `H264parse` is used to parse the captured data. `Mp4mux` is used to multiplex the data.

Multi-Stream

Description: The gst-multi-stream-example The gst-multi-stream-example application allows you to access several streams from a single camera and either display them simultaneously on a screen or save them in MP4 format. For example, you can use this application for a camera security system where the video is captured from a camera. This video can be simultaneously streamed over a network, displayed on a monitor, and archived for future review. The figure shows the pipeline, which is run with flags to specify width, height, stream count, and output file. For more information, see Pipeline flow.

Figure: gst-multi-stream-example pipeline Application: gst-multi-stream-example

Try me

Run the application

Complete prerequisites

Ensure that you complete the Prerequisites.

In the terminal of the target device, run the use case:

gst-multi-stream-example -w 1920 -h 1080 -n 2

In the terminal of the target device, run the following command to display the available help options:

gst-multi-stream-example --help

In the terminal of the host computer, run the following command to pull the files from the target device:

scp root@<IP address of target device>:/etc/media/<filename to pull>

Expected output

The output streams are displayed simultaneously on the screen or saved to a file.

Figure: Expected output for gst-multi-stream-example application - Preview

Pipeline flow

The following table lists the plugins used in the multistream pipeline:

Plugin	Description
qticamsrc	`qticamsrc` captures the video stream and feeds it into the capsfilter.
Capsfilter	Branches the source into two separate pipelines. In one pipeline, the output is sent to `waylandsink` for display. In the other pipeline, the video undergoes the following: Encoded using `v4l2h264enc`. Parsed with `h264parse`. Multiplexed into MP4 format with `mp4mux`. Written to a file using `filesink`.

Known Issues

The default bit rate for the encoded video is configured to a low setting, leading to reduced quality at higher resolutions.
The application may not work due to v4l2h264enc encoder issue.

Wide dynamic range and electronic image stabilization

Description: The gst-camera-shdr-ldc-eis-example application shows the staggered high dynamic range (sHDR), lens distortion correction (LDC), and electronic image stabilization (EIS) camera imaging features. For more information, see Overview of camera imaging features.

Note: This sample application isn’t supported on Dragonwing IQ-8275 and Dragonwing IQ-9075.

The following figure shows the pipeline, where the input is received from the camera and the output is displayed using Waylandsink. For more information about the pipeline, see Pipeline flow.

For sHDR, ensure that the Sensor mode for Digital Overlap mode is enabled and tuned.
For LDC and EIS, ensure that the sensor and hardware are tuned. Application: gst-camera-shdr-ldc-eis-example

Try me

Run the application

Complete prerequisites

Ensure that you complete the Prerequisites.

In the terminal of the target device, run the respective command for any of the following use cases to display the output:

sHDR

gst-camera-shdr-ldc-eis-example -s 1 -w 1920 -h 1080

Note: This use case isn’t supported on the Ubuntu Server.

LDC

gst-camera-shdr-ldc-eis-example -l 1 -w 1920 -h 1080

EIS

gst-camera-shdr-ldc-eis-example -e 1 -w 1920 -h 1080

To Stop the running use case

Press Ctrl + C.

In the terminal of the target device, run the following command to display the available help options:

gst-camera-shdr-ldc-eis-example --help

Expected output

sHDR Figure: Output without sHDR Figure: Output with sHDR
LDC Figure: Output without LDC Figure: Output with LDC

Pipeline flow

The following table lists the plugins used in the gst-camera-shdr-ldc-eis-example pipeline:

Plugin	Description
qticamsrc	Captures the camera feed and sends it to the capsfilter.
Capsfilter	Passes the feed for display.
waylandsink	Shows the output on the Wayland display.

Overview of camera imaging features

The following table provides a brief description of the supported imaging features:Table : Camera imaging features

Feature	Description
sHDR	The video high dynamic range feature is supported on camera sensors that operate in the Digital Overlap mode. The fusion of the low and high exposure frames is enabled in the processor GPU. Use the sHDR flag to enable the sHDR mode.
LDC	A GPU-based LDC algorithm is supported to rectify barrel distortion artifacts that are inherent with wide field of vision (FoV) lens. Use the LDC flag to enable the LDC mode.
EIS	A GPU-based EIS algorithm is supported. Use the EIS flag to enable the EIS mode.

Snapshot Stream

Description: The gst-snapshot-stream-example application allows you to generate two streams: a preview stream for live display and a snapshot stream that captures images and saves them as JPEG files. For the snapshot stream, you can determine the number of snapshots.

Note: This sample application isn’t supported on Dragonwing IQ-8275 and Dragonwing IQ-9075.

The following figure shows the pipeline, which allows for simultaneous live preview and snapshot capture from the same video source. For more information, see Pipeline flow.

Figure: gst-snapshot-stream-example pipeline Application: gst-snapshot-stream-example

Try me

Run the application

Complete prerequisites

Ensure that you complete the Prerequisites.

In the terminal of the target device, use the following command to run the use case. Specify the application with parameters, such as camera width, height, and the maximum snapshot count for dumping JPEG files.

gst-snapshot-stream-example -W 1280 -H 720 -w 3840 -h 2160 -c 5

To Stop the running use case

Press Ctrl + C.

In the terminal of the target device, run the following command to display the available help options:

gst-snapshot-stream-example --help

Expected output

The application generates an output for preview on the display and dumps the JPEG files in the respective location on the device. After the use case stops, the snapshot output files are available at the /etc/media/ path.

Pipeline flow

The following table lists the plugins used in the live snapshot pipeline:

Plugin	Description
qticamsrc	`qticamsrc` captures the video stream and feeds it into the capsfilter.
Capsfilter	Branches the source into two separate pipelines. In one pipeline, the constraints on the raw video data are enforced, and then the video data is sent to `waylandsink` for live preview. In the other pipeline: The frames are written to multiple files using multi filesink. The snapshots or frames from the video stream are saved to a disk.

Camera frame processing

Description: The gst-appsink-example application allows you to extract the buffer samples from a camera pipeline. These buffers can configure the camera resolution with the options such as width and height. For example, the appsink plugin can be used for a multimedia application that processes the video frame samples in real-time. Filters or transformations can be applied to the data before playing it back. The following figure shows the pipeline, which configures the camera resolution using parameters such as width and height. When appsink emits a signal indicating the availability of a new sample in the pipeline, the application extracts the buffer from this sample. For information about the plugins used in this pipeline, see Pipeline flow.

Appsink pipeline : gst-appsink-example pipeline

Figure : gst-appsink-example pipeline Application: gst-appsink-example

Try me

Run the application

Complete prerequisites

Ensure that you complete the Prerequisites.

In the terminal of the target device, run the use case

gst-appsink-example -w 1920 -h 1080

To stop the running use case

Press CTRL + C.

In the terminal of the target device, run the following command to display the available help options:

gst-appsink-example --help

Expected output

The user interface displays the following message:

Received a buffer, doing some processing.

Appsink output : Expected output for gst-appsink-example application

Figure : Expected output for gst-appsink-example application

Pipeline flow

The following table lists the plugins used in the camera frame processing pipeline:

Plugin	Description
`qticamsrc`	Captures the camera feed and sends it into the capsfilter.
`Capsfilter`	Ensures that the video stream is in the correct format.
`Appsink`	Provides the video data as an output to an application.

Camera metadata

Description: The gst-camera-metadata-example application starts with a menu that allows you to choose between multiple properties of the camera plugin such as video-metadata, static-metadata, and image-metadata. A practical use case for this application is handling camera metadata such as white balance, International Organization for Standardization (ISO) sensitivity, and exposure to tune the camera. The following figure shows the execution of appsink when a new sample is available in the pipeline. The application is run from the command-line with specific options for custom pipeline, display type, and timestamp location. For information about the plugins used in this pipeline, see Pipeline flow.

Figure : gst-camera-metadata-example pipeline Application: gst-camera-metadata-example

Try me

Run the application

Complete prerequisites

Ensure that you complete the Prerequisites.

In the terminal of the target device, run the following use case:

Camera metadata use case with output on Waylandsink:

gst-camera-metadata-example

Camera metadata use case without Waylandsink:

gst-camera-metadata-example -d 0

Dump metadata of saturation:

gst-camera-metadata-example -d 0 -r "org.codeaurora.qcamera3. saturation use_saturation"

Dump metadata of exposureTime:

gst-camera-metadata-example -d 0 -r "spectra.sensor exposureTime"

To stop the use case

Press CTRL + C.

In the terminal of the target device, run the following command to display the available help options:

gst-camera-metadata-example --help

Expected output

During the execution, the application requires the command-line options to output the camera metadata and options available to set or get the vendor tags.
On successful execution of the application, you can find and dump the respective metadata information.

Figure : Expected output for gst-camera-metadata-example application

Pipeline flow

The following table lists the plugins used in the camera metadata pipeline:

Plugin	Description
`qticamsrc`	Captures the camera feed and sends it into the capsfilter.
`Capsfilter`	Passes the feed to waylandsink for display. If display option isn’t provided, passes the feed to appsink.

The application extracts the buffer from the sample for further processing. During execution, the application expects an argument on which the following options should be displayed on the console.

Video metadata
Image metadata
Static metadata
Session metadata

Based on the options selected, the application displays the output on the console.

Known issue

When the GetTag menu option is selected, the capture options don’t work.

Camera stream configuration

Description: The gst-add-streams-as-bundle-example application allows you to activate and configure camera streams as a bundle instead of configuring each stream one at a time thus reducing the delay in configuring new streams. The following figure shows the pipeline, which takes the configuration parameters such as width and height, processes to either display on a screen, or encodes and saves to a file. For more information about the plugins used in the pipeline, see Pipeline flow.

add stream as bundle pipeline: gst-add-streams-as-bundle-example pipeline

Figure : gst-add-streams-as-bundle-example pipeline Application: gst-add-streams-as-bundle-example

Try me

Run the application

Complete prerequisites

Ensure that you complete the Prerequisites.

In the terminal of the target device,run the use case:

gst-add-streams-as-bundle-example <OPTION>

Run for various use cases

Preview the stream:

gst-add-streams-as-bundle-example -o Display

Encode the stream and save it to a file:

gst-add-streams-as-bundle-example -o File

The encoded stream is saved at /etc/media/video_%d.mp4.

To stop the running use case

Use CTRL + C.

In the terminal of the target device, run the following command to display the available help options:

gst-add-streams-as-bundle-example --help

In the terminal of the host computer, run the following command to pull the files from the target device:

scp root@<IP address of target device>:/etc/media/<filename to pull>

Expected output

One stream is sent to display and the other stream is saved to a file.

Figure : Expected output for gst-add-streams-as-bundle-example application

Pipeline flow

The following table lists the plugins used in the camera stream configuration pipeline:

Plugin	Description
`qticamsrc`	Captures the video stream and feeds it into the capsfilter.
`Capsfilter`	Branches the source into two separate pipelines. In one pipeline, the output is sent to waylandsink for display. In the other pipeline, the video undergoes the following: 1. Encoded using v4l2h264enc. 2. Parsed with h264parse. 3. Multiplexed into an MP4 format with mp4mux. 4. Written to a file using filesink.

Known issue

The application doesn’t work due to video encoder issue.

Camera Burst Capture

Description: The gst-camera-burst-capture-example application allows you to capture snapshots in the Burst mode. It captures five burst snapshots delayed by a 10-second timer and displays the snapshots on a screen.

Note: This sample application isn’t supported on Dragonwing IQ-8275 and Dragonwing IQ-9075.

The following figure shows the pipeline, which captures the snapshots and processes them in using one of the following three branches:

They’re displayed on the screen.
They’re encoded and saved to a file.
They’re saved to a buffer and based on the capture format (BAYER/RAW/JPEG), saved to the device storage. For more information about the plugins used in the pipeline, see Pipeline flow. Figure: gst-camera-burst-capture-example pipeline

Application: gst-camera-burst-capture-example

Try me

Run the application

Complete prerequisites

Ensure that you complete the Prerequisites.

In the terminal of the target device, run the respective command for any of the following use cases:

Display stream and JPEG capture:

gst-camera-burst-capture-example -w 1280 -h 720 -p 1 -c 0

Encode stream and JPEG capture:

gst-camera-burst-capture-example -w 1280 -h 720 -p 0 -c 0

To Stop the running use case

Press Ctrl + C.

In the terminal of the target device, run the following command to display the available help options:

gst-camera-burst-capture-example --help

In the terminal of the host computer, run the following command to pull the files from the target device:

scp root@<IP address of target device>:/etc/media/<filename to pull>

Expected output

If you choose to display the stream, the snapshots are displayed on the screen.
If you choose to encode the stream, the encoded stream is saved at /etc/media/video_%d.mp4.

Pipeline flow

The following table lists the plugins used in the camera burst capture pipeline:

Pipeline	Description
Display preview	`qticamsrc` captures the camera live stream and uses capsfilter to create a branch for preview. The data is sent to the Wayland display sink (`waylandsink`) for preview.
Encode and save to a file	`qticamsrc` captures the camera live stream and uses capsfilter to create a branch for encoding. The snapshots are encoded. H264parse is used to parse the captured data. Mp4mux is used to multiplex the data.
BAYER/RAW/JPEG dump	`qticamsrc` captures the camera live stream and uses capsfilter to create a branch for preview. Appsink is used to capture callback and save the snapshots in the storage dump.

Known issue

The test cases for 144p and 240p resolutions are failing.

RTMP Streaming

Description: The gst-rtmp-stream-example application uses the input from ISP or RTSP camera source to generate RTMP streams on a Waylandsink. You can use this application as a reference to develop and configure new applications. This application is primarily targeted for security systems and media broadcasting, where a video from RTSP source or camera source streams across a network. You can either view the stream or archive it for future review. The following figure shows the pipeline involved in RTMP streaming with different sources. For more information about the elements used in this pipeline, see Pipeline flow.

Figure: gst-rtmp-stream-example pipeline Application: gst-rtmp-stream-example

Try me

Run the application

Complete prerequisites

Ensure that you complete the Prerequisites.

In the terminal of the target device, download and run the RTMP server:

docker pull tiangolo/nginx-rtmp
docker run -d -p 1935:1935 --name nginx-rtmp tiangolo/nginx-rtmp

In the terminal of the target device, select any of the following use cases and run the respective command:

RTSP camera AVC to RTMP use case:

gst-rtmp-stream-example  -u 0 -d rtmp://<192.168.1.171/>live/01 -r <rtsp://admin:qualcomm1@192.168.1.252:554/Streaming/Channels/101 ! >

ISP camera to RTMP use case:

gst-rtmi-camera-stream-example  -u 1 -d rtmp://192.168.1.171/live/01

In the terminal of the target device, run the following command to connect VLC to the stream:

rtmp://192.168.1.171/live/01

In this command, replace rtmp://192.168.1.171/live/01 with the IP address of your device.

In the terminal of the target device, run the following commands to stop the RTMP server:

docker stop nginx-rtmp
docker rm nginx-rtmp

To Stop the running use case

Press Ctrl + C.

In the terminal of the target device, run the following command to display the available help options:

gst-rtmp-stream-example --help

Expected output

The output is displayed on a Waylandsink.

Pipeline flow

The following table lists the plugins used in the RTMP stream pipeline:

Plugin	Description
qticamsrc	`qticamsrc` Captures the camera feed and sends it into the capsfilter.
Capsfilter	Passes the feed to `waylandsink` for display. If display option isn’t provided, passes the feed to appsink.
v4l2h264enc	Encodes the video for AVC (H.264) format.
h264parse	Parses the H.264 video.
flvmux	Wraps the video into an FLV container
rtmp2sink	Transmits to an RTMP server for live streaming.

Smart Codec

Description: The gst-smartcodec-example application reduces bandwidth and storage resources based on scene change estimate thresholds. The smart codec plugin, qtismartvencbin, uses smart fps, smart I-frame, smart bit rate, and smart ROI to do the following:

Decrease the bit rate without compromising the quality.
Enable efficient use of network and storage resources.

The following figure shows the pipeline flow for a two-stream use case. For more information, see Pipeline flow.

Figure: gst-smartcodec-example pipeline for two streams Application: gst-smartcodec-example

Try me

Run the application

Complete prerequisites

Ensure that you complete the Prerequisites.

In the terminal of the target device, run the use case with a camera source:

gst-smartcodec-example -w 1920 -h 1080 -o /etc/media/video.mp4

To Stop the running use case

Press Ctrl + C.

In the terminal of the target device, run the following command to display the available help options:

gst-smartcodec-example --help

Expected output

The application generates the output as an encoded MP4 file. The following table lists the file sizes of a 1080p video with and without smart codec. Table:Smart codec–file sizes

Smart codec	File size
Enabled	3718 KB.
Disabled	17706 KB.

Pipeline flow

The following table lists the plugins used in the smart codec pipeline:

Plugin	Description
qticamsrc	Captures the video stream. Feeds the stream to capsfilter.
Capsfilter	Branches the source into two separate pipelines. In one pipeline, qtismartvencbin processes the output. In the other pipeline, the video undergoes the following: Encoded using v4l2h264enc. Parsed with h264parse. Multiplexed into an MP4 format with mp4mux. Written to a file using filesink.

GstD single camera streaming

Description: The GstD-camera-single-stream-example shows how to use a menu‑driven GstD application to create and control a single‑camera pipeline with configurable resolution. The application supports live preview, MP4 recording, and raw YUV dumping, and manages the GStreamer pipeline lifecycle cleanly for capture and archival use cases. The following figure shows the pipeline flow for various single camera stream use case. For more information, see Pipeline flow.

Figure: GstD-camera-single-stream-example pipeline Application: GstD-camera-single-stream-example

Try me

Run the application

Complete prerequisites

Ensure that you complete the Prerequisites.

In the terminal of the target device, select any of the following use cases and run the respective command:

Live camera use case: View the live camera on Waylandsink:

GstD-camera-single-stream-example -o 0 -W 1280 -H 720

MP4 use case: Dump the MP4 to the /opt folder:

GstD-camera-single-stream-example -o 1

YUV use case: Dump the YUV data to the /opt folder:

GstD-camera-single-stream-example -o 2 -W 640 -H 480

To Stop the running use case

Press Ctrl + C.

In the terminal of the target device, run the following command to display the available help options:

GstD-camera-single-stream-example --help

In the terminal of the host computer, run the following command to pull the files from the target device:

scp root@<IP address of target device>:/opt/<filename to pull> <destination directory>

Expected output

The following table lists the expected output based on the selected use case.

Figure: Expected output for gst-camera-single-stream-exampleTable: Expected output

Output Type	Description
Display	Preview the camera stream on display
MP4	Saves encoded video to `/opt`
YUV	Dumps raw YUV data to `/opt`
RTSP	View the stream on a remote display

Pipeline flow

The following table lists the pipelines and their descriptions:

Pipeline	Description
Live camera preview on display	1. `qtiqmmfsrc` captures the camera live stream. 2. Capsfilter is applied to enforce constraints on the raw video data. 3. The data is sent to the Wayland display sink (`waylandsink`) for live preview.
Video encoding	1. `qtiqmmfsrc` captures the camera live stream. 2. Capsfilter is applied to enforce constraints on the raw video data. 3. The stream is sent to (`waylandsink`) for preview. 4. `v4l2h264enc` encodes the video using the H.264 format. 5. `h264parse` parses the encoded stream. 6. `mp4mux` multiplexes the stream into an MP4 container. 7. `filesink` writes the video to a file.
Dump the camera YUV to a filesink	1. `qtiqmmfsrc` captures the camera live stream. 2. Capsfilter is applied to enforce constraints on the raw video data. 3. `filesink` dumps the YUV data into a file.

Dynamic configuration

A common scenario in Edge AI applications is changing the image resolution at runtime for minimal frame loss or gap in the display or adding and removing several streams for camera reconfiguration. The dynamic configuration sample applications support these runtime changes.

Activate-deactivate streams at runtime

The gst-activate-deactivate-streams-runtime-example application allows you to change the resolution dynamically without the need to reconfigure the camera sensor. This application can be used in surveillance systems, videoconferencing, and broadcasting where several live streams are processed in parallel and you can control the active streams dynamically. The following figure shows the pipeline, where the application creates two streams of different resolutions, and activates and deactivates them at runtime. For information about the plugins used in this pipeline, see Pipeline flow.

activate deactivate at runtime : gst-activate-deactivate-streams-runtime-example pipeline

Figure : gst-activate-deactivate-streams-runtime-example pipeline Application: gst-activate-deactivate-streams-runtime-example

Try me

Run the application

The following table lists the application use case modes. The NULL state is the initial state of a GStreamer element.

Note:: Currently, the application is compatible only with the Basic mode. Full mode will be supported in a future release.

Use case mode	Description
`Basic`	Activates or deactivates several streams without entering the NULL state.
`Full`	Activates or deactivates several streams by entering the NULL state.

For more information, see GStreamer states.

Complete prerequisites

Ensure that you complete the Prerequisites.

In the terminal of the target device, run the respective command for any of the following use cases. Before running the application, ensure to configure the use cases and output type parameters.

Basic use casesStream on Waylandsink:

gst-activate-deactivate-streams-runtime-example -u Basic -o Display

Encode to MP4 file:

gst-activate-deactivate-streams-runtime-example -u Basic -o File

Full use casesStream on Waylandsink:

gst-activate-deactivate-streams-runtime-example -u Full -o Display

Encode to MP4 file:

gst-activate-deactivate-streams-runtime-example -u Full -o File

To Stop the running use case

Press Ctrl + C.

In the terminal of the target device, run the following command in the SSH shell to display the available help options:

gst-activate-deactivate-streams-runtime-example --help

In the terminal of the host computer, run the following command to pull the files from the target device:

scp root@<IP address of target device>:/etc/media/<destination directory>

Expected output

During the execution, the first stream is activated followed by the second stream. After a while, the streams deactivate.

If the output option is set to Display, the output is displayed as a preview.
If the output option is set to File, the encoded video files are generated.

activate deactivate output : Expected output for gst-activate-deactivate-streams-runtime-example application

Figure : Expected output for gst-activate-deactivate-streams-runtime-example application

Pipeline flow

The following table lists the plugins used in the gst-activate-deactivate-streams-runtime-example pipeline:

Plugin	Description
`qticamsrc`	Captures the camera feed and sends it into the capsfilter.
`Capsfilter`	Passes the feed to waylandsink for display. Passes the feed to filesink to save it to a file.

Add-remove streams at runtime

The gst-add-remove-streams-runtime application allows you to add or remove several camera streams dynamically during runtime. The following figure shows the pipeline, which demonstrates the ability of the application to add or remove the streams at runtime with camera reconfiguration. It creates three streams and adds or removes them in different order. For information about the plugins used in the pipeline, see Pipeline flow.

Figure : gst-add-remove-streams-runtime pipeline Application: gst-add-remove-streams-runtime

Try me

Run the application

Complete prerequisites

Ensure that you complete the Prerequisites.

In the terminal of the target device, run the following command to verify if many streams can run simultaneously:

gst-add-remove-streams-runtime <OPTION>

Use the following commands to run the application for various use cases:

Default mode preview on the display:

gst-add-remove-streams-runtime

Preview the stream:

gst-add-remove-streams-runtime -o Display

Encode the stream:

gst-add-remove-streams-runtime -o File

To Stop the running use case

Press Ctrl + C.

In the terminal of the target device, run the following command to display the available help options:

gst-add-remove-streams-runtime --help

In the terminal of the host computer, run the following command to pull the files from the target device:

scp root@<IP address of target device>:/etc/media/<destination directory>

Expected output

If you choose to preview, the stream is displayed on the screen.
If you choose to encode the stream, the encoded stream is saved at /etc/media/video_%d.mp4.

Figure : Expected output for gst-add-remove-streams-runtime application–Preview

Pipeline flow

The following table lists the plugins used in the gst-add-remove-streams-runtime pipeline:

`Plugin`	`Description`
qticamsrc	Captures the video stream and feeds it into the `capsfilter`.
Capsfilter	Branches the source into two separate pipelines. In one pipeline, the output is sent to `waylandsink` for display.
	In the other pipeline, the video undergoes the following:
	Encoded using `v4l2h264enc`. Parsed with `h264parse`. Multiplexed into an MP4 format with `mp4mux`.

Video and Audio Applications

Video playback

Description: The gst-video-playback-example application allows you to decode and playback a video file. The standard playback options such as play, pause, resume, and fast forward are supported. Application: gst-video-playback-example The following figure shows the pipeline workflow of the video playback GStreamer application. The file is read, demultiplexed, parsed, decoded, and displayed on a screen.For more information about the plugins, see pipeline flow.

Figure: gst-video-playback-example pipeline

Try me

Run the application

Complete prerequisites

Ensure that you complete the Prerequisites.

Input the AVC.mp4 (H.264) and HEVC.mp4 (H.265) video files on the target device.

To Stop the running use case

Press Ctrl + C.

In the terminal of the target device, run the following command to display the available help options:

gst-video-playback-example --help

Expected output

The video is captured, decoded, and played on the local display.

Figure : Expected output for gst-video-playback-example application

Pipeline flow

The following table lists the plugins used to run the video playback pipeline:

Plugin	Description
filesrc	Reads the AVC or HEVC MP4 file.
qtdemux	Demultiplexes the AVC or HEVC MP4 file.
h264parse	Parses the video stream for H.264 format.
h265parse	Parses the video stream for H.265 format.
v4l2h264dec	Decodes the video stream for H.264 format.
v4l2h265dec	Decodes the video stream for H.265 format.
waylandsink	Displays the video in full-screen mode on a Wayland display.

Known IssueThe rewind playback option isn’t supported.

Video Transcode

Description: The gst-video-transcode-example application supports video transcoding of AVC-to-HEVC or HEVC-to-AVC format. A few use cases can be video editing, security camera, and augmented reality. The following figure shows the pipeline, which reads the incoming stream, decodes and encodes it, and either displays the transcoded output on a screen, or saves it to a file. For information about the plugins used in this pipeline, see Pipeline flow

Figure: gst-video-transcode-example pipeline Application: gst-video-transcode-example

Try me

Run the application

The following table lists the actions that can be performed for the common use cases of gst-videotranscode- example application:

Use case	Actions
Video editing	Rotate, flip, or resize videos.
Security camera	Invert or rotate the video image to compensate for situations where the camera is improperly mounted.
Augmented reality	Scale or rotate the video feed to synchronize with the virtual components.

Complete prerequisites

Ensure that you complete the Prerequisites.

Input AVC.mp4 and HEVC.mp4 video files on the target device.

In the terminal of the target device, transcode the video file:

gst-video-transcode-example -i /etc/media/<input-file>.mp4 -c <format> -o /etc/media/<output-file>.mp4

The following table provides a description of the command-line parameters:

Command-line parameter	Description
`-1`	Specifies the absolute path to the MP4 input file.
`-c`	Specifies the video codec of the input file.
`-o`	Specifies the absolute path to where the transcoded MP4 file is saved.

Run the use cases for different video formats:

Transcode from AVC to HEVC:

gst-video-transcode-example -i /etc/media/video_avc.mp4 -c 1 -o /etc/media/hevc_transcode_out.mp4

Transcode from HEVC to AVC:

gst-video-transcode-example -i /etc/media/video_hevc.mp4 -c 2 -o /etc/media/avc_transcode_out.mp4

To Stop the running use case

Press Ctrl + C.

In the terminal of the target device, run the following command to display the available help options:

gst-video-transcode-example --help

Expected output

The video stream is transcoded from H.264 to H.265 and back again to the H.264 format.

Pipeline flow

The following table lists the plugins used in the video transcoding pipeline:

Plugin	Description
filesrc	Reads the video streams.
qtdemux	Demultiplexes the streams into separate audio and video streams.
h264parse	Parses the H.264 video stream.
h265parse	Parses the H.265 video stream.
v4l2h264dec	Decodes the H.264 video stream using Video4Linux.
v4l2h265dec	Decodes the H.265 video stream using Video4Linux.
waylandsink	Displays the H.264 video stream on Wayland display.
filesink	Writes the H.265 video to a YUV file for playback.

The following table provides information on how the decoded video streams are handled to generate the transcoded output:

Plugin	Description
H.264	v4l2h265enc encodes the decoded H.264 video into H.265 format. h265parse parses the H.265 video stream. mp4mux multiplexes the stream into an MP4 container. filesink writes the stream data to a file. The AVC (H.264) video is transcoded to HEVC (H.265) format.
H.265	v4l2h264enc encodes the decoded H.265 video into H.264 format. h264parse parses the H.264 video stream. mp4mux multiplexes the stream into an MP4 container. filesink writes the stream data to a file. The HEVC (H.265) video is transcoded to AVC (H.264) format.

Video Transform

Description: The gst-transform-example application supports rotation, flipping, and scaling operations. You can use this application for a security camera. Adjust the orientation of the displayed image using the flip and mirror operations on the CCTV camera. For more information about the transformation properties, see qtivtransform. The following figure shows the pipeline, which takes the captured video stream, transforms it, and either displays it or saves it to a file. For information about the plugins used in the pipeline, see Pipeline flow.

Figure: gst-transform-example pipeline Application: gst-transform-example

Try me

Complete prerequisites

Ensure that you complete the Prerequisites.

In the terminal of the target device, run the following use cases to rotate, downscale, or flip the video:

Use camera source and save output to a file.

Perform operations such as rotate, flip, upscale/downscale simultaneously:

  gst-transform-example -r 90 -f 2 --output_width 1920 --output_height 1080  -o /etc/media/video_transform.mp4 -i <inputfile>.mp4

Rotate:

  gst-transform-example -r 90 --input_width 3840 --input_height 2160 -o <output_file>.mp4

Downscale/upscale:

  gst-transform-example --input_width 3840 --input_height 2160 --output_width 1920 --output_height 1080 -o <output_file>.mp4

Flip:

  gst-transform-example -f 2 --input_width 3840 --input_height 2160 -o <output_file>.mp4

Use file source and save output to a file.

Perform operations such as rotate, flip, upscale/downscale simultaneously:

gst-transform-example -r 90 -f 2 –i /etc/media/video_avc.mp4 --output_width 1920 --output_height 1080 -o <output_file>.mp4

Rotate:

gst-transform-example -r 90 -i /etc/media/video_avc.mp4 -o <output_file>.mp4

Downscale/upscale:

gst-transform-example -i /etc/media/video_avc.mp4 --output_width 1920 --output_height 1080 -o <output_file>.mp4

Flip:

gst-transform-example -f 2 -i /etc/media/video_avc.mp4 -o <output_file>.mp4

To Stop the running use case

Press Ctrl + C.

In the terminal of the target device, run the following command to display the available help options:

gst-transform-example --help

In the terminal of the host computer, run the following command to pull the files from the target device:

scp root@<IP address of target device>:/etc/media/<filename to pull>

The following table provides a description of the command-line parameters:

Parameter	Description
`-h`, `--help`	Help
`--output_width`	Downscale/upscale width. - Default value: 1920 - Supports scaling: 1080 → 720 or 720 → 1080
`--output_height`	Downscale/upscale height. Default value: 1080
`-o`	Absolute path to generate the file
`-r`	Image rotation. Supported values: `0`, `90`, `180`, `270`
`--input_width`	Camera width. Default value: 1920
`--input_height`	Camera height. Default value: 1080
`-f`	Flip the image: - `0`: Don’t flip (Default) - `1`: Flip horizontally - `2`: Flip vertically - `3`: Flip horizontally and vertically
`-I`, `--input_file`	Absolute path of the input file. Example: `-I /etc/media/inputfileh264.mp4`
`-o`, `--output_file`	Absolute path of the output file. Example: `-o /etc/media/video_transform.mp4`

Expected output

This pipeline supports simultaneous live preview and recording of the transformed video stream. It provides two outputs:

A video recording that’s stored as an MP4 file in the specified path.
A preview on a display device. Figure: Expected output for gst-transform-example application–Preview

Pipeline flow

The following table lists the plugins used to run the video transforming pipeline:

Plugins Overview

Plugin	Description
qticamsrc and filesrc	Captures the video data.
capsfilter	Enforces constraints on the raw video data.
qtivtransform	Transforms the video data. After transformation, capsfilter is applied again.
tee	Splits the transformed video data into two output paths: Live preview. Save to file.
waylandsink	Displays the video in live preview.
h264parse	Parses the encoded H.264 video stream.
mp4mux	Multiplexes the stream into an MP4 container.
filesink	Writes the video data to a file.

Concurrent Playback & Composition(video wall)

The gst-concurrent-videoplay-composition application supports concurrent video playback for MP4 AVC (H.264) videos and performs composition on a video wall display. In the concurrent video playback and composition pipeline, four decode and composition pipelines run concurrently. For more information about the plugins used in this pipeline, see Pipeline flow.

Figure: gst-concurrent-videoplay-composition pipeline with 4 streams Application: gst-concurrent-videoplay-composition

Try me

Run the application

The following table lists the use cases that are supported through the gst-concurrent-videoplay-composition application:

Use case	Description
Video conferencing	Simultaneously decodes the video streams of several participants. Useful in a multi-party video conference scenario where the application manages several video streams.
Surveillance systems	Concurrently decodes video feeds from two separate cameras.
Digital signage	Allows simultaneous changes or updates to centrally managed information that is electronically displayed across several locations.

To run the use cases, do the following:

Complete prerequisites

Ensure that you complete the Prerequisites.

Prepare input video files

Add at least one MP4 AVC (H.264) video file on the target device.

Run the use case

In the terminal of the target device, run:

gst-concurrent-videoplay-composition -c <number-of-streams> -i <input-file-1>.mp4 -i <input-file-2>.mp4

Command-line parameters

Parameter	Description
`-c`	Number of streams to decode for composition. Supported values: `2`, `4`, `8`, `16`.
`-i`	Absolute path to an input video file.

Examples

Concurrent playback of two sessions

gst-concurrent-videoplay-composition -c 2 -i /etc/media/video_avc.mp4 -i /etc/media/video_avc.mp4

Concurrent playback of four sessions

gst-concurrent-videoplay-composition -c 4 -i /etc/media/video_avc.mp4 -i /etc/media/video_avc.mp4 -i /etc/media/video_avc.mp4 -i /etc/media/video_avc.mp4

4. Stop the running use case

Press Ctrl + C.

5. In the terminal of the target device, run the following command to display the available help options:

gst-concurrent-videoplay-composition --help

Expected output

The individual composed streams are tiled together to display as a unified stream.

Figure: Expected output for gst-concurrent-videoplay-composition application

Pipeline flow

The following table lists the plugins used to run the video wall pipeline:

Plugin	Description
filesrc	Reads the video data.
qtdemux	Demultiplexes the video data.
h264parse	Parses the H.264 video streams.
v4l2h264dec	Decodes the H.264 video streams.
qtivcomposer	Composes the decoded streams and sends them over the display.
waylandsink	Displays the composed video streams on Wayland display.

Known issue

Video playback closes or freezes. About an hour later, it continues running with 16 H.264 720p concurrent video playback sessions.

Weston Composition

The gst-weston-composition-example application allows you to arrange live camera input or offline files on the screen using the picture-in-picture (PiP) or side by side display formats. A few use cases of this application are video conferencing and online educational courses. This application involves several pipelines and elements. For more information, see Pipeline flow.

Figure: gst-weston-composition-example pipeline Application: gst-weston-composition-example

Try me

Run the application

The following table lists the use cases that supported through the gst-weston-composition-example application:

Use case	Description
Video conferencing	• Simultaneously displays several video streams from many participants. • Videos are displayed in either PiP or side-by-side format. • Useful in a multi-party video conference scenario where the application manages several video streams.
Online education	Enhances the learning experience by enabling the PiP or side-by-side display of the instructional video.

Complete prerequisites

Ensure that you complete the Prerequisites.

Input a <h264_file>.mp4 video file on the target device.

In the terminal of the target device, run any of the following use cases:

Compose PiP:

gst-weston-composition-example -t 0 -i /etc/media/video_avc.mp4

Compose side by side:

gst-weston-composition-example -t 1 -i /etc/media/video_avc.mp4

To Stop the running use case

Press Ctrl + C.

In the terminal of the target device, run the following command to display the available help options:

gst-weston-composition-example --help

The following table provides a description of the command-line parameters:

Command-line Parameters

Command-line parameter	Description
`-t`	Specifies the type of composition: • `0` = Picture in picture • `1` = Side by side
`-i`	Specifies the absolute path to the input video file.

Expected output

The video is rendered as PiP and side by side display on the screen.

Figure: Expected output for gst-weston-composition-example application

Pipeline flow

The following table lists the plugins used to run the video composition pipeline:

Plugin	Description
`qticamsrc`	Captures the video from a source.
`capsfilter`	Filters the video file.
`filesrc`	Reads the video data.
`qtdemux`	Demultiplexes the video data.
`h264parse`	Parses the H.264 video.
`v4l2h264dec`	Decodes H.264 video from a file.
`qtivcomposer`	Composes the decoded video.

Multi format decode

Description: The gst-videocodec-concurrent-playback application supports concurrent video playback for AVC (H.264) and HEVC (H.265) videos. It demonstrates the concurrent decoding capabilities of the Qualcomm Linux video engine, which can process different video codecs simultaneously. As shown in the following figure, the sample application uses H.264 and H.265 videos, both encapsulated in MP4 containers. The pipeline involves several elements for concurrent video playback. For more information about these elements, see Pipeline flow.

Figure: gst-videocodec-concurrent-playback pipeline Application: gst-videocodec-concurrent-playback

Try me

Run the application

The following table lists the use cases that are supported through the gst-videocodec-concurrent-playback application:

Use case	Description
Video conferencing	• Simultaneously decodes the video streams of several participants. • Useful in a multi-party video conference scenario where the application manages several video streams.
Surveillance systems	Concurrently decodes video feeds from two separate cameras, each employing a different encoding method.

Complete prerequisites

Ensure that you complete the Prerequisites.

Input MP4 AVC (H.264) and HEVC (H.265) video files on the target device.

In the terminal of the target device, run the use case:

gst-videocodec-concurrent-playback -i /etc/media/video_avc.mp4 -i /etc/media/video_hevc.mp4 -o /etc/media/h265_dump.yuv

To Stop the running use case

Press Ctrl + C.

In the terminal of the target device, run the following command to display the available help options:

gst-videocodec-concurrent-playback --help

Note: Due to the large size of YUV data, it’s recommended to run this use case for a short duration (less than 5 seconds) to avoid exhausting the storage space.

Command-line parameters

Command-line parameter	Description
`-i`	Specifies the absolute path to the H.264 and H.265 input video files. Ensure that the H.264 input file is specified before H.265.
`-o`	Specifies the absolute path to the H.265 output YUV file.

Expected output

The H.264 video is displayed on the Wayland display.
The H.265 video is written to the YUV file.

To play the YUV file:

Pull the YUV dump file from the device.
Use a YUV player running on the host to play the video. For example, select YUV420planarNV12 format, and use 1920 and 1088 as the stride and scan lines.

Note: If a default YUV player is not available on the host computer, use Raw YUV Player.

Pipeline flow

The following table lists the plugins used to run the videocodec concurrent playback pipeline:

Plugin	Description
`filesrc`	Reads the video data.
`qtdemux`	Demultiplexes the video data.
`h264parse`	Parses the H.264 video stream.
`h265parse`	Parses the H.265 video stream.
`v4l2h264dec`	Decodes the H.264 video stream using Video4Linux.
`v4l2h265dec`	Decodes the H.265 video stream using Video4Linux.
`waylandsink`	Displays the H.264 video stream on Wayland display.
`filesink`	Writes the H.265 video to a YUV file for playback.

Known issues

Video playback may not work as expected when the stream is generated from a file source.

JPEG Decode

Description: The gst-jpg-decode-example application allows you to decode JPEG images and view the decoded images on a screen. The following figure shows the pipeline for gst-jpg-decode-example. For information about the plugins used in the pipeline flow, see Pipeline flow.

Figure: gst-jpg-decode-example pipeline Application: gst-jpg-decode-example

Try me

Run the application

Complete prerequisites

Ensure that you complete the Prerequisites.

Input at least one imagefiles_%d.jpg on the target device.

In the terminal of the target device, run the use case. Specify the application with parameters such as width and height.

gst-jpg-decode-example -w 1920 -h 1080 -i /etc/media/imagefiles_%d.jpg

To Stop the running use case

Press Ctrl + C.

In the terminal of the target device, run the following command to display the available help options:

gst-jpg-decode-example --help

Expected output

The decoded image is displayed on the screen for preview.

Pipeline flow

The following table lists the plugins used to run the decode JPEG images pipeline:

Plugin	Description
multifilesrc	Reads the video data from sequentially named files.
capsfilter	Enforces constraints on the video data.
jpegdec	Decodes the JPEG video stream.
videoconvert	Converts the video frames from one format to another.
waylandsink	Displays the video stream on the Wayland display.

Audio Playback

Description: The gst-audio-decode-example application demonstrates the playback of audio files in WAV, MP3, and FLAC formats. Separate pipelines for each audio format ensure efficient processing and compatibility. This sample application provides a basic template that can be extended and adapted. The following figure shows the pipeline, which reads and parses the audio file, and then decodes and plays the audio. For information about the plugins used in the pipeline, see Pipeline flow.

Figure: gst-audio-decode-example pipeline Application: gst-audio-decode-example

Try me

Run the application

Complete prerequisites

Ensure that you complete the Prerequisites.

Input at least one of these audio files on the target device: <filename>.mp3, <filename>.wav, or <filename>.flac.

In the terminal of the target device, run the application:

gst-audio-decode-example -i /etc/media/<filename> --audio_format=<format>

To stop the running use case

Press Ctrl + C.

In the terminal of the target device, run the following command to display available help options:

gst-audio-decode-example --help

Command-line parameters

Command-line parameter	Description
`-i`	Specifies the absolute path to the input audio file to decode.
`--audio_format`	Specifies the format of the audio file: • `1` = MP3 • `2` = WAV • `3` = FLAC

For example, run the application as follows:

Decode MP3:

gst-audio-decode-example -i /etc/media/audio.mp3 --audio_format=1

Decode WAV:

gst-audio-decode-example -i /etc/media/audio.wav --audio_format=2

Decode FLAC:

gst-audio-decode-example -i /etc/media/audio.flac --audio_format=3

Expected output

Based on the selected format (MP3, WAV, or FLAC), the decoded audio file is played back.

Pipeline flow

The following table lists the plugins used to run the audio decode pipeline:

Plugin	Description
filesrc	Reads the audio files.
mpegaudioparse	Parses MPEG audio files.
wavparse	Parses WAV audio files.
flacparse	Parses FLAC audio files.
mpg123audiodec	Decodes MP3 files.
flacdec	Decodes FLAC files.
pulsesink	Plays the decoded audio.

Known issues

Audio playback may not work if the audio sink isn’t selected using wpctl.

Audio Encode

Description: The gst-audio-encode-example application encodes audio in FLAC or WAV format. A common use case for this application can be to encode live audio data for streaming or storage. The following figure shows the pipeline, which captures the audio, converts the stream into FLAC or WAV formats, encodes the stream, and stores it in a file. For more information about the plugins used in the pipeline, see Pipeline flow.

Figure: gst-audio-encode-example pipeline Application: gst-audio-encode-example

Try me

Run the application

For this use case, if you have a live audio source (such as a microphone or an internet radio stream), you can encode this audio data into a more compact or standardized format for efficient transmission or storage.

Complete prerequisites

Ensure that you complete the Prerequisites.

In the terminal of the target device, run the application for different audio formats:

gst-audio-encode-example -o <path-file> --audio_format=<format>

To stop the running use case

Press Ctrl + C.

In the terminal of the target device, run the following command to display available help options:

gst-audio-encode-example --help

In the terminal of the host computer,run the following command to pull files from the target device:

scp root@<IP address of target device>:/etc/media/<destination directory>

Command-line parameters

Command-line parameter	Description
`-o`	Specifies the absolute path to the output audio file.
`--audio_format`	Specifies the format of the audio file: • `1` = FLAC • `2` = WAV

For example, use the following commands to run the application for FLAC and WAV audio formats::

Encode FLAC audio:

gst-audio-encode-example -o /etc/media/audio_record.flac.flac --audio_format=1

Encode WAV audio:

gst-audio-encode-example -o /etc/media/audio_record.flac.wav --audio_format=2

Expected output

The captured audio is encoded and written to a file based on the selected format (FLAC or WAV).

Pipeline flow

In audio encoding, the pipeline differs slightly based on the encoding format. The following table lists the plugins used to run the audio recording pipeline:

Plugin	Description
pulsesrc	Captures audio from the system sound device.
capsfilter	Ensures that the audio is in a format suitable for the encoder.
audioconvert	Converts the audio to the required format.
flacenc	Encodes audio into FLAC format.
wavenc	Encodes audio into WAV format.
filesink	Writes the encoded audio to a file.

Known issues

Audio encode may not work if the audio sink is not selected using wpctl.

Audio-Video Encode

Description: The gst-audio-video-encode application encodes audio and video data, which is stored in a user-specified output file. A few use cases for this application are streaming services, video conferencing, and surveillance systems. The following figure shows the pipeline, which receives the video and audio feed, encodes, and parses the video stream. It then captures the audio stream and encodes it in MP3 format. Both streams are multiplexed and written to a file. For information about the plugins used in the pipeline, see Pipeline flow.

Figure: gst-audio-video-encode pipeline Application: gst-audio-video-encode

Try me

Run the application

The following table lists the actions that can be performed for common use cases of the gst-audio-video-encode application:

Use case	Actions
Streaming services	Encode audio and video streams for live internet streaming.
Security camera	Invert or rotate the video image to compensate for situations where the camera is improperly mounted.
Augmented reality	Scale or rotate the video feed to synchronize with virtual components.

Complete prerequisites

Ensure that you complete the Prerequisites.

In the terminal of the target device, run the use case:

gst-audio-video-encode -w <width> -h <height> -c <inputcodec> -o /etc/media/<output-file>.mp4

To stop the running use case

Press Ctrl + C.

In the terminal of the target device, run the following command to display available help options:

gst-audio-video-encode --help

In the terminal of the host computer,run the following command to pull files from the target device:

scp root@<IP address of target device>:/etc/media/<destination directory>

Command-line parameters

Command-line parameter	Description
`-w`	Specifies the camera width.
`-h`	Specifies the camera height.
`-c`	Specifies the video codec: • `1` = AVC (H.264) • `2` = HEVC (H.265)
`-o`	Provides the absolute path to the output file.

For example, run different video encode use cases as follows:

AVC:

gst-audio-video-encode -w 1920 -h 1080 -c 1 -o /etc/media/audiovideo.mp4

HEVC:

gst-audio-video-encode -w 1920 -h 1080 -c 2 -o /etc/media/audiovideo.mp4

Expected output

The encoded audio and video streams are multiplexed into an MP4 container and written to a file.

Pipeline flow

The following table lists the plugins used to run the audio and video stream encoding pipeline:

Plugin	Description
`qticamsrc` and `capsfilter`	Captures the media stream.
`v4l2h264enc`	Encodes video for AVC (H.264) format.
`v4l2h265enc`	Encodes video for HEVC (H.265) format.
`h264parse`	Parses the H.264 video stream.
`h265parse`	Parses the H.265 video stream.
`pulsesrc`	Captures the audio stream, filtered using capsfilter.
`audioconvert`	Converts the audio for encoding.
`lamemp3enc`	Encodes the audio in MP3 format.
`mp4mux`	Multiplexes encoded video and audio into an MP4 container.
`filesink`	Writes the stream into a file.

Known issues

Audio and video encode may not work if the sink is not selected using wpctl.

Audio-Video Playback

The gst-audio-video-playback application supports simultaneous audio and video decoding and playback of FLAC/MP3 audio and AVC/HEVC video. The following are the applicable scenarios to use the gst-audio-video-playback application:

Media player applications
Streaming services
Video conferencing
Surveillance systems
Educational platforms
Entertainment platforms that can play back audio and video files in various formats

As shown in the following figure, the pipeline involves several elements for audio and video playback. For more information about these elements, see Pipeline flow.

Figure: gst-audio-video-playback pipeline Application: gst-audio-video-playback

Try me

Run the application

Complete prerequisites

Ensure that you complete the Prerequisites.

Input these video files on the target device: <avc_flac_file>.mp4, <hevc_flac_file>.mp4, <avc_mp3_file>.mp4, and <hevc_mp3_file>.mp4.

In the terminal of the target device, run the use case:

gst-audio-video-playback -v 1 -a 1 -i /etc/media/<filename>.mp4

To stop the running use case

Press Ctrl + C.

In the terminal of the target device, run the following command to display available help options:

gst-audio-video-playback --help

Command-line parameters

Command-line parameter	Description
`-v`	Specifies the video codec: `1` = AVC (H.264) `2` = HEVC (H.265)
`-a`	Specifies the audio codec: `1` = FLAC `2` = MP3
`-i`	Specifies the absolute path to the MP4 input file.

For example, run the following for different audio/video formats:

AVC video with FLAC audio:

gst-audio-video-playback -v 1 -a 1 -i /etc/media/avc_flac.mp4

AVC video with MP3 audio:

gst-audio-video-playback -v 1 -a 2 -i /etc/media/avc_mp3.mp4

HEVC video with FLAC audio:

gst-audio-video-playback -v 2 -a 1 -i /etc/media/hevc_flac.mp4

HEVC video with MP3 audio:

gst-audio-video-playback -v 2 -a 2 -i /etc/media/hevc_mp3.mp4

Expected output

The AVC/HEVC video is displayed on the screen.
The audio plays through the device speaker. Figure: Expected output for gst-audio-video-playback application

Pipeline flow

The following table lists the plugins used to run the audio-video playback pipeline:

Plugin	Description
`filesrc`	Reads the video data.
`qtdemux`	Demultiplexes the video data into separate audio and video streams.
`h264parse`	Parses the H.264 video stream.
`h265parse`	Parses the H.265 video stream.
`v4l2h264dec`	Decodes the H.264 video stream.
`v4l2h265dec`	Decodes the H.265 video stream.
`waylandsink`	Displays the video stream on Wayland display.
`flacparse` or `mpegaudioparse`	Parses the audio stream.
`flacdec` or `mpg123audiodec`	Decodes the audio stream.
`pulsesink`	Allows playback of the audio stream.

Known issues

Audio and video playback may not work if the sink is not selected using wpctl.

​Prerequisites

​Camera Applications

​Single Camera Streaming

​Run the application

​Expected output

Pipeline flow

​Multi-Camera

​Run the application

​Expected Output

Pipeline flow

​Camera Switch

​Run the application

​Expected output

Pipeline flow

​Multi-Stream

​Run the application

​Expected output

Pipeline flow

​Known Issues

​Wide dynamic range and electronic image stabilization

​Run the application

​Expected output

Pipeline flow

​Overview of camera imaging features

​Snapshot Stream

​Run the application

​Expected output

Pipeline flow

​Camera frame processing

​Run the application

​Expected output

Pipeline flow

​Camera metadata

​Run the application

​Expected output

Pipeline flow

​Known issue

​Camera stream configuration

​Run the application

​Expected output

Pipeline flow

​Known issue

​Camera Burst Capture

​Run the application

​Expected output

Pipeline flow

​Known issue

​RTMP Streaming

​Run the application

​Expected output

Pipeline flow

​Smart Codec

​Run the application

​Expected output

Pipeline flow

​GstD single camera streaming

​Run the application

​Expected output

Pipeline flow

​Dynamic configuration

​Activate-deactivate streams at runtime

​Run the application

​Expected output

Pipeline flow

​Add-remove streams at runtime

​Run the application

​Expected output

Pipeline flow

​Video and Audio Applications

​Video playback

​Run the application

​Expected output

Pipeline flow

​Video Transcode

​Run the application

​Expected output

Pipeline flow

​Video Transform

​Expected output

Pipeline flow

Prerequisites

Camera Applications

Single Camera Streaming

Run the application

Expected output

Multi-Camera

Run the application

Expected Output

Camera Switch

Run the application

Expected output

Multi-Stream

Run the application

Expected output

Known Issues

Wide dynamic range and electronic image stabilization

Run the application

Expected output

Overview of camera imaging features

Snapshot Stream

Run the application

Expected output

Camera frame processing

Run the application

Expected output

Camera metadata

Run the application

Expected output

Known issue

Camera stream configuration

Run the application

Expected output

Known issue

Camera Burst Capture

Run the application

Expected output

Known issue

RTMP Streaming

Run the application

Expected output

Smart Codec

Run the application

Expected output

GstD single camera streaming

Run the application

Expected output

Dynamic configuration

Activate-deactivate streams at runtime

Run the application

Expected output

Add-remove streams at runtime

Run the application

Expected output

Video and Audio Applications

Video playback

Run the application

Expected output

Video Transcode

Run the application

Expected output

Video Transform

Expected output

Plugins Overview

Concurrent Playback & Composition(video wall)

Run the application

Command-line parameters

Examples

Expected output

Known issue

Weston Composition

Run the application

Command-line Parameters

Expected output

Multi format decode

Run the application

Command-line parameters

Expected output

Known issues

JPEG Decode

Run the application