LLMs/VLMs using GenieX - Qualcomm Dragonwing Documentation

GenieX is the Qualcomm AI Hub command line and serving layer for generative models. On a Dragonwing ARM64 Linux host it runs models natively against the device NPU through the Qualcomm driver libraries. This page takes you from a freshly imaged IQ9 board to Gemma 4 E4B answering prompts on the NPU. Expect around 15 minutes of setup, most of it spent downloading the ~5 GB model bundle. You will:

Install the system and Qualcomm driver packages, then the geniex CLI.
Pull the Gemma 4 E4B bundle from AI Hub.
Run text, image, and audio inference, and optionally serve an OpenAI-compatible API.

Every step on this page runs directly on the IQ9 device, not on a development host.

Open a session on the board:

ssh ubuntu@<device-ip>

Prerequisites

Requirement	Value
Architecture	Linux ARM64 (`aarch64`)
Operating system	Ubuntu image for Dragonwing IQ-9075 EVK, or any device with a Qualcomm BSP
Chipset	Dragonwing IoT chipset — IQ-9075 / QCS9075 in this blog
Free storage	≥ 12 GB
RAM	≥ 12 GB
Network	Outbound HTTPS to `qaihub-public-assets.s3.us-west-2.amazonaws.com`

Confirm the architecture and available space before continuing:

uname -m      # expected: aarch64
df -h /       # confirm at least 12 GB available

If the board is not yet brought up, complete device setup first — see the IQ-9075 EVK setup guide. This is only applicable to IQ9 and please refer GenieX page for other platforms and select the supported model.

Install GenieX

Install standard APT packages

sudo apt update
sudo apt install -y libatomic1 libglib2.0-0 ocl-icd-libopencl1

ocl-icd-libopencl1 is replaced by qcom-adreno1 in the next step; installing it first keeps the dependency resolver happy on minimal images.

Install the Qualcomm driver packages

geniex reaches the NPU through the Qualcomm proprietary driver libraries, delivered from the ubuntu-qcom-iot PPA.

sudo apt-get install -y qcom-adreno1 qcom-fastrpc1 libqnn1

Library	Provided by
`libOpenCL_adreno.so.1`, `libCB.so.1`, `libadreno_utils.so.1`, `libgsl.so.1`	`qcom-adreno1`
`libcdsprpc.so.1.0.0`	`qcom-fastrpc1`
QNN backend libraries (`libQnnHtp.so`, `libQnnSystem.so`)	`libqnn1`

The PPA (ppa:ubuntu-qcom-iot/qcom-ppa) is pre-configured on the IQ9075 EVK Ubuntu image. On a custom image, add it with sudo add-apt-repository -y ppa:ubuntu-qcom-iot/qcom-ppa followed by sudo apt update.

These three packages are the most common cause of installation failure. If a later step reports a missing .so, return here.

Run the installer

If HOME is unset — common in minimal containers or sudo -s sessions — export it first:

export HOME=/home/ubuntu

curl -fsSL https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-geniex/install.sh | sh

The script downloads the latest stable release, verifies its SHA256 checksum, and installs without sudo.

Add GenieX to your PATH

The installer prints the exact export line if the launcher directory is not already on your PATH. For example:

echo 'export PATH="/usr/local/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc

Optional: install SoX for microphone input

Needed only for the /mic command in an interactive session. Loading audio from disk works without it.

sudo apt install -y sox

Verify the installation

geniex --help

If geniex is not found, open a new shell or apply the PATH line the installer printed. If the command fails on a missing shared library such as libCB.so.1 or libcdsprpc.so.1.0.0, the Qualcomm driver packages are not installed — see step 2. Add --log to any command to raise the log level. The flag is equivalent to the GENIEX_LOG environment variable and takes precedence over it.

Level	Emits
`none`	nothing (default)
`error`	errors only
`warn`	warnings and errors
`info`	informational messages and above
`debug`	debug messages and above
`trace`	everything

Download the model

geniex pull google/gemma-4-E4B-it-qat-q4_0-gguf

This transfers roughly 5 GB, so allow several minutes on a typical link. If the transfer is interrupted, re-run the same command — it resumes rather than starting over. The general syntax is:

geniex pull <model-name>[:<precision>]

Flag	Purpose
`--model-hub`	Model source: `aihub`, `hf`, or `localfs`. Auto-detected when omitted.
`--local-path`	Register a bundle already present on disk.

For GGUF models the CLI prompts for a precision when more than one is published. Choose Q4_0 for IQ9 — it is the quantization-aware-trained build and gives the best accuracy per byte on the NPU. Pin it inline to skip the prompt in scripts:

geniex pull google/gemma-4-E4B-it-qat-q4_0-gguf:Q4_0

Confirm the result:

geniex list

pull copies files into the GenieX cache. After a successful --local-path pull you can delete the source directory rather than keeping two copies of a ~5 GB model.

Bundle contents

GenieX manages the GGUF bundle for you. It contains the quantized *.gguf weights, an mmproj-*.gguf multimodal projector for image and audio input, tokenizer metadata embedded in the GGUF container, and a manifest recording precision, runtime, and compute defaults. A Genie/QAIRT bundle — the w4a16 build, or output from the Jupyter path — is instead explicit, and must contain:

File	Purpose
`genie_config.json`	Backend selection, model and tokenizer paths, context and token limits
`htp_backend_ext_config.json`	HTP/NPU backend settings (SoC ID, DSP architecture, performance mode)
`tokenizer.json`	Tokenizer vocabulary and merge rules
`*.bin`	Prompt-processor and token-generator context binaries

Sample Genie configurations are published in the AI Hub Apps repository; for field definitions see the Genie dialog JSON reference.

Run inference

Start an interactive chat session:

geniex infer google/gemma-4-E4B-it-qat-q4_0-gguf

The model loads onto the NPU and you get a prompt. Type a message and press Enter. Pass a single prompt and exit instead — useful for scripts and smoke tests:

geniex infer google/gemma-4-E4B-it-qat-q4_0-gguf \
  -p "Summarize what a Hexagon NPU does in three sentences."

Common flags

Flag	Effect
`-p "<prompt>"`	Run one prompt and exit instead of opening a session.
`--think`	Show intermediate reasoning before the answer.
`--think=false`	Answer directly. Recommended in production.
`--compute npu`	Run on the Hexagon NPU. This is the default.
`--compute cpu` / `--compute gpu`	Run on CPU or GPU. Valid for GGUF builds only; useful for A/B comparison or isolating an NPU driver problem.
`--log <level>`	Raise the log level for diagnostics.

Multimodal prompts

The q4_0 bundle ships an audio-capable projector, so a single prompt can carry both an image and an audio clip. Download two sample files into your home directory:

cd ~
curl -L -o jfk.wav https://github.com/ggml-org/whisper.cpp/raw/master/samples/jfk.wav
curl -L -o landmark.jpg "https://images.pexels.com/photos/402028/pexels-photo-402028.jpeg?w=1024"

Reference them by absolute path in the prompt:

geniex infer google/gemma-4-E4B-it-qat-q4_0-gguf \
  -p "Describe the image and transcribe the audio. Image: $HOME/landmark.jpg Audio: $HOME/jfk.wav"

Expected output:

**Image Description:**
This is a scenic, panoramic photograph that features a traditional Japanese temple ...

**Audio Transcription:**
And so my fellow Americans, ask not what your country can do for you, ask what you
can do for your country.

Always use absolute paths for image and audio inputs. Relative paths resolve against the process working directory and are a frequent source of “file not found” errors.

In an interactive session, /mic records a clip instead of loading one from disk; Ctrl-C stops recording and transcribes it. This requires SoX on your PATH.

QAIRT models report audio: false. Passing one an audio file fails with GenieXError(-201201): Multimodal generation failed. Use the GGUF build for audio.

Serve an OpenAI-compatible endpoint

For application integration, run the local server instead of the interactive CLI:

geniex serve

The server prints the address and port it is listening on at startup. Substitute those values below:

curl http://<device-ip>:<port>/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "google/gemma-4-E4B-it-qat-q4_0-gguf",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "What is the capital of Italy?"}
    ],
    "stream": false
  }'

Any OpenAI-compatible client or framework works, including LangChain and Open WebUI. Set the per-request reasoning_format field to move a thinking model’s chain-of-thought out of message.content into message.reasoning_content, which keeps the rendered answer clean while preserving the reasoning for logs.

Test with curl from the device itself first. If the endpoint works locally but not remotely, the server is bound to loopback or a firewall is blocking the port.

Performance and best practices

On IQ9 the reference figures are a 4096-token context, ~660 tokens/s prefill, and ~17.9 tokens/s decode. Because decode is roughly 37× slower than prefill, output length dominates response time far more than prompt length.

Reuse the loaded model. Model load is the largest fixed cost — use geniex serve or a long-lived interactive session rather than invoking geniex infer per request.
Cap output length. Ask for “three bullet points” rather than “explain in detail” when latency matters.
Keep prompts short. Prefill is fast, but prompt tokens still consume context that generation needs.
Disable thinking mode (--think=false) unless you consume the reasoning trace; it can multiply the number of generated tokens.
Right-size the context. KV-cache memory scales with context length, so do not configure a large window for a task that never needs it.
Watch thermals. Sustained generation raises SoC temperature and triggers throttling; measure steady-state throughput, not just the first request.
Keep storage headroom. Provision roughly double the ~5 GB bundle size to allow for upgrades.

Verify the result

#	Check	Command	Expected
1	Correct architecture	`uname -m`	`aarch64`
2	Drivers installed	`apt list --installed qcom-adreno1 qcom-fastrpc1 libqnn1`	All three present
3	CLI works	`geniex --help`	Usage text, no library errors
4	Model in cache	`geniex list`	Gemma 4 E4B listed
5	Inference returns text	`geniex infer google/gemma-4-E4B-it-qat-q4_0-gguf -p "Reply with OK."`	A short coherent reply within seconds of model load
6	NPU backend selected	`geniex --log debug infer google/gemma-4-E4B-it-qat-q4_0-gguf -p "Hi"`	NPU backend in the log, not a CPU fallback

If any check fails, see Troubleshooting.

Next step

The published bundle is now running on the NPU. If it meets your accuracy, context length, and licensing requirements, you are done — move on to integrating it with your application. If you need a custom quantization recipe, a longer context, or your own checkpoint, continue to the Jupyter notebook path. Otherwise go to Troubleshooting and next steps.

​Prerequisites

​Install GenieX

​Verify the installation

​Download the model

​Bundle contents

​Run inference

​Common flags

​Multimodal prompts

​Serve an OpenAI-compatible endpoint

​Performance and best practices

​Verify the result

​Next step

Prerequisites

Install GenieX

Verify the installation

Download the model

Bundle contents

Run inference

Common flags

Multimodal prompts

Serve an OpenAI-compatible endpoint

Performance and best practices

Verify the result

Next step