> ## Documentation Index
> Fetch the complete documentation index at: https://dragonwingdocs.qualcomm.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Overview
> An overview of the tools, runtimes, and frameworks for building AI applications on Qualcomm platforms, with a guided path for choosing how to prepare and run your model.
This guide covers AI application development using the tools, runtimes, and frameworks supported on Qualcomm Dragonwing IoT platforms. It is intended for developers who want to train or fine-tune models, prepare them for deployment, and build AI applications on Qualcomm hardware.
Most decisions in this guide come down to three questions:
* **What are you building?** Classic ML/vision, generative AI (LLM/VLM), or voice/speech AI — each has its own tools and quick-start paths.
* **How will you prepare your model?** Convert, quantize, compile, or fine-tune it for the target hardware.
* **How will you run inference?** The runtime, container, or integration path that executes the model on the device.
You can bring pretrained models from ONNX, PyTorch, TensorFlow, or LiteRT — or start from Qualcomm's ready-to-run LLM, VLM, ASR, and TTS models — and run them efficiently across the Qualcomm Kryo™ CPU, Adreno™ GPU, and Hexagon™ NPU (HTP).
We assume device setup is done. If not please follow these links to setup ([IQ-9075 EVK](https://dragonwingdocs.qualcomm.com/Ubuntu/devices/iq9075-evk/set-up-the-device) or [IQ-8275 EVK](https://dragonwingdocs.qualcomm.com/Ubuntu/devices/iq8275-evk/set-up-the-device)) the device and update ([IQ-9075 EVK](https://dragonwingdocs.qualcomm.com/Ubuntu/devices/iq9075-evk/update-software/overview) or [IQ-8275 EVK](https://dragonwingdocs.qualcomm.com/Ubuntu/devices/iq8275-evk/update-software/overview)) the software.
## Choose your journey
Use the flowchart below to find the path that fits your application. It walks you through whether you already have a model, what you're building, how to prepare it, and how to run inference, then links you to the right Ubuntu workflow page. Every highlighted box is clickable.
```mermaid theme={null}
%%{init: {"flowchart": {"rankSpacing": 110, "nodeSpacing": 65}} }%%
flowchart TD
Q1{"Do you already have
a trained model?"}
Q1 -->|"No, just show me
what it can do"| RunPrebuilt["Run prebuilt AI models & apps
Qdemo UI · prebuilt samples · IM SDK"]
Q1 -->|"I want to fine tune a model
with my own data"| EI["Edge Impulse"]
Q1 -->|"Yes / bring my own
ONNX · PyTorch · TF · LiteRT"| TypeQ{"What are you building?"}
TypeQ -->|"Classic ML / vision"| Prep{"Prepare the model"}
TypeQ -->|"Generative AI
LLM · VLM"| GenAI["On-device GenAI workflows
Genie · Llama.cpp · OpenAI-compatible container"]
TypeQ -->|"Voice / speech AI"| VoiceQ{"What do you need?"}
VoiceQ -->|"ASR + TTS +
translation"| VoiceContainer["Audio Analytics container"]
VoiceQ -->|"Specifically
Whisper ASR"| Whisper["Whisper
VoiceAI ASR (NPU) or whisper.cpp (CPU/GPU)"]
Prep -->|"Ready model, or BYOM
in the cloud"| AIHub["Qualcomm AI Hub
ready models · BYOM
(optimize in the cloud)"]
Prep -->|"Convert + quantize
my own (local)"| QAIRT["Qualcomm AI Runtime SDK
(QAIRT)"]
Prep -->|"Recover quantized
accuracy"| AIMET["AIMET (PTQ / QAT)"]
AIHub --> Runtime{"Run inference"}
QAIRT --> Runtime
AIMET --> Runtime
Runtime -->|"Python / quick .tflite"| LiteRT["LiteRT / TFLite
AI Engine Direct delegate"]
Runtime -->|"ONNX models"| ONNXRun["ONNX Runtime
AI Engine Direct"]
Runtime -->|"C++ / low-level control
SNPE or QNN"| QAIRTcpp["QAIRT SDK C++ APIs"]
Runtime -->|"Camera / video +
AI pipeline"| IMSDK["Qualcomm IM SDK
GStreamer · zero-copy · GPU pre/post"]
click RunPrebuilt "../map/run-prebuilt-models-and-apps"
click EI "../topic/edge-impulse"
click GenAI "../map/run-on-device-genai"
click VoiceContainer "../map/run-on-device-voice-ai"
click Whisper "../topic/whisper"
click AIHub "../topic/ai-hub"
click QAIRT "../topic/qairt"
click AIMET "../topic/aimet"
click LiteRT "../topic/litert-overview"
click ONNXRun "../topic/onnxruntime"
click QAIRTcpp "../topic/develop-your-own-application-qairt-cpp"
click IMSDK "../topic/develop-your-own-application-im-sdk"
classDef prep fill:#31017D,stroke:#31017D,color:#fff;
classDef run fill:#3253DC,stroke:#3253DC,color:#fff;
classDef genai fill:#B45309,stroke:#B45309,color:#fff;
classDef voice fill:#0F766E,stroke:#0F766E,color:#fff;
class AIHub,QAIRT,AIMET,EI prep;
class RunPrebuilt,LiteRT,ONNXRun,QAIRTcpp,IMSDK run;
class GenAI genai;
class VoiceContainer,Whisper voice;
```
On-device generative AI availability depends on your Dragonwing development board and model. Start with [LLMs using Genie](../topic/use-genai-model-with-genie), and use [Llama.cpp](../topic/llama-cpp) as a fallback where Genie model support is not available. For voice/speech AI, see the [Voice AI workflow](../map/run-on-device-voice-ai) and [Whisper](../topic/whisper).
## Application development & execution flow summary
| Flow | Purpose |
| ---------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Edge Impulse](../topic/edge-impulse) | Build and train AI models using audio, image, and other sensor data, or bring your own model in a variety of formats. |
| [Qualcomm AI Hub](../topic/ai-hub) | Download preoptimized models, or bring your own model (BYOM) and optimize, validate, and deploy on hosted Qualcomm platform devices within minutes. |
| [Qualcomm AI Runtime SDK (QAIRT)](../topic/qairt) | Convert, quantize, and compile models from TensorFlow, PyTorch, LiteRT, or ONNX locally for deployment on the NPU. |
| [AIMET (PTQ / QAT)](../topic/aimet) | Recover accuracy lost during quantization using post-training quantization or quantization-aware training. |
| [LiteRT / TFLite](../topic/litert-overview) | Run `.tflite` models on the NPU (Python or C++) using AI Engine Direct delegates. Works with models from AI Hub, QAIRT, or Edge Impulse. |
| [ONNX Runtime](../topic/onnxruntime) | Run ONNX models on the NPU using ONNX Runtime with AI Engine Direct. |
| [Run models with SNPE or QNN](../topic/run-models) | Run converted `.dlc` or `.so` files on CPU, GPU, or HTP using the QAIRT SDK C/C++ APIs. |
| [Qualcomm IM SDK](../topic/develop-your-own-application-im-sdk) | Build high-performance camera and video AI pipelines using GStreamer, zero-copy buffers, and GPU pre/post-processing. |
| [Llama.cpp](../topic/llama-cpp) | Run LLMs and VLMs locally using a C++ backend optimized for CPU/GPU and quantized formats. |
| [Genie / GenieX](../topic/use-genai-model-with-genie) | Orchestrate on-device generative AI (LLM/VLM) workflows and run text, image, and audio inference on the NPU. |
| [LLM/VLM using Qualcomm OpenAI container](../topic/llm-vlm-qcom-setup) | Run LLMs/VLMs behind a drop-in OpenAI-compatible API on the NPU. |
| [Whisper](../topic/whisper) | Speech transcription, translation, and language identification using the NPU (VoiceAI ASR) or CPU (whisper.cpp). |
| [Audio Analytics container](../topic/audio-analytics-overview) | Speech recognition (ASR), text-to-speech (TTS), and language translation on the NPU. |
## Prepare your model
Before a model runs efficiently on Qualcomm hardware, it is converted to an executable format and, for the Hexagon NPU (HTP), quantized to a supported precision. (LiteRT models are an exception: they run directly through the AI Engine Direct delegate.) Choose the preparation tool that matches your starting point.
| Tool | Use it to | Output |
| ------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------ |
| [Qualcomm AI Hub](../topic/ai-hub) | Download a preoptimized model, or **bring your own model (BYOM)** and have QAIRT **compile, convert, and quantize it in the cloud** for your target chipset, with no local toolchain required. | Ready-to-run LiteRT or Qualcomm AI Engine Direct model |
| [Qualcomm AI Runtime SDK (QAIRT)](../topic/qairt) | The **local** alternative to AI Hub BYOM: convert, quantize, and compile models from TensorFlow, PyTorch, LiteRT, or ONNX yourself. Integrates AI Engine Direct and the Neural Processing SDK. | Quantized model / compiled context binary |
| [Qualcomm AI Model Efficiency Toolkit](../topic/aimet) | Recover accuracy lost during quantization using post-training quantization (PTQ) and quantization-aware training (QAT). | Higher-accuracy quantized model |
| [Edge Impulse](../topic/edge-impulse) | Build, train, or fine-tune models from your own audio, image, and sensor data. | Trained model in your chosen format |
## Run inference
After your model is prepared, choose how to execute it on the device. The runtime you pick depends on your language, model format, and whether you are building a full camera/video pipeline.
| Runtime / integration | Best for |
| ------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [LiteRT](../topic/litert-overview) | High-performance on-device inference from Python or C++, using Qualcomm AI Engine Direct delegates. |
| [QAIRT SDK C++ APIs](../topic/develop-your-own-application-qairt-cpp) | Low-level C++ control over model execution and the inference backend. |
| [Qualcomm Intelligent Multimedia SDK (IM SDK)](../topic/develop-your-own-application-im-sdk) | High-performance camera, video, and vision pipelines that combine capture, preprocessing, inference, and rendering, with zero-copy buffers and GPU pre/post-processing. |
| [Qualcomm® GenAI Inference Engine (Genie)](https://docs.qualcomm.com/doc/80-63442-10/topic/index_Genie.html) | Running and orchestrating on-device generative AI (LLMs, multimodal) workflows. |
| [LLM using Geniex](../topic/geniex) | Fastest way to run LLM/VLM text, image, and audio inference on the NPU via the GenieX CLI. |
| [Llama.cpp](../topic/llama-cpp) | Run LLMs and VLMs locally using a C++ backend optimized for CPU/GPU and quantized formats. |
Building a robotics application? The [Qualcomm® Intelligent Robotics (QIR) SDK](https://www.thundercomm.com/rubik-pi-3/en/docs/rubik-pi-3-user-manual/1.0.0-u/Application%20Development%20and%20Execution%20Guide/Robotics-Sample-Applications/Robotics%20Sample%20Applications/) adds ROS-based modules and hardware-accelerated nodes on top of these runtimes.
## AI hardware
Qualcomm platforms include the following hardware accelerators for AI inference:
* **Qualcomm Kryo™ CPU** — High-performance CPU with best-in-class power efficiency.
* **Qualcomm Adreno™ GPU**: Balanced power and performance for AI workloads, accelerated with OpenCL kernels. Also used for model **pre- and post-processing** (for example, the IM SDK runs resize, color conversion, and overlay on the GPU).
* **Qualcomm Hexagon™ Tensor Processor (HTP)**: Also known as NPU/DSP/HMX. Optimized for low-power, high-performance AI **inference**. For best performance, quantize pretrained models to a supported precision.