Classical vision migration is mostly about tensors, preprocessing, quantization, and camera pipelines. GenAI migration adds another layer:
Raw
llama.cpp is still useful when you need low-level debugging or a direct GGUF experiment. For a migration story, GenieX is the cleaner first recommendation.
This post is a decision guide, not a full recipe for every modality. In the running case study, it covers the optional local LLM/VLM sidecar that may sit beside the migrated camera pipeline.
Before you start:
Start with the runtime, not the model name
On Jetson, a GenAI prototype might be built around vLLM, TensorRT-LLM, llama.cpp, Python Transformers, or a custom service. The first Qualcomm decision is not “which exact framework replaces this?” It is:LLMs: measure prefill and decode separately
A single “tokens per second” number hides too much. For user-facing LLM workloads, track at least:
A model can have good decode speed and still feel slow if TTFT is high. A model can run a short demo and still fail when the production context length doubles.
Use GenieX benchmarking or runtime logs to capture the same measurements across models and OS images.
geniex-bench is a standalone binary, separate from the geniex CLI. Install it from the GenieX benchmarking tutorial, set BENCH to the extracted binary, then run examples like these:
llama_cpp, context length can be adjusted at runtime within model and memory limits. For qairt bundles, context length is fixed by the compiled bundle; use a longer-context bundle if the product prompt requires it.
For a simple first run, using a current GenieX docs example:
Q4_0 is the usual Day 0 choice for Hexagon NPU support.
Pick models that match the board
Model support changes quickly, so use AI Hub and GenieX docs as the source of truth. The current validated model families include examples like:
A smaller model that runs reliably on-device is often more useful than a larger model that only works in a lab prompt.
VLMs: split the text path from the vision path
Vision-language models are easy to over-simplify. They are not just LLMs with an image parameter. A VLM has at least two performance paths:Audio: treat streaming as the product path
Audio demos often pass as batch jobs and fail as products. For ASR models such as Whisper, measure:VLA and robotics: latency is a safety input
Vision-language-action models and robotics policies have a different success condition. A nice answer is not enough. The action loop has to be stable. For ACT, Pi0.5, or similar workloads, measure the full loop:
The model runtime is only one piece. The product is the closed loop.
Minimum deliverable before calling VLA migrated: a logged replay or closed-loop run that records input timestamps, inference timestamps, action timestamps, missed deadlines, and the safe fallback behavior.
Migration patterns from Jetson
The practical migration is not a rewrite. It is preserving the service boundary that already works.
Benchmark fairly
Before comparing Jetson and Qualcomm numbers, record the setup:
A Qualcomm result that uses less power at acceptable latency can be the better product result even if a Jetson GPU wins a narrow synthetic benchmark.
Takeaway
For GenAI and robotics workloads, the model file is only part of the migration. Runtime choice, serving API, tokenizer behavior, context length, memory, and sustained power all matter. Start with GenieX. Use thellama_cpp lane for broad GGUF coverage and fast experiments. Use the qairt lane when AI Hub provides a supported NPU-optimized bundle. Keep your existing service boundary when possible, measure TTFT and decode separately, and validate the full product loop before calling the migration done.