> ## Documentation Index
> Fetch the complete documentation index at: https://dragonwingdocs.qualcomm.com/llms.txt
> Use this file to discover all available pages before exploring further.

# 使用 QAIRT C++ API 开发 AI 应用

> 在 Qualcomm Dragonwing IoT 平台上,使用 Qualcomm AI Runtime(QAIRT)SDK C++ API 配合 QNN 或 SNPE 构建和运行 AI 应用。

Qualcomm AI Runtime(QAIRT)SDK 提供用于示例应用开发的 C++ API。Qualcomm AI Engine Direct(QNN)
和 Qualcomm Neural Processing Engine SDK(SNPE)均提供示例。这些示例可帮助您开始应用
开发。以下说明介绍如何构建、运行和浏览源代码,并演示了使用 QNN 或 SNPE API
运行模型的工作流。

## 构建并运行 QNN 示例应用

`qnn-sample-app` 位于 `${QNN_SDK_ROOT}/examples/QNN/SampleApp`,
其中 `QNN_SDK_ROOT` 指 QNN SDK 解压所在的路径。

### 设置 QAIRT SDK

要为 QNN 示例应用设置工具链,请完成以下步骤:

<Steps>
  <Step title="下载 Qualcomm AI Runtime SDK">
    [单击此处](https://softwarecenter.qualcomm.com/api/download/software/sdks/Qualcomm_AI_Runtime_Community/All/2.46.0.260424/v2.46.0.260424.zip)下载 Qualcomm AI Runtime SDK。

    <Note>
      如果您想下载特定的 QAIRT 版本,请遵循[此处](../topic/qairt-install#qairt-qpm)的说明。
    </Note>
  </Step>

  <Step title="提取并解压 SDK">
    ```shell theme={null}
    unzip v2.46.0.260424.zip
    ```

    ```shell theme={null}
    cd qairt/2.46.0.260424
    ```

    ```shell theme={null}
    export QNN_SDK_ROOT=`pwd`
    ```
  </Step>

  <Step title="安装 SDK">
    按照 [Build QIM SDK](https://imsdkdocs.qualcomm.com/advanced/yocto-build#download-and-install-the-sdk) 安装 SDK,其中包含
    所需的交叉编译工具链。

    * 这些库使用 GCC-11.2 编译。
    * 使用 SDK 安装路径设置 SDK\_PATH 环境变量。后续步骤会使用该安装路径(/path/to/extracted/toolchain)进行编译。

    ```shell theme={null}
    export SDK_PATH="/path/to/extracted/toolchain"
    ```
  </Step>
</Steps>

### 构建 QNN 示例应用

完成以下步骤以构建 QNN 示例应用。

<Steps>
  <Step title="进入示例应用目录">
    ```shell theme={null}
    cd ${QNN_SDK_ROOT}/examples/QNN/SampleApp/SampleApp/
    ```
  </Step>

  <Step title="为 GCC 工具链设置环境变量">
    ```shell theme={null}
    export QNN_AARCH64_LINUX_OE_GCC_112=$SDK_PATH
    ```
  </Step>

  <Step title="构建应用">
    ```shell theme={null}
    make CXX="$SDK_PATH/sysroots/x86_64-qcomsdk-linux/usr/bin/aarch64-qcom-linux/aarch64-qcom-linux-g++ --sysroot=$SDK_PATH/sysroots/armv8a-qcom-linux/" all_linux_oe_aarch64_gcc112
    ```

    这会创建两个文件夹。

    * `bin`:在各平台对应的目录中包含 `qnn-sample-app` 二进制文件。
    * `obj`:包含构建和链接可执行文件时使用的所有目标文件。
  </Step>
</Steps>

### 在 Ubuntu 上运行 QNN 示例应用

构建好的 `qnn-sample-app` 可执行文件可以使用任何
QNN 后端运行模型。对于基于 Yocto scarthgap 的设备,后端可用于
`aarch64-oe-linux-gcc11.2`。

<Steps>
  <Step title="将工件推送到目标设备">
    ```shell theme={null}
    scp ${QNN_SDK_ROOT}/examples/QNN/SampleApp/SampleApp/bin/aarch64-oe-linux-gcc11.2/qnn-sample-app ubuntu@<IP_ADDRESS_OF_TARGET_DEVICE>:/home/ubuntu/qnn-sample-app
    ssh ubuntu@<ip-address>
    sudo cp /home/ubuntu/qnn-sample-app /etc/apps/qnn-sample-app
    ```

    <Note>
      如果设备上尚不存在 `/etc/apps/` 目录,请创建它。
    </Note>
  </Step>

  <Step title="在主机上,使用 AI Hub 导出模型">
    使用 [AI Hub](../topic/ai-hub) 导出模型。

    例如,要导出 InceptionV3 QNN 模型,请运行以下命令:

    ```shell theme={null}
    pip3 install qai-hub-models
    ```

    ```shell theme={null}
    python -m qai_hub_models.models.inception_v3.export --quantize w8a8 --target-runtime=qnn_context_binary --device="Dragonwing RB3 Gen 2 Vision Kit" --compile-options="--qairt_version 2.45" --profile-options "--qairt_version 2.45"
    ```

    <Note>
      请为目标设备上使用的相同 SDK 版本生成上下文二进制文件。
    </Note>
  </Step>

  <Step title="将导出的 InceptionV3 QNN 模型推送到目标设备">
    将模型保存到 `export_assets/inception_v3-qnn_context_binary-w8a8-<CHIPSET>`。以下示例使用 `QCS6490` 作为芯片组。

    ```shell theme={null}
    scp export_assets/inception_v3-qnn_context_binary-w8a8-qualcomm_qcs6490/inception_v3.bin ubuntu@<IP_ADDRESS_OF_TARGET_DEVICE>:/home/ubuntu/
    ssh ubuntu@<ip-address>
    sudo cp /home/ubuntu/inception_v3.bin /etc/apps/
    ```
  </Step>

  <Step title="在主机上,生成用于推理的虚拟输入文件并将其传输到目标设备">
    <Steps>
      <Step title="在 Python 环境中运行以下命令">
        ```python theme={null}
        python3
        ```

        ```python theme={null}
        import numpy as np
        np.random.random((1, 3, 299, 299)).astype(np.float32).tofile("input.raw")
        ```
      </Step>

      <Step title="将 input.raw 文件传输到目标设备">
        ```shell theme={null}
        scp input.raw ubuntu@<IP_ADDRESS_OF_TARGET_DEVICE>:/home/ubuntu/
        ssh ubuntu@<ip-address>
        sudo cp /home/ubuntu/input.raw /etc/apps/
        ```
      </Step>
    </Steps>
  </Step>

  <Step title="从主机通过 SSH 登录到目标设备">
    ```shell theme={null}
    ssh ubuntu@<IP_ADDRESS_OF_TARGET_DEVICE>
    ```

    ```shell theme={null}
    cd /home/ubuntu
    ```
  </Step>

  <Step title="创建 input_list.txt">
    ```shell theme={null}
    echo "input.raw" > /home/ubuntu/input_list.txt
    sudo cp input_list.txt /etc/apps/
    ```
  </Step>

  <Step title="运行应用">
    ```shell theme={null}
    cd /etc/apps/
    sudo chmod +x qnn-sample-app
    ```

    ```shell theme={null}
    sudo ./qnn-sample-app --retrieve_context inception_v3.bin \
                    --backend libQnnHtp.so \
                    --input_list input_list.txt \
                    --system_library libQnnSystem.so
    ```

    <Note>
      请根据所选模型更新模型名称和 input\_list。
    </Note>

    要获取帮助信息,请运行:

    ```shell theme={null}
    ./qnn-sample-app --help
    ```
  </Step>
</Steps>

### 命令行参数

**必需参数**

* `--model`:QNN 网络模型的路径。与 `--retrieve_context` 互斥。
* `--retrieve_context`:用于加载已保存上下文和执行图的
  缓存二进制文件的路径。与 `--model` 互斥。
* `--backend`:用于运行模型的 QNN 后端的路径。
* `--input_list`:列出网络输入的文件的路径。对于多个
  图,请提供以逗号分隔的输入文件列表。

**可选参数**

* `--debug`:保存所有网络层的输出。
* `--output_dir`:输出目录(默认:./output)。
* `--output_data_type`:输出数据类型(float\_only、native\_only、float\_and\_native)。
* `--input_data_type`:输入数据类型(float 或 native)。
* `--op_packages`:以逗号分隔的算子包和接口提供程序列表。
* `--profiling_level`:性能分析级别(basic 或 detailed)。
* `--save_context`:将后端上下文和图元数据保存到二进制文件。
* `--num_inferences`:要执行的推理次数。
* `--log_level`:最大日志级别(error、warn、info、verbose)。
* `--system_library`:用于在上下文加载期间使用反射 API 的 libQnnSystem.so 的路径。
* `--version`:打印 QNN SDK 版本。
* `--help`:显示帮助信息。

## 工作流和 API 用法

使用以下推荐模式,通过 QNN API 开发 C++ 应用。

<Steps>
  <Step title="步骤 1">
    [加载先决共享库](#load-prerequisite-shared-libraries)
  </Step>

  <Step title="步骤 2">
    [使用 QNN API](https://docs.qualcomm.com/nav/home/sample_app.html?product=1601111740009302#usage-of-qnn-apis)

    <Steps>
      <Step title="步骤 2.1">
        [使用 QNN 接口获取函数指针](https://docs.qualcomm.com/nav/home/sample_app.html?product=1601111740009302#use-qnn-interface-to-obtain-function-pointers)
      </Step>

      <Step title="步骤 2.2">
        [设置日志记录](https://docs.qualcomm.com/nav/home/sample_app.html?product=1601111740009302#set-up-logging)
      </Step>

      <Step title="步骤 2.3">
        [初始化后端](https://docs.qualcomm.com/nav/home/sample_app.html?product=1601111740009302#initialize-backend)
      </Step>

      <Step title="步骤 2.4">
        [初始化性能分析](https://docs.qualcomm.com/nav/home/sample_app.html?product=1601111740009302#initialize-profiling)
      </Step>

      <Step title="步骤 2.5">
        [创建设备](https://docs.qualcomm.com/nav/home/sample_app.html?product=1601111740009302#create-device)
      </Step>

      <Step title="步骤 2.6">
        [注册算子包](https://docs.qualcomm.com/nav/home/sample_app.html?product=1601111740009302#register-op-packages)
      </Step>

      <Step title="步骤 2.7">
        [创建上下文](https://docs.qualcomm.com/nav/home/sample_app.html?product=1601111740009302#create-context)
      </Step>

      <Step title="步骤 2.8">
        [准备图](https://docs.qualcomm.com/nav/home/sample_app.html?product=1601111740009302#prepare-graphs)
      </Step>

      <Step title="步骤 2.9">
        [完成图](https://docs.qualcomm.com/nav/home/sample_app.html?product=1601111740009302#finalize-graphs)
      </Step>

      <Step title="步骤 2.10">
        [将上下文保存到二进制文件](https://docs.qualcomm.com/nav/home/sample_app.html?product=1601111740009302#save-context-into-a-binary)
      </Step>

      <Step title="步骤 2.11">
        [从缓存的二进制文件加载上下文](https://docs.qualcomm.com/nav/home/sample_app.html?product=1601111740009302#load-context-from-a-cached-binary)
      </Step>

      <Step title="步骤 2.12">
        [运行图](https://docs.qualcomm.com/nav/home/sample_app.html?product=1601111740009302#execute-graphs)
      </Step>

      <Step title="步骤 2.13">
        [释放上下文](https://docs.qualcomm.com/nav/home/sample_app.html?product=1601111740009302#free-context)
      </Step>

      <Step title="步骤 2.14">
        [终止后端](https://docs.qualcomm.com/nav/home/sample_app.html?product=1601111740009302#terminate-backend)
      </Step>
    </Steps>
  </Step>
</Steps>

<h3 id="load-prerequisite-shared-libraries">
  加载先决共享库
</h3>

QNN SDK 提供了各种共享库用于访问后端,
应用必须根据需要加载它们才能运行网络。

可通过以下方式之一在 QNN 中创建网络。

* 使用 QNN API 直接在您的应用中构建网络。
* 使用 QNN 转换器生成 QNN 网络的共享库。

`qnn-sample-app` 使用共享库方式。该网络可以
使用 SDK 中提供的任一 QNN 转换器生成,并
使用 `qnn-model-lib-generator` 编译为共享库。

<Note>
  对于 Windows 用户,请在以下说明中将所有 `.so` 文件替换为
  对应的 `.dll` 文件。有关更多详情,请参阅
  平台差异。
</Note>

#### 加载后端

QNN SDK 中提供了包括 CPU、GPU、HTP 和 DSP 在内的各种后端的
共享库。每个实现了 QNN API 的后端
都会暴露所有必要的符号,可通过动态加载
机制进行访问。

假设有一个名为 *libQnnSampleBackend.so* 的示例后端共享库,
可以按如下方式动态加载它:

```cpp theme={null}
void* libBackendHandle = pal::dynamicloading::dlOpen(
  "libQnnSampleBackend.so", pal::dynamicloading::DL_NOW | pal::dynamicloading::DL_LOCAL);

if (nullptr == libBackendHandle) {
  QNN_ERROR("Unable to load backend. pal::dynamicloading::dlError(): %s",
            pal::dynamicloading::dlError());
  return StatusCode::FAIL_LOAD_BACKEND;
```

要以共享库形式加载模型,我们假设有一个名为
*libQnnSampleModel.so* 的示例模型共享库,可以按如下方式
动态加载它:

```cpp theme={null}
void* libModelHandle = pal::dynamicloading::dlOpen(
    "libQnnSampleModel.so", pal::dynamicloading::DL_NOW | pal::dynamicloading::DL_LOCAL);

if (nullptr == libModelHandle) {
  QNN_ERROR("Unable to load model. pal::dynamicloading::dlError(): %s",
            pal::dynamicloading::dlError());
  return StatusCode::FAIL_LOAD_MODEL;
}
```

另外,如果要从缓存的二进制文件创建上下文并执行图,
应用可以利用 QnnSystem API 检索与上下文
关联的元数据。可以按如下方式加载
*libQnnSystem.so* 共享库来访问 QnnSystem API:

```cpp theme={null}
void* systemLibraryHandle = pal::dynamicloading::dlOpen(
  "libQnnSystem.so", pal::dynamicloading::DL_NOW | pal::dynamicloading::DL_LOCAL);

if (nullptr == systemLibraryHandle) {
  QNN_ERROR("Unable to load system library. pal::dynamicloading::dlError(): %s",
            pal::dynamicloading::dlError());
  return StatusCode::FAIL_LOAD_SYSTEM_LIB;
}
```

#### 解析共享库中的符号

共享库成功加载后,我们可以继续
解析访问 QNN API 所需的所有符号。

以下代码片段展示了一个在共享库中解析符号的
模板:

```cpp theme={null}
// A generic function to resolve symbols in a library
template <class T>
static inline T resolveSymbol(void* libHandle, const char* symName) {
T ptr = (T)pal::dynamicloading::dlSym(libHandle, symName);
if (ptr == nullptr) {
  QNN_ERROR("Unable to access symbol [%s]. pal::dynamicloading::dlError(): %s", symName, pal::dynamicloading::dlError());
}
return ptr;
}
// Template for resolving a function of type SampleFnHandleType_t
typedef ReturnType_t (*SampleFnHandleType_t)(FunctionParameterTypes_t ...);
SampleFnHandleType_t sampleFn = nullptr;
sampleFnHandle = resolveSymbol<SampleFnHandleType_t>(libBackendHandle, "QnnSample_API");
if (nullptr == sampleFnHandle) {
// Error code indicating failure in symbol resolution
return StatusCode::FAIL_SYM_FUNCTION;
}
```

以下代码片段展示了如何解析实际 QNN
API 的示例:

```cpp theme={null}
/* Resolve the symbol for Qnn_ErrorHandle_t QnnInterface_getProviders(const QnnInterface_t*** providerList,
                                                                    uint32_t* numProviders)
  API */

typedef Qnn_ErrorHandle_t (*QnnInterfaceGetProvidersFn_t)(const QnnInterface_t*** providerList,
                                                        uint32_t* numProviders);

QnnInterfaceGetProvidersFn_t getInterfaceProviders {nullptr};

getInterfaceProviders =
resolveSymbol<QnnInterfaceGetProvidersFn_t>(libBackendHandle, "QnnInterface_getProviders");
if (nullptr == getInterfaceProviders) {
return StatusCode::FAIL_SYM_FUNCTION;
}
```

在 *qnn-sample-app* 源代码中,所有必要的符号都会被解析并
存储在类型为 QnnFunctionPointers 的结构体中,如下所示:

```cpp theme={null}
typedef struct QnnFunctionPointers {
  // APIs from model output from converters
  // QnnModel_composeGraphs
  ComposeGraphsFnHandleType_t composeGraphsFnHandle;
  // QnnModel_freeGraphsInfo
  FreeGraphInfoFnHandleType_t freeGraphInfoFnHandle;
  // QNN Interface function table containing pointers to all necessary QNN APIs
  // in a backend
  QNN_INTERFACE_VER_TYPE qnnInterface;
  // QNN System Interface function table containing pointers to all QNN System APIs
  QNN_SYSTEM_INTERFACE_VER_TYPE qnnSystemInterface;
} QnnFunctionPointers;
```

上述结构体可以在
`${QNN_SDK_ROOT}/examples/QNN/SampleApp/SampleApp/src/SampleApp.hpp` 中找到。
本教程的其余部分将假定存在一个名为
*m\_qnnFunctionPointers*、类型为 *QnnFunctionPointers* 的变量,
其中包含有效的函数指针。

### QNN API 的用法

本节演示 QNN API 在客户端应用中的用法。

#### 使用 QNN 接口获取函数指针

可以使用 QNN 接口机制来设置指向后端中 QNN API 的函数
指针表,而不必手动逐个解析
每个 API 的符号,这使得符号解析变得简单。QNN
接口的使用方式如下:

```cpp theme={null}
QnnInterface_t** interfaceProviders{nullptr};
uint32_t numProviders{0};
// Query for al available interfaces
if (QNN_SUCCESS !=
  getInterfaceProviders((const QnnInterface_t***)&interfaceProviders, &numProviders)) {
QNN_ERROR("Failed to get interface providers.");
return StatusCode::FAIL_GET_INTERFACE_PROVIDERS;
}
// Check for validity of returned interfaces
if (nullptr == interfaceProviders) {
QNN_ERROR("Failed to get interface providers: null interface providers received.");
return StatusCode::FAIL_GET_INTERFACE_PROVIDERS;
}
if (0 == numProviders) {
QNN_ERROR("Failed to get interface providers: 0 interface providers.");
return StatusCode::FAIL_GET_INTERFACE_PROVIDERS;
}
bool foundValidInterface{false};
// Loop through all available interface providers and pick the one that suits the current API
// version
for (size_t pIdx = 0; pIdx < numProviders; pIdx++) {
if (QNN_API_VERSION_MAJOR == interfaceProviders[pIdx]->apiVersion.coreApiVersion.major &&
      QNN_API_VERSION_MINOR <= interfaceProviders[pIdx]->apiVersion.coreApiVersion.minor) {
  foundValidInterface                 = true;
  m_qnnFunctionPointers.qnnInterface = interfaceProviders[pIdx]->QNN_INTERFACE_VER_NAME;
  break;
}
}
if (!foundValidInterface) {
QNN_ERROR("Unable to find a valid interface.");
libBackendHandle = nullptr;
return StatusCode::FAIL_GET_INTERFACE_PROVIDERS;
}
```

可以使用 QNN 系统接口来解析与 QNN
系统 API 相关的所有符号,如下所示:

```cpp theme={null}
typedef Qnn_ErrorHandle_t (*QnnSystemInterfaceGetProvidersFn_t)(
  const QnnSystemInterface_t*** providerList, uint32_t* numProviders);

QnnSystemInterfaceGetProvidersFn_t getSystemInterfaceProviders{nullptr};
getSystemInterfaceProviders = resolveSymbol<QnnSystemInterfaceGetProvidersFn_t>(
  systemLibraryHandle, "QnnSystemInterface_getProviders");
if (nullptr == getSystemInterfaceProviders) {
  return StatusCode::FAIL_SYM_FUNCTION;
}
QnnSystemInterface_t** systemInterfaceProviders{nullptr};
uint32_t numProviders{0};
if (QNN_SUCCESS != getSystemInterfaceProviders(
                    (const QnnSystemInterface_t***)&systemInterfaceProviders, &numProviders)) {
  QNN_ERROR("Failed to get system interface providers.");
  return StatusCode::FAIL_GET_INTERFACE_PROVIDERS;
}
if (nullptr == systemInterfaceProviders) {
  QNN_ERROR("Failed to get system interface providers: null interface providers received.");
  return StatusCode::FAIL_GET_INTERFACE_PROVIDERS;
}
if (0 == numProviders) {
  QNN_ERROR("Failed to get interface providers: 0 interface providers.");
  return StatusCode::FAIL_GET_INTERFACE_PROVIDERS;
}
bool foundValidSystemInterface{false};
for (size_t pIdx = 0; pIdx < numProviders; pIdx++) {
  if (QNN_SYSTEM_API_VERSION_MAJOR == systemInterfaceProviders[pIdx]->systemApiVersion.major &&
      QNN_SYSTEM_API_VERSION_MINOR <= systemInterfaceProviders[pIdx]->systemApiVersion.minor) {
  foundValidSystemInterface = true;
  m_qnnFunctionPointers->qnnSystemInterface =
      systemInterfaceProviders[pIdx]->QNN_SYSTEM_INTERFACE_VER_NAME;
  break;
  }
}
```

#### 设置日志记录

可以在初始化后端之前、动态加载后端
共享库之后设置日志记录。

要初始化日志记录,必须定义一个类型为 *QnnLog\_Callback\_t* 的
回调。示例定义如下:

```cpp theme={null}
void logStdoutCallback(const char* fmt,
                        QnnLog_Level_t level,
                        uint64_t timestamp,
                        va_list argp) {
  const char* levelStr = "";
  switch (level) {
  case QNN_LOG_LEVEL_ERROR:
  levelStr = " ERROR ";
  break;
  case QNN_LOG_LEVEL_WARN:
  levelStr = "WARNING";
  break;
  case QNN_LOG_LEVEL_INFO:
  levelStr = "  INFO ";
  break;
  case QNN_LOG_LEVEL_DEBUG:
  levelStr = " DEBUG ";
  break;
  case QNN_LOG_LEVEL_VERBOSE:
  levelStr = "VERBOSE";
  break;
  case QNN_LOG_LEVEL_MAX:
  levelStr = "UNKNOWN";
  break;
  }
  fprintf(stdout, "%8.1fms [%-7s] ", ms, levelStr);
  vfprintf(stdout, fmt, argp);
  fprintf(stdout, "\n");
}
```

可以将上述回调与最大日志级别一起注册到后端。
以下示例代码以最大日志级别
QNN\_LOG\_LEVEL\_INFO 进行初始化:

```cpp theme={null}
Qnn_LogHandle_t logHandle;
if (QNN_SUCCESS !=
      m_qnnFunctionPointers.qnnInterface.logCreate(logStdoutCallback, QNN_LOG_LEVEL_INFO, &logHandle)) {
QNN_ERROR("Unable to initialize logging in the backend.");
return StatusCode::FAILURE;
}
```

#### 初始化后端

成功初始化日志记录后,可以按如下方式初始化
后端:

```cpp theme={null}
Qnn_BackendHandle_t backendHandle;
const QnnBackend_Config_t* backendConfigs;
/* Set up any necessary backend configurations */
if (QNN_BACKEND_NO_ERROR != m_qnnFunctionPointers.qnnInterface.backendCreate(logHandle,
                                                                            &backendConfigs,
                                                                            &backendHandle)) {
  QNN_ERROR("Could not initialize backend");
  return StatusCode::FAILURE;
}
```

#### 初始化性能分析

如果需要性能分析,在初始化后端之后,可以设置一个性能分析
句柄。此性能分析句柄可以在之后
用于任何支持性能分析的 API。

可以在后端以 basic 性能分析级别创建性能分析句柄,
如下所示:

```cpp theme={null}
Qnn_ProfileHandle_t profileHandle;
if (QNN_PROFILE_NO_ERROR != m_qnnFunctionPointers.qnnInterface.profileCreate(
                                  backendHandle, QNN_PROFILE_LEVEL_BASIC, &profileHandle)) {
  QNN_WARN("Unable to create profile handle in the backend.");
  return StatusCode::FAILURE;
}
```

#### 创建设备

可以按如下方式创建设备:

```cpp theme={null}
Qnn_DeviceHandle_t deviceHandle {nullptr};
const QnnDevice_Config_t* devConfigArray[] = {&devConfig, nullptr};
Qnn_ErrorHandle_t ret = m_qnnFunctionPointers.qnnInterface.deviceCreate(logHandle,
                                                                        devConfigArray,
                                                                        &deviceHandle);
if (QNN_SUCCESS != ret) {
  QNN_ERROR("Failed to create device: %u", qnnStatus);
  return StatusCode::FAILURE;
}
```

按照 QNN HTP Backend API 中的定义设置 devConfig

#### 注册算子包

算子包是向后端提供包含算子的库的一种方式。
可以按如下方式注册它们:

```cpp theme={null}
uint32_t opPackageCount;
char* opPackagePath[opPackageCount];
char* opPackageInterfaceProvider[opPackageCount];
/* Set up required op package paths and interface providers as necessary */
for(uint32_t idx = 0; idx < opPackageCount; idx++) {
  if (QNN_BACKEND_NO_ERROR !=
        m_qnnFunctionPointers.qnnInterface.backendRegisterOpPackage(backendHandle,
                                                                    opPackagePath[idx],
                                                                    opPackageInterfaceProvider[idx])) {
    QNN_ERROR("Could not register Op Package: %s and interface provider: %s",
            opPackagePath[idx],
            opPackageInterfaceProvider[idx]);
    return StatusCode::FAILURE;
  }
}
```

#### 创建上下文

可以在后端中按如下方式创建上下文:

```cpp theme={null}
Qnn_ContextHandle_t context;
Qnn_DeviceHandle_t deviceHandle {nullptr};
const QnnContext_Config_t* contextConfigs;
/* Set up any context configs that are necessary */
if (QNN_CONTEXT_NO_ERROR !=
      m_qnnFunctionPointers.qnnInterface.contextCreate(backendHandle,
                                                        deviceHandle,
                                                        &contextConfigs,
                                                        &context)) {
  QNN_ERROR("Could not create context");
  return StatusCode::FAILURE;
}
```

#### 准备图

*qnn-sample-app* 依赖某个转换器的输出在后端中
创建 QNN 网络。*composeGraphsFnHandle* 映射到
模型共享库中的 *QnnModel\_composeGraphs* API,该 API
以 *qnn\_wrapper\_api::GraphInfo\_t*\*\*\* 作为参数之一。
函数 *composeGraphsFnHandle* 会对后端进行必要的调用
以创建网络。它还会将执行图所需的所有必要
信息(例如与图相关的输入和输出张量的信息)写入
结构体 *graphsInfo*,如以下代码块所示:

```cpp theme={null}
/* Structure to retrieve information about graphs, like graph name,
  details about input and output tensors preset in libQnnSampleModel.so */
qnn_wrapper_api::GraphInfo_t** graphsInfo;
// No. of graphs present in libQnnSampleModel.so
uint32_t graphsCount;
// true to enable intermediate outputs, false for network outputs only
bool debug;
if (qnn_wrapper_api::ModelError_t::MODEL_NO_ERROR !=
        m_qnnFunctionPointers.composeGraphsFnHandle(backendHandle,
                                                    m_qnnFunctionPointers.qnnInterface,
                                                    context,
                                                    &graphsInfo,
                                                    &graphsCount,
                                                    debug)) {
  QNN_ERROR("Failed in composeGraphs()");
  return StatusCode::FAILURE;
}
```

此时,上下文将包含 *libQnnSampleModel.so* 中存在的
所有图。

#### 完成图

可以按如下方式完成上一步中添加的
图:

```cpp theme={null}
// information about graphs obtained in the previous step
qnn_wrapper_api::GraphInfo_t** graphsInfo;
// No. of graphs obtained in the previous step
uint32_t graphsCount;
/* A valid profile handle if profiling is desired,
  nullptr if profiling is not needed */
Qnn_ProfileHandle_t profileHandle;

for (size_t graphIdx = 0; graphIdx < m_graphsCount; graphIdx++) {
  if (QNN_GRAPH_NO_ERROR !=
    m_qnnFunctionPointers.qnnInterface.graphFinalize(
        (*graphsInfo)[graphIdx].graph, profileBackendHandle, nullptr)) {
    return StatusCode::FAILURE;
  }
  /* Extract profiling information if desired and if a valid handle was supplied to finalize
    graphs API */
}
```

#### 将上下文保存到二进制文件

当上下文中的所有图都完成后,用户应用
可以选择将上下文保存为二进制文件以供将来使用。保存
上下文的优点是,将来可以检索它
以执行其中包含的图,而无需再次完成
它们。这将大大节省网络执行期间的
初始化时间。

可以按如下方式保存上下文:

```cpp theme={null}
// Get the expected size of the buffer from the backend in which the context can be saved
if (QNN_CONTEXT_NO_ERROR !=
  m_qnnFunctionPointers.qnnInterface.contextGetBinarySize(context, &requiredBufferSize)) {
  QNN_ERROR("Could not get the required binary size.");
  return StatusCode::FAILURE;
}

// Allocate a buffer of the required size
saveBuffer = (uint8_t*)malloc(requiredBufferSize * sizeof(uint8_t));
if (nullptr == saveBuffer) {
  QNN_ERROR("Could not allocate buffer to save binary.");
  return StatusCode::FAILURE;
}

auto status = StatusCode::SUCCESS;
uint32_t writtenBufferSize{0};
// Pass the allocated buffer and obtain a copy of the context binary written into the buffer
if (QNN_CONTEXT_NO_ERROR !=
  m_qnnFunctionPointers.qnnInterface.contextGetBinary(context,
                                                      reinterpret_cast<void*>(saveBuffer),
                                                      requiredBufferSize,
                                                      &writtenBufferSize)) {
QNN_ERROR("Could not get binary.");
status = StatusCode::FAILURE;
}

// Check if the supplied buffer size is at least as big as the amount of data witten by the backend
if (requiredBufferSize < writtenBufferSize) {
  QNN_ERROR(
    "Illegal written buffer size [%d] bytes. Cannot exceed allocated memory of [%d] bytes",
    writtenBufferSize,
    requiredBufferSize);
  status = StatusCode::FAILURE;
}

// Use caching utility to save metadata along with the binary buffer from the backend
if (status == StatusCode::SUCCESS &&
  tools::datautil::StatusCode::SUCCESS != tools::datautil::writeBinaryToFile(outputPath,
                                                                            saveBinaryName + ".bin",
                                                                            (uint8_t*)saveBuffer,
                                                                            writtenBufferSize)) {
  QNN_ERROR("Could not serialize to file.");
  status = StatusCode::FAILURE;
}
```

#### 从缓存的二进制文件加载上下文

像上一步那样保存为二进制文件的上下文,可以
被加载,作为每次创建新上下文的替代方案。
以下代码片段演示了此步骤:

```cpp theme={null}
auto returnStatus   = StatusCode::SUCCESS;
std::shared_ptr<uint8_t> buffer{nullptr};
uint32_t graphsCount {0};
buffer = std::shared_ptr<uint8_t>(new uint8_t[bufferSize], std::default_delete<uint8_t[]>());
if (!buffer) {
    QNN_ERROR("Failed to allocate memory.");
    return StatusCode::FAILURE;
}

if (tools::datautil::StatusCode::SUCCESS !=
    tools::datautil::readBinaryFromFile(
    cachedBinaryPath, reinterpret_cast<uint8_t*>(buffer.get()), bufferSize)) {
    QNN_ERROR("Failed to read binary file.");
    returnStatus = StatusCode::FAILURE;
}

/* Create a QnnSystemContext handle to access system context APIs. */
QnnSystemContext_Handle_t sysCtxHandle{nullptr};
if (QNN_SUCCESS != m_qnnFunctionPointers.qnnSystemInterface.systemContextCreate(&sysCtxHandle)) {
  QNN_ERROR("Could not create system handle.");
  returnStatus = StatusCode::FAILURE;
}

/* Retrieve metadata from the context binary through QNN System Context API. */
QnnSystemContext_BinaryInfo_t* binaryInfo{nullptr};
uint32_t binaryInfoSize{0};
if (StatusCode::SUCCESS == returnStatus &&
    QNN_SUCCESS != m_qnnFunctionPointers.qnnSystemInterface.systemContextGetBinaryInfo(
                    sysCtxHandle,
                    static_cast<void*>(buffer.get()),
                    bufferSize,
                    &binaryInfo,
                    &binaryInfoSize)) {
    QNN_ERROR("Failed to get context binary info");
    returnStatus = StatusCode::FAILURE;
}

qnn_wrapper_api::GraphInfo_t** graphsInfo;
/* Make a copy of the metadata. */
if (StatusCode::SUCCESS == returnStatus &&
    !copyMetadataToGraphsInfo(binaryInfo, graphsInfo, graphsCount)) {
  QNN_ERROR("Failed to copy metadata.");
  returnStatus = StatusCode::FAILURE;
}

/* Release resources associated with previously created QnnSystemContext handle. */
m_qnnFunctionPointers.qnnSystemInterface.systemContextFree(sysCtxHandle);
sysCtxHandle = nullptr;

/* readBuffer contains the binary data that was previously obtained from a backend. Pass this
  cached binary data to the backend to recreate the same context. */
if (StatusCode::SUCCESS == returnStatus &&
    m_qnnFunctionPointers.qnnInterface.contextCreateFromBinary(backendHandle,
                                                              deviceHandle,
                                                              (const QnnContext_Config_t**)&contextConfig,
                                                              reinterpret_cast<void*>(readBuffer),
                                                              bufferSize,
                                                              &context,
                                                              profileBackendHandle)) {
  QNN_ERROR("Could not create context from binary.");
  returnStatus = StatusCode::FAILURE;
}

// Optionally, extract profiling numbers if desired
if (ProfilingLevel::OFF != m_profilingLevel) {
  extractBackendProfilingInfo(profileBackendHandle);
}

/* Obtain and save graph handles for each graph present in the context based on the saved graph
  names in the metadata */
if (StatusCode::SUCCESS == returnStatus) {
  for (size_t graphIdx = 0; graphIdx < m_graphsCount; graphIdx++) {
    if (QNN_SUCCESS !=
        m_qnnFunctionPointers.qnnInterface.graphRetrieve(
            context, (*graphsInfo)[graphIdx].graphName, &((*graphsInfo)[graphIdx].graph))) {
      QNN_ERROR("Unable to retrieve graph handle for graph Idx: %d", graphIdx);
      returnStatus = StatusCode::FAILURE;
    }
  }
}
```

#### 运行图

在创建了上下文、添加并完成了图之后,
或者在从二进制文件检索了上下文之后,可以执行上下文中的一个
或多个图。

运行图涉及以下操作:

<Steps>
  <Step title="设置输入和输出张量" />

  <Step title="将输入数据填充到输入张量中" />

  <Step title="调用后端中的执行方法" />

  <Step title="获取输出并保存" />
</Steps>

以下代码片段演示了这一过程:

```cpp theme={null}
// Select a graph from graphsInfo if there are more than one graph in this context
uint32_t graphIdx;
QNN_DEBUG("Starting execution for graphIdx: %d", graphIdx);
  Qnn_Tensor_t* inputs  = nullptr;
  Qnn_Tensor_t* outputs = nullptr;
  // IOTensor utility is used to set up input and output tensor structures
  if (iotensor::StatusCode::SUCCESS !=
        ioTensor.setupInputAndOutputTensors(&inputs, &outputs, (*graphsInfo)[graphIdx])) {
    QNN_ERROR("Error in setting up Input and output Tensors for graphIdx: %d", graphIdx);
    returnStatus = StatusCode::FAILURE;
    break;
  }
  
  // Grab input raw file paths to read input data
  auto inputFileList = inputFileLists[graphIdx];
  auto graphInfo     = (*graphsInfo)[graphIdx];
  if (!inputFileList.empty()) {
    /* *qnn-sample-app* reads data based on the batch size until the whole buffer is filled.
        If there isn't sufficient data, it pads the rest with zeroes. */
    size_t totalCount = inputFileList[0].size();
    while (!inputFileList[0].empty()) {
      size_t startIdx = (totalCount - inputFileList[0].size());
  
      // IOTensor utility is used to populate input tensors with input data
      if (iotensor::StatusCode::SUCCESS !=
            m_ioTensor.populateInputTensors(
              graphIdx, inputFileList, inputs, graphInfo, inputDataType)) {
        returnStatus = StatusCode::FAILURE;
      }
  
      if (StatusCode::SUCCESS == returnStatus) {
        // Execute the graph in the backend with optional profile handle
        QNN_DEBUG("Successfully populated input tensors for graphIdx: %d", graphIdx);
        Qnn_ErrorHandle_t executeStatus = QNN_GRAPH_NO_ERROR;
        executeStatus = m_qnnFunctionPointers.qnnInterface.graphExecute(graphInfo.graph,
                                                                        inputs,
                                                                        graphInfo.numInputTensors,
                                                                        outputs,
                                                                        graphInfo.numOutputTensors,
                                                                        profileBackendHandle,
                                                                        nullptr);
        if (QNN_GRAPH_NO_ERROR != executeStatus) {
          returnStatus = StatusCode::FAILURE;
        }
        if (StatusCode::SUCCESS == returnStatus) {
          QNN_DEBUG("Successfully executed graphIdx: %d ", graphIdx);
          // IOTensor utility is used to write output tensors to raw files
          if (iotensor::StatusCode::SUCCESS !=
                ioTensor.writeOutputTensors(graphIdx,
                                            startIdx,
                                            graphInfo.graphName,
                                            outputs,
                                            graphInfo.outputTensors,
                                            graphInfo.numOutputTensors,
                                            outputDataType,
                                            graphsCount,
                                            outputPath)) {
              returnStatus = StatusCode::FAILURE;
            }
          }
        }
        if (StatusCode::SUCCESS != returnStatus) {
          QNN_ERROR("Execution of Graph: %d failed!", graphIdx);
          break;
        }
      }
    }
  
    // Clean up all the tensors after execution is completed
    ioTensor.tearDownInputAndOutputTensors(
        inputs, outputs, graphInfo.numInputTensors, graphInfo.numOutputTensors);
    inputs  = nullptr;
    outputs = nullptr;
    if (StatusCode::SUCCESS != returnStatus) {
      break;
    }
}
```

IOTensor 是随源代码提供的实用工具,位于
`${QNN_SDK_ROOT}/examples/QNN/SampleApp/SampleApp/src/Utils/IOTensor.cpp`。
它公开了一些有助于执行图的方法,
这些方法在前面的代码片段中已被使用:

1. *setupInputAndOutputTensors*,用于设置与输入
   和输出张量相关的结构。

2. *populateInputTensors*,用于将输入数据复制到输入张量
   结构中。

3. *tearDownInputAndOutputTensors*,用于清理与
   输入和输出张量关联的资源。

有关这些 API 的更多详情,请参阅 IOTensor 源代码。

#### 释放上下文

所有执行完成后,可以按如下方式释放
上下文:

```cpp theme={null}
if (QNN_CONTEXT_NO_ERROR !=
      m_qnnFunctionPointers.qnnInterface.contextFree(context, profileBackendHandle)) {
  QNN_ERROR("Could not free context");
  return StatusCode::FAILURE;
}
```

#### 终止后端

可以按如下方式终止后端:

```cpp theme={null}
if (QNN_BACKEND_NO_ERROR != m_qnnFunctionPointers.qnnInterface.backendFree(backendHandle)) {
  QNN_ERROR("Could not free backend");
  return StatusCode::FAILURE;
}
```

## SNPE 示例应用

有关使用 SNPE 的 C++ API 和示例应用执行,请参阅 [Qualcomm AI Runtime SDK 文档](https://docs.qualcomm.com/nav/home/usergroup8.html?product=1601111740009302)。