> ## Documentation Index
> Fetch the complete documentation index at: https://dragonwingdocs.qualcomm.com/llms.txt
> Use this file to discover all available pages before exploring further.
# 转换 TensorFlow 模型
> 将 TensorFlow/Keras 模型量化并转换为 TFLite 格式,以便通过 LiteRT 在 NPU 上运行。
本页介绍如何将 TensorFlow 模型**转换**为量化的 TFLite 格式。转换完成后,您可以使用 [LiteRT](/zh/ai-workflows/lite-rt) 在 NPU 上运行生成的 `.tflite` 文件。
```mermaid theme={null}
flowchart LR
A["TF / Keras model
(.keras, .h5)"] --> B["Quantize
(post-training)"]
B --> C[".tflite file
(int8)"]
C --> D["Run on NPU
via LiteRT"]
click D "/ai-workflows/lite-rt"
style D fill:#31017D,stroke:#31017D,color:#fff
```
**想要运行已经转换好的模型?** 请跳过本页,直接前往[运行 LiteRT / TFLite 模型](/zh/ai-workflows/lite-rt)。只有当您拥有尚未量化的 TensorFlow 模型时才需要阅读本页。
TensorFlow 是 Google 开发的开源机器学习框架,提供构建、训练和部署神经网络的工具。要在 Dragonwing 开发板的 NPU 上运行 TensorFlow 模型,您需要将模型转换为量化的 TFLite 模型。然后,您可以使用 [LiteRT](/zh/ai-workflows/lite-rt) 以完整的硬件加速运行该模型。
## 量化并转换模型
TensorFlow 模型的权重和激活值使用 32 位浮点数。而开发板上的 NPU 仅支持 8 位整数,因此必须对 TensorFlow 模型进行*量化* — 将浮点值转换为定点值。这会使模型更小、运行更快(并能够在 NPU 上运行),但会对精度产生一定影响。
量化模型最简单的方法是使用训练后量化(post-training quantization)。您使用一个已经训练好的模型,然后借助代表性数据集对权重和激活值进行量化。这意味着训练循环不受任何影响。您也可以选择使用 TensorFlow 内置的[量化感知训练](https://www.tensorflow.org/model_optimization/guide/quantization/training)来降低量化误差(但这需要修改训练循环)。
我们通过量化一个 Keras 模型来演示。在开发板上打开终端,或通过 SSH 连接到开发板,然后:
1. 创建一个新的 `venv` 并安装一些基础软件包:
```
mkdir -p ~/post-training-quantization-tf
cd ~/post-training-quantization-tf
python3 -m venv .venv
source .venv/bin/activate
pip3 install tensorflow==2.20.0 tf_keras==2.20.1 ai-edge-litert==1.3.0
```
2. 下载 `.keras` 格式的演示模型以及测试集。
```
mkdir -p models
wget -O models/cats.keras https://cdn.edgeimpulse.com/qc-ai-docs/models/cats.keras
wget -O models/cats_X_val.npy https://cdn.edgeimpulse.com/qc-ai-docs/models/cats_X_val.npy
wget -O models/cats_y_val.npy https://cdn.edgeimpulse.com/qc-ai-docs/models/cats_y_val.npy
```
3. 创建一个新文件 `quantize.py`,并添加:
```python theme={null}
import tensorflow as tf, numpy as np, os, time, tf_keras as keras
from ai_edge_litert.interpreter import Interpreter, load_delegate
# Shape: (444, 160, 160, 3)
X_val = np.load('models/cats_X_val.npy')
# Shape: (444, 1) -> with class 1..6 -> scale to 0..5
y_val = np.load('models/cats_y_val.npy') - 1
# Load Keras model
model = keras.models.load_model("models/cats.keras")
# Calculate accuracy of the TF model
tf_start = time.perf_counter()
y_pred = model.predict(X_val)
tf_end = time.perf_counter()
preds = np.argmax(y_pred, axis=1)
acc_tf = (preds == y_val).mean()
print(f"TF/Keras accuracy: {acc_tf*100:.2f}% (time per inference: {(tf_end - tf_start) * 1000 / X_val.shape[0]:.4g}ms)")
print('')
# Convert to quantized TFLite file... Uses the dataset earlier as a representative dataset to improve accuracy.
TFLITE_FILE = 'cats_i8.tflite'
if not os.path.exists(TFLITE_FILE):
print(f'Converting to TFLite file ({TFLITE_FILE})...')
def rep_dataset():
for i in range(X_val.shape[0]):
yield [X_val[i:i+1]]
# Build a fixed batch=1 input signature (QNN cannot handle dynamic dims)
specs = []
for t in model.inputs:
if None in t.shape[1:]:
raise ValueError(f"Non-batch dims must be known; got {t.shape}")
specs.append(tf.TensorSpec([1, *t.shape[1:]], dtype=t.dtype, name=t.name.split(':')[0]))
@tf.function(input_signature=specs)
def serve(*xs):
y = model(*xs)
return y if isinstance(y, (tuple, list)) else (y,) # keep output order stable
concrete = serve.get_concrete_function()
converter = tf.lite.TFLiteConverter.from_concrete_functions([concrete], model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = rep_dataset
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8
tflite_model = converter.convert()
with open(TFLITE_FILE, "wb") as f:
f.write(tflite_model)
print(f"TFLite written: {TFLITE_FILE} ({os.path.getsize(TFLITE_FILE)/1e6:.2f} MB)")
else:
print(f'TFLite file already exists ({TFLITE_FILE})')
print('')
def run_tflite_model(model_path, use_npu):
# Use QNN to run this model on NPU
experimental_delegates = []
if use_npu:
experimental_delegates = [load_delegate("libQnnTFLiteDelegate.so", options={"backend_type": "htp"})]
# Get accuracy for the quantized TFLite file, construct the interpreter
interpreter = Interpreter(model_path=model_path, experimental_delegates=experimental_delegates)
interpreter.allocate_tensors()
in_details = interpreter.get_input_details()[0]
out_details = interpreter.get_output_details()[0]
# You need to scale the input / output yourself using quantization params
in_scale, in_zp = in_details["quantization"]
out_scale, out_zp = out_details["quantization"]
# Loop through one-by-one (most TFLite files have a fixed batch size of 1)
preds_tflite = []
tflite_start = time.perf_counter()
for i in range(X_val.shape[0]):
# Scale input and invoke
x = X_val[i:i+1]
x_q = np.round(x / in_scale + in_zp).astype(in_details['dtype'])
interpreter.set_tensor(in_details["index"], x_q)
interpreter.invoke()
# Scale output back to f32
out = interpreter.get_tensor(out_details["index"])
out = (out.astype(np.float32) - out_zp) * out_scale
# And add the outcome to the predictions
preds_tflite.append(np.argmax(out, axis=1)[0])
tflite_end = time.perf_counter()
# Compare accuracy in the same way as above
acc_tflite = (np.array(preds_tflite) == y_val).mean()
if use_npu:
print(f"Quantized TFLite accuracy (NPU): {acc_tflite*100:.2f}% (time per inference: {(tflite_end - tflite_start) * 1000 / X_val.shape[0]:.4g}ms)")
else:
print(f"Quantized TFLite accuracy (CPU): {acc_tflite*100:.2f}% (time per inference: {(tflite_end - tflite_start) * 1000 / X_val.shape[0]:.4g}ms)")
run_tflite_model(TFLITE_FILE, False)
run_tflite_model(TFLITE_FILE, True)
```
4. 运行示例:
```
python3 quantize.py
# TF/Keras accuracy: 94.37% (time per inference: 12.9ms)
#
# Converting to TFLite file (cats_i8.tflite)...
# ...
# Quantized TFLite accuracy (CPU): 87.16% (time per inference: 10.37ms)
# Quantized TFLite accuracy (NPU): 88.51% (time per inference: 3.809ms)
```
太棒了!您现在拥有了 `cats_i8.tflite`,它在 NPU 上的运行速度快约 4 倍(但会有一定的精度损失)。有关 LiteRT 运行时的更多详细信息(包括 C++ 示例),请参见[在 NPU 上运行 LiteRT/TFLite 模型](/zh/ai-workflows/lite-rt)。