Whisper - Qualcomm Dragonwing Documentation

Whisper 是 OpenAI 推出的通用自动语音识别 (ASR) 模型。您可以将其用于音频转写、翻译和语种识别。您可以借助 Qualcomm 的 VoiceAI ASR 在 Dragonwing 开发板的 NPU 上运行 Whisper,也可以借助 whisper.cpp 在 CPU 上运行 Whisper。

通过 AIHUB 对 NPU 上 Whisper 模型的支持正在推进中,不久将更新。

使用 VoiceAI ASR 在 NPU 上运行 Whisper

1. 安装 SDK

准备基础环境

在开发板上打开终端,并为本示例准备基础依赖:

sudo apt install -y cmake pulseaudio-utils

安装 VoiceAI ASR - Community Edition

安装 VoiceAI ASR - Community Edition:

cd ~/

wget https://softwarecenter.qualcomm.com/api/download/software/sdks/VoiceAI_ASR_Community/All/2.3.0.0/VoiceAI_ASR_Community_v2.3.0.0.zip
unzip VoiceAI_ASR_Community_v2.3.0.0.zip -d voiceai_asr
rm VoiceAI_ASR_Community_v2.3.0.0.zip

cd voiceai_asr/2.3.0.0/

# Put the path to VoiceAI ASR in your bash_profile (so it's available under VOICEAI_ROOT)
echo "" >> ~/.bash_profile
echo "# Begin VoiceAI ASR" >> ~/.bash_profile
echo "export VOICEAI_ROOT=$PWD" >> ~/.bash_profile
echo "# End VoiceAI ASR" >> ~/.bash_profile
echo "" >> ~/.bash_profile

# Re-load the environment variables
source ~/.bash_profile

# Symlink Whisper libraries
cd $VOICEAI_ROOT/whisper_sdk/libs/npu/rpc_libraries/linux/whisper_all_quantized/
sudo ln -s $PWD/*.so /usr/lib/

2. 从 AI Hub 下载模型

安装好 SDK 后,您可以从 AI Hub 下载已编译的 Whisper 模型。下载模型时,请选择以下设备:

开发板	AI Hub 设备
RB3 Gen 2 Vision Kit	Qualcomm QCS6490 (Proxy)
RUBIK Pi 3	Qualcomm QCS6490 (Proxy)
IQ-9075 EVK	Qualcomm QCS9075 (Proxy)

下载后,将 encoder 模型重命名为 encoder_model_htp.bin,将 decoder 模型重命名为 decoder_model_htp.bin。要直接在开发板上下载 Whisper-Small-Quantized 模型:

RB3 Gen 2 Vision Kit / Rubik Pi 3
IQ-9075 EVK

mkdir -p ~/whisper_models/ai_hub_small_quantized/
cd ~/whisper_models/ai_hub_small_quantized/

# Models from https://aihub.qualcomm.com/models/whisper_small_quantized for QCS6490 (Proxy) target
wget -O encoder_model_htp.bin https://huggingface.co/qualcomm/Whisper-Small-Quantized/resolve/0e21411/precompiled/qualcomm-qcs6490-proxy/Whisper-Small-Quantized_WhisperSmallEncoderQuantizable_w8a16.bin
wget -O decoder_model_htp.bin https://huggingface.co/qualcomm/Whisper-Small-Quantized/resolve/0e21411/precompiled/qualcomm-qcs6490-proxy/Whisper-Small-Quantized_WhisperSmallDecoderQuantizable_w8a16.bin

# Vocab file is not in AI Hub yet, grab from our CDN
wget -O vocab.bin https://cdn.edgeimpulse.com/qc-ai-docs/models/whisper/vocab.bin

mkdir -p ~/whisper_models/ai_hub_small_quantized/
cd ~/whisper_models/ai_hub_small_quantized/

# Models from https://aihub.qualcomm.com/models/whisper_small_quantized for QCS9075 (Proxy) target
wget -O encoder_model_htp.bin https://huggingface.co/qualcomm/Whisper-Small-Quantized/resolve/0e21411/precompiled/qualcomm-qcs9075-proxy/Whisper-Small-Quantized_WhisperSmallEncoderQuantizable_w8a16.bin
wget -O decoder_model_htp.bin https://huggingface.co/qualcomm/Whisper-Small-Quantized/resolve/0e21411/precompiled/qualcomm-qcs9075-proxy/Whisper-Small-Quantized_WhisperSmallDecoderQuantizable_w8a16.bin

# Vocab file is not in AI Hub yet, grab from our CDN
wget -O vocab.bin https://cdn.edgeimpulse.com/qc-ai-docs/models/whisper/vocab.bin

3. 编译并运行示例

构建 npu_rpc_linux_sample/voice-ai-ref 示例

cd $VOICEAI_ROOT/whisper_sdk/sampleapp/npu_rpc_linux_sample/voice-ai-ref

# Overwrite the LogUtil.h function to log to stdout
wget -O include/LogUtil.h https://cdn.edgeimpulse.com/qc-ai-docs/code/voiceai_asr/2.3.0.0/whisper_sdk/sampleapp/npu_rpc_linux_sample/voice-ai-ref/include/LogUtil.h
# Overwrite the main.cpp example to add microphone selection
wget -O src/main.cpp https://cdn.edgeimpulse.com/qc-ai-docs/code/voiceai_asr/2.3.0.0/whisper_sdk/sampleapp/npu_rpc_linux_sample/voice-ai-ref/src/main.cpp

# Symlink Whisper libraries for build
mkdir -p libs/arm64-v8a/
cd libs/arm64-v8a/
ln -s $VOICEAI_ROOT/whisper_sdk/libs/npu/rpc_libraries/linux/whisper_all_quantized/*.so .
cd ../../

mkdir -p build
cd build
cmake ..
make -j`nproc`

转写 .WAV 文件

您现在可以转写 .WAV 文件:

cd $VOICEAI_ROOT/whisper_sdk/sampleapp/npu_rpc_linux_sample/voice-ai-ref/build

# Download sample file
wget -O jfk.wav https://raw.githubusercontent.com/ggml-org/whisper.cpp/refs/heads/master/samples/jfk.wav

# Transcribe:
./voice-ai-ref -f jfk.wav -l en -t transcribe -m ~/whisper_models/ai_hub_small_quantized/

# ... Expected result:
# VoiceAIRef final result =  And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country. [language: English]

# Press Control-C to exit the running application.

或者进行实时转写

将麦克风连接到开发板

将麦克风连接到开发板。

查找您的麦克风名称

pactl list short sources
# 49	alsa_output.platform-sound.stereo-fallback.monitor	PipeWire	s24-32le 2ch 48000Hz	SUSPENDED
# 76	alsa_input.usb-046d_C922_Pro_Stream_Webcam_C72F6EDF-02.analog-stereo	PipeWire	s16le 2ch 32000Hz	SUSPENDED

# To use the USB webcam, use "alsa_input.usb-046d_C922_Pro_Stream_Webcam_C72F6EDF-02.analog-stereo" as the name

运行实时转写

./voice-ai-ref -r -l en -t transcribe -m ~/whisper_models/ai_hub_small_quantized/ -d "alsa_input.usb-046d_C922_Pro_Stream_Webcam_C72F6EDF-02.analog-stereo"

# VoiceAIRef final result =  Hi, this is to see if I can do live transcription. [language: English]

在 VAD 判定当前无语音后,实时转写会立即报错退出,希望在未来更新中得到修复。

🚀 您现在已经可以在开发板上完全离线地进行音频转写了!VoiceAI ASR 没有面向更高级语言(如 Python)的绑定,因此如果您希望在应用中使用 Whisper,最简单的方法就是启动 voice-ai-ref 二进制文件,并从其 stdout 中读取数据。

使用 whisper.cpp 在 CPU 上运行 Whisper

或者,您也可以借助 whisper.cpp(或其他流行的 Whisper 库)在 CPU 上运行 Whisper(性能稍差)。下面是 whisper.cpp 的使用说明。在开发板上打开终端,或通过 ssh 连接到开发板,然后运行:

安装构建依赖

sudo apt update
sudo apt install -y libsdl2-dev libsdl2-2.0-0 libasound2-dev

构建 whisper.cpp

mkdir -p ~/dev/llm/
cd ~/dev/llm/

git clone https://github.com/ggml-org/whisper.cpp.git
cd whisper.cpp
git checkout v1.7.6

# Build (CPU)
cmake -B build-cpu -DWHISPER_SDL2=ON
cmake --build build-cpu -j`nproc` --config Release

将 whisper.cpp 路径添加到 PATH

cd ~/dev/llm/whisper.cpp/build-cpu/bin

echo "" >> ~/.bash_profile
echo "# Begin whisper.cpp" >> ~/.bash_profile
echo "export PATH=\$PATH:$PWD" >> ~/.bash_profile
echo "# End whisper.cpp" >> ~/.bash_profile
echo "" >> ~/.bash_profile

# To use the whisper.cpp files in your current session
source ~/.bash_profile

使用 whisper.cpp 转写一段音频

# Download model
cd ~/dev/llm/whisper.cpp
sh ./models/download-ggml-model.sh tiny.en-q5_1

# Transcribe text
whisper-cli -m models/ggml-tiny.en-q5_1.bin -f samples/jfk.wav

# [00:00:00.000 --> 00:00:10.480]
# and so my fellow Americans ask not what your country can do for you ask what you can do for your country

实时转写音频

将麦克风连接到开发板

将麦克风连接到开发板。

查找您的麦克风 ID

SDL_AUDIODRIVER=alsa whisper-stream -m models/ggml-tiny.en-q5_1.bin
# init: found 2 capture devices:
# init:    - Capture device #0: 'qcm6490-rb3-vision-snd-card, '
# init:    - Capture device #1: 'Yeti Stereo Microphone, USB Audio'

# If you want "Yeti Stereo Microphone, USB Audio" then the ID is 1

开始实时转写

SDL_AUDIODRIVER=alsa whisper-stream -m models/ggml-tiny.en-q5_1.bin -c 1

# main: processing 48000 samples (step = 3.0 sec / len = 10.0 sec / keep = 0.2 sec), 4 threads, lang = en, task = transcribe, timestamps = 0 ...
# main: n_new_line = 2, no_context = 1
#
# [Start speaking]
# This is a test to see if you can transcribe text live on your Qualcomm device

通过 OpenCL 在 GPU 上运行

您也可以构建在 GPU 上运行的二进制文件:

安装 OpenCL 头文件和 ICD 加载库

首先按 llama.cpp 中”安装 OpenCL 头文件和 ICD 加载库”的步骤操作。

构建带 OpenCL 的二进制文件

cd ~/dev/llm/whisper.cpp

cmake -B build-gpu -DGGML_OPENCL=ON  -DWHISPER_SDL2=ON
cmake --build build-gpu -j`nproc` --config Release

# Find the binary in:
#     build-gpu/bin/whisper-cli

​使用 VoiceAI ASR 在 NPU 上运行 Whisper

​1. 安装 SDK

​2. 从 AI Hub 下载模型

​3. 编译并运行示例

​使用 whisper.cpp 在 CPU 上运行 Whisper

​通过 OpenCL 在 GPU 上运行

使用 VoiceAI ASR 在 NPU 上运行 Whisper

1. 安装 SDK

2. 从 AI Hub 下载模型

3. 编译并运行示例

使用 whisper.cpp 在 CPU 上运行 Whisper

通过 OpenCL 在 GPU 上运行