Whisper - Qualcomm Dragonwing Documentation

Whisper 是 OpenAI 的通用自动语音识别(ASR)模型。您可以将其用于音频转录、翻译和语言识别。您可以使用 Qualcomm 的 VoiceAI ASR 在 Dragonwing 开发板的 NPU 上运行 Whisper,或使用 whisper.cpp 在 CPU 上运行。

使用 VoiceAI ASR 在 NPU 上运行 Whisper

1. 安装 SDK

在开发板上打开终端,并为此示例设置基本依赖:
sudo apt install -y cmake pulseaudio-utils

安装 AI Runtime SDK - Community Edition:

# Install the SDK
wget -qO- https://cdn.edgeimpulse.com/qc-ai-docs/device-setup/install_ai_runtime_sdk.sh | bash

# Use the SDK in your current session
source ~/.bash_profile

安装 VoiceAI ASR - Community Edition:

cd ~/

wget https://softwarecenter.qualcomm.com/api/download/software/sdks/VoiceAI_ASR_Community/All/2.3.0.0/VoiceAI_ASR_Community_v2.3.0.0.zip
unzip VoiceAI_ASR_Community_v2.3.0.0.zip -d voiceai_asr
rm VoiceAI_ASR_Community_v2.3.0.0.zip

cd voiceai_asr/2.3.0.0/

# Put the path to VoiceAI ASR in your bash_profile (so it's available under VOICEAI_ROOT)
echo "" >> ~/.bash_profile
echo "# Begin VoiceAI ASR" >> ~/.bash_profile
echo "export VOICEAI_ROOT=$PWD" >> ~/.bash_profile
echo "# End VoiceAI ASR" >> ~/.bash_profile
echo "" >> ~/.bash_profile

# Re-load the environment variables
source ~/.bash_profile

# Symlink Whisper libraries
cd $VOICEAI_ROOT/whisper_sdk/libs/npu/rpc_libraries/linux/whisper_all_quantized/
sudo ln -s $PWD/*.so /usr/lib/

2. 从 AI Hub 下载模型

安装好 SDK 后,您可以从 AI Hub 下载预编译的 Whisper 模型。下载模型时,请选择以下设备:

RB3 Gen 2 Vision Kit:‘Qualcomm QCS6490 (Proxy)’
RUBIK Pi 3:‘Qualcomm QCS6490 (Proxy)’
IQ-9075 EVK:‘Qualcomm QCS9075 (Proxy)’

下载后,将编码器模型重命名为 encoder_model_htp.bin,将解码器模型重命名为 decoder_model_htp.bin。要在开发板上直接下载 Whisper-Small-Quantized 模型:

RB3 Gen 2 Vision Kit / Rubik Pi 3:

mkdir -p ~/whisper_models/ai_hub_small_quantized/
cd ~/whisper_models/ai_hub_small_quantized/

# Models from https://aihub.qualcomm.com/models/whisper_small_quantized for QCS6490 (Proxy) target
wget -O encoder_model_htp.bin https://huggingface.co/qualcomm/Whisper-Small-Quantized/resolve/0e21411/precompiled/qualcomm-qcs6490-proxy/Whisper-Small-Quantized_WhisperSmallEncoderQuantizable_w8a16.bin
wget -O decoder_model_htp.bin https://huggingface.co/qualcomm/Whisper-Small-Quantized/resolve/0e21411/precompiled/qualcomm-qcs6490-proxy/Whisper-Small-Quantized_WhisperSmallDecoderQuantizable_w8a16.bin

# Vocab file is not in AI Hub yet, grab from our CDN
wget -O vocab.bin https://cdn.edgeimpulse.com/qc-ai-docs/models/whisper/vocab.bin

IQ-9075 EVK:

mkdir -p ~/whisper_models/ai_hub_small_quantized/
cd ~/whisper_models/ai_hub_small_quantized/

# Models from https://aihub.qualcomm.com/models/whisper_small_quantized for QCS9075 (Proxy) target
wget -O encoder_model_htp.bin https://huggingface.co/qualcomm/Whisper-Small-Quantized/resolve/0e21411/precompiled/qualcomm-qcs9075-proxy/Whisper-Small-Quantized_WhisperSmallEncoderQuantizable_w8a16.bin
wget -O decoder_model_htp.bin https://huggingface.co/qualcomm/Whisper-Small-Quantized/resolve/0e21411/precompiled/qualcomm-qcs9075-proxy/Whisper-Small-Quantized_WhisperSmallDecoderQuantizable_w8a16.bin

# Vocab file is not in AI Hub yet, grab from our CDN
wget -O vocab.bin https://cdn.edgeimpulse.com/qc-ai-docs/models/whisper/vocab.bin

3. 编译并运行示例

构建 npu_rpc_linux_sample/voice-ai-ref 示例:

cd $VOICEAI_ROOT/whisper_sdk/sampleapp/npu_rpc_linux_sample/voice-ai-ref

# Overwrite the LogUtil.h function to log to stdout
wget -O include/LogUtil.h https://cdn.edgeimpulse.com/qc-ai-docs/code/voiceai_asr/2.3.0.0/whisper_sdk/sampleapp/npu_rpc_linux_sample/voice-ai-ref/include/LogUtil.h
# Overwrite the main.cpp example to add microphone selection
wget -O src/main.cpp https://cdn.edgeimpulse.com/qc-ai-docs/code/voiceai_asr/2.3.0.0/whisper_sdk/sampleapp/npu_rpc_linux_sample/voice-ai-ref/src/main.cpp

# Symlink Whisper libraries for build
mkdir -p libs/arm64-v8a/
cd libs/arm64-v8a/
ln -s $VOICEAI_ROOT/whisper_sdk/libs/npu/rpc_libraries/linux/whisper_all_quantized/*.so .
cd ../../

mkdir -p build
cd build
cmake ..
make -j`nproc`

现在您可以转录 .WAV 文件:

cd $VOICEAI_ROOT/whisper_sdk/sampleapp/npu_rpc_linux_sample/voice-ai-ref/build

# Download sample file
wget -O jfk.wav https://raw.githubusercontent.com/ggml-org/whisper.cpp/refs/heads/master/samples/jfk.wav

# Transcribe:
./voice-ai-ref -f jfk.wav -l en -t transcribe -m ~/whisper_models/ai_hub_small_quantized/

# ... Expected result:
# VoiceAIRef final result =  And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country. [language: English]

# Press Control-C to exit the running application.

甚至可以进行实时转录: a. 将麦克风连接到开发板。 b. 找到麦克风的名称:

pactl list short sources
# 49	alsa_output.platform-sound.stereo-fallback.monitor	PipeWire	s24-32le 2ch 48000Hz	SUSPENDED
# 76	alsa_input.usb-046d_C922_Pro_Stream_Webcam_C72F6EDF-02.analog-stereo	PipeWire	s16le 2ch 32000Hz	SUSPENDED

# To use the USB webcam, use "alsa_input.usb-046d_C922_Pro_Stream_Webcam_C72F6EDF-02.analog-stereo" as the name

运行实时转录:

./voice-ai-ref -r -l en -t transcribe -m ~/whisper_models/ai_hub_small_quantized/ -d "alsa_input.usb-046d_C922_Pro_Stream_Webcam_C72F6EDF-02.analog-stereo"

# VoiceAIRef final result =  Hi, this is to see if I can do live transcription. [language: English]

当 VAD 判断没有语音后,实时转录会立即报错退出,希望这一问题会在未来的更新中得到修复。

🚀 您现在可以在开发板上完全离线地转录音频了!VoiceAI ASR 没有面向高级语言(如 Python)的绑定,因此如果您想在应用程序中使用 Whisper,最简单的方法是直接启动 voice-ai-ref 二进制文件,并从 stdout 读取数据。

使用 whisper.cpp 在 CPU 上运行 Whisper

您也可以使用 whisper.cpp(或其他任何流行的 Whisper 库)在 CPU 上运行 Whisper(性能较低)。以下是 whisper.cpp 的操作说明。在开发板上打开终端,或通过 SSH 会话连接到开发板,然后运行:

安装构建依赖项:

sudo apt update
sudo apt install -y libsdl2-dev libsdl2-2.0-0 libasound2-dev

构建 whisper.cpp:

mkdir -p ~/dev/llm/
cd ~/dev/llm/

git clone https://github.com/ggml-org/whisper.cpp.git
cd whisper.cpp
git checkout v1.7.6

# Build (CPU)
cmake -B build-cpu -DWHISPER_SDL2=ON
cmake --build build-cpu -j`nproc` --config Release

将 whisper.cpp 路径添加到您的 PATH:

cd ~/dev/llm/whisper.cpp/build-cpu/bin

echo "" >> ~/.bash_profile
echo "# Begin whisper.cpp" >> ~/.bash_profile
echo "export PATH=\$PATH:$PWD" >> ~/.bash_profile
echo "# End whisper.cpp" >> ~/.bash_profile
echo "" >> ~/.bash_profile

# To use the whisper.cpp files in your current session
source ~/.bash_profile

现在您可以使用 whisper.cpp 转录一些音频:

# Download model
cd ~/dev/llm/whisper.cpp
sh ./models/download-ggml-model.sh tiny.en-q5_1

# Transcribe text
whisper-cli -m models/ggml-tiny.en-q5_1.bin -f samples/jfk.wav

# [00:00:00.000 --> 00:00:10.480]
# and so my fellow Americans ask not what your country can do for you ask what you can do for your country

您还可以实时转录音频: a. 将麦克风连接到开发板。 b. 找到麦克风 ID:

SDL_AUDIODRIVER=alsa whisper-stream -m models/ggml-tiny.en-q5_1.bin
# init: found 2 capture devices:
# init:    - Capture device #0: 'qcm6490-rb3-vision-snd-card, '
# init:    - Capture device #1: 'Yeti Stereo Microphone, USB Audio'

# If you want "Yeti Stereo Microphone, USB Audio" then the ID is 1

c. 开始实时转录:

SDL_AUDIODRIVER=alsa whisper-stream -m models/ggml-tiny.en-q5_1.bin -c 1

# main: processing 48000 samples (step = 3.0 sec / len = 10.0 sec / keep = 0.2 sec), 4 threads, lang = en, task = transcribe, timestamps = 0 ...
# main: n_new_line = 2, no_context = 1
#
# [Start speaking]
# This is a test to see if you can transcribe text live on your Qualcomm device

使用 OpenCL 在 GPU 上运行

您还可以构建在 GPU 上运行的二进制文件:

首先按照 llama.cpp 中”安装 OpenCL 头文件和 ICD 加载器库”下的步骤操作。

使用 OpenCL 构建二进制文件:

cd ~/dev/llm/whisper.cpp

cmake -B build-gpu -DGGML_OPENCL=ON  -DWHISPER_SDL2=ON
cmake --build build-gpu -j`nproc` --config Release

# Find the binary in:
#     build-gpu/bin/whisper-cli

​使用 VoiceAI ASR 在 NPU 上运行 Whisper

​1. 安装 SDK

​2. 从 AI Hub 下载模型

​3. 编译并运行示例

​使用 whisper.cpp 在 CPU 上运行 Whisper

​使用 OpenCL 在 GPU 上运行

使用 VoiceAI ASR 在 NPU 上运行 Whisper

1. 安装 SDK

2. 从 AI Hub 下载模型

3. 编译并运行示例

使用 whisper.cpp 在 CPU 上运行 Whisper

使用 OpenCL 在 GPU 上运行