Commit Graph
3 Commits
Author SHA1 Message Date
ryan4yin | 二花 9f092808fd feat(aquamarine): serve Whisper transcription from the NPU (#472)
The goal is speech recognition on the NPU for vinput, and sherpa-onnx cannot
deliver it: every model it loads keeps a dynamic time axis, which the NPU
compiler rejects outright. Qwen3-ASR and SenseVoice both core-dumped in
vpux-compiler ("Got non broadcastable dimensions pair"), and reshaping the ONNX
inputs does not help because the dynamic dimensions sit inside the graph
(WA-037). Whisper is the only family OpenVINO publishes fully static IRs for,
and its fixed 30 s window is exactly what the compiler accepts.

The host now runs an OpenAI-compatible /v1/audio/transcriptions service backed
by whisper-large-v3-turbo int8 on the NPU: 1.5 s for 11 s of audio against
2.1 s on CPU, with the compiled graph cached in /var/cache so only a cold cache
pays the ~60 s NPU compile. vinput on the desktop sends raw PCM to it through a
small provider script.

Only the NPU driver and compiler still come from the unmerged branch in the
`nixpkgs-npu` input (NixOS/nixpkgs#570230), whose nixpkgs also happens to be
the one where openvino-genai 2026.4.0.0 builds at all.

Assisted-by: DeepSeek Harness 0.2.1-rc.2 (deepseek-v4.1-flash)
2026-10-18 08:34:50 +08:00
Ryan Yin 4587c6bd14 chore(vinput): update key words 2026-10-17 00:36:10 +08:00
ryan4yin | 二花 e8819de40c feat(vinput): own the config here and read keys from the environment (#445)
* feat(vinput): read provider keys from an EnvironmentFile
* feat(vinput): manage the config from this repository
2026-10-15 13:54:20 +08:00