Commit Graph
7 Commits
Author SHA1 Message Date
ryan4yin | 二花 9f092808fd feat(aquamarine): serve Whisper transcription from the NPU (#472)
The goal is speech recognition on the NPU for vinput, and sherpa-onnx cannot
deliver it: every model it loads keeps a dynamic time axis, which the NPU
compiler rejects outright. Qwen3-ASR and SenseVoice both core-dumped in
vpux-compiler ("Got non broadcastable dimensions pair"), and reshaping the ONNX
inputs does not help because the dynamic dimensions sit inside the graph
(WA-037). Whisper is the only family OpenVINO publishes fully static IRs for,
and its fixed 30 s window is exactly what the compiler accepts.

The host now runs an OpenAI-compatible /v1/audio/transcriptions service backed
by whisper-large-v3-turbo int8 on the NPU: 1.5 s for 11 s of audio against
2.1 s on CPU, with the compiled graph cached in /var/cache so only a cold cache
pays the ~60 s NPU compile. vinput on the desktop sends raw PCM to it through a
small provider script.

Only the NPU driver and compiler still come from the unmerged branch in the
`nixpkgs-npu` input (NixOS/nixpkgs#570230), whose nixpkgs also happens to be
the one where openvino-genai 2026.4.0.0 builds at all.

Assisted-by: DeepSeek Harness 0.2.1-rc.2 (deepseek-v4.1-flash)
2026-10-18 08:34:50 +08:00
Ryan Yin 4587c6bd14 chore(vinput): update key words 2026-10-17 00:36:10 +08:00
ryan4yin | 二花 e8819de40c feat(vinput): own the config here and read keys from the environment (#445)
* feat(vinput): read provider keys from an EnvironmentFile
* feat(vinput): manage the config from this repository
2026-10-15 13:54:20 +08:00
Ryan Yin 8e3f12aa19 feat(vinput): open the command palette with right Ctrl
Pin keyd's right-hand modifiers (rightcontrol/rightshift/rightmeta) to their own keycodes; keyd otherwise folds them into the left keys. Set Vinput's MenuKey to right Ctrl and clear CommandKeys, which would shadow it.
2026-10-05 22:41:48 +08:00
ryan4yin | 二花 6b8c5bfae4 feat(vinput): drop NPU/OpenVINO support (#384)
Vinput's streaming Zipformer models have unbounded dynamic shapes, so
OpenVINO cannot compile them for the GPU or the NPU; the `openvino`
provider and ai's Intel NPU enablement could never run on the NPU (noted
as unverified in WORKAROUNDS WA-017). Remove the backend option, the
daemon's OpenVINO library path, and the intel-npu module enable, and
update the eval test, README and WORKAROUNDS (drop WA-016 and WA-017).

Keep Vinput's stock package and its bundled sherpa-onnx (1.13.4): the
fork-based and nixpkgs-master sherpa overrides existed only to provide the
`openvino` provider, so drop package.nix with them.

Assisted-by: opencode (DeepSeek V4.1 Flash)
2026-10-05 15:42:48 +08:00
ryan4yin | 二花 502d0c85c7 fix(preservation): keep vinput config and models across reboots (#381)
Vinput stores its core config in ~/.config/vinput/config.json and its
downloaded sherpa-onnx models in ~/.local/share/vinput/models/. Neither
was bind-mounted from /persistent on the tmpfs-root hosts, so a reboot
wiped both and vinput-daemon started with "Local ASR provider model is
not configured". Persist both directories and document the one-shot CLI
model setup in the vinput README.
2026-10-05 00:18:01 +08:00
Ryan Yin a695a3ce3d feat(vinput): add NPU speech input on ai 2026-10-04 19:51:22 +08:00