Files
nix-config/home/linux/gui/base/vinput/default.nix
T
ryan4yin | 二花 9f092808fd feat(aquamarine): serve Whisper transcription from the NPU (#472)
The goal is speech recognition on the NPU for vinput, and sherpa-onnx cannot
deliver it: every model it loads keeps a dynamic time axis, which the NPU
compiler rejects outright. Qwen3-ASR and SenseVoice both core-dumped in
vpux-compiler ("Got non broadcastable dimensions pair"), and reshaping the ONNX
inputs does not help because the dynamic dimensions sit inside the graph
(WA-037). Whisper is the only family OpenVINO publishes fully static IRs for,
and its fixed 30 s window is exactly what the compiler accepts.

The host now runs an OpenAI-compatible /v1/audio/transcriptions service backed
by whisper-large-v3-turbo int8 on the NPU: 1.5 s for 11 s of audio against
2.1 s on CPU, with the compiled graph cached in /var/cache so only a cold cache
pays the ~60 s NPU compile. vinput on the desktop sends raw PCM to it through a
small provider script.

Only the NPU driver and compiler still come from the unmerged branch in the
`nixpkgs-npu` input (NixOS/nixpkgs#570230), whose nixpkgs also happens to be
the one where openvino-genai 2026.4.0.0 builds at all.

Assisted-by: DeepSeek Harness 0.2.1-rc.2 (deepseek-v4.1-flash)
2026-10-18 08:34:50 +08:00

71 lines
2.5 KiB
Nix

{
inputs,
config,
lib,
pkgs,
...
}:
let
cfg = config.modules.desktop.vinput;
vinput = inputs.fcitx5-vinput.packages.${pkgs.stdenv.hostPlatform.system}.fcitx5-vinput;
in
{
options.modules.desktop.vinput.enable = lib.mkEnableOption "Fcitx5 Vinput voice input";
config = lib.mkIf cfg.enable {
i18n.inputMethod.fcitx5.addons = [ vinput ];
# MenuKey opens the command palette; set it to right Ctrl. `CommandKeys`
# (voice command on the selection) defaults to right Ctrl and outranks
# `MenuKey`, so clear it or the palette never opens. List options use
# fcitx5's `[Option]` + `0=` form, not upstream's `[Trigger]` shorthand.
xdg.configFile."fcitx5/conf/vinput.conf" = {
text = ''
[MenuKey]
0=Control_R
[CommandKeys]
'';
# fcitx5 rewrites this file at runtime, like `profile` and `config`.
force = true;
};
# The daemon rewrites this itself (the GUI, `vinput scene edit`) and resolves symlinks before
# writing, so the link keeps those edits in the checkout instead of a read-only store copy.
# `force` retakes the path if a write ever leaves a real file there. See the README.
xdg.configFile."vinput/config.json" = {
source = config.lib.file.mkOutOfStoreSymlink "${config.home.homeDirectory}/nix-config/home/linux/gui/base/vinput/config.json";
force = true;
};
# The OpenAI-transcription client vinput runs as a provider. config.json points at this
# absolute path, so the two must stay in sync.
xdg.configFile."vinput/aqua-asr-provider".source =
pkgs.writers.writePython3 "vinput-aqua-asr-provider" { }
./aqua-provider.py;
systemd.user.services.vinput-daemon = {
Unit = {
Description = "Fcitx5 Vinput voice input daemon";
After = [ "pipewire.service" ];
PartOf = [ "graphical-session.target" ];
};
Service = {
Type = "dbus";
BusName = "org.fcitx.Vinput";
ExecStart = "${vinput}/bin/vinput-daemon";
# Provider keys are `{{env.*}}` references the daemon resolves from its own environment,
# and a systemd user service inherits neither the interactive shell's environment nor any
# app's env file. No `-` prefix: a host without the file is misconfigured, not
# half-working, and the daemon logs "unset or empty" without it.
EnvironmentFile = "%h/.secrets/nushell-secrets.env";
Restart = "on-failure";
RestartSec = 3;
};
Install.WantedBy = [ "graphical-session.target" ];
};
};
}