starred/nix-config
1
0 Fork 0
mirror of https://github.com/ryan4yin/nix-config.git synced 2026-11-13 12:13:46 +01:00
Code Issues Packages Projects Releases 10 Wiki Activity

feat(voice): add NPU voice input on ai (#377)

Browse Source 
* feat(vinput): add NPU speech input on ai

* docs(agents): scope nixpkgs review guidance
This commit is contained in:
ryan4yin | 二花 authored and GitHub committed 2026-10-04 19:52:44 +08:00
2 parents fdafca930b ac18edd273
commit 0a672060a4
15 files changed
+273 -36
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.agents/skills/nixpkgs-patched/SKILL.md
+14 -5
View File
@@ -51,17 +51,22 @@ PR, review both changes and record their order.
## 3. Validate before publishing
In the nixpkgs checkout, run the smallest relevant checks first:
For a local patched-branch update, validate only the package(s) the user intends to use. Use normal
Nix commands from the nixpkgs checkout, such as `nix build .#<package>` or a specifically requested
`passthru.tests.<name>`. These commands build their required dependencies; do not separately build
or test unrelated packages just because they changed on the branch.
`nixpkgs-review` is for reviewing an upstream nixpkgs PR. Do not use it to validate ordinary local
package use or a patched-branch lock update. For this workflow, inspect the diff and commit list:
```bash
git diff --check
git log --oneline --decorate upstream/nixos-unstable..HEAD
nixpkgs-review rev HEAD --no-shell --print-result
```
Build the affected package and its relevant tests. A cherry-pick that applies cleanly can still be
invalid against the current unstable base. Record the source PR, commit IDs, base revision, package,
and test result before pushing.
Run package builds or tests only when the user requests them or when the selected package is needed
to complete the requested local verification. Record which package and checks were actually run;
never imply that unrelated packages or platforms were covered.
## 4. Publish and consume it
@@ -82,6 +87,10 @@ just eval-host <affected-host>
just build-host <affected-host>
```
The nix-config eval and host-build commands validate this configuration. They are not a reason to
build every package in the patched nixpkgs branch. Run a host build only for a host that consumes the
changed package and only when the user asks for that validation.
Keep the nixpkgs fork change, lock update, and unrelated configuration changes separate. Do not run
`just niri`, `just local`, or a remote deployment from this skill; activation is a separate
authorized step in `nix-config-update`.
.agents/skills/nixpkgs-review/SKILL.md
+18 -12
View File
@@ -1,17 +1,21 @@
---
name: nixpkgs-review
description:
Use when reviewing or testing a NixOS/nixpkgs pull request, including packages, passthru tests,
dependencies or closure size, local vs CI results, and regressions before they reach this flake.
Use when reviewing an upstream NixOS/nixpkgs pull request before it is merged, including its
package changes, passthru tests, dependencies, or CI results.
---
# Reviewing nixpkgs changes
Prefer a local review: faster for a small change, the result stays available, and it leaves no
GitHub Actions state. Use the repository's GHA workflow when a review needs several architectures, a
clean remote environment, or more build capacity than this x86_64-linux desktop has.
**`nixpkgs-review` is the first test, not the only test:** follow it with focused package,
test-quality, or runtime checks when the change has user-visible behavior.
Use `nixpkgs-review` to review an upstream nixpkgs PR before it is merged. It compares package
changes against a PR base and can build selected packages and passthru tests. It is not the normal
way to build a package for local use or validate a nix-config lock update. For local use, build the
needed package or test directly with the project's Nix commands.
Review only the package(s) changed by the PR that are relevant to the review question. Add
`--tests` when the selected package's passthru tests are part of the review. Do not broaden a review
to unrelated packages; selecting a large source package can trigger substantial downloads and
builds.
## Choose the runner
@@ -21,7 +25,7 @@ test-quality, or runtime checks when the change has user-visible behavior.
| Need to inspect the package interactively | local review shell |
| Need aarch64/Darwin or a reproducible remote build | `just pkg-review <pr>` |
| Only one package's passthru tests matter | `just pkg-test <pr> <pname>` or local `--package`/`--tests` |
| Check a previous local commit or worktree | `nixpkgs-review rev <rev>` or `wip` |
| Review a local commit proposed for an upstream PR | `nixpkgs-review rev <rev>` |
The local tool defaults to the current system. This machine is `x86_64-linux`; do not imply that a
successful local result covers Darwin or aarch64. Use `--systems` explicitly when builders or
@@ -167,9 +171,11 @@ them for cross-architecture coverage, large reviews, or when local capacity cann
relevant target. Read the workflow summary and distinguish evaluation, build, passthru-test, and
architecture-specific failures.
## 6. Bring a result back to this flake
## 6. Keep local package use separate
If the package is used here, validate the affected `nixpkgs` input or package override separately:
When an upstream PR has been merged or carried on a local patched branch, do not use
`nixpkgs-review` just to consume a package. Update the relevant flake input, then use the smallest
configuration check that answers the local question:
```bash
just test
@@ -177,6 +183,6 @@ just eval-host <affected-host>
just build-host <affected-host>
```
A nixpkgs-review result proves only the reviewed nixpkgs build/test set. It does not prove this
flake's overlays, hardening wrappers, host configuration, or runtime behavior. Use
An upstream PR review result proves only the selected nixpkgs packages and systems. It does not
prove this flake's overlays, hardening wrappers, host configuration, or runtime behavior. Use
`nix-config-debug` for failures and `nix-config-update` before changing a locked input.
WORKAROUNDS.md
+11 -9
View File
@@ -16,15 +16,16 @@ obsolete workarounds get rolled back instead of lingering.
A version or revision pinned for a reason that can go away.
| ID | What & where | Why | Removal condition | Added | Revisit | Status |
| ------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | ---------- | ---------------------- | ------ |
| WA-001 | smartctl_exporter pinned to upstream `6274decc`, which carries PR [#329](https://github.com/prometheus-community/smartctl_exporter/pull/329) — `overlays/smartctl-exporter/` | `/metrics` returns HTTP 500 once a device becomes readable after startup ([#305](https://github.com/prometheus-community/smartctl_exporter/issues/305), [#326](https://github.com/prometheus-community/smartctl_exporter/issues/326)) | Drop once nixpkgs provides a version containing the #329 fix | 2026-10-03 | next nixpkgs bump | active |
| WA-002 | `nixpkgs-2505` — `flake.nix`; consumer `home/base/tui/container.nix` | `kubernetes-helm` regressed in newer nixpkgs | Drop the input and its consumer once helm works again in current nixpkgs | 2025-12-03 | when helm is revisited | active |
| WA-003 | `rust-overlay` follow — `flake.nix` | lanzaboote v1.1.0's pinned rust-overlay uses the deprecated `stdenv.isLinux`/`stdenv.isDarwin` | Drop the follow once lanzaboote updates its pin (or we drop the lanzaboote pin) | 2026-09-25 | on lanzaboote bump | active |
| WA-004 | `catppuccin` on `main` — `flake.nix` | `main` carries the nodejs→nodejs-slim deprecation fix, which is not in a release | Switch to a release tag once it includes the fix | 2026-09-25 | on catppuccin release | active |
| WA-005 | `tuios` input — `flake.nix`; `home/base/tui/tuios/` | nixpkgs lags behind upstream | Use nixpkgs' `tuios` once it is recent enough | 2026-10-03 | on nixpkgs bump | active |
| WA-006 | `nixpkgs-blender` — `flake.nix` | Blender 5.2 LTS + CUDA/OptiX pin; keeps `just up` from triggering a full source rebuild | Bump deliberately when needed; drop the pin if the current nixpkgs build is acceptable | 2026-07-24 | deliberate | active |
| WA-007 | WeChat from `nixpkgs-master` — `hardening/bwraps/default.nix` | newest builds; the pinned channel lags | Fall back to the stable channel once it has a good build | 2026-10-03 | on nixpkgs bump | active |
| ID | What & where | Why | Removal condition | Added | Revisit | Status |
| ------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | ---------- | -------------------------- | ------ |
| WA-001 | smartctl_exporter pinned to upstream `6274decc`, which carries PR [#329](https://github.com/prometheus-community/smartctl_exporter/pull/329) — `overlays/smartctl-exporter/` | `/metrics` returns HTTP 500 once a device becomes readable after startup ([#305](https://github.com/prometheus-community/smartctl_exporter/issues/305), [#326](https://github.com/prometheus-community/smartctl_exporter/issues/326)) | Drop once nixpkgs provides a version containing the #329 fix | 2026-10-03 | next nixpkgs bump | active |
| WA-002 | `nixpkgs-2505` — `flake.nix`; consumer `home/base/tui/container.nix` | `kubernetes-helm` regressed in newer nixpkgs | Drop the input and its consumer once helm works again in current nixpkgs | 2025-12-03 | when helm is revisited | active |
| WA-003 | `rust-overlay` follow — `flake.nix` | lanzaboote v1.1.0's pinned rust-overlay uses the deprecated `stdenv.isLinux`/`stdenv.isDarwin` | Drop the follow once lanzaboote updates its pin (or we drop the lanzaboote pin) | 2026-09-25 | on lanzaboote bump | active |
| WA-004 | `catppuccin` on `main` — `flake.nix` | `main` carries the nodejs→nodejs-slim deprecation fix, which is not in a release | Switch to a release tag once it includes the fix | 2026-09-25 | on catppuccin release | active |
| WA-005 | `tuios` input — `flake.nix`; `home/base/tui/tuios/` | nixpkgs lags behind upstream | Use nixpkgs' `tuios` once it is recent enough | 2026-10-03 | on nixpkgs bump | active |
| WA-006 | `nixpkgs-blender` — `flake.nix` | Blender 5.2 LTS + CUDA/OptiX pin; keeps `just up` from triggering a full source rebuild | Bump deliberately when needed; drop the pin if the current nixpkgs build is acceptable | 2026-07-24 | deliberate | active |
| WA-007 | WeChat from `nixpkgs-master` — `hardening/bwraps/default.nix` | newest builds; the pinned channel lags | Fall back to the stable channel once it has a good build | 2026-10-03 | on nixpkgs bump | active |
| WA-016 | Sherpa-ONNX 1.13.8 override and Nix CMake patches — `flake.nix`, `home/linux/gui/base/vinput/package.nix` | Vinput pins Sherpa 1.13.4, which does not recognize the `openvino` provider; newer upstream source needs Nix-compatible dependency linking | Drop the override and carried patches once Vinput's Sherpa dependency is at least 1.13.8 and builds with upstream Nix patches | 2026-10-04 | on Vinput or Sherpa update | active |
## Workarounds
@@ -49,6 +50,7 @@ A known limitation with no fix yet.
| ------ | ---------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | ---------- | -------------------------------------- | ------ |
| WA-012 | `muvm` (x86_64 apps/games via microVM) disabled — `hosts/12kingdoms-shoukei/apple-silicon.nix` | Apple Silicon's 16k page size means x86_64 apps need `muvm`; it is off because it does not build/run on nixos-apple-silicon yet ([nixos-apple-silicon#237](https://github.com/nix-community/nixos-apple-silicon/issues/237)) | Re-enable once `muvm` builds and works | — | on nixos-apple-silicon or muvm updates | active |
| WA-015 | Darwin secret ownership — `secrets/darwin.nix` | Blanket `chown /etc/agenix/*` can override declared root ownership through symlinks; source concern, not a verified runtime leak | Verify consumers/runtime ownership, replace blanket ownership with explicit per-file ownership, and recheck target modes | 2026-10-03 | next Darwin activation review | active |
| WA-017 | NPU inference for X-ASR — `home/linux/gui/base/vinput/` | OpenVINO and the NPU plugin load and the daemon opens `/dev/accel/accel0`, but no profiling evidence confirms model computation on the NPU; `npu_busy_time_us` did not increase in the sample test | Remove once per-layer NPU profiling confirms execution on `ai` | 2026-10-04 | when model/provider changes | active |
## WIP
flake.lock
Generated
+47 -3
View File
@@ -166,6 +166,28 @@
"type": "github"
}
},
"fcitx5-vinput": {
"inputs": {
"nixpkgs": [
"nixpkgs"
],
"sherpa-onnx": "sherpa-onnx"
},
"locked": {
"lastModified": 1790069778,
"narHash": "sha256-Fs3dTHfde74C4MXZLJWvhDY6fQUIrY2Qwcgnq59aKJ0=",
"owner": "xifan2333",
"repo": "fcitx5-vinput",
"rev": "eada49c62ddcb1a4efd7473794bad71a7cacc075",
"type": "github"
},
"original": {
"owner": "xifan2333",
"ref": "v2.4.0",
"repo": "fcitx5-vinput",
"type": "github"
}
},
"flake-compat": {
"flake": false,
"locked": {
@@ -850,11 +872,11 @@
},
"nixpkgs-patched": {
"locked": {
"lastModified": 1786862044,
"narHash": "sha256-zd5dQC9Y6Q8FtfqAbNJTFmbwpyqaeFuZ9Ww5YjpIsaQ=",
"lastModified": 1791111712,
"narHash": "sha256-HJ5klsgKTTS7HhhUgttbYtluiW4gf5ye1j9LbghtSWw=",
"owner": "ryan4yin",
"repo": "nixpkgs",
"rev": "5d9f7ea174a0cb08dcc36eba9b8f67615962ece9",
"rev": "365fea9e582610470f2bec94da5c54f122fbeb73",
"type": "github"
},
"original": {
@@ -1018,6 +1040,7 @@
"catppuccin": "catppuccin",
"colmena": "colmena",
"disko": "disko",
"fcitx5-vinput": "fcitx5-vinput",
"haumea": "haumea",
"home-manager": "home-manager",
"lanzaboote": "lanzaboote",
@@ -1066,6 +1089,27 @@
"type": "github"
}
},
"sherpa-onnx": {
"inputs": {
"nixpkgs": [
"fcitx5-vinput",
"nixpkgs"
]
},
"locked": {
"lastModified": 1785564218,
"narHash": "sha256-B1/qLagKXemMgyrmd7RqPEgz2polrZMUkeBT1dv3hso=",
"owner": "xifan2333",
"repo": "sherpa-onnx-flake",
"rev": "a8ef134239c923420becfa6870538dcc8e9c8eb8",
"type": "github"
},
"original": {
"owner": "xifan2333",
"repo": "sherpa-onnx-flake",
"type": "github"
}
},
"spectrum": {
"flake": false,
"locked": {
flake.nix
+5 -5
View File
@@ -21,17 +21,17 @@
# https://hydra.nixos.org/jobset/nixpkgs/unstable
# update via nix flake update nixpkgs --override-input nixpkgs github:NixOS/nixpkgs/<commit-hash>
nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
fcitx5-vinput = {
url = "github:xifan2333/fcitx5-vinput/v2.4.0";
inputs.nixpkgs.follows = "nixpkgs";
};
nixpkgs-stable.url = "github:nixos/nixpkgs/nixos-26.05";
# Kept for pinning a package that regressed in newer nixpkgs. Currently only
# kubernetes-helm uses it (home/base/tui/container.nix); extend or drop this
# input together with that consumer.
nixpkgs-2505.url = "github:nixos/nixpkgs/nixos-25.05";
# nixpkgs with some custom patches.
# Carried but intentionally unused: the `.agents/skills/nixpkgs-patched`
# workflow consumes `pkgs-patched` when a temporary patch is needed, and the
# input is already wired in outputs/default.nix so that workflow never has to
# edit the flake. See .agents/skills/nixpkgs-patched/SKILL.md.
# nixpkgs with some custom patches, including the NPU-capable sherpa-onnx package.
nixpkgs-patched.url = "github:ryan4yin/nixpkgs/nixos-unstable-patched";
# get some latest packages from the master branch
nixpkgs-master.url = "github:nixos/nixpkgs/master";
home/hosts/linux/12kingdoms-shoukei.nix
+5
View File
@@ -9,6 +9,11 @@ in
./12kingdoms-shoukei/wluma.nix
];
modules.desktop.vinput = {
enable = true;
backend = "cpu";
};
programs.ssh.settings."github.com".IdentityFile = "${config.home.homeDirectory}/.ssh/${hostName}";
modules.desktop.gaming.enable = false;
home/hosts/linux/idols-ai.nix
+12 -2
View File
@@ -1,9 +1,19 @@
{ config, ... }:
{
config,
...
}:
let
mkSymlink = config.lib.file.mkOutOfStoreSymlink;
in
{
imports = [ ../../linux/gui.nix ];
imports = [
../../linux/gui.nix
];
modules.desktop.vinput = {
enable = true;
backend = "openvino";
};
programs.ssh.settings."github.com".IdentityFile = "${config.home.homeDirectory}/.ssh/idols-ai";
home/linux/gui/base/README.md
+1
View File
@@ -77,6 +77,7 @@ Noctalia consolidates functionality that previously required multiple tools:
- **[`fcitx5/`](./fcitx5/)**: Fcitx5 input method with Rime (Chinese), Mozc (Japanese), and Hangul
(Korean) engines
- **[`vinput/`](./vinput/)**: Voice input on `ai` (OpenVINO) and `shoukei` (CPU)
### Applications
home/linux/gui/base/fcitx5/default.nix
+6
View File
@@ -12,6 +12,12 @@
config.lib.file.mkOutOfStoreSymlink "${config.home.homeDirectory}/nix-config/home/linux/gui/base/fcitx5/mozc-config1.db";
};
xdg.dataFile."fcitx5/rime/default.custom.yaml".text = ''
patch:
ascii_composer/switch_key/Shift_L: noop
ascii_composer/switch_key/Shift_R: commit_code
'';
i18n.inputMethod = {
enable = true;
type = "fcitx5";
home/linux/gui/base/vinput/README.md
+31
View File
@@ -0,0 +1,31 @@
# Fcitx5 Vinput
On `ai` and `shoukei`, install a Sherpa-ONNX model in Vinput under **Resources → Models**, then
select it in **Control → ASR Providers**. Hold and release **right Alt** to dictate. **Right Shift**
toggles Chinese/English; **left Shift** remains a normal modifier. Open Vinput from the application
launcher for settings.
Models are stored in `~/.local/share/vinput/models/sherpa-onnx/`. Review their source and license
before installing.
## Backends
- `shoukei` uses CPU. Leave the model's provider at its default.
- `ai` uses OpenVINO when the model's `vinput-model.json` sets `"provider": "openvino"`; restart
`vinput-daemon` after changing it. The user needs the `render` group and a new login after
deployment.
**NPU execution status: unverified.** On `ai`, the OpenVINO runtime and NPU plugin load, the daemon
opens `/dev/accel/accel0`, and the bundled WAV sample transcribes. However, the NPU busy-time
counter did not increase, and no NPU profiling data has been collected. These observations do not
confirm that model computation ran on the NPU; CPU fallback or partial fallback remains possible.
## Diagnostics
```sh
journalctl --user -b -u vinput-daemon
```
Check the Sherpa/OpenVINO logs for the selected device. A definitive check needs NPU profiling
enabled for inference and non-zero per-layer NPU timings; a loaded plugin or device busy-time
counter alone is not conclusive.
home/linux/gui/base/vinput/default.nix
+60
View File
@@ -0,0 +1,60 @@
{
inputs,
config,
lib,
pkgs,
pkgs-patched,
...
}:
let
cfg = config.modules.desktop.vinput;
vinput = import ./package.nix {
inherit inputs pkgs pkgs-patched;
backend = cfg.backend;
};
in
{
options.modules.desktop.vinput = {
enable = lib.mkEnableOption "Fcitx5 Vinput voice input";
backend = lib.mkOption {
type = lib.types.enum [
"cpu"
"openvino"
];
default = "cpu";
description = "ASR backend used by Vinput: CPU or OpenVINO.";
};
};
config = lib.mkIf cfg.enable {
i18n.inputMethod.fcitx5.addons = [ vinput ];
xdg.configFile."fcitx5/conf/vinput.conf" = {
text = ''
[Trigger]
MenuKey=Super_R
'';
force = true;
};
systemd.user.services.vinput-daemon = {
Unit = {
Description = "Fcitx5 Vinput voice input daemon";
After = [ "pipewire.service" ];
PartOf = [ "graphical-session.target" ];
};
Service = {
Type = "dbus";
BusName = "org.fcitx.Vinput";
ExecStart = "${vinput}/bin/vinput-daemon";
Environment = lib.optionals (cfg.backend == "openvino") [
# OpenVINO dlopens Level Zero and discovers the NPU UMD through these paths.
"LD_LIBRARY_PATH=${pkgs.level-zero}/lib:/run/opengl-driver/lib"
];
Restart = "on-failure";
RestartSec = 3;
};
Install.WantedBy = [ "graphical-session.target" ];
};
};
}
home/linux/gui/base/vinput/package.nix
+25
View File
@@ -0,0 +1,25 @@
{
inputs,
pkgs,
pkgs-patched,
backend,
}:
let
system = pkgs.stdenv.hostPlatform.system;
sherpaOnnx = inputs.fcitx5-vinput.inputs.sherpa-onnx.packages.${system}.sherpa-onnx;
vinput = inputs.fcitx5-vinput.packages.${system}.fcitx5-vinput;
in
if backend == "openvino" then
let
sherpaOnnxOpenVino = pkgs-patched.sherpa-onnx;
in
vinput.overrideAttrs (old: {
buildInputs = map (
dependency: if dependency == sherpaOnnx then sherpaOnnxOpenVino else dependency
) old.buildInputs;
passthru = (old.passthru or { }) // {
sherpaOnnxVersion = sherpaOnnxOpenVino.version;
};
})
else
vinput
hosts/idols-ai/default.nix
+3
View File
@@ -62,6 +62,9 @@ in
modules.btrbk.enable = true;
# Permit the desktop user to access Intel accelerator devices for local ASR.
users.users.${myvars.username}.extraGroups = [ "render" ];
# systemd-journal - cap disk usage
# https://www.freedesktop.org/software/systemd/man/latest/journald.conf.html
services.journald.settings.Journal = {
outputs/x86_64-linux/tests/idols-ai-voice-input/expected.nix
+9
View File
@@ -0,0 +1,9 @@
{
enablesLocalAsr = true;
enablesLitePackage = false;
sherpaOnnxVersion = "1.13.8";
hasVinputDaemon = true;
hasOpenvinoServer = false;
exposesNpuRuntimeLibraries = true;
enablesIntelNpu = true;
}
outputs/x86_64-linux/tests/idols-ai-voice-input/expr.nix
+26
View File
@@ -0,0 +1,26 @@
{
outputs,
...
}:
let
cfg = outputs.nixosConfigurations.ai-niri.config;
hm = cfg.home-manager.users.ryan;
packageNames = map (package: package.pname or "") hm.i18n.inputMethod.fcitx5.addons;
vinputPackage = builtins.head (
builtins.filter (
package: (package.pname or "") == "fcitx5-vinput"
) hm.i18n.inputMethod.fcitx5.addons
);
daemonEnvironment = hm.systemd.user.services.vinput-daemon.Service.Environment;
in
{
enablesLocalAsr = builtins.elem "fcitx5-vinput" packageNames;
enablesLitePackage = builtins.elem "fcitx5-vinput-lite" packageNames;
sherpaOnnxVersion = vinputPackage.passthru.sherpaOnnxVersion or "";
hasVinputDaemon = hm.systemd.user.services ? vinput-daemon;
hasOpenvinoServer = hm.systemd.user.services ? openvino-npu-stt;
exposesNpuRuntimeLibraries = builtins.any (
variable: builtins.match "LD_LIBRARY_PATH=.*:/run/opengl-driver/lib" variable != null
) daemonEnvironment;
enablesIntelNpu = cfg.hardware.cpu.intel.npu.enable;
}