diff --git a/.agents/skills/nix-config-debug/SKILL.md b/.agents/skills/nix-config-debug/SKILL.md index 3cf93e73..afc804e0 100644 --- a/.agents/skills/nix-config-debug/SKILL.md +++ b/.agents/skills/nix-config-debug/SKILL.md @@ -26,7 +26,7 @@ bump, and the commands that destroy rollback points are in the `nix-config-updat | Layer | Symptom | Look with | | ------------- | ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | -| eval | `just test` prints `false`, an eval error | `just eval-host ` (already passes `--show-trace`) | +| eval | `just test` fails, an eval error | `just eval-host ` (already passes `--show-trace`) | | build | build error, hash mismatch, "marked as broken" | `just build-host `, then `nix log ` | | activation | the deploy fails after building | the deploy output; `journalctl -u home-manager-$USER -b` for Home Manager | | runtime | a unit is failed or restarting | `just list-failed`, `systemctl status `, `journalctl -u -b` | @@ -39,7 +39,8 @@ bump, and the commands that destroy rollback points are in the `nix-config-updat ## 2. Read the evaluated value -When `just test` prints `false`, re-run the suite directly to get the trace: +When `just test` fails, re-run the suite directly to get the trace (`nix eval` exits 0 even when the +suite returns `false`, so read its output): ```bash nix eval .#evalTests --show-trace diff --git a/.agents/skills/nix-config-new-host/SKILL.md b/.agents/skills/nix-config-new-host/SKILL.md index 916db663..f9651b88 100644 --- a/.agents/skills/nix-config-new-host/SKILL.md +++ b/.agents/skills/nix-config-new-host/SKILL.md @@ -79,7 +79,7 @@ check every configuration and which list hosts by name: - `security-*`, `kernel`, `nix-system-features`: apply to every configuration automatically. - `home-manager`, `btrbk`, and the other host-listing tests: add the host if it should be covered. -`just test` must print `true`; an exit code of `0` with `false` is a failure. +`just test` fails (non-zero) unless the suite returns `true`. ## 4. Install and deploy diff --git a/.agents/skills/nix-config-update/SKILL.md b/.agents/skills/nix-config-update/SKILL.md index 1dedf712..45c35662 100644 --- a/.agents/skills/nix-config-update/SKILL.md +++ b/.agents/skills/nix-config-update/SKILL.md @@ -75,7 +75,7 @@ Treat an update as a supply-chain event. Run these before any host is touched: -- `just test` — must print `true`. An exit code of `0` with `false` is a failed suite. +- `just test` — fails (non-zero) unless the suite returns `true`. - `just eval-host ` — fast, evaluation only. - `just build-host ` — builds the full system closure and catches broken packages or build failures that eval misses. diff --git a/AGENTS.md b/AGENTS.md index b851983f..4d12340b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -38,8 +38,9 @@ deployments. Keep repository guidance here; reusable global rules live in `agent `nixfmt` with width 100. - For supported non-Nix files, use `prettier --write ` and `prettier --check `; configuration lives in `.prettierrc.yaml`. Spelling checks use `typos` and `.typos.toml`. -- Run `just test` for configuration changes. It evaluates `.#evalTests` across Linux and Darwin; the - output must be `true`. Exit code zero with `false` is a failed suite. +- Run `just test` for configuration changes; it evaluates `.#evalTests` across Linux and Darwin and + exits non-zero unless the result is `true`. The bare `nix eval .#evalTests` exits 0 even on + `false`, so run `just test` or compare its `--json` output yourself. - Eval tests are `expr.nix` / `expected.nix` pairs under `outputs//tests/`. Update focused cases when changing behavior covered by those tests. - Use `nix flake check` for broader flake checks. A host build can validate changes beyond eval: diff --git a/README.md b/README.md index cf5cd532..89cbde66 100644 --- a/README.md +++ b/README.md @@ -159,7 +159,7 @@ of the same flake, e.g. `just col `, `just k3s-test`, `just lab`. `nix develop` provides the formatters and linters used by the repository. The most useful commands: ```bash -just test # eval tests across Linux & Darwin; the output must be `true` +just test # eval tests across Linux & Darwin (fails unless the result is `true`) just fmt # format all Nix files with nixfmt just --list # all recipes diff --git a/SECURITY.md b/SECURITY.md index fa1fff5e..f43956d5 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -109,8 +109,8 @@ See [application hardening](hardening/README.md) for wrapper details. 1. Check the advisory and affected package/kernel; prioritize active exploitation and reachable attack paths. Review changed inputs and cache trust instead of treating updates as inherently safe. -2. Run `just test` (must return `true`), build affected systems/runners and review closure - differences. Evaluate tests cannot prove runtime enforcement or workload compatibility. +2. Run `just test` (must pass), build affected systems/runners and review closure differences. + Evaluate tests cannot prove runtime enforcement or workload compatibility. 3. Record the old system generation/runner and ensure backups and recovery access exist. Use `boot` mode for bridge/network-stack changes or broad VM-host updates. 4. Deploy MicroVM closures to the **physical host**, not the guest's read-only store. Install the diff --git a/hosts/12kingdoms-youko/homelab-services/immich/README.md b/hosts/12kingdoms-youko/homelab-services/immich/README.md index c9dce3c6..f4fdfe66 100644 --- a/hosts/12kingdoms-youko/homelab-services/immich/README.md +++ b/hosts/12kingdoms-youko/homelab-services/immich/README.md @@ -106,7 +106,7 @@ document is the agreed design and the rationale. ## Validation -- `just fmt`; `just test` -> `true`; `nix build` the youko toplevel. +- `just fmt`; `just test` passes; `nix build` the youko toplevel. - Postgres: `\dx` lists `vectorchord`/`pgvector`; `SHOW shared_preload_libraries` includes `vchord.so`; the `immich` DB exists. - Units: `immich-server`, `immich-machine-learning`, `redis-shared`, `postgresql` are active. diff --git a/outputs/README.md b/outputs/README.md index 137ba65a..bbb828fa 100644 --- a/outputs/README.md +++ b/outputs/README.md @@ -31,9 +31,13 @@ set for each NixOS and nix-darwin host. How to run all the eval tests: ```bash -nix eval .#evalTests --show-trace --print-build-logs --verbose +just test ``` +`just test` exits non-zero unless the suite returns `true`. The underlying +`nix eval .#evalTests --show-trace --print-build-logs --verbose` prints the result but exits 0 even +on `false`, so read its output when you need the failing trace. + Each test is a directory `outputs//tests//` holding `expr.nix` and `expected.nix`; the suite passes when every `expr` equals its `expected`.