feat(e2e): windows-nextest runs the Windows unit tests in the guest

The Windows crates' unit tests could only run in the (future) hosted Windows
CI. They run in the existing Windows e2e guest: nix run .#windows-nextest
boots the golden image like a scenario (restoring a saved state when
KOMOREBI_E2E_STATE is set), uploads the test executables that the check
windows-nextest-build cross-compiled, runs every test in a process of its own
(nextest.exe is not needed: komorebi's tests share global state, so one
process per test is what nextest does too) and prints a line per executable
and the failures, with OUT/cargo-tests.json for every result.

The driver e2e/platforms/qemu_windows/cargo_tests.py already existed and was
wired to nothing; it now takes the check output (tests.tsv) as well as cargo
messages, creates the saved state itself, prints the panic message of a
failure, and uploads the data files the tests read. Those are found by the
CARGO_MANIFEST_DIR baked into the executables, a path of the Nix build, so
the check also keeps schemas/ and the tests' fixtures and the driver puts them
at that path under C:. Without them 20 tests failed for the missing files.

docs/dev/build-and-test.md and docs/dev/flake-checks.md describe the check,
the app and the neutral crates in linux-nextest.

Verified: nix run .#windows-nextest on the golden image g8 (scale 1.5, state
~/e2e-w0/state-nextest): 964 tests in 37 executables, 964 passed, 0 failed,
202 s with the state restored (first run 163 s more to save the state).
One run only; not soaked. No e2e scenario, the golden image or the existing
e2e-windows behaviour changed.
This commit is contained in:
LGUG2Z committed 2026-11-05 15:03:27 -08:00
1 parent d872c9ec43
commit 3e21ebd3a4
4 files changed
+115 -21

No files matched your search

+2 -1
View File
@@ -58,7 +58,7 @@ shared by all of them):
```sh
nix build -L --no-link .#checks.x86_64-linux.windows-clippy
# (there is no windows-nextest: the Windows crates' tests run on Windows, `cargo test` of .github/workflows/windows.yaml)
nix build -L --no-link .#checks.x86_64-linux.windows-nextest-build # cross-compiles the Windows crates' test executables (48), runs none
nix build -L --no-link .#checks.x86_64-linux.windows-fmt .#checks.x86_64-linux.windows-toml-fmt .#checks.x86_64-linux.windows-deny
nix build -L --no-link .#checks.x86_64-linux.windows-e2e-tests # the Python of the e2e (pytest), no guest needed
```
@@ -186,6 +186,7 @@ Full text: `e2e/platforms/qemu_windows/README.md` (building the golden image, ev
```sh
E2E_GOLDEN=golden nix run .#e2e-windows -- OUT SCENARIO... # e.g. OUT tiling focus commands
E2E_GOLDEN=golden KOMOREBI_E2E_SCALE=1.5 nix run .#e2e-windows -- OUT bar # the komorebi-bar scenario
E2E_GOLDEN=golden KOMOREBI_E2E_STATE=state-nextest nix run .#windows-nextest -- OUT # the Windows crates' unit tests in the guest
```
The app uses this checkout's binaries, built with the constant stamp `e2e` (so a docs-only commit reuses them);
+29 -11
View File
@@ -75,7 +75,7 @@ members, and every set is also one derivation (`pkgs.linkFarm`) that builds its
| `fmt` | `treefmt`, `pre-commit`, `windows-fmt`, `windows-toml-fmt`, `linux-fmt`, `linux-toml-fmt` (macOS systems: `macos-fmt`, `macos-toml-fmt`) | cheap |
| `deny` | `windows-deny`, `linux-deny` (`macos-deny`) | cheap |
| `windows-lint` | `windows-clippy` | cheap |
| `windows-test` | `windows-e2e-tests` (the Rust tests of the Windows crates cannot run on a Linux builder; the old `windows-nextest` ran none, see below) | cheap |
| `windows-test` | `windows-e2e-tests`, `windows-nextest-build` (the Windows test executables, cross-built and not run; they run in the guest with `nix run .#windows-nextest`, below) | cheap |
| `linux-lint` | `linux-clippy`, `linux-doc` | cheap |
| `linux-test` | `linux-nextest`, `linux-bridge-js`, `linux-keybindings-fixtures`, `vm-lib`, `linux-wsl2-probe`, `linux-wsl2-probe-dry-run`, the two effect install checks | cheap |
| `linux-demo` | `linux-run-kde`, `linux-run-kde-doctor`, `linux-run-kde-dry-run` | 1.5 GB |
@@ -85,7 +85,7 @@ members, and every set is also one derivation (`pkgs.linkFarm`) that builds its
| `vm-fast` | the suites whose `tier` is `fast` (16), computed from `suite.tier` | 9 GB |
| `vm-slow` | the suites of tier `slow` (18) | 10 GB |
The Windows end-to-end run is an app (`nix run .#e2e-windows`), not a check: it needs a golden image. The
The Windows end-to-end run is an app (`nix run .#e2e-windows`), and so is `windows-nextest`, not a check: it needs a golden image. The
sets that exist on a system are the ones with members on it (a Mac has no `windows-*` build host except
Apple silicon's cross build, which the member lists already say).
@@ -174,18 +174,36 @@ where its two checks took 10.3 + 9.0 GB alone before. Every set of one platform
set was killed.
Nothing is lost: the check names of every system (`checks` plus what `heavyChecks` had) before, and the union
of `checkMembers` after, are the same sets except `windows-nextest` (below): 61 names on x86_64-linux and
aarch64-linux, 27 on aarch64-darwin, 9 on x86_64-darwin. The `drvPath` of every one of them is the same before and
of `checkMembers` after, are the same sets except `windows-nextest` (below), which is now an app, and the new check
`windows-nextest-build`: that was 61 names on x86_64-linux and aarch64-linux, 27 on aarch64-darwin, 9 on
x86_64-darwin, and each system that builds the Windows crates has one more now (not re-counted). The `drvPath` of every one of them is the same before and
after, except `treefmt` and `pre-commit`, which hash the whole source tree and so change with any edit of it.
## `windows-nextest` and `macos-nextest` ran no test
`cargoNextest` of crane runs the tests in its check phase, and the Windows and macOS `individualCrateArgs`
have `doCheck = false` (Linux passes `doCheck = true`), so the derivation ran `cargo nextest --version` and
nothing else. A Windows program's tests cannot run on a Linux builder, so `windows-nextest` is gone (a check
that runs nothing under the name of one that runs tests is worse than no check) and `windows-clippy
--all-targets` is what compiles them; the Windows CI runs `cargo test`. `macos-nextest` is the same shape and is
kept, unchanged and still named for what it does not do, only until a Mac run decides whether `doCheck = true`
is right there (the tests of `komorebi-macos` run natively on an Apple machine). Open: the tests of the
platform-neutral crates `komorebi-layouts` and `komorebi-scrolling` run in the Windows CI and nowhere in nix
(`linux-nextest` names `-p komorebi -p komorebi-linux -p komorebic -p komorebi-client -p komorebi-hotkeys`).
nothing else. A Windows program's tests cannot run on a Linux builder, so the check `windows-nextest` was
removed (a check that runs nothing under the name of one that runs tests is worse than no check). The tests
now run in the Windows e2e guest, which is the answer to "can the Windows nextest run in a VM from NixOS":
- `windows-nextest-build` (a check, set `windows-test`) cross-compiles every test executable of the Windows
workspace (`cargo test --workspace --release --no-run`, xwin, 48 executables) and keeps them, so that they
are proved to compile and link; about 90 s with warm dependencies.
- `windows-nextest` (an app, the name of the old check) boots the golden image like a scenario, uploads the
executables and the data files they read at the path the build baked in (the schemas, the fixtures), runs
every test in a process of its own (as nextest does) and prints a line per executable and the failures;
`OUT/cargo-tests.json` has every result. The guest needs KVM and the golden image, so it is not a check:
```sh
E2E_GOLDEN=golden KOMOREBI_E2E_SCALE=1.5 KOMOREBI_E2E_STATE=state-nextest nix run .#windows-nextest -- OUT
```
964 tests ran and passed (the first run also saves the guest state, 163 s; after that about 3.5 minutes,
of which the tests are 171 s). It is a release build: debug assertions are off, as in `linux-nextest`.
The neutral crates (`komorebi-layouts`, `komorebi-scrolling`, `komorebi-themes`, the shared `crates/*`) now
also run in `linux-nextest` (1532 tests with the Linux packages), which needs the schemas of every platform
and `docs/macos/cli` in the Linux source (`nix/workspace-src.nix`). `macos-nextest` is the same shape as the
old Windows one and is kept, unchanged and still named for what it does not do, only until a Mac run decides
whether `doCheck = true` is right there (the tests of `komorebi-macos` run natively on an Apple machine).
+48 -8
View File
@@ -1,10 +1,13 @@
"""Runs the cargo tests of the workspace in the Windows guest.
usage: python -m platforms.qemu_windows.cargo_tests GOLDEN OUT STATE MESSAGES
usage: python -m platforms.qemu_windows.cargo_tests GOLDEN OUT STATE TESTS
MESSAGES is the `--message-format json` output of
`cargo test --workspace --no-run` for the Windows target, which names every
test executable. Each is uploaded to the guest, asked for its tests, and every
TESTS is a directory with a `tests.tsv` (name, tab, file under `tests/`: the
output of the flake check `windows-nextest-build`), or the
`--message-format json` output of `cargo test --workspace --no-run` for the
Windows target, which names every test executable. STATE is a directory to
restore the guest from (made if missing, as `KOMOREBI_E2E_STATE` does for the
scenarios), or `-` to boot. Each is uploaded to the guest, asked for its tests, and every
test is run in a process of its own (as nextest does: komorebi's tests share
global state), with its stdout kept. Prints a line per executable and the
tests that failed; OUT/cargo-tests.json has every result. Exits 1 if any test
@@ -17,11 +20,18 @@ import time
from pathlib import Path
from .guest import Guest
from .state import ensure
GUEST_DIR = "C:\\e2e\\tests"
def executables(messages: Path) -> list[tuple[str, Path]]:
if messages.is_dir():
listing = (messages / "tests.tsv").read_text().splitlines()
return sorted(
(name, messages / "tests" / file)
for name, file in (line.split("\t") for line in listing)
)
found = {}
for line in messages.read_text().splitlines():
try:
@@ -43,12 +53,43 @@ def tests_of(guest: Guest, exe: str) -> list[str]:
]
def upload_data(guest: Guest, tests: Path) -> None:
"""Puts the data files the tests read (the schemas, fixtures) in the guest
at the path the build baked into the executables (CARGO_MANIFEST_DIR, the
build directory of the check, as a path on the guest's C: drive)."""
if not (tests.is_dir() and (tests / "srcroot").exists()):
return
root = "C:" + (tests / "srcroot").read_text().strip().replace("/", "\\")
data = tests / "data"
files = sorted(p for p in data.rglob("*") if p.is_file())
made: set[str] = set()
for path in files:
remote = root + "\\" + str(path.relative_to(data)).replace("/", "\\")
directory = remote.rsplit("\\", 1)[0]
if directory not in made:
guest.agent.exec("cmd", "/c", f"mkdir {directory} 2>nul")
made.add(directory)
guest.agent.put_file(path, remote)
print(f"data: {len(files)} files under {root}", flush=True)
def reason(output: str) -> str:
"""The lines that say why a test failed: the panic message, else the end."""
lines = [l for l in output.splitlines() if l.strip()]
for i, line in enumerate(lines):
if "panicked at" in line:
return " | ".join(lines[i : i + 2])
return " | ".join(lines[-3:])
def main(argv: list[str]) -> int:
golden, out, state, messages = (Path(a) for a in argv[:4])
out.mkdir(parents=True, exist_ok=True)
saved = ensure(golden, state) if str(state) != "-" else None
results: dict[str, dict] = {}
with Guest(golden, out / "guest", state=state if str(state) != "-" else None) as guest:
with Guest(golden, out / "guest", state=saved) as guest:
guest.agent.exec("cmd", "/c", f"mkdir {GUEST_DIR} 2>nul")
upload_data(guest, messages)
for name, path in executables(messages):
exe = f"{GUEST_DIR}\\{name}.exe"
began = time.monotonic()
@@ -62,7 +103,7 @@ def main(argv: list[str]) -> int:
else:
failed.append({
"test": test, "status": run["status"],
"output": (run["stdout"] + run["stderr"])[-1500:],
"output": (run["stdout"] + run["stderr"])[-4000:],
})
results[name] = {
"tests": len(names), "passed": passed, "failed": failed,
@@ -72,8 +113,7 @@ def main(argv: list[str]) -> int:
f"{'' if not failed else f', {len(failed)} FAILED'}"
f" ({results[name]['seconds']} s)", flush=True)
for f in failed:
first = [l for l in f["output"].splitlines() if l.strip()][-3:]
print(f" FAIL {f['test']} (status {f['status']}): {' | '.join(first)}", flush=True)
print(f" FAIL {f['test']} (status {f['status']}): {reason(f['output'])}", flush=True)
(out / "cargo-tests.json").write_text(json.dumps(results, indent=2))
total = sum(r["tests"] for r in results.values())
failed = sum(len(r["failed"]) for r in results.values())
+36 -1
View File
@@ -62,6 +62,25 @@ in
"$E2E_GOLDEN" "$bins" "$out" "$@"
'';
};
# the Windows unit tests in the guest: every test of every executable in a process of its own
windowsNextest = pkgs.writeShellApplication {
name = "windows-nextest";
runtimeInputs = [
pkgs.qemu
pkgs.swtpm
(pkgs.python3.withPackages (p: [ p.pillow ]))
];
text = ''
: "''${E2E_GOLDEN:?set E2E_GOLDEN to the golden image directory}"
out=''${1:?usage: windows-nextest OUT}
export OVMF_FD=${ovmfWithTpm}
export PYTHONPATH=${lib'.repoFile "e2e/lib"}:${lib'.repoFile "e2e"}
exec python -m platforms.qemu_windows.cargo_tests \
"$E2E_GOLDEN" "$out" "''${KOMOREBI_E2E_STATE:--}" ${windowsNextestBuild}
'';
};
# the test executables of the Windows workspace, with a list of them (name, tab, file) for the
# guest runner; the check `windows-nextest-build` and the app `windows-nextest` use it
# the members that are stubs here (the Linux and macOS packages) have no test and do not build
excluded = pkgs.lib.concatMapStringsSep " " (p: "--exclude ${p}") build.stubbedPackages;
windowsNextestBuild = build.craneLib.mkCargoDerivation (
@@ -69,7 +88,10 @@ in
// {
pnameSuffix = "-nextest-build";
doInstallCargoArtifacts = false;
nativeBuildInputs = build.individualCrateArgs.nativeBuildInputs ++ [ pkgs.jq ];
nativeBuildInputs = build.individualCrateArgs.nativeBuildInputs ++ [
pkgs.jq
pkgs.cpio
];
buildPhaseCargoCommand = ''
cargo test --locked --workspace ${excluded} --release --no-run --message-format json > messages.json
'';
@@ -81,6 +103,13 @@ in
cp "$exe" "$out/tests/$name.exe"
printf '%s\t%s.exe\n' "$name" "$name" >> $out/tests.tsv
done
# the data files the tests read by the absolute path the build baked into them
# (CARGO_MANIFEST_DIR): the guest gets them at the same path under C:
pwd > $out/srcroot
mkdir -p $out/data
find . -path ./target -prune -o -type f \
\( -path './schemas/*' -o -path '*/tests/*' -o -path '*/fixtures/*' \) \
! -name '*.rs' -print | cpio -pdm $out/data 2>/dev/null
'';
}
);
@@ -223,6 +252,12 @@ in
type = "app";
program = pkgs.lib.getExe e2eWindows;
};
# the unit tests of the Windows crates in the guest (E2E_GOLDEN as above):
# E2E_GOLDEN=<golden> KOMOREBI_E2E_SCALE=1.5 nix run .#windows-nextest -- OUT
windows-nextest = {
type = "app";
program = pkgs.lib.getExe windowsNextest;
};
};
};
}