diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 5059bfde..7a794518 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -5,3 +5,8 @@ By opening this PR, you confirm that you have read, understood and agree to this project's `CONTRIBUTING.md`. --> + + diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 2b129d43..9209dddd 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -42,6 +42,13 @@ warnings denied. `just fmt` formats the Rust sources, and `nix fmt` every file type the repository has. `nix flake check` is what CI runs; it builds the same checks, plus `cargo deny`, as derivations. +## VM tests + +A change that needs a real KWin to be checked comes with a VM test, next to +the Rust test that covers the same behavior against the fake desktop. +[`docs/dev/vm-tests.md`](docs/dev/vm-tests.md) says how they are laid out, how +to run one (`just vm NAME`) and how to add one. + ## Continuous integration [`ci.yaml`](.github/workflows/ci.yaml) runs on every pull request and on every diff --git a/README.md b/README.md index 883344a9..01d29827 100644 --- a/README.md +++ b/README.md @@ -33,10 +33,15 @@ and the tests under `cargo nextest`, on Linux and on an Apple silicon Mac. A Mac runs the tests against the fake desktop, since KWin only exists on Linux. `nix fmt` formats every file the checks look at. +The VM tests boot NixOS with Plasma and run a real KWin. `just vm NAME` runs +one; [`docs/dev/vm-tests.md`](docs/dev/vm-tests.md) has the details, including +how to run them on a Mac. + ## Documentation `cargo doc --workspace --no-deps --open` builds the API documentation. -[`CONTRIBUTING.md`](CONTRIBUTING.md) says how to work on the code. +[`CONTRIBUTING.md`](CONTRIBUTING.md) says how to work on the code, and +[`docs/dev/vm-tests.md`](docs/dev/vm-tests.md) how the VM tests work. ## License diff --git a/docs/dev/vm-tests.md b/docs/dev/vm-tests.md new file mode 100644 index 00000000..02bd441d --- /dev/null +++ b/docs/dev/vm-tests.md @@ -0,0 +1,116 @@ +# VM tests + +The unit tests run against a fake desktop. They cannot say whether KWin does +what the daemon asks of it, so a second kind of test boots a NixOS virtual +machine with Plasma in it, starts a real KWin, and drives it. This page says how +they are laid out, how to run them, and how to add one. + +## What runs where + +| Where | How | Accelerator | +|---|---|---| +| CI, `x86_64-linux` | `nix build -L .#checks.x86_64-linux.vm-NAME` | KVM | +| A Linux machine with KVM | `just vm NAME` | KVM | +| A Mac | `just vm NAME`, which runs `scripts/vm-mac` | HVF | +| A Linux machine with no KVM | `nix build -L .#legacyPackages.SYSTEM.vmTcg.NAME` | TCG | + +TCG is software emulation. It is slow enough that only short suites are worth +running that way, and the library's `scale` argument (below) is 8 for it. + +Each suite is a directory under +[`nix/tests/vm/suites`](../../nix/tests/vm/suites). The generator in +[`nix/tests/vm/default.nix`](../../nix/tests/vm/default.nix) lists that +directory, so a new suite is a new directory and nothing else: it appears as +`checks..vm-NAME` on Linux and as `legacyPackages..vm.NAME` +everywhere. `legacyPackages..vmTiers` lists the names of each tier as +JSON, which is what CI reads. + +A suite has two files: + +- `default.nix` sets `tier` (`"fast"` or `"slow"`), `kind` (the node it runs on) + and `timeoutMinutes`. +- `test.py` is the test, a Python script for the NixOS test driver. It is + prefixed with `SCALE = `, how many times slower than KVM the machine + is. + +## Nodes + +`kind` picks a file under +[`nix/tests/vm/nodes`](../../nix/tests/vm/nodes). + +`headless` is a machine with Plasma installed and no login. Each KWin a test +starts is a pair of user units of the test user, `session-bus@NAME` (a private +D-Bus session bus) and `kwin-virtual@NAME` (`kwin_wayland --virtual` with its +own Wayland socket), so a test can run several sessions side by side and none +touches another. A virtual KWin has no real output, so it cannot draw: it can +place windows and answer questions about them, but it cannot take a screenshot. + +KWin wants a DRM render node, which needs a GPU. The test driver gives every aarch64 guest one +`virtio-gpu-pci`; on x86_64 the node adds it, unless `komorebi.vm.renderNode` is false. + +## The library + +Suites import from `komorebi_vm`, in +[`nix/tests/vm/lib`](../../nix/tests/vm/lib). It does not depend on the test +driver. It depends on the small `Machine` protocol, and its unit tests pass a +fake machine, so the `vm-lib` check tests it without booting anything. The +check also runs `ruff` and `ty` over it. The driver type checks every +`test.py` with `ty` and `ruff` too, so a typo in a suite fails the build, not +the run. + +- `Checks` counts checks and fails once, at the end, so that one failure does + not hide the ones after it. `finish()` prints `passed: N failed: M`. +- `Kde` is one KWin session. `Kde.headless(machine, name, size=, outputs=, + scale=)` starts the units and waits for KWin to own its name on the private + bus. `wid`, `rect`, `title` and `active` read windows through `kdotool`, + `outputs` counts the enabled outputs through `kscreen-doctor`, `spawn` starts + a client, and `wait_for(predicate, timeout)` polls, with the timeout + multiplied by `scale`. + +Poll for what the test needs with `wait_for`. A fixed sleep is too long on +KVM and too short on TCG. + +## Running a test on a Mac + +QEMU has to run on the Mac to use HVF, and the guest is Linux, so the guest's +closure is built on a Linux builder and the driver runs on the Mac. The nix +sandbox on macOS cannot give QEMU the hypervisor, so `scripts/vm-mac` runs the +driver directly instead of building the check. + +```sh +export KOMOREBI_BUILDER=ssh-ng://builder@localhost +export NIX_SSHOPTS="-p 31022 -i $HOME/.ssh/builder_ed25519 -o IdentitiesOnly=yes" +just vm kwin +``` + +`KOMOREBI_BUILDER` is the store URL of an `aarch64-linux` builder you can +reach. `nix run nixpkgs#darwin.linux-builder` starts one, with one core and 3 GB +of memory by default; a Plasma guest needs more than that (6 cores and 10 GB +worked, at time of writing, 2026-10-27), and the builder's 20 GB disk holds one +Plasma system closure of about 10 GB, so run `nix-collect-garbage` on it +between builds of different guests. + +The script prints where the artifacts (screenshots, logs) are, under +`.vm/out/NAME`. + +## Writing a test + +1. Create `nix/tests/vm/suites/NAME/default.nix` and `test.py`. Start from + [`suites/kwin`](../../nix/tests/vm/suites/kwin). +2. Pick the tier. `fast` suites are the pull request gate and should take + minutes. `slow` suites run on `master` and every night. +3. Run it: `just vm NAME`. +4. If the test covers behavior the daemon has, write the Rust test that covers + the same behavior against the fake desktop in the same commit, and say in + the VM test's first comment what only a real KWin can tell. + +Add `git add -N` for new files first: a flake only sees files git knows about. + +## Looking inside a failed run + +`nix build -L` prints the guest's console. Run the driver by hand, as +`scripts/vm-mac` does, to keep the screenshots. For an interactive session +build `driverInteractive` instead of `driver`, run it, and call `start_all()` +and `machine.shell_interact()` at its prompt. In the guest, KWin's own log is +`journalctl --user -u kwin-virtual@NAME` as the test user, and a KWin script's +`print()` goes to the same journal. diff --git a/flake.nix b/flake.nix index 7a4f69af..09f742a8 100644 --- a/flake.nix +++ b/flake.nix @@ -112,6 +112,12 @@ let pkgs = mkPkgs system; build = mkKomorebiPackages { inherit pkgs; }; + isLinux = pkgs.stdenv.hostPlatform.isLinux; + # a Mac runs the guests under HVF, which is slower than KVM + vm = import ./nix/tests/vm { + hostPkgs = pkgs; + scale = if isLinux then 1 else 1.5; + }; nightlyRustfmt = pkgs.rust-bin.nightly.latest.rustfmt; nightlyToolchain = pkgs.rust-bin.nightly.latest.default; rustToolchainWithNightlyRustfmt = pkgs.symlinkJoin { @@ -169,6 +175,30 @@ komorebi-workspace-nextest = build.craneLib.cargoNextest ( build.individualCrateArgs // { doCheck = true; } ); + + vm-lib = vm.lib; + } + # The VM tests need QEMU with KVM, so only Linux checks them. A Mac runs the same tests + # through legacyPackages..vm, see docs/dev/vm-tests.md. + // lib.optionalAttrs isLinux ( + lib.mapAttrs' (name: test: lib.nameValuePair "vm-${name}" test) vm.suites + ); + + legacyPackages = { + # every suite, as a test with a `driver`, on every system + vm = vm.suites; + # the suites of each tier, for the CI matrix + vmTiers = vm.tiers; + # the guests of every suite, to build on a Linux builder when the host is a Mac + vmGuests = vm.guests; + } + // lib.optionalAttrs isLinux { + # the same suites on a machine with no KVM, which is slow but works anywhere + vmTcg = + (import ./nix/tests/vm { + hostPkgs = pkgs; + scale = 8; + }).suites; }; packages = lib.optionalAttrs pkgs.stdenv.hostPlatform.isLinux { @@ -202,6 +232,11 @@ pkgs.jq pkgs.just pkgs.ripgrep + + # the VM test library + pkgs.ruff + pkgs.ty + (pkgs.python3.withPackages (ps: [ ps.pytest ])) ]; }; diff --git a/justfile b/justfile index 50230bb6..2120a566 100644 --- a/justfile +++ b/justfile @@ -17,5 +17,25 @@ lint: test -z "$(rg 'dbg!' --type rust .)" || (echo "dbg! macro not allowed"; false) test -z "$(rg 'println!' --type rust ./komorebi/src)" || (echo "println! macro not allowed"; false) +# one VM test, on this machine: `just vm kwin`. Linux boots the guest under KVM through the +# flake check; a Mac builds the guest on a builder and runs the driver (see docs/dev/vm-tests.md) +vm name: + #!/usr/bin/env bash + set -euo pipefail + if [ "$(uname)" = Darwin ]; then + scripts/vm-mac {{ name }} + else + nix build -L .#checks.{{ arch() }}-linux.vm-{{ name }} + fi + +# every VM test, in the order of the flake +vm-all: + #!/usr/bin/env bash + set -euo pipefail + system=$(nix eval --raw --impure --expr builtins.currentSystem) + for name in $(nix eval --json ".#legacyPackages.$system.vm" --apply builtins.attrNames | jq -r '.[]'); do + just vm "$name" + done + run target *args: cargo run --bin {{ target }} --locked -- {{ args }} diff --git a/nix/tests/vm/default.nix b/nix/tests/vm/default.nix new file mode 100644 index 00000000..96255143 --- /dev/null +++ b/nix/tests/vm/default.nix @@ -0,0 +1,100 @@ +# The NixOS VM tests. Every directory under `suites/` is one test, `checks..vm-` +# (Linux) and `legacyPackages..vm.` (everywhere), so a new suite needs no edit to +# the flake. See docs/dev/vm-tests.md. +# +# `hostPkgs` is the pkgs of the machine that runs QEMU; the guests are always Linux. `scale` says +# how much slower than KVM that machine is (1 for KVM, 1.5 for HVF on a Mac, 8 for TCG), and every +# wait in the suites is multiplied by it. +{ + hostPkgs, + scale ? 1, +}: +let + inherit (hostPkgs) lib; + + # the library the suites import, built for the python of the test driver + komorebiVm = + ps: + ps.buildPythonPackage { + pname = "komorebi-vm"; + version = "0.1.0"; + pyproject = true; + src = lib.fileset.toSource { + root = ./lib; + fileset = ./lib; + }; + build-system = [ ps.setuptools ]; + doCheck = false; + }; + + # unit tests of the library against a fake machine, and the linters over it + libCheck = + let + python = hostPkgs.python3.withPackages (ps: [ + ps.pytest + (komorebiVm ps) + ]); + in + hostPkgs.runCommand "vm-lib" + { + nativeBuildInputs = [ + python + hostPkgs.ruff + hostPkgs.ty + ]; + src = lib.fileset.toSource { + root = ./lib; + fileset = ./lib; + }; + } + '' + cp -r $src lib && chmod -R u+w lib && cd lib + ruff check . + ruff format --check . + ty check --python ${python} . + pytest -q -p no:cacheprovider + touch $out + ''; + + suiteNames = lib.attrNames ( + lib.filterAttrs (_: type: type == "directory") (builtins.readDir ./suites) + ); + + suiteDefs = lib.genAttrs suiteNames (name: import (./suites + "/${name}")); + + mkSuite = + name: + let + dir = ./suites + "/${name}"; + suite = suiteDefs.${name}; + in + hostPkgs.testers.runNixOSTest { + name = "komorebi-${name}"; + extraPythonPackages = ps: [ (komorebiVm ps) ]; + globalTimeout = builtins.ceil (suite.timeoutMinutes * 60 * scale); + passthru.tier = suite.tier; + + nodes.machine = { + imports = [ (./nodes + "/${suite.kind}.nix") ]; + }; + + testScript = '' + SCALE = ${toString scale} + '' + + builtins.readFile (dir + "/${suite.script or "test.py"}"); + }; + + suites = lib.genAttrs suiteNames mkSuite; +in +{ + inherit suites; + lib = libCheck; + + # the suites of each tier, for the CI matrix: { fast = [ "kwin" ... ]; slow = [ ... ]; } + tiers = lib.groupBy (name: suiteDefs.${name}.tier) suiteNames; + + # the guests of every suite, so that a Mac can build them on a Linux builder first + guests = lib.mapAttrs ( + _: test: lib.mapAttrs (_: node: node.system.build.toplevel) test.nodes + ) suites; +} diff --git a/nix/tests/vm/lib/komorebi_vm/__init__.py b/nix/tests/vm/lib/komorebi_vm/__init__.py new file mode 100644 index 00000000..4d2c4b4a --- /dev/null +++ b/nix/tests/vm/lib/komorebi_vm/__init__.py @@ -0,0 +1,18 @@ +"""Helpers for the NixOS VM tests of komorebi for KDE. + +A suite imports what it needs from here: + +* [`Checks`](komorebi_vm.checks.Checks) counts checks and fails at the end. +* [`Kde`](komorebi_vm.kde.Kde) is a KWin session inside a test machine, with + the tools to look at it (`kdotool`, `busctl`, `kscreen-doctor`). + +The library depends on the small [`Machine`](komorebi_vm.machine.Machine) +protocol, not on the test driver, so `nix/tests/vm/lib/tests` can run it +against a fake machine (the `vm-lib` check does). +""" + +from komorebi_vm.checks import Checks +from komorebi_vm.kde import Kde, Rect +from komorebi_vm.machine import Machine + +__all__ = ["Checks", "Kde", "Machine", "Rect"] diff --git a/nix/tests/vm/lib/komorebi_vm/checks.py b/nix/tests/vm/lib/komorebi_vm/checks.py new file mode 100644 index 00000000..a14616a8 --- /dev/null +++ b/nix/tests/vm/lib/komorebi_vm/checks.py @@ -0,0 +1,41 @@ +"""Counted checks, so that one failure does not hide the ones after it.""" + +from collections.abc import Callable + + +class Checks: + """Collects the outcome of every check and fails once, at the end. + + A suite is a long scenario: a failure in the middle should not stop the + rest from running, because the rest says what else is broken. `finish` + prints the totals and raises if any check failed. + """ + + def __init__(self) -> None: + self.passed = 0 + self.failed = 0 + + def ok(self, description: str) -> None: + """Records a check that passed.""" + self.passed += 1 + print(f" ok {description}") + + def bad(self, description: str) -> None: + """Records a check that failed.""" + self.failed += 1 + print(f" FAIL {description}") + + def check(self, description: str, condition: bool | Callable[[], bool]) -> bool: + """Records `condition` (or what calling it returns) and returns it.""" + holds = condition() if callable(condition) else condition + if holds: + self.ok(description) + else: + self.bad(description) + return holds + + def finish(self) -> None: + """Prints `passed: N failed: M` and raises if any check failed.""" + print(f"passed: {self.passed} failed: {self.failed}") + if self.failed: + raise AssertionError(f"{self.failed} checks failed") diff --git a/nix/tests/vm/lib/komorebi_vm/kde.py b/nix/tests/vm/lib/komorebi_vm/kde.py new file mode 100644 index 00000000..8b0faa4b --- /dev/null +++ b/nix/tests/vm/lib/komorebi_vm/kde.py @@ -0,0 +1,183 @@ +"""A KWin session inside a test machine, and the tools to look at it.""" + +import shlex +import time +from collections.abc import Callable +from dataclasses import dataclass + +from komorebi_vm.machine import Machine + + +@dataclass(frozen=True) +class Rect: + """A window's position and size, in pixels.""" + + x: int + y: int + w: int + h: int + + def overlaps(self, other: "Rect") -> bool: + """Returns true if and only if the two rectangles share at least one pixel.""" + return ( + self.x < other.x + other.w + and other.x < self.x + self.w + and self.y < other.y + other.h + and other.y < self.y + self.h + ) + + +class Kde: + """One KWin session of the test user, with its own session bus and Wayland socket. + + The node runs `kwin_wayland` and the bus as user units (see + `nix/tests/vm/nodes/headless.nix`), so several sessions can run side by + side under different names and none touches another. Every command is run + as the test user, with the environment of the named session. + + `scale` multiplies every wait: 1 under KVM, more where the machine is + slower (HVF on a Mac, TCG anywhere). + """ + + def __init__( + self, + machine: Machine, + name: str, + *, + user: str = "komorebi", + uid: int = 1000, + scale: float = 1.0, + ) -> None: + self.machine = machine + self.name = name + self.user = user + self.uid = uid + self.scale = scale + + @classmethod + def headless( + cls, + machine: Machine, + name: str, + *, + size: tuple[int, int] = (1920, 1080), + outputs: int = 1, + scale: float = 1.0, + user: str = "komorebi", + uid: int = 1000, + ) -> "Kde": + """Starts `kwin_wayland --virtual` with `outputs` outputs of `size`, and waits for it.""" + kde = cls(machine, name, user=user, uid=uid, scale=scale) + machine.succeed(f"loginctl show-user {user} >/dev/null") + width, height = size + args = f"--width {width} --height {height} --output-count {outputs}" + kde.as_user(f"echo KWIN_ARGS={shlex.quote(args)} > {kde.runtime}/kwin-virtual-{name}.env") + kde.systemctl("start", f"session-bus@{name}.service", f"kwin-virtual@{name}.service") + if not kde.wait_for(lambda: kde.has_name("org.kde.KWin"), timeout=60): + log = machine.execute(f"journalctl _UID={uid} --no-pager -n 40")[1] + raise AssertionError(f"KWin did not come up:\n{log}") + return kde + + @property + def runtime(self) -> str: + """The test user's runtime directory.""" + return f"/run/user/{self.uid}" + + @property + def env(self) -> str: + """The `VAR=value` words that put a command inside this session.""" + pairs = { + "XDG_RUNTIME_DIR": self.runtime, + "DBUS_SESSION_BUS_ADDRESS": f"unix:path={self.runtime}/session-{self.name}.bus", + "WAYLAND_DISPLAY": f"wayland-{self.name}", + "KDE_SESSION_VERSION": "6", + "XDG_CURRENT_DESKTOP": "KDE", + "XDG_SESSION_TYPE": "wayland", + "QT_QPA_PLATFORM": "wayland", + "GDK_BACKEND": "wayland", + } + return " ".join(f"{key}={shlex.quote(value)}" for key, value in pairs.items()) + + def as_user(self, command: str) -> str: + """Runs `command` as the test user, and fails the test unless it succeeds.""" + return self.machine.succeed(self._wrap(command, session=False)) + + def run(self, command: str) -> tuple[int, str]: + """Runs `command` inside the session and returns its status and output.""" + return self.machine.execute(self._wrap(command)) + + def sh(self, command: str) -> str: + """Runs `command` inside the session, and fails the test unless it succeeds.""" + return self.machine.succeed(self._wrap(command)) + + def spawn(self, command: str) -> None: + """Starts `command` in the background inside the session, as a client of KWin.""" + self.sh(f"setsid nohup {command} >/dev/null 2>&1 < /dev/null &") + + def systemctl(self, *args: str) -> str: + """Runs `systemctl --user` as the test user.""" + return self.as_user("systemctl --user " + " ".join(args)) + + def stop(self) -> None: + """Stops the session's KWin and its bus.""" + self.systemctl( + "stop", f"kwin-virtual@{self.name}.service", f"session-bus@{self.name}.service" + ) + + def wait_for( + self, + predicate: Callable[[], bool], + timeout: float = 10, + interval: float = 0.1, + ) -> bool: + """Polls `predicate` until it holds or `timeout` seconds (times `scale`) pass.""" + deadline = time.monotonic() + timeout * self.scale + while True: + if predicate(): + return True + if time.monotonic() >= deadline: + return False + time.sleep(interval) + + def has_name(self, name: str) -> bool: + """Returns true if and only if `name` is owned on the session bus.""" + status, _ = self.run(f"busctl --user --no-pager list | grep -q '^{name} '") + return status == 0 + + def wid(self, title: str) -> str: + """The id of the first window whose title is exactly `title`, or the empty string.""" + pattern = shlex.quote(f"^{title}$") + return self.run(f"kdotool search --name {pattern} | head -1")[1].strip() + + def rect(self, wid: str) -> Rect: + """The position and size KWin reports for the window.""" + out = self.sh(f"kdotool getwindowgeometry {shlex.quote(wid)}") + position = size = None + for line in out.splitlines(): + words = line.split() + if words[:1] == ["Position:"]: + position = words[1] + elif words[:1] == ["Geometry:"]: + size = words[1] + if position is None or size is None: + raise AssertionError(f"no geometry in {out!r}") + x, y = (int(v) for v in position.split(",")) + w, h = (int(v) for v in size.split("x")) + return Rect(x, y, w, h) + + def title(self, wid: str) -> str: + """The title of the window.""" + return self.sh(f"kdotool getwindowname {shlex.quote(wid)}").strip().splitlines()[-1] + + def active(self) -> str: + """The id of the window KWin has focused, or the empty string.""" + lines = self.run("kdotool getactivewindow")[1].strip().splitlines() + return lines[-1] if lines else "" + + def outputs(self) -> int: + """The number of outputs KWin has enabled.""" + return int(self.sh("kscreen-doctor -j | jq '[.outputs[] | select(.enabled)] | length'")) + + def _wrap(self, command: str, session: bool = True) -> str: + env = self.env if session else f"XDG_RUNTIME_DIR={shlex.quote(self.runtime)}" + return f"runuser -u {self.user} -- env {env} sh -c {shlex.quote(command)}" diff --git a/nix/tests/vm/lib/komorebi_vm/machine.py b/nix/tests/vm/lib/komorebi_vm/machine.py new file mode 100644 index 00000000..6162ecf8 --- /dev/null +++ b/nix/tests/vm/lib/komorebi_vm/machine.py @@ -0,0 +1,32 @@ +"""The part of the NixOS test driver's `Machine` that the library uses. + +The driver's own class is not importable outside a test run, so the library +depends on this protocol instead, and its unit tests pass a fake. The methods +take only what the library passes; the driver's accept more (a timeout, whether +to check the exit status), which does not stop its class from satisfying the +protocol. +""" + +from pathlib import Path +from typing import Protocol + + +class Machine(Protocol): + """A virtual machine the test can run commands on, as root.""" + + @property + def out_dir(self) -> Path: + """The directory that screenshots and other artifacts go to.""" + ... + + def execute(self, command: str, /) -> tuple[int, str]: + """Runs `command` and returns its exit status and its output.""" + ... + + def succeed(self, *commands: str) -> str: + """Runs each command and fails the test unless it exits with status 0.""" + ... + + def fail(self, *commands: str) -> str: + """Runs each command and fails the test if it exits with status 0.""" + ... diff --git a/nix/tests/vm/lib/komorebi_vm/py.typed b/nix/tests/vm/lib/komorebi_vm/py.typed new file mode 100644 index 00000000..e69de29b diff --git a/nix/tests/vm/lib/pyproject.toml b/nix/tests/vm/lib/pyproject.toml new file mode 100644 index 00000000..110f6bf6 --- /dev/null +++ b/nix/tests/vm/lib/pyproject.toml @@ -0,0 +1,24 @@ +[build-system] +requires = ["setuptools"] +build-backend = "setuptools.build_meta" + +[project] +name = "komorebi-vm" +version = "0.1.0" +description = "Helpers for the NixOS VM tests of komorebi for KDE" +requires-python = ">=3.12" + +[tool.setuptools] +packages = ["komorebi_vm"] + +[tool.setuptools.package-data] +komorebi_vm = ["py.typed"] + +[tool.ruff] +line-length = 100 + +[tool.ruff.lint] +select = ["E", "F", "I", "UP", "B", "SIM"] + +[tool.pytest.ini_options] +testpaths = ["tests"] diff --git a/nix/tests/vm/lib/tests/__init__.py b/nix/tests/vm/lib/tests/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/nix/tests/vm/lib/tests/conftest.py b/nix/tests/vm/lib/tests/conftest.py new file mode 100644 index 00000000..089a9f9b --- /dev/null +++ b/nix/tests/vm/lib/tests/conftest.py @@ -0,0 +1,46 @@ +"""A fake machine that records commands and answers from a script.""" + +from collections.abc import Callable +from pathlib import Path + +import pytest + + +class FakeMachine: + """Answers `execute` with the first matching reply, and records every command.""" + + out_dir = Path("/tmp") + + def __init__(self) -> None: + self.commands: list[str] = [] + self.replies: list[tuple[Callable[[str], bool], tuple[int, str]]] = [] + + def reply(self, contains: str, status: int = 0, output: str = "") -> None: + """Makes any command containing `contains` answer with `status` and `output`.""" + self.replies.append((lambda command: contains in command, (status, output))) + + def execute(self, command: str, /) -> tuple[int, str]: + self.commands.append(command) + for matches, answer in self.replies: + if matches(command): + return answer + return 0, "" + + def succeed(self, *commands: str) -> str: + output = "" + for command in commands: + status, output = self.execute(command) + assert status == 0, f"{command!r} exited with {status}: {output}" + return output + + def fail(self, *commands: str) -> str: + output = "" + for command in commands: + status, output = self.execute(command) + assert status != 0, f"{command!r} succeeded: {output}" + return output + + +@pytest.fixture +def machine() -> FakeMachine: + return FakeMachine() diff --git a/nix/tests/vm/lib/tests/test_checks.py b/nix/tests/vm/lib/tests/test_checks.py new file mode 100644 index 00000000..168ac0d2 --- /dev/null +++ b/nix/tests/vm/lib/tests/test_checks.py @@ -0,0 +1,33 @@ +import pytest + +from komorebi_vm import Checks + + +def test_counts_passes_and_failures(capsys: pytest.CaptureFixture[str]) -> None: + checks = Checks() + + assert checks.check("a holds", True) + assert not checks.check("b does not hold", False) + + assert (checks.passed, checks.failed) == (1, 1) + out = capsys.readouterr().out + assert " ok a holds" in out + assert " FAIL b does not hold" in out + + +def test_a_callable_condition_is_called() -> None: + checks = Checks() + + assert checks.check("lazy", lambda: 1 + 1 == 2) + + +def test_finish_raises_only_when_something_failed(capsys: pytest.CaptureFixture[str]) -> None: + checks = Checks() + checks.check("fine", True) + checks.finish() + assert "passed: 1 failed: 0" in capsys.readouterr().out + + checks.check("broken", False) + with pytest.raises(AssertionError, match="1 checks failed"): + checks.finish() + assert "passed: 1 failed: 1" in capsys.readouterr().out diff --git a/nix/tests/vm/lib/tests/test_kde.py b/nix/tests/vm/lib/tests/test_kde.py new file mode 100644 index 00000000..e96752e7 --- /dev/null +++ b/nix/tests/vm/lib/tests/test_kde.py @@ -0,0 +1,102 @@ +import shlex + +import pytest + +from komorebi_vm import Kde, Rect + +from .conftest import FakeMachine + + +def inner(command: str) -> str: + """The script a wrapped command hands to `sh -c`, with one level of quoting removed.""" + return shlex.split(command)[-1] + + +def test_commands_run_as_the_user_inside_the_named_session(machine: FakeMachine) -> None: + kde = Kde(machine, "main") + + kde.sh("kdotool getactivewindow") + + (command,) = machine.commands + assert command.startswith("runuser -u komorebi -- env ") + assert "WAYLAND_DISPLAY=wayland-main" in command + assert "DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/session-main.bus" in command + assert "XDG_RUNTIME_DIR=/run/user/1000" in command + + +def test_two_sessions_do_not_share_a_socket(machine: FakeMachine) -> None: + assert "wayland-a" in Kde(machine, "a").env + assert "wayland-b" in Kde(machine, "b").env + assert "wayland-a" not in Kde(machine, "b").env + + +def test_as_user_has_no_session_in_its_environment(machine: FakeMachine) -> None: + Kde(machine, "main").as_user("systemctl --user is-system-running") + + (command,) = machine.commands + assert "WAYLAND_DISPLAY" not in command + assert "XDG_RUNTIME_DIR=/run/user/1000" in command + + +def test_headless_writes_the_arguments_and_starts_the_units(machine: FakeMachine) -> None: + machine.reply("busctl", status=0) + + Kde.headless(machine, "main", size=(1280, 720), outputs=2) + + scripts = [inner(command) for command in machine.commands if " sh -c " in command] + assert ( + "echo KWIN_ARGS='--width 1280 --height 720 --output-count 2'" + " > /run/user/1000/kwin-virtual-main.env" + ) in scripts + assert "systemctl --user start session-bus@main.service kwin-virtual@main.service" in scripts + + +def test_headless_reports_the_journal_when_kwin_does_not_come_up(machine: FakeMachine) -> None: + machine.reply("busctl", status=1) + machine.reply("journalctl", output="kwin: no GPU") + + with pytest.raises(AssertionError, match="kwin: no GPU"): + Kde.headless(machine, "main", scale=0.01) + + +def test_wait_for_scales_its_timeout(machine: FakeMachine) -> None: + kde = Kde(machine, "main", scale=0.0) + + assert not kde.wait_for(lambda: False, timeout=100) + assert kde.wait_for(lambda: True, timeout=0) + + +def test_rect_parses_what_kdotool_prints(machine: FakeMachine) -> None: + machine.reply( + "getwindowgeometry", output="Window {abc}\n Position: 10,20\n Geometry: 300x400\n" + ) + + assert Kde(machine, "main").rect("{abc}") == Rect(10, 20, 300, 400) + + +def test_rect_fails_on_output_it_does_not_understand(machine: FakeMachine) -> None: + machine.reply("getwindowgeometry", output="nothing useful") + + with pytest.raises(AssertionError, match="no geometry"): + Kde(machine, "main").rect("{abc}") + + +def test_overlap() -> None: + assert Rect(0, 0, 10, 10).overlaps(Rect(5, 5, 10, 10)) + assert not Rect(0, 0, 10, 10).overlaps(Rect(10, 0, 10, 10)) + assert not Rect(0, 0, 10, 10).overlaps(Rect(0, 10, 10, 10)) + + +def test_wid_is_the_first_match(machine: FakeMachine) -> None: + machine.reply("kdotool search", output="{one}\n") + + assert Kde(machine, "main").wid("alpha") == "{one}" + + +def test_active_is_the_last_line(machine: FakeMachine) -> None: + machine.reply("getactivewindow", output="noise\n{two}\n") + + assert Kde(machine, "main").active() == "{two}" + machine.replies.clear() + machine.reply("getactivewindow", output="") + assert Kde(machine, "main").active() == "" diff --git a/nix/tests/vm/nodes/headless.nix b/nix/tests/vm/nodes/headless.nix new file mode 100644 index 00000000..72b237ee --- /dev/null +++ b/nix/tests/vm/nodes/headless.nix @@ -0,0 +1,67 @@ +# A machine with Plasma installed and no login: `kwin_wayland --virtual` runs as the test user's +# own user units, once for every session a test starts, each with its own private session bus and +# Wayland socket (`session-bus@NAME`, `kwin-virtual@NAME`). A virtual KWin has no real output, so it +# cannot draw: pixel and pointer tests use the session node instead. +{ + config, + lib, + pkgs, + ... +}: +{ + imports = [ ./options.nix ]; + + # KWin, its plugins and the environment a session expects + services.desktopManager.plasma6.enable = true; + hardware.graphics.enable = true; + + users.users.komorebi = { + isNormalUser = true; + uid = 1000; + # keeps the user manager, and so /run/user/1000, alive without a login + linger = true; + }; + + virtualisation = { + memorySize = 3072; + cores = 2; + # KWin wants a DRM render node, which needs a GPU. The test driver adds one on aarch64 only. + qemu.options = lib.optionals (config.komorebi.vm.renderNode && pkgs.stdenv.hostPlatform.isx86_64) [ + "-device virtio-gpu-pci" + ]; + }; + + environment.systemPackages = with pkgs; [ + foot + jq + kdotool + kdePackages.libkscreen + kdePackages.qttools + ]; + + systemd.user.services = { + "session-bus@" = { + description = "The private session bus of the test session %i"; + serviceConfig.ExecStart = "${pkgs.dbus}/bin/dbus-daemon --session --address=unix:path=%t/session-%i.bus --nofork --nopidfile"; + }; + + "kwin-virtual@" = { + description = "A headless KWin for the test session %i"; + requires = [ "session-bus@%i.service" ]; + after = [ "session-bus@%i.service" ]; + environment = { + DBUS_SESSION_BUS_ADDRESS = "unix:path=%t/session-%i.bus"; + XDG_SESSION_TYPE = "wayland"; + # software rendering makes KWin skip every animating effect unless told otherwise + KWIN_EFFECTS_FORCE_ANIMATIONS = "1"; + # script print() goes through qCDebug, which is off by default + QT_LOGGING_RULES = "kwin_scripting.debug=true;js.debug=true;qml.debug=true"; + }; + serviceConfig = { + # the library writes KWIN_ARGS (size, number of outputs) before it starts the unit + EnvironmentFile = "-%t/kwin-virtual-%i.env"; + ExecStart = "${pkgs.kdePackages.kwin}/bin/kwin_wayland --virtual --socket wayland-%i --no-lockscreen --no-global-shortcuts $KWIN_ARGS"; + }; + }; + }; +} diff --git a/nix/tests/vm/nodes/options.nix b/nix/tests/vm/nodes/options.nix new file mode 100644 index 00000000..95be6131 --- /dev/null +++ b/nix/tests/vm/nodes/options.nix @@ -0,0 +1,14 @@ +# The options the test nodes share. +{ lib, ... }: +{ + options.komorebi.vm.renderNode = lib.mkOption { + type = lib.types.bool; + default = true; + description = '' + Give an x86_64 guest a virtio-gpu, so that KWin has a DRM render node. A node that draws sets + this to false: the virtio-gpu would be a second display, KWin would draw on it, and a + screenshot, which is of the other one, would show nothing. The test driver adds the one + virtio-gpu an aarch64 guest has, whatever this says. + ''; + }; +} diff --git a/nix/tests/vm/suites/kwin/default.nix b/nix/tests/vm/suites/kwin/default.nix new file mode 100644 index 00000000..bb57ac90 --- /dev/null +++ b/nix/tests/vm/suites/kwin/default.nix @@ -0,0 +1,5 @@ +{ + tier = "fast"; + kind = "headless"; + timeoutMinutes = 10; +} diff --git a/nix/tests/vm/suites/kwin/test.py b/nix/tests/vm/suites/kwin/test.py new file mode 100644 index 00000000..a861e743 --- /dev/null +++ b/nix/tests/vm/suites/kwin/test.py @@ -0,0 +1,31 @@ +# What the harness itself promises: a headless KWin comes up with the outputs asked for, a client +# gets a window on it, and the library sees both. +from komorebi_vm import Checks, Kde + +start_all() +machine.wait_for_unit("multi-user.target") +machine.wait_for_unit("user@1000.service") + +checks = Checks() + +kde = Kde.headless(machine, "main", size=(1280, 720), outputs=2, scale=SCALE) +checks.check("KWin owns its name on the private bus", kde.has_name("org.kde.KWin")) +checks.check("KWin has the two outputs that were asked for", lambda: kde.outputs() == 2) + +kde.spawn("foot -T alpha sleep 3000") +checks.check("a window appears", kde.wait_for(lambda: kde.wid("alpha") != "", timeout=30)) +wid = kde.wid("alpha") +checks.check("the library reads its title back", lambda: kde.title(wid) == "alpha") +rect = kde.rect(wid) +print(f"alpha is at {rect}") +checks.check("it has a size", rect.w > 0 and rect.h > 0) + +other = Kde.headless(machine, "other", size=(800, 600), scale=SCALE) +checks.check("a second session is its own KWin", lambda: other.outputs() == 1) +checks.check("it does not see the first session's window", other.wid("alpha") == "") +other.stop() + +kde.stop() +checks.check("stopping the session releases its name", not kde.has_name("org.kde.KWin")) + +checks.finish() diff --git a/scripts/vm-mac b/scripts/vm-mac new file mode 100755 index 00000000..537fdb99 --- /dev/null +++ b/scripts/vm-mac @@ -0,0 +1,38 @@ +#!/usr/bin/env bash +# Runs one VM test on a Mac, where the guest is Linux and the host is not. +# +# scripts/vm-mac NAME [driver arguments] +# +# The guest's closure is built on a Linux builder, copied back, and the test driver then runs +# on the Mac under HVF. The driver is run directly, not through `nix build`, because the nix +# sandbox on macOS cannot give QEMU the hypervisor. Artifacts (screenshots, logs) land in +# `.vm/out/NAME`. +# +# The builder is yours: set KOMOREBI_BUILDER to its store URL (for example +# `ssh-ng://builder@localhost`) and, if it needs them, NIX_SSHOPTS. docs/dev/vm-tests.md +# says what a builder has to be. +set -euo pipefail + +name="${1:?usage: scripts/vm-mac NAME [driver arguments]}" +shift +builder="${KOMOREBI_BUILDER:?set KOMOREBI_BUILDER to the store URL of a Linux builder}" +cd "$(dirname "$0")/.." +test="path:$PWD#legacyPackages.aarch64-darwin.vm.$name" +[ -d .git ] && test="git+file://$PWD#legacyPackages.aarch64-darwin.vm.$name" + +echo "== building the guests on $builder" +nix eval --json "$test.nodes" --apply 'nodes: builtins.mapAttrs (_: node: node.system.build.toplevel.drvPath) nodes' | + jq -r '.[]' | + while read -r drv; do + out=$(nix build "$drv^out" --eval-store auto --store "$builder" --no-link --print-out-paths) + nix copy --from "$builder" --substitute-on-destination --no-check-sigs "$out" + done + +echo "== building the driver" +driver=$(nix build "$test.driver" --builders '' --no-link --print-out-paths) + +out="$PWD/.vm/out/$name" +mkdir -p "$out" +echo "== running $name (artifacts in $out)" +cd "$out" +exec "$driver/bin/nixos-test-driver" -o . "$@"