From 0185fd0f85d23b8e755d17dd9892cfa4a4329874 Mon Sep 17 00:00:00 2001 From: Ryan Yin Date: Mon, 21 Sep 2026 21:31:27 +0800 Subject: [PATCH] docs(computer-use): document the headless i3 session --- home/README.md | 2 ++ home/linux/gui/i3/README.md | 39 +++++++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+) create mode 100644 home/linux/gui/i3/README.md diff --git a/home/README.md b/home/README.md index a85503c0..fbf26ba4 100644 --- a/home/README.md +++ b/home/README.md @@ -22,6 +22,7 @@ home/ ├── linux/ # Linux-specific home manager configurations │ ├── base/ # Linux base configurations │ ├── gui/ # Linux GUI applications +│ │ ├── i3/ # i3 — headless computer-use session (see i3/README.md) │ │ ├── niri/ # Niri window manager │ │ └── ... │ ├── editors/ # Linux-specific editors @@ -42,6 +43,7 @@ home/ 2. **linux**: Linux-specific configuration - Desktop environments (Noctalia Shell, Niri compositor) + - Headless computer-use session (`gui/i3`, Xvfb + i3; see `gui/i3/README.md`) - Linux-specific GUI applications - System integration tools diff --git a/home/linux/gui/i3/README.md b/home/linux/gui/i3/README.md new file mode 100644 index 00000000..b9d6de2d --- /dev/null +++ b/home/linux/gui/i3/README.md @@ -0,0 +1,39 @@ +# i3 — headless computer-use session + +This module provides the headless X11 session used by the computer-use VMs (`idols-ruby`, +`idols-kana`). It is **not** a normal desktop configuration and is not meant to be used +interactively by a person. + +## Purpose + +Give a computer-use agent a virtual 1920x1080 X display it can observe and control: screenshots, +window discovery/activation, synthetic input, and the AT-SPI accessibility tree. The agent itself +runs elsewhere and connects over MCP; this module is only the "computer" side. + +The session is built on Xvfb plus i3 because the drivers support native X11 at their highest tier +(cua-driver "Linux X11 — Supported"; computer-use-linux has an i3 backend). cua-driver's Sway lane +is XWayland/X11-based, so a native X11 session is simpler and better supported than Sway/Wayland. + +## How it is wired + +- `home/linux/gui/i3/default.nix` defines the option `modules.desktop.computerUse` and the user + services: `xvfb` → `i3` → (`x11vnc`, `cua-driver serve`). +- `modules/nixos/desktop/computer-use.nix` provides the system side (`linger`, AT-SPI, fonts, Mesa, + Clash Verge proxy, proxy-region timezone). +- `computer-use-init.sh` (exec'd by i3) publishes `DISPLAY`/`XDG_SESSION_TYPE` to the systemd user + manager, clears any stale `WAYLAND_DISPLAY`, and enables the AT-SPI bridge via a gsettings key. +- Drivers live in `overlays/cua-driver.nix` and `overlays/computer-use-linux.nix`. +- Host wiring: `home/hosts/linux/idols-ruby.nix`, `home/hosts/linux/idols-kana.nix`. + +## The i3 config + +The generated config is intentionally minimal: + +- `font pango:monospace 10` +- `default_border normal` — keeps title bars, so the client area is slightly smaller than the + screen. That matches a real desktop and avoids a full-screen-sized viewport. +- `exec --no-startup-id .../computer-use-init.sh` — one-time session setup, see above. + +There are no keybindings, bars or autostarted apps: agents drive the session through the drivers, +not the keyboard. Manual inspection is done over VNC (`x11vnc`, bound to `127.0.0.1`; reach it +through an SSH tunnel).