Commit Graph
39 Commits
Author SHA1 Message Date
LGUG2Z 93a024f0f1 test: skip what needs komorebi for Mac's source when there is none
Sixteen tests read the source of komorebi for Mac, through the catalog that
build.rs indexes from KOMOREBI_MAC_SRC (which nix develop sets) or the checkout
itself, and panicked or failed their assertions on a machine without it: every
Windows machine, and a Mac outside nix develop. That is not a failure of
anything, so they now say why and stop, as the tests that need ffmpeg do:

  skipped: KOMOREBI_MAC_SRC is not set, or this build was made without it

testkit::komorebi_mac_source_available says whether both the variable and the
indexed catalog are there, and skip_without_komorebi_mac_source! is its use at
the top of a test. With the source they run exactly as before.

On Windows the whole suite now passes.
2026-10-28 16:24:29 -07:00
LGUG2Z 7c692138c7 docs(usage): proposing a rule for a ghost tile
What the viewer offers under a bug region that was marked over a ghost tile, what
sending does and which credentials it uses, where the rule goes for each komorebi
(applications.json or applications.mac.json), how narrow it is, and what a
recording needs for there to be any ghost tile to propose on each host.
2026-10-28 16:24:29 -07:00
LGUG2Z 4bc35ef5d2 feat(viewer): offer to propose a rule for a ghost tile
Under a bug region that was marked while a ghost tile was there, the Bugs tab
says so and offers a rule. The button opens a form with what the recording showed
of the window, the application's name to change, and the rule as it would be
written, narrow enough to leave the program's other windows alone; if no such
rule can be written (nothing tells the ghost from a window that is on screen) it
says why and sends nothing. Nothing is sent until the user presses the button.

Sending runs on a background thread, and the form lists the steps as they happen
and ends with the link to the pull request, a link to open one, or, if something
failed, where the clone is kept (with a button to open the folder) and the
instructions to finish by hand. Escape closes the form, but not while it is
sending.

The recording's ghost tiles are found when it is opened, for the platform the
recording is of: the words are those of that komorebi, and the rule goes in its
file, whatever the viewer runs on.
2026-10-28 16:24:29 -07:00
LGUG2Z 9638dc44c5 feat(asc): propose a rule to komorebi-application-specific-configuration
A rule written from what a recording shows, and the pull request that sends it,
for either komorebi: the rule goes in the file of the komorebi the recording is
of (applications.json or applications.mac.json), not of the machine it is looked
at on.

asc::rule is the rule. It is an ignore rule, always composite so that never the
whole program is ignored: the program and the class on Windows, the application
and the title on a Mac, which has no class. When the program's other windows in
the recording share the ghost's class, the title is added; when nothing tells the
ghost from a window that is on screen no rule is written, and it says why. It
puts the rule into the file changing the lines of the rule and no others: a new
application is a pure insertion where it sorts (case ignored, as just sort does
it), a known one gets the rule in its ignore list and its comma. The result is
checked to be the old file and the rule and nothing more, and a file not laid out
as jq prints it is refused instead of reprinted. Applied to the real
applications.json: 16 lines added and none removed.

asc::submit clones the repository into a temporary directory, branches, commits
with a message that says what was seen, why an ignore rule and why one this
narrow, and pushes. With gh signed in it pushes to the repository if the user may
and to their fork if not (making one), and opens the pull request; without, it
pushes with git and gives a link. git is used for every push, so it is the user's
own credential helper that is asked. gh is looked for where Homebrew puts it, for
a program started from Finder. Anything that fails leaves the clone, with the
commit in it, and says what is left to do. Every location is injectable, and the
tests send to local repositories, a fake gh and a hook that refuses, never to
GitHub.
2026-10-28 16:24:29 -07:00
LGUG2Z 22a1930a48 feat(analysis): ghost tiles
A ghost tile is a tile of the layout holding a window a person cannot see: one
komorebi has in front on a workspace that is showing, and that was never on
screen, on at least two reads over half a second or still so at the last. Anything
less is a transition (a workspace switch, an application starting up) or
something a person did.

How the recording shows it depends on the host, since only one can be asked:

- komorebi for Windows: Windows says the window is cloaked or not visible on
  every probe of it, and never on screen or minimized. This needs a recording
  made with --probe-windows; without probes there are none here, which is not
  knowing and not none.
- komorebi for Mac: macOS cannot be asked, but komorebi's state has each window's
  frame, read live from the accessibility API. A ghost is a window whose frame is
  zero-sized or outside every display on every read, and never one a person can
  see. That is weaker evidence than Windows saying so, and is documented as such.
  It was not run against a real Mac recording.

A ghost carries the program, class and title komorebi has for it, and the other
windows of the same program that the recording saw (siblings), so that a rule for
it can be narrow enough to leave them alone. overlaps() says whether it was there
while a bug region was marked.
2026-10-28 16:24:29 -07:00
LGUG2Z 53c5d50786 feat(analysis): the class of a window
komorebi for Windows writes a window as {hwnd, title, exe, class, rect} and the
state read dropped the class. It is what komorebi-application-specific-configuration
rules are mostly written against, so a Window now has it: empty on a Mac, which
has none, and for a window komorebi could not read, whose error text is not a
class.
2026-10-28 16:24:29 -07:00
LGUG2Z f36f0ea951 build(just): build both executables before run and record
The recorder opens the viewer as komorebi-recorder-gui, a separate executable
beside it, and `cargo run --bin komorebi-recorder` builds only the recorder.
The viewer was left as it was last built, which meant fixes to it did not
show up in `just run komorebi-recorder` until the other was built by hand.
2026-10-27 22:22:57 -07:00
LGUG2Z 5a4d89ffd5 test(session): do not assume a Mac, or a lock that can be read
Four tests only passed on a Mac:

- the pid is kept out of the locked file: Windows locks are mandatory, so the
  test could not read the file it was holding; it checks the pid file only
- an older build's pid in the lock file: an older build only ever ran on a
  Mac, and Windows cannot read it back while the file is locked; unix only
- the README describes the OS trace the host's session has: the Mac side of
  it took the build host's manifest; it now says it is one
- the README embeds the session path: the quoting it looks for is the
  host's own (single quotes in a unix shell, double on Windows)
2026-10-27 22:22:57 -07:00
LGUG2Z d4fc229a77 fix(viewer): tell the system's groups apart in the events pane
On a Windows session several groups of the OS trace share the one source,
oslog:system (one per program), so the id of their collapsing headers,
built from the event and the source, was the same for all of them and egui
warned "First use of widget ID" under "the system around it". The
label is the program, and it tells them apart.

macOS gives every group a source of its own, which is why it never showed.
2026-10-27 22:22:57 -07:00
LGUG2Z fa631e7eea fix(av): let the source reader resize frames
Every command that decodes the video (sheet, changes, view) failed on
Windows with 0xC00D36B4, for any video: the source reader was opened with
MF_SOURCE_READER_ENABLE_VIDEO_PROCESSING, which converts colour but refuses a
frame size other than the video's, and they ask for BGRA at a size of their
own. MF_SOURCE_READER_ENABLE_ADVANCED_VIDEO_PROCESSING resizes too.

MF_MT_DEFAULT_STRIDE is no longer set on the requested type: frame_of reads
the pitch the buffer reports, negative for a bottom-up picture, so it asked
for nothing it needed.
2026-10-27 22:22:57 -07:00
LGUG2Z b38a6f4dc6 fix(capture): scale a display H.264 cannot take
An H.264 encoder refuses frames bigger than 4096 x 2304 (either way round).
On a 5120 x 2160 display the sink writer failed at SetInputMediaType with
0xC00D36B4 (MF_E_INVALIDMEDIATYPE), reported as "the encoder cannot take
frames of this format", and no display could be recorded.

segment::h264_size gives the size to encode at: the display's own, made even,
when H.264 can take it, and otherwise the largest even size of the same shape
that it can (5120 x 2160 is recorded at 4096 x 1728). When that is not the
display's size, a Direct3D 11 video processor scales each frame on the GPU
before it is handed to the encoder. Displays that fit are untouched.

Recorded and decoded on a 5120 x 2160 display: all three screens, undistorted.
2026-10-27 22:22:57 -07:00
LGUG2Z 8a203f0da7 feat(icon): a watermelon for every window
The recorder window and the viewer carry an icon of their own: a slice of
watermelon, drawn as komorebi-recorder/assets/watermelon.svg. `just icon`
(assets/generate.py, with resvg from the devShell) renders the PNG the windows
set and the ICO that Windows embeds. Until now eframe's default egui icon
showed.

macOS: eframe sets the PNG as the Dock icon while a window is open. The program
is a bare executable and not an app bundle, so Finder is unchanged.

Windows: eframe sets the same PNG as the window and taskbar icon. build.rs
compiles the ICO into both executables with embed-resource (rc.exe on Windows,
llvm-rc when cross-building from a Mac), and the installer shows it in Apps &
features through ARPPRODUCTICON. The taskbar, Explorer and the installer have
not been looked at on Windows: W4 and W5 in docs/dev/windows-checkpoints.md now
say what to check.

embed-resource adds seven crates to the lockfile and moves toml_datetime and
toml_parser up by a patch release.
2026-10-27 22:22:56 -07:00
LGUG2Z df4ab2d2a5 docs(repo): how it fits together, per platform, and why
The model (the host axis and the flavor axis, the master clock, the facade, the
rules and the layer that enforces each), the same questions answered for each
host, the decisions behind the design, and the checkpoints to run on a Windows
machine, including the installer.
2026-10-27 22:22:56 -07:00
LGUG2Z 28cfdaa463 feat(viewer): view a recording
`komorebi-recorder view <session>` opens a recording in a window: the video of
every display, a timeline with a lane for the bug regions, events, log errors,
the machine and the findings, and an inspector for the state at the playhead,
one event traced end to end, and komorebi's log. The window is untitled so
komorebi leaves it alone, and out of any recording being made. It is where a bug
is marked: the form takes the region from the selection or the playhead and asks
what went wrong and what was expected. It opens from the recorder window as
well.
2026-10-27 22:22:56 -07:00
LGUG2Z 278fde6650 feat(playback): play forwards and backwards
A transport clock that runs forwards or backwards at any speed, which is plain
arithmetic and tested to the millisecond, and an engine that decodes on its own
thread and keeps the frame of every display in step with it.
2026-10-27 22:22:56 -07:00
LGUG2Z 52a45aa17b feat(winprobe): ask Windows what it thinks of a window
komorebi for Windows hides a window by cloaking it in place, so its state cannot
show whether a window is really hidden. `record --probe-windows` follows each
window event with a read of the window's real state from Windows and, now and
then, asks komorebi whether it manages the window. The analysis compares the
two: a window komorebi shows that Windows hides, one it hides that Windows
shows, and a visible window it does not manage. macOS refuses the flag with a
message.
2026-10-27 22:22:56 -07:00
LGUG2Z e2fdd54a54 feat(analysis): drill into the OS trace around an event
From a komorebi event into what the OS trace recorded around it: komorebi's own
calls, the system, and each application, while the handler ran and for a second
after. On Windows the WinEvents are grouped per program. `trace --os` prints it.
2026-10-27 22:22:56 -07:00
LGUG2Z 069347060d feat(analysis): see what the screen did
When and where on each display the picture changed, found by decoding the video,
and a contact sheet of frames with the times, events and bug region marked. A
display with no video for part of a range has a gap there instead of an invented
frame.
2026-10-27 22:22:56 -07:00
LGUG2Z 26963391a3 feat(analysis): investigate from what you marked
The brief starts from what the user marked as the bug, in their words, and ranks
the events most worth tracing first; findings an agent concludes are saved with
the evidence they cite, which must exist. The commands arrive: `state`, `diff`,
`events`, `window`, `trace`, `code`, `brief`, `overview`, `search`, `annotate`,
`annotations`, `guide` and `schema`, all with `--json`, and a search is bounded
by a budget.
2026-10-27 22:22:56 -07:00
LGUG2Z 079f1ac6d0 feat(analysis): explain one event, follow one window
One event end to end: what its handler logged, what other threads logged
meanwhile, and every change with who made it and how sure that is. The state,
the diff and the events are answered as plain values, an overview ranks what
stands out in a recording, and invariants flag states that are wrong: a window
in two containers, a window parked off screen on the workspace that is showing
(a rule that holds only where komorebi hides windows by moving them, so it is
skipped on Windows).
2026-10-27 22:22:56 -07:00
LGUG2Z 3a325470eb feat(analysis): trace a log line to its source
Every `tracing` statement in komorebi's source is indexed at build time, for
each flavor, so a line of its log can be traced to the statement that wrote it
with the code around it. A recording from a different komorebi than the one
indexed is said to be skewed, decided by the files that differ between the two
commits rather than by commit equality. The `Analysis` facade that every
question is asked of arrives with it, and one window's history through a
recording.
2026-10-27 22:22:56 -07:00
LGUG2Z 72419d89a2 feat(analysis): what komorebi believed, and what changed
A komorebi `State` is read into monitors, workspaces and windows through each
flavor's rules, and what changed between two states is said in terms a person
would use. `at`, `range`, `summary` and `locate` ask what the recording holds
around a video time, and the other way round, with the text and JSON the later
commands share.
2026-10-27 22:22:56 -07:00
LGUG2Z af6e332fed feat(analysis): read komorebi's log as handlers
komorebi's log says what happened but not which handler was running. The log is
parsed into lines with their span chain, and a handler's lines are gathered into
one block, from the line that opens it to the one that ends it, as each flavor's
log marks them: komorebi for Mac logs a header, komorebi for Windows has none
and a block is found from its span.
2026-10-27 22:22:56 -07:00
LGUG2Z e3a1f01fd7 feat(doctor): check what a recording leans on
`doctor` says what the machine can do for a recording: which komorebi runs and
how it logs, whether the screen can be captured and played back, the displays
and their scale, and the host clock. Probes measure one thing: the clock across
a sleep, a short capture, a decode, whether komorebi would tile a window of
ours.
2026-10-27 22:22:56 -07:00
LGUG2Z 3e30d81bc5 feat(av): decode the recorded video
Reading the video back: AVFoundation on macOS, the Media Foundation source
reader on Windows, a clear error elsewhere. A request for the frames between two
times is answered with the frame that was on screen at the start, even when the
decoder had to seek to an earlier key frame. `selftest` records a rendered
on-screen clock and compares the clock the video shows with the time the session
predicts.
2026-10-27 22:22:56 -07:00
LGUG2Z 5291a24fd4 feat(app): the recorder window
Run with no command, `komorebi-recorder` opens a small window with Record and
Stop. It is untitled and borderless because komorebi does not manage windows
without a title, so opening it does not move the layout under investigation, and
it keeps itself out of the video (by process name on macOS, with
`WDA_EXCLUDEFROMCAPTURE` on Windows). `komorebi-recorder-gui` is the same
program built for the GUI subsystem on Windows, so that a shortcut opens no
console; both are in the zips and the installer.
2026-10-27 22:22:56 -07:00
LGUG2Z 05d87908c5 feat(recorder): orchestrate a recording
`komorebi-recorder record` starts every source in the order that makes the
non-video sources a superset of the video, stops them in reverse, and
checkpoints the manifest as it goes, so that a recorder that is killed leaves a
session `reindex` can recover. `--no-video` records everything but the screen,
and Ctrl-C or `--duration` stops and finalizes a recording. The format of
`annotations.jsonl`, the append-only file that bug regions, notes and findings
are written to, is defined here too. The command table arrives with it, with
`displays` and `reindex`, and with the text-and-JSON conventions the later
commands share.
2026-10-27 22:22:56 -07:00
LGUG2Z 1006ec2570 feat(timeline): index a session
`timeline.jsonl` is a merged, time-sorted index that points into the raw files
by byte range, so the raw files are never rewritten and the index can be rebuilt
at any time. It places komorebi's log, events, the OS trace and the recorder's
own incidents on one timeline, and answers what was around a video time and
where in the video a log line or event is. `frames` extracts PNG frames from the
recorded video with ffmpeg, which is never used to capture.
2026-10-27 22:22:56 -07:00
LGUG2Z 383cc3c3db feat(power): ride out sleep, lock and display changes
Docking, sleep and lock are where komorebi bugs show up, so the recording
carries on through them. On Windows a hidden window hears suspend, resume, the
lid, the displays and session lock as Windows delivers them; on a Mac a sleep is
found because the host clock pauses while the wall clock does not, and the lock
is polled on both hosts. A tracker turns these into reactions: a display that is
gone has its video finalized at once, one that returns starts a new segment, and
every stream is started afresh after a wake instead of trusting the old one.
2026-10-27 22:22:56 -07:00
LGUG2Z fb11cb2db9 feat(capture): record the screen
Screen capture behind a `CaptureBackend`: ScreenCaptureKit and AVFoundation on
macOS (HEVC by default), Windows Graphics Capture and a Media Foundation writer
on Windows (H.264). Each display is written as fragmented mp4 in segments with
every frame's master-clock timestamp, and a manager decides, from what the
backend reports, when a segment ends and another begins. The segment logic is
shared by both backends and tested without a screen.
2026-10-27 22:22:56 -07:00
LGUG2Z 9d35869f88 feat(sources): the OS trace
What the OS says while komorebi acts: the unified log (`log stream`) on macOS,
one file per managed application, komorebi's process and the system; on Windows
the WinEvents komorebi hears, folded when repeated, and the Event Log's warnings
and its display, power and GPU providers. Both end up as `oslog/*.ndjson` on the
master clock, and a host with no OS trace says so instead of failing.
2026-10-27 22:22:56 -07:00
LGUG2Z 4724d831ba feat(session): the session directory
A session is a directory and its manifest, `session.json`: the host, the
komorebi flavor, the clock anchors, the log segments and the applications seen,
written atomically. komorebi's own log is recorded as the exact byte range
written while recording (`%TEMP%` on Windows, its data directory on a Mac),
together with the version of the komorebi that wrote it. A generated README in
each session says what every file is, quoting paths the way the host's shell
reads them.
2026-10-27 22:22:56 -07:00
LGUG2Z c748237741 feat(komorebi_link): record what komorebi says
The recorder subscribes to komorebi over its socket (a unix socket on macOS,
`uds_windows` on Windows) and writes every notification with the state it came
with, stamped on the master clock. Its accept loop only hands off to a writer
thread, since komorebi notifies subscribers while holding its locks. What
happens to the recording itself (a komorebi that exits, a stream that dies) is
written as incidents, and a lock file says whether a recorder is still running
in a session.
2026-10-27 22:22:56 -07:00
LGUG2Z 0de0f82840 feat(flavor): read komorebi for Mac and for Windows
Which komorebi produced a session is a separate axis from the host the recorder
ran on. `Flavor` names the two and `Rules` holds what each knows: what its
events look like on the wire, which of them move a window, and which are
komorebi making things up. komorebi for Mac and komorebi for Windows each
implement it, and a missing method is a compile error. The Windows rules were
written from komorebi's source and say so until a recording confirms them.

Events are classified through the rules, and the names of things in a session
(`event:12`, `log:7`) are parsed here too, with the envelope and exit codes
every command will answer in.
2026-10-27 22:22:56 -07:00
LGUG2Z d5eacecef8 feat(clock): the master clock
Every record in a session is stamped on one monotonic clock:
`mach_absolute_time` on macOS and `QueryPerformanceCounter` on Windows, behind a
`platform` facade with a third `unsupported` host that builds anywhere. Anchors
pair that clock with the wall clock, a map interpolates between them, and a jump
detector notices sleep and clock steps by comparing how far each moved. Video
timecodes are parsed and printed here too.

`Host` and `ClockKind` say what a session was recorded on.
2026-10-27 22:22:56 -07:00
LGUG2Z ab0b943ba0 build(wix): an installer for Windows
An MSI for Windows built with WiX and `cargo wix`, for x86_64 and aarch64,
attached to every release and, on tags, offered to winget. It is its own
product: its upgrade code and the GUID of its PATH component are new, since
reusing komorebi's would make the two installers upgrade and uninstall each
other.

WiX only runs on Windows, so `tests/packaging.rs` checks what a Mac can: every
binary the crate builds is in the installer, and the installer is not
komorebi's.
2026-10-27 22:22:56 -07:00
LGUG2Z 928ee8ebaa ci(github): build and attach artifacts for macOS and Windows
One workflow builds both platforms and publishes what they built. macOS builds
and checks through nix on the self-hosted runner; Windows builds natively on
`windows-latest` for x86_64 and aarch64, with fmt, clippy and the tests on
x86_64. A publish job packages the executables and checksums and writes the
release notes with kokai: a `v*` tag makes a release, a scheduled or manual run
on master replaces the rolling nightly, and any other push to master makes a
draft.

There is no nix on the Windows runner, so it checks out the komorebi sources at
the revisions `flake.lock` pins.
2026-10-27 22:22:56 -07:00
LGUG2Z 76bae0630e build(nix): build and check for macOS and Windows
A flake that builds the recorder for macOS and cross-compiles it for Windows
from a Mac (xwin for the SDK, LLVM for the toolchain), with a dev shell for
each, and `nix flake check` running clippy for both targets, rustfmt, taplo,
cargo-deny and the tests.

The sources of komorebi for Mac and komorebi for Windows are flake inputs,
pinned in `flake.lock`: the recorder indexes them so that a line of komorebi's
log can be traced to the statement that wrote it, and the dev shells and every
derivation export where they are.
2026-10-27 22:22:56 -07:00
LGUG2Z af3c080ace chore(repo): a workspace of one crate
A cargo workspace with one member, `komorebi-recorder`, and the files a project
needs around it: toolchain, rustfmt and cargo-deny configuration (both the macOS
and the Windows target are in the dependency graph), a justfile whose `check`
recipe is the one command CONTRIBUTING names, the license, and README and
CONTRIBUTING.

The crate is a skeleton that prints its version; the recorder arrives in the
commits after it.
2026-10-27 22:22:56 -07:00