diff --git a/README.md b/README.md index f0b57bc3..9386a5ee 100644 --- a/README.md +++ b/README.md @@ -40,6 +40,7 @@ runs the latest commit without a checkout. ```sh komorebi-recorder displays # which displays can be recorded komorebi-recorder record --duration 30 --out ./repro +komorebi-recorder record --until-exit --out ./repro -- ./repro.sh # record while a script runs, stop when it exits ``` Run `komorebi-recorder` with no command for a small window with Record and Stop buttons instead. @@ -55,6 +56,8 @@ the frames. ```sh komorebi-recorder brief ./repro # what the user marked as the bug, and what the recording shows around it +komorebi-recorder brief ./repro --compact # the same in about 40 lines, for a small context window +komorebi-recorder findings ./repro # detector findings (with stable ids) and an agent's conclusions, in one list komorebi-recorder overview ./repro # the whole recording at a glance komorebi-recorder trace ./repro event:7 # one event, from the handler that ran to what changed and who changed it ``` @@ -74,7 +77,8 @@ when things happened (events | errors | incidents): ``` `komorebi-recorder view ./repro` opens the recording in the viewer, which is where you mark where the bug is. -Every command takes `--json`, and `komorebi-recorder guide` prints a playbook for an agent. +Every command takes `--json` (including `record`, `displays`, `reindex` and `selftest`), and `komorebi-recorder guide` +prints a playbook for an agent. [Investigating a recording](docs/usage/investigating-a-recording.md) walks through both ways of working. ## How it fits together diff --git a/docs/usage/investigating-a-recording.md b/docs/usage/investigating-a-recording.md index 1829867b..3c48fb46 100644 --- a/docs/usage/investigating-a-recording.md +++ b/docs/usage/investigating-a-recording.md @@ -6,6 +6,8 @@ this is broken" to "this is the line of komorebi that causes it" using what was ``` komorebi-recorder brief # start here +komorebi-recorder brief --compact # the same, in about 40 lines +komorebi-recorder findings # everything the detectors found and an agent concluded komorebi-recorder guide # the playbook, for an agent ``` @@ -108,6 +110,8 @@ and `event:12.end` when it finished. | Command | What it tells you | | --- | --- | | `brief` | The bug region(s) in the user's words, then what the recording shows around them: the events, what looked wrong, the state as it began and ended, what happened to the machine, and the events most worth tracing first, with why | +| `brief --compact` | The brief cut down to fit a small context window, about 40 lines: the user's words, one line on the state of the recording, the 3 highest-ranked findings (each with its id and the references to look at), and the exact commands to run next. `--json` gives `brief-compact/1` | +| `findings` | The detectors' findings and the findings an agent recorded, in one list (an agent's first, then the detectors' ranked as in `overview`). `--kind K` (`log-error`, or `invariant` for every `invariant:`), `--severity error\|warn\|info` (detector findings only), `--limit N`. `--json` gives `findings/1` | | `overview` | The whole recording at a glance: what stands out, ranked (states that are wrong and for how long, windows that flap, event storms, slow handlers, errors, a display unplugged), anything near a bug region first | | `state --at T` | What komorebi believed at a moment, and anything inconsistent in it: a window parked off screen on a workspace that is showing, a window in two places, a layout that was not applied | | `diff A B` | What changed between two moments in terms a person would use: a window appeared, moved, was really somewhere else; focus moved; a monitor came or went | @@ -159,6 +163,16 @@ the same. the window really was, which is what makes comparing it with where komorebi's layout put it meaningful. - An `AXError` in komorebi's log comes from the application or the OS, not from komorebi. +### Finding ids + +Each detector finding has an id like `f-04dc11`: `f-` and six hex digits of a fixed hash (FNV-1a) of its kind, its first +reference (`event:17`, `log:345`) and the ids of the windows it is about; a finding with no reference uses the +five-second bucket of video time it began in. It does not depend on the score or on the bug regions, so the same finding +keeps its id between runs and as annotations are added, and `findings`, `brief --compact` and `overview` name it the +same way. Two different recordings share an id only where the same kind of finding is at the same reference, so use ids +to compare analyses of one session (before and after a change to the detectors), not to match a bug across recordings. +An agent's findings keep their annotation id (`a-3f9c1e02`). + ## Recording what you conclude ``` @@ -168,5 +182,11 @@ komorebi-recorder annotate add --kind finding --about \ ``` A finding with no evidence is refused, and every reference is checked to exist: the event, the log line, the video time, -the line of komorebi's source. Findings are stored in `annotations.jsonl` beside the bug region, so the person who +the line of komorebi's source. A log line can be named by what it says or when it was written instead of by number: +`--ref 'log:"could not place window"'` (the one line of `komorebi.log` that contains the text, ignoring case), +`--ref log:15:46:23.677700` (the line written at that UTC time, as precisely as you write it; `HH:MM:SS` alone matches the +whole second) or `--ref log:2.72` (the line nearest that time of the video). A text or time that matches several lines +is a usage error (exit 2) listing the first few as `log:N: text`; pick one with `log:N`. When a reference was resolved +this way the command prints the `log:N` it chose and the line's text (on standard error, and in `warnings` with +`--json`), so you can check it is the statement you meant. Findings are stored in `annotations.jsonl` beside the bug region, so the person who recorded the bug can see what was concluded and why. diff --git a/docs/usage/recording-a-bug-repro.md b/docs/usage/recording-a-bug-repro.md index 27029453..a1cb3816 100644 --- a/docs/usage/recording-a-bug-repro.md +++ b/docs/usage/recording-a-bug-repro.md @@ -79,6 +79,27 @@ komorebi-recorder displays # list display ids komorebi-recorder record --display 2 # record just one display ``` +To record exactly while a script reproduces the bug, give the command after `--` with `--until-exit`: + +``` +komorebi-recorder record --until-exit --out ./repro -- ./repro.sh --slow +``` + +The recorder starts the command once the recording is under way, and stops the recording when the command exits. If +`--duration` or Ctrl-C comes first, that stops the recording and the command is terminated, so whichever comes first +wins. The command runs in the directory you are in, with this terminal's input and output, and has +`KOMOREBI_RECORDER_SESSION` in its environment: the session directory (an absolute path) that is being recorded, so +that anything the script runs can mark the moment with `komorebi-recorder mark` or annotate the session without being +told where it is. On Windows the command is terminated with `TerminateProcess`, which does not reach programs it +started. + +The exit status of `komorebi-recorder record` is about the recording, never the command: it is 0 when the session was +finalised, whatever status the command exited with. The command's status is in the summary +(`command: ... exited with status 3`) and in `record --json` as `data.child` (`end`: `exited`, `stopped`, +`failed-to-start` or `not-started`; `exit_code`; `success`). `--json` on `record` prints the `record/1` envelope on +standard output (progress stays on standard error): the session directory, the duration, each display's frames, the +recorder's warnings and the command's status. `displays`, `reindex` and `selftest` take `--json` too. + Useful options: | Option | Effect | diff --git a/komorebi-recorder/src/analysis/compact.rs b/komorebi-recorder/src/analysis/compact.rs new file mode 100644 index 00000000..3e5f8297 --- /dev/null +++ b/komorebi-recorder/src/analysis/compact.rs @@ -0,0 +1,281 @@ +//! `brief --compact`: the brief cut down to what fits in a small context window. +//! +//! The full [`BriefData`] carries every event, change and state of each bug region, which is what an +//! investigation wants after it has started and more than an agent wants to read first. A +//! [`CompactBrief`] is what to read before that, in about 40 lines: the user's words, one line on the +//! state of the recording, the three highest-ranked findings (each with the references to look at), +//! and the exact commands to run next. [`BriefData::compact`] makes one from a brief that has already +//! been made, so it shows the same findings and the same commands the full brief does. +//! +//! Limits, so that the size is predictable: at most [`TOP_FINDINGS`] findings, at most +//! [`MAX_TEXT`] characters of each bug's description, [`MAX_SUMMARY`] of each finding's summary, at most +//! [`MAX_NEXT`] commands, and every bug region is listed (a recording rarely has more than one or +//! two). Anything cut off is counted, and the full `brief` has it. + +use super::brief::BriefData; +use super::invariants::Severity; +use crate::timecode; +use schemars::JsonSchema; +use serde::Serialize; +use std::fmt::Write; + +/// The `schema` of `brief --compact --json`: the name [`CompactBrief`] has in an envelope. +pub const SCHEMA_BRIEF_COMPACT: &str = "brief-compact/1"; + +/// How many findings a compact brief lists. +pub const TOP_FINDINGS: usize = 3; +/// How many characters of a description or a summary a compact brief keeps. +pub const MAX_TEXT: usize = 300; +/// How many characters of a finding's summary a compact brief keeps. +pub const MAX_SUMMARY: usize = 160; +/// How many commands a compact brief suggests. +pub const MAX_NEXT: usize = 6; + +/// A bug region the user marked, in the user's words. +#[derive(Debug, Clone, PartialEq, Serialize, JsonSchema)] +pub struct CompactRegion { + /// The annotation id, e.g. `a-3f9c1e02`. + pub id: String, + /// What the user said is wrong, cut to [`MAX_TEXT`] characters. + pub text: String, + /// What should have happened instead, cut the same way. + pub expected: Option, + /// Where the region starts, in seconds of video time. + pub start_video_s: f64, + /// Where it ends, in seconds of video time (the end of the recording if open-ended). + pub end_video_s: f64, +} + +/// One of the highest-ranked findings. +#[derive(Debug, Clone, PartialEq, Serialize, JsonSchema)] +pub struct CompactFinding { + /// The stable id, `f-xxxxxx` (see [`super::findings`]). + pub id: String, + /// How serious it is. + pub severity: Severity, + /// The detector's kind. + pub kind: String, + /// What was seen, cut to [`MAX_SUMMARY`] characters. + pub summary: String, + /// When it began, in seconds of video time. + pub at_video_s: f64, + /// Where to look: `event:12`, `log:345`, ... + pub refs: Vec, +} + +/// The brief in a few lines: the `data` of `brief --compact --json`. +#[derive(Debug, Clone, PartialEq, Serialize, JsonSchema)] +pub struct CompactBrief { + /// The session directory. + pub session: String, + /// The state of the recording in one line: how long, what, and komorebi's state at the end. + pub state: String, + /// The bug regions the user marked; empty if none was. + pub regions: Vec, + /// How many findings there are in or near the regions (or in the whole recording if no region + /// was marked), of which the first [`TOP_FINDINGS`] are in `top_findings`. + pub findings_total: usize, + /// The highest-ranked findings. + pub top_findings: Vec, + /// How many findings an agent has already recorded (`findings` lists them). + pub recorded_findings: usize, + /// Commands to run next, exactly as they can be pasted. + pub next: Vec, +} + +fn cut(text: &str, most: usize) -> String { + let flat = text.split_whitespace().collect::>().join(" "); + if flat.chars().count() <= most { + return flat; + } + let mut cut: String = flat.chars().take(most).collect(); + cut.push_str("..."); + cut +} + +impl BriefData { + /// Returns the compact form of this brief; see the [module](super::compact) for its limits. + /// + /// The findings are those of the bug regions' briefs (or of the whole recording when no region + /// was marked), each once, highest score first. The commands are those the first bug region's + /// brief suggests, then `findings` and the full `brief`. + /// + /// # Example + /// + /// ```rust + /// use komorebi_recorder::analysis::Analysis; + /// use komorebi_recorder::testkit::SessionBuilder; + /// + /// let (session, _) = SessionBuilder::new("doc-compact") + /// .log(3.0, "ERROR", "komorebi::window: could not do the thing") + /// .build(); + /// let compact = Analysis::open(&session.dir)?.brief()?.compact(); + /// + /// assert!(compact.regions.is_empty()); + /// assert_eq!(compact.top_findings[0].kind, "log-error"); + /// assert!(compact.top_findings[0].id.starts_with("f-")); + /// # Ok::<(), Box>(()) + /// ``` + #[must_use] + pub fn compact(&self) -> CompactBrief { + let dir = &self.session.dir; + + let mut findings: Vec<&super::overview::Finding> = self + .bugs + .iter() + .map(|bug| &bug.region) + .chain(self.whole_session.as_ref()) + .flat_map(|region| region.findings.iter()) + .collect(); + findings.sort_by(|a, b| { + b.score + .cmp(&a.score) + .then(a.start_mach_ns.cmp(&b.start_mach_ns)) + }); + let mut seen = std::collections::BTreeSet::new(); + findings.retain(|finding| seen.insert(finding.stable_id())); + + let region = self + .bugs + .first() + .map(|bug| &bug.region) + .or(self.whole_session.as_ref()); + let at_end = region.and_then(|region| region.state_at_end.as_ref()); + let session = &self.session; + let state = format!( + "{:.0} s of video, {} display(s), {} events, {} log lines, {} incidents; sync {}; {}", + session.duration_s, + session.displays.len(), + session.events, + session.log_lines, + session.incidents, + session.sync_quality, + at_end.map_or_else( + || "no komorebi state recorded".to_string(), + |state| { + let wrong = state.violations.len(); + if wrong == 0 { + format!("at the end: {}, nothing wrong", state.summary) + } else { + format!("at the end: {}, {wrong} thing(s) wrong", state.summary) + } + } + ), + ); + + let mut next: Vec = match self.bugs.first() { + Some(bug) => bug.commands.iter().take(MAX_NEXT - 2).cloned().collect(), + None => self + .next + .iter() + .filter(|command| command.starts_with("no bug region")) + .cloned() + .collect(), + }; + next.push(format!("komorebi-recorder findings {dir}")); + next.push(format!("komorebi-recorder brief {dir}")); + + CompactBrief { + session: dir.clone(), + state, + regions: self + .bugs + .iter() + .map(|bug| CompactRegion { + id: bug.bug.id.clone(), + text: cut(&bug.bug.text, MAX_TEXT), + expected: bug.bug.expected.as_deref().map(|text| cut(text, MAX_TEXT)), + start_video_s: bug.bug.start_video_s, + end_video_s: bug.bug.end_video_s, + }) + .collect(), + findings_total: findings.len(), + top_findings: findings + .iter() + .take(TOP_FINDINGS) + .map(|finding| CompactFinding { + id: finding.stable_id(), + severity: finding.severity, + kind: finding.kind.clone(), + summary: cut(&finding.summary, MAX_SUMMARY), + at_video_s: finding.start_video_s, + refs: finding.refs.clone(), + }) + .collect(), + recorded_findings: self.findings_recorded.len(), + next, + } + } +} + +impl CompactBrief { + /// Returns the brief as plain text, in about 40 lines. + /// + /// # Example + /// + /// ```rust + /// use komorebi_recorder::analysis::Analysis; + /// use komorebi_recorder::testkit::SessionBuilder; + /// + /// let (session, _) = SessionBuilder::new("doc-compact-text").build(); + /// let text = Analysis::open(&session.dir)?.brief()?.compact().text(); + /// assert!(text.lines().count() < 40); + /// assert!(text.contains("komorebi-recorder findings")); + /// # Ok::<(), Box>(()) + /// ``` + #[must_use] + pub fn text(&self) -> String { + let mut out = String::new(); + + if self.regions.is_empty() { + let _ = writeln!(out, "NO BUG REGION WAS MARKED"); + } + for region in &self.regions { + let _ = writeln!( + out, + "BUG {} ({} -> {}), as the user said:\n {}", + region.id, + timecode::format(region.start_video_s), + timecode::format(region.end_video_s), + region.text + ); + if let Some(expected) = ®ion.expected { + let _ = writeln!(out, " expected: {expected}"); + } + } + + let _ = writeln!(out, "\nrecording {}\n {}", self.session, self.state); + + let _ = writeln!( + out, + "\nfindings ({} shown of {}; {} recorded by an agent):", + self.top_findings.len(), + self.findings_total, + self.recorded_findings + ); + if self.top_findings.is_empty() { + let _ = writeln!(out, " none: no detector fired"); + } + for finding in &self.top_findings { + let severity = serde_json::to_value(finding.severity) + .ok() + .and_then(|value| value.as_str().map(str::to_string)) + .unwrap_or_default(); + let _ = writeln!( + out, + " {} {severity} {} at {}: {} [{}]", + finding.id, + finding.kind, + timecode::format(finding.at_video_s), + finding.summary, + finding.refs.join(" ") + ); + } + + let _ = writeln!(out, "\nnext:"); + for command in &self.next { + let _ = writeln!(out, " {command}"); + } + out + } +} diff --git a/komorebi-recorder/src/analysis/findings.rs b/komorebi-recorder/src/analysis/findings.rs new file mode 100644 index 00000000..29d1cd86 --- /dev/null +++ b/komorebi-recorder/src/analysis/findings.rs @@ -0,0 +1,323 @@ +//! Every finding in one list: what the detectors found and what an agent concluded. +//! +//! `overview` ranks what the detectors found, and `annotations --kind finding` lists what an +//! agent wrote down. [`Analysis::merged_findings`] is both together, which is what the `findings` +//! command prints: an agent coming back to a session sees its own conclusions next to what the +//! detectors say, and can filter by kind and severity. +//! +//! # Stable ids +//! +//! A detector finding has no id of its own, so [`Finding::stable_id`] makes one: `f-` and six hex +//! digits of an FNV-1a hash of the finding's `kind`, its first reference (`event:12`, `log:345`, +//! `incident:2`) and the ids of the windows it is about (so that two windows wrong at the same event +//! are two ids); a finding with no reference uses the five-second bucket of video time it began in +//! instead of the reference. The hash is fixed (it does not change between runs, machines +//! or versions of the recorder), and it does not depend on the score, the summary or the bug +//! regions, so the same finding keeps its id when a bug region is added and the ranking changes. +//! It is the same for the same session however many times it is analysed. Two *different* +//! sessions share an id only where the same kind of finding is at the same reference, which for +//! numbered references is by coincidence of numbering: ids compare findings across analyses of one +//! session (before and after a fix to the detectors, say), and are not a fingerprint of a bug across +//! recordings. Agent findings keep the annotation id (`a-3f9c1e02`) they were given. + +use super::Analysis; +use super::invariants::Severity; +use super::overview::Finding; +use crate::annotations::Kind; +use color_eyre::eyre; +use schemars::JsonSchema; +use serde::Serialize; + +/// The `schema` of the `findings` command's output: the name [`FindingsData`] has in an envelope. +pub const SCHEMA_FINDINGS: &str = "findings/1"; + +/// The `kind` of a finding an agent wrote down with `annotate add --kind finding`. +pub const AGENT_KIND: &str = "agent-finding"; + +/// Who a finding is from. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, JsonSchema)] +#[serde(rename_all = "kebab-case")] +pub enum FindingSource { + /// Found by one of the detectors (see `overview`). + Detector, + /// Written down by an agent or a person with `annotate add --kind finding`. + Agent, +} + +/// One finding, from either source. +#[derive(Debug, Clone, PartialEq, Serialize, JsonSchema)] +pub struct FindingItem { + /// `f-xxxxxx` for a detector finding (see [`Finding::stable_id`]), the annotation id for an + /// agent's. + pub id: String, + /// Where it came from. + pub source: FindingSource, + /// The detector's kind (`invariant:`, `log-error`, ...), or `agent-finding`. + pub kind: String, + /// How serious the detector thinks it is. `None` for an agent's finding, which is a conclusion + /// and has no severity. + pub severity: Option, + /// The detector's ranking score (see `overview`); `None` for an agent's finding. + pub score: Option, + /// What was seen (a detector), or the conclusion (an agent). + pub summary: String, + /// When it began, in seconds of video time. + pub start_video_s: f64, + /// When it ended, in seconds of video time. + pub end_video_s: f64, + /// Where to look: `event:12`, `log:345`, `incident:2`, `code::`, ... + pub refs: Vec, + /// The bug regions it overlaps (a detector), or the one it explains (an agent). + pub bugs: Vec, + /// Who wrote an agent's finding. + pub author: Option, + /// A command that digs into it, if there is one. + pub next: Option, +} + +/// What [`Analysis::merged_findings`] is asked for. +#[derive(Debug, Clone, Default)] +pub struct FindingsQuery { + /// Only findings of this kind: the whole `kind`, or the part before the colon (`invariant` + /// matches every `invariant:`). + pub kind: Option, + /// Only detector findings of this severity. Agent findings have none, so a severity leaves them + /// out. + pub severity: Option, + /// The most findings to return; `None` for all of them. + pub limit: Option, +} + +/// The `data` of the `findings` command. +#[derive(Debug, Clone, PartialEq, Serialize, JsonSchema)] +pub struct FindingsData { + /// Agent findings first (earliest first), then detector findings, ranked as in `overview`. + pub findings: Vec, + /// How many findings matched the filters, including any cut off by the limit. + pub total: usize, + /// How many of those were written by an agent. + pub agent: usize, + /// How many of those were found by a detector. + pub detector: usize, +} + +/// FNV-1a over `bytes`: fixed forever, unlike the standard library's hashers. +fn fnv1a(bytes: &[u8]) -> u64 { + bytes.iter().fold(0xcbf2_9ce4_8422_2325, |hash, byte| { + (hash ^ u64::from(*byte)).wrapping_mul(0x0000_0100_0000_01b3) + }) +} + +impl Finding { + /// Returns the finding's stable id, `f-` and six hex digits. + /// + /// It is made from the kind, the first reference (or, for a finding with none, the five-second + /// bucket of video time it began in) and the windows it is about; see the [module](self) for what + /// that does and does not promise. + /// + /// # Example + /// + /// ```rust + /// use komorebi_recorder::analysis::Analysis; + /// use komorebi_recorder::testkit::SessionBuilder; + /// + /// let (session, _) = SessionBuilder::new("doc-stable-id") + /// .log(3.0, "ERROR", "komorebi::window: could not do the thing") + /// .build(); + /// let found = Analysis::open(&session.dir)?.findings()?; + /// + /// let id = found[0].stable_id(); + /// assert!(id.starts_with("f-") && id.len() == 8); + /// // the same finding, the same id + /// assert_eq!(id, found[0].clone().stable_id()); + /// # Ok::<(), Box>(()) + /// ``` + #[must_use] + pub fn stable_id(&self) -> String { + let place = self.refs.first().map_or_else( + || format!("t{}", (self.start_video_s.max(0.0) / 5.0).floor() as u64), + Clone::clone, + ); + let mut windows = self.window_ids.clone(); + windows.sort_unstable(); + let hash = fnv1a(format!("{}\0{place}\0{windows:?}", self.kind).as_bytes()); + format!("f-{:06x}", hash & 0x00ff_ffff) + } +} + +impl Analysis { + /// Returns the detectors' findings and the agent's, together, filtered by `query`. + /// + /// The detector findings are those of [`Analysis::findings`], ranked against the bug regions by + /// [`Analysis::rank`]; the agent's are the `finding` annotations. Agent findings come first. + /// + /// # Errors + /// + /// Returns an error if the annotations cannot be read, or a recorded state cannot (see + /// [`Analysis::findings`]). + /// + /// # Example + /// + /// ```rust + /// use komorebi_recorder::analysis::Analysis; + /// use komorebi_recorder::analysis::findings::FindingsQuery; + /// use komorebi_recorder::testkit::SessionBuilder; + /// + /// let (session, _) = SessionBuilder::new("doc-merged-findings") + /// .log(3.0, "ERROR", "komorebi::window: could not do the thing") + /// .build(); + /// let data = Analysis::open(&session.dir)?.merged_findings(&FindingsQuery::default())?; + /// assert_eq!(data.detector, 1); + /// assert_eq!(data.findings[0].kind, "log-error"); + /// # Ok::<(), Box>(()) + /// ``` + pub fn merged_findings(&self, query: &FindingsQuery) -> eyre::Result { + let bugs = self.bugs()?; + let bug_refs: Vec<&crate::annotations::Annotation> = bugs.iter().collect(); + let ranked = self.rank(self.findings()?, &bug_refs); + let loaded = crate::annotations::load(&self.session.dir)?; + + let mut items: Vec = loaded + .annotations + .iter() + .filter(|annotation| annotation.kind == Kind::Finding) + .map(|annotation| { + let view = annotation.view(&self.manifest); + FindingItem { + id: view.id, + source: FindingSource::Agent, + kind: AGENT_KIND.to_string(), + severity: None, + score: None, + summary: view.text, + start_video_s: view.start_video_s, + end_video_s: view.end_video_s, + refs: view.refs.iter().map(ToString::to_string).collect(), + bugs: view.about.into_iter().collect(), + author: Some(view.author), + next: None, + } + }) + .collect(); + items.sort_by(|a, b| a.start_video_s.total_cmp(&b.start_video_s)); + + items.extend(ranked.into_iter().map(|finding| FindingItem { + id: finding.stable_id(), + source: FindingSource::Detector, + kind: finding.kind, + severity: Some(finding.severity), + score: Some(finding.score), + summary: finding.summary, + start_video_s: finding.start_video_s, + end_video_s: finding.end_video_s, + refs: finding.refs, + bugs: finding.bugs, + author: None, + next: finding.next, + })); + + items.retain(|item| { + query.kind.as_deref().is_none_or(|kind| { + item.kind == kind + || item + .kind + .strip_prefix(kind) + .is_some_and(|rest| rest.starts_with(':')) + }) && query + .severity + .is_none_or(|severity| item.severity == Some(severity)) + }); + + let total = items.len(); + let agent = items + .iter() + .filter(|item| item.source == FindingSource::Agent) + .count(); + items.truncate(query.limit.unwrap_or(usize::MAX)); + + Ok(FindingsData { + findings: items, + total, + agent, + detector: total - agent, + }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::analysis::invariants::Severity; + + fn finding(kind: &str, refs: &[&str], start_video_s: f64) -> Finding { + Finding { + kind: kind.to_string(), + severity: Severity::Warn, + score: 0, + summary: String::new(), + start_video_s, + end_video_s: start_video_s, + start_mach_ns: 0, + end_mach_ns: 0, + refs: refs.iter().map(ToString::to_string).collect(), + window_ids: Vec::new(), + persisted_to_end: false, + bugs: Vec::new(), + next: None, + } + } + + #[test] + fn the_id_is_fixed_and_follows_the_kind_and_first_reference() { + let a = finding("log-error", &["log:12"], 3.0); + // pinned: the hash must not change between versions + assert_eq!( + a.stable_id(), + finding("log-error", &["log:12"], 9.0).stable_id() + ); + assert_ne!( + a.stable_id(), + finding("log-error", &["log:13"], 3.0).stable_id() + ); + assert_ne!( + a.stable_id(), + finding("log-warning", &["log:12"], 3.0).stable_id() + ); + // the score and the summary do not matter + let mut ranked = a.clone(); + ranked.score = 1300; + ranked.summary = "different".to_string(); + assert_eq!(a.stable_id(), ranked.stable_id()); + assert_eq!(a.stable_id().len(), 8); + assert_eq!(fnv1a(b"a"), 0xaf63_dc4c_8601_ec8c); + } + + #[test] + fn two_windows_wrong_at_the_same_event_are_two_ids() { + let mut one = finding("invariant:layout-mismatch", &["event:4"], 3.0); + one.window_ids = vec![7]; + let mut other = one.clone(); + other.window_ids = vec![9]; + assert_ne!(one.stable_id(), other.stable_id()); + + // the order the windows are listed in does not matter + let mut both = one.clone(); + both.window_ids = vec![9, 7]; + let mut reversed = one.clone(); + reversed.window_ids = vec![7, 9]; + assert_eq!(both.stable_id(), reversed.stable_id()); + } + + #[test] + fn a_finding_with_no_reference_uses_a_five_second_bucket() { + let early = finding("event-storm", &[], 1.0); + assert_eq!( + early.stable_id(), + finding("event-storm", &[], 4.9).stable_id() + ); + assert_ne!( + early.stable_id(), + finding("event-storm", &[], 5.1).stable_id() + ); + } +} diff --git a/komorebi-recorder/src/analysis/mod.rs b/komorebi-recorder/src/analysis/mod.rs index 5ef29200..65a81757 100644 --- a/komorebi-recorder/src/analysis/mod.rs +++ b/komorebi-recorder/src/analysis/mod.rs @@ -16,11 +16,13 @@ pub mod catalog; pub mod changes; pub mod classify; pub mod code; +pub mod compact; pub mod compare; pub mod diff; pub mod envelope; pub mod events; pub mod facade; +pub mod findings; pub mod flavor; pub mod framediff; pub mod ghost; diff --git a/komorebi-recorder/src/analysis/schemas.rs b/komorebi-recorder/src/analysis/schemas.rs index f2d5e344..0d1db284 100644 --- a/komorebi-recorder/src/analysis/schemas.rs +++ b/komorebi-recorder/src/analysis/schemas.rs @@ -159,6 +159,36 @@ pub const ALL: &[Entry] = &[ about: "`doctor --probe`: what one measurement found", schema: of::, }, + Entry { + name: "brief-compact/1", + about: "`brief --compact`: the brief cut down to the user's words, the top findings and what to run next", + schema: of::, + }, + Entry { + name: "findings/1", + about: "`findings`: detector findings (with stable ids) and an agent's findings, together", + schema: of::, + }, + Entry { + name: "record/1", + about: "`record --json`: the finished session, its displays and frames, warnings and the --until-exit command's status", + schema: of::, + }, + Entry { + name: "displays/1", + about: "`displays --json`: the displays that can be recorded", + schema: of::, + }, + Entry { + name: "reindex/1", + about: "`reindex --json`: what was recovered and how many records the index has", + schema: of::, + }, + Entry { + name: "selftest/1", + about: "`selftest --json`: the video-to-clock readings of a passing self-test", + schema: of::, + }, Entry { name: "error/1", about: "any failed command: the envelope with `ok: false` and an `error`", diff --git a/komorebi-recorder/src/cli/agent.rs b/komorebi-recorder/src/cli/agent.rs index b2498175..c42480b4 100644 --- a/komorebi-recorder/src/cli/agent.rs +++ b/komorebi-recorder/src/cli/agent.rs @@ -41,6 +41,10 @@ pub const GUIDE: &str = include_str!("../guide.md"); pub struct BriefArgs { /// Session directory pub session: PathBuf, + /// A trimmed brief for a small context window (about 40 lines): the user's words, a line on the + /// state, the top 3 findings with their references, and the commands to run next + #[arg(long)] + pub compact: bool, /// Machine-readable JSON output #[arg(long)] pub json: bool, @@ -114,6 +118,20 @@ pub(super) fn brief(args: &BriefArgs) -> eyre::Result<()> { let analysis = Analysis::open(&args.session)?; let data = analysis.brief()?; + if args.compact { + let mut warnings = analysis.warnings(); + warnings.extend(data.skew.warning.clone()); + let compact = data.compact(); + if args.json { + return print_json( + &Envelope::ok(crate::analysis::compact::SCHEMA_BRIEF_COMPACT, compact) + .warnings(warnings), + ); + } + print!("{}", compact.text()); + return Ok(()); + } + if args.json { let mut warnings = analysis.warnings(); warnings.extend(data.skew.warning.clone()); diff --git a/komorebi-recorder/src/cli/annotate.rs b/komorebi-recorder/src/cli/annotate.rs index 79432fd2..b66e640f 100644 --- a/komorebi-recorder/src/cli/annotate.rs +++ b/komorebi-recorder/src/cli/annotate.rs @@ -128,7 +128,9 @@ pub struct AddArgs { #[arg(long, value_name = "ID")] pub about: Option, /// Evidence (repeatable; a finding needs at least one): event:, log:, - /// log::, code::, frame: