feat: full Copilot SDK feature parity — custom agents, hooks, image input, skills, steering, session persistence

Backend (sidecar):
- Extended ProtocolModels with DTOs for custom agents, hooks, skills,
  infinite sessions, session lifecycle, and 9 new event types
- Added CopilotManagedSessionIds for stable SDK session ID mapping
- Added CopilotSessionManager/ICopilotSessionManager for session lifecycle
- Added CopilotSessionHooks for hook registration
- Added CopilotMessageOptionsMetadata for mid-turn steering
- Extended CopilotAgentBundle to wire custom agents, hooks, skills,
  infinite sessions, and stable session IDs
- Extended CopilotTurnExecutionState to project 13 new SDK event types
- Widened ITurnWorkflowRunner callback to accept SidecarEventDto
- Added list/delete/disconnect session commands to SidecarProtocolHost
- Added AryxCopilotAgentMessageOptionsTests (14 new tests, 142 total)

Frontend (renderer + main + shared):
- Added ChatMessageAttachment type and helpers (attachment.ts)
- Extended sidecar contracts with MessageMode, 3 new command types,
  9 new event types, and agent/session config DTOs
- Extended SessionEventRecord with 6 new event kinds and ~20 fields
- Added PatternAgentCopilotConfig to pattern domain
- Added attachments support to ChatMessageRecord
- Updated sidecar client with session lifecycle methods and
  turn-scoped event routing via onTurnScopedEvent callback
- Updated main process: handleTurnScopedEvent(), deleteSession(),
  steering bypass for mid-turn messages, attachment passthrough
- Added deleteSession IPC handler and preload binding
- Added TurnEventLog state tracker with format/apply/prune helpers
- ChatPane: always-enabled composer, steering indicator, attachment
  picker with preview, image thumbnails in message history,
  context-usage bar, amber steer mode for send button
- ActivityPanel: turn events section with sub-agent, hook, skill,
  and compaction event rendering
- Sidebar: delete session action in context menu
- App.tsx: wired sessionUsage, turnEventLogs, and deleteSession

Documentation:
- AGENTS.md: added glob safety rule for node_modules
- README.md: added steering, image input, and richer observability
- ARCHITECTURE.md: added turn-scoped events, steering, and attachments
- Website: added steering and image input feature cards, updated
  live visibility and session cards

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
David Kaya
2026-03-28 12:28:20 +01:00
co-authored by Copilot
parent 0c2973c599
commit f1fa52f9c3
40 changed files with 2515 additions and 92 deletions
+20
View File
@@ -202,6 +202,17 @@ This is a structured stdio protocol used for:
This protocol boundary keeps the AI execution runtime replaceable and prevents the Electron main process from becoming overloaded with workflow-specific behavior.
The protocol also carries **turn-scoped lifecycle events** alongside output deltas. These events let the UI visualize execution internals without the main process having to interpret AI workflow semantics:
- **Sub-agent events**: started, completed, failed, selected, deselected — surfaced when custom agents are defined
- **Skill invocation events**: emitted when an agent-side skill is triggered
- **Hook lifecycle events**: start and end of registered hooks (e.g. pre-turn, post-turn)
- **Session compaction events**: start and complete, with token-reduction metrics when infinite sessions trigger context trimming
- **Session usage events**: current token count and context-window limit for usage-bar rendering
- **Pending-messages-modified events**: emitted when mid-turn steering changes the pending message queue
These events flow through a single `onTurnScopedEvent` callback on the `runTurn` command, avoiding per-event-type callback proliferation. The main process maps each event to a `SessionEventRecord` and pushes it to the renderer, where lightweight state maps (activity, usage, turn-event log) consume them without touching the persisted workspace.
## Security model
Security in this system is mostly about **desktop trust boundaries**.
@@ -273,11 +284,20 @@ The architecture treats execution as observable by design:
- partial output is streamed
- agent activity is surfaced
- turn-scoped lifecycle events (sub-agent, hook, skill, compaction, usage) are streamed
- runs are persisted as timeline history
- failures are represented explicitly
This improves trust and debuggability, especially for multi-agent workflows.
### Mid-turn steering
Aryx supports sending user messages while a turn is actively running. These messages are delivered with a `messageMode` flag (`immediate` or `enqueue`) that tells the sidecar to inject the content into the current Copilot session rather than starting a new turn. This enables real-time steering without waiting for turn completion. The main process allows the IPC call even when the session is in `running` status, and the renderer keeps the composer enabled throughout.
### Image attachments
User messages can carry image attachments as base64-encoded blobs. These flow from the renderer through IPC, the main process, and the sidecar protocol as `ChatMessageAttachmentDto` objects alongside the text content. The sidecar maps them into the Copilot SDK's `DataPart` model. Attachment metadata is persisted on the `ChatMessageRecord` so thumbnail previews render correctly when revisiting a session.
## Persistence and repair
Workspace persistence is intentionally simple: the app stores a durable workspace document and repairs or normalizes it when loading.