starred/aryx
1
0
Fork 0
mirror of https://github.com/davidkaya/aryx.git synced 2026-08-27 13:23:57 +02:00
Code Issues 3 Packages Projects Releases 10 Wiki Activity

docs: refresh roadmap completion status

Browse Source 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
David Kaya
2026-03-23 20:01:16 +01:00
co-authored by Copilot
parent 32a6ecfb9a
commit 8d140b6972
1 changed files with 167 additions and 165 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ROADMAP.md
+167 -165
View File
@@ -19,16 +19,16 @@ Modern AI chat apps set a clear baseline for user expectations: projects, memory
Based on the current codebase, Kopaya already has: Based on the current codebase, Kopaya already has:
- persistent local workspace state for projects, patterns, and sessions in `src/main/persistence/workspaceRepository.ts` - [x] persistent local workspace state for projects, patterns, and sessions in `src/main/persistence/workspaceRepository.ts`
- built-in orchestration modes in `src/shared/domain/pattern.ts`: `single`, `sequential`, `concurrent`, `handoff`, `group-chat`, with `magentic` reserved for future support - [x] built-in orchestration modes in `src/shared/domain/pattern.ts`: `single`, `sequential`, `concurrent`, `handoff`, `group-chat`, with `magentic` reserved for future support
- a dynamic model catalog with provider metadata and reasoning-effort support in `src/shared/domain/models.ts` - [x] a dynamic model catalog with provider metadata and reasoning-effort support in `src/shared/domain/models.ts`
- scratchpad-specific in-chat model overrides in `src/main/KopayaAppService.ts` - [x] scratchpad-specific in-chat model overrides in `src/main/KopayaAppService.ts`
- real-time turn streaming and agent activity events in `src/shared/contracts/sidecar.ts`, `src/shared/domain/event.ts`, and `sidecar/src/Kopaya.AgentHost/Services/CopilotWorkflowRunner.cs` - [x] real-time turn streaming and agent activity events in `src/shared/contracts/sidecar.ts`, `src/shared/domain/event.ts`, and `sidecar/src/Kopaya.AgentHost/Services/CopilotWorkflowRunner.cs`
- a right-side activity panel that already surfaces per-agent state, model, and effort in `src/renderer/components/ActivityPanel.tsx` - [x] a right-side activity panel that already surfaces per-agent state, model, and effort in `src/renderer/components/ActivityPanel.tsx`
- a pattern editor and settings flow in `src/renderer/components/SettingsPanel.tsx` - [x] a pattern editor and settings flow in `src/renderer/components/SettingsPanel.tsx`
- Copilot CLI-backed runtime access via the system-installed `copilot` command, with Kopaya sanitizing inherited runtime env vars before spawning the sidecar - [x] Copilot CLI-backed runtime access via the system-installed `copilot` command, with Kopaya sanitizing inherited runtime env vars before spawning the sidecar
- refreshable Copilot connection diagnostics and settings UI for `ready`, `copilot-cli-missing`, `copilot-auth-required`, and `copilot-error` states in `src/renderer/components/CopilotStatusCard.tsx`, `src/renderer/components/SettingsPanel.tsx`, `src/main/KopayaAppService.ts`, and `sidecar/src/Kopaya.AgentHost/Services/SidecarProtocolHost.cs` - [x] refreshable Copilot connection diagnostics and settings UI for `ready`, `copilot-cli-missing`, `copilot-auth-required`, and `copilot-error` states in `src/renderer/components/CopilotStatusCard.tsx`, `src/renderer/components/SettingsPanel.tsx`, `src/main/KopayaAppService.ts`, and `sidecar/src/Kopaya.AgentHost/Services/SidecarProtocolHost.cs`
- an OS secret store wrapper in `src/main/secrets/secretStore.ts` that can support future non-Copilot secrets and integrations - [x] an OS secret store wrapper in `src/main/secrets/secretStore.ts` that can support future non-Copilot secrets and integrations
That is a strong base. The biggest gaps are not around "can it run agents?" but around: That is a strong base. The biggest gaps are not around "can it run agents?" but around:
@@ -61,14 +61,14 @@ That is a strong base. The biggest gaps are not around "can it run agents?" but
These are the improvements users will expect from any serious AI desktop app. These are the improvements users will expect from any serious AI desktop app.
| Priority | Initiative | Why users need it | Likely layers | | Priority | Initiative | Why users need it | Likely layers |
| --- | --- | --- | --- | | -------- | ------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------- |
| Highest | Copilot connection and account status management | Users need a clear way to see whether Copilot is installed, authenticated, healthy, and able to serve the expected models. | Renderer, main, sidecar | | Highest | Copilot connection and account status management | Users need a clear way to see whether Copilot is installed, authenticated, healthy, and able to serve the expected models. | Renderer, main, sidecar |
| Highest | Conversation organization and search | Users need to find old work quickly, pin important threads, archive noise, and search by project, title, agent, and content. | Renderer, persistence | | Highest | Conversation organization and search | Users need to find old work quickly, pin important threads, archive noise, and search by project, title, agent, and content. | Renderer, persistence |
| Highest | Session export and sharing | Users will want to export runs to Markdown/JSON/PDF, share patterns, and preserve outcomes outside the app. | Renderer, main, persistence | | Highest | Session export and sharing | Users will want to export runs to Markdown/JSON/PDF, share patterns, and preserve outcomes outside the app. | Renderer, main, persistence |
| High | Attachments and artifact handling | Modern chat apps let users drop files into a thread and keep generated artifacts nearby. This is table stakes for research and coding workflows. | Renderer, main, sidecar | | High | Attachments and artifact handling | Modern chat apps let users drop files into a thread and keep generated artifacts nearby. This is table stakes for research and coding workflows. | Renderer, main, sidecar |
| High | Chat branching and session forking | Users need to explore alternate solution paths without losing the original conversation. | Renderer, persistence | | High | Chat branching and session forking | Users need to explore alternate solution paths without losing the original conversation. | Renderer, persistence |
| High | Better error and diagnostics UX | Sidecar/runtime issues need clear explanation, retry actions, and debug details instead of vague failure states. | Renderer, main, sidecar | | High | Better error and diagnostics UX | Sidecar/runtime issues need clear explanation, retry actions, and debug details instead of vague failure states. | Renderer, main, sidecar |
### What this should look like ### What this should look like
@@ -97,59 +97,61 @@ If Kopaya later adds direct OpenAI, Anthropic, Google, MCP, or team-managed secr
- [x] global search across sessions, messages, projects, and patterns - [x] global search across sessions, messages, projects, and patterns
- [x] pinned sessions, archived sessions - [x] pinned sessions, archived sessions
- [x] filters like "running", "errored", "scratchpad", "project X", "pattern Y" - [ ] filters like "running", "errored", "scratchpad", "project X", "pattern Y"
- [x] duplicate session, rename session, and favorite pattern - [x] duplicate session, rename session, and favorite pattern
- [ ] lightweight tags - [ ] lightweight tags
- [ ] recent activity views and "resume where I left off" - [ ] recent activity views and "resume where I left off"
The shared/backend query layer supports these filters, but the dedicated filter UI is currently deferred.
#### Session export and sharing #### Session export and sharing
- export a full run to Markdown - [ ] export a full run to Markdown
- export machine-readable JSON for debugging and replay - [ ] export machine-readable JSON for debugging and replay
- copy/share a clean transcript without internal activity noise - [ ] copy/share a clean transcript without internal activity noise
- export a pattern with its agents, instructions, and model selections - [ ] export a pattern with its agents, instructions, and model selections
#### Attachments and artifacts #### Attachments and artifacts
- drag-and-drop files into chat - [ ] drag-and-drop files into chat
- inline preview for code, Markdown, images, and PDFs - [ ] inline preview for code, Markdown, images, and PDFs
- artifact shelf for generated outputs - [ ] artifact shelf for generated outputs
- "open in project", "save as", and "promote to workspace artifact" - [ ] "open in project", "save as", and "promote to workspace artifact"
#### Chat branching and session forking #### Chat branching and session forking
- fork from any message - [ ] fork from any message
- compare branch A vs branch B - [ ] compare branch A vs branch B
- keep separate titles and summaries for each branch - [ ] keep separate titles and summaries for each branch
- optionally turn a fork into a new pattern experiment - [ ] optionally turn a fork into a new pattern experiment
#### Better error and diagnostics UX #### Better error and diagnostics UX
- collapsible run diagnostics panel - [ ] collapsible run diagnostics panel
- sidecar logs per session - [ ] sidecar logs per session
- retry failed turn - [ ] retry failed turn
- "why did this fail?" summaries - [ ] "why did this fail?" summaries
- copyable debug bundle for issue reports - [ ] copyable debug bundle for issue reports
## 2. Project-aware coding improvements ## 2. Project-aware coding improvements
Kopaya should feel much smarter about the project it is attached to. Kopaya should feel much smarter about the project it is attached to.
| Priority | Initiative | Why it matters | Likely layers | | Priority | Initiative | Why it matters | Likely layers |
| --- | --- | --- | --- | | -------- | -------------------------------- | --------------------------------------------------------------------------------- | ------------------------------ |
| Highest | Project context controls | Users need to know what the agents can see and what is excluded. | Renderer, main, sidecar | | Highest | Project context controls | Users need to know what the agents can see and what is excluded. | Renderer, main, sidecar |
| High | Git-aware context | Branch, diff, dirty state, and commit context are essential for coding workflows. | Main, renderer | | High | Git-aware context | Branch, diff, dirty state, and commit context are essential for coding workflows. | Main, renderer |
| High | Workspace indexing and summaries | Large projects need a fast, understandable overview before orchestration starts. | Main, sidecar, persistence | | High | Workspace indexing and summaries | Large projects need a fast, understandable overview before orchestration starts. | Main, sidecar, persistence |
| High | File pinning and working sets | Users need to constrain attention to a selected set of files or folders. | Renderer, persistence, sidecar | | High | File pinning and working sets | Users need to constrain attention to a selected set of files or folders. | Renderer, persistence, sidecar |
| Medium | Project presets | Teams will want reusable project-level context, rules, and exclusion templates. | Persistence, renderer | | Medium | Project presets | Teams will want reusable project-level context, rules, and exclusion templates. | Persistence, renderer |
### Recommended features ### Recommended features
- show project metadata: repo name, branch, dirty state, languages, package managers, solution files - [ ] show project metadata: repo name, branch, dirty state, languages, package managers, solution files
- explicit include/exclude controls using gitignore-aware defaults plus manual overrides - [ ] explicit include/exclude controls using gitignore-aware defaults plus manual overrides
- working set support: "only reason about these files/folders" - [ ] working set support: "only reason about these files/folders"
- project summary card: architecture snapshot, detected stack, test commands, important entry points - [ ] project summary card: architecture snapshot, detected stack, test commands, important entry points
- saved session context packs, such as "frontend only", "API layer", or "build pipeline" - [ ] saved session context packs, such as "frontend only", "API layer", or "build pipeline"
This is where Kopaya can beat generic chat apps: not just talking about a project, but acting like a focused control surface for that project. This is where Kopaya can beat generic chat apps: not just talking about a project, but acting like a focused control surface for that project.
@@ -157,14 +159,14 @@ This is where Kopaya can beat generic chat apps: not just talking about a projec
This is the highest-leverage product area. If done well, it becomes Kopaya's signature advantage. This is the highest-leverage product area. If done well, it becomes Kopaya's signature advantage.
| Priority | Initiative | Why users need it | Likely layers | | Priority | Initiative | Why users need it | Likely layers |
| --- | --- | --- | --- | | -------- | ------------------------------ | ---------------------------------------------------------------------------------------- | ------------------------------ |
| Highest | Rich run timeline | Users need to see the exact sequence of thinking, handoffs, tool calls, and outputs. | Renderer, sidecar | | Highest | Rich run timeline | Users need to see the exact sequence of thinking, handoffs, tool calls, and outputs. | Renderer, sidecar |
| Highest | Replayable run traces | Users need to inspect how a result was produced, not just read the final answer. | Sidecar, persistence, renderer | | Highest | Replayable run traces | Users need to inspect how a result was produced, not just read the final answer. | Sidecar, persistence, renderer |
| High | Pattern versioning | Users need to know which pattern version produced which session. | Persistence, renderer | | High | Pattern versioning | Users need to know which pattern version produced which session. | Persistence, renderer |
| High | Run comparison lab | Users need to compare models, patterns, prompts, or reasoning settings side by side. | Renderer, persistence, sidecar | | High | Run comparison lab | Users need to compare models, patterns, prompts, or reasoning settings side by side. | Renderer, persistence, sidecar |
| High | Guardrails and policy controls | Users need caps for cost, tools, runtime, file access, and escalation behavior. | Renderer, sidecar, persistence | | High | Guardrails and policy controls | Users need caps for cost, tools, runtime, file access, and escalation behavior. | Renderer, sidecar, persistence |
| High | Approval checkpoints | Users need pause-and-approve steps before risky tool use, file writes, or final actions. | Renderer, sidecar | | High | Approval checkpoints | Users need pause-and-approve steps before risky tool use, file writes, or final actions. | Renderer, sidecar |
### Rich run timeline ### Rich run timeline
@@ -172,75 +174,75 @@ The current activity model already exposes `thinking`, `tool-calling`, `handoff`
Build on that with: Build on that with:
- a vertical run timeline in the side panel - [ ] a vertical run timeline in the side panel
- event cards with timestamps, agent identity, tool name, and result status - [ ] event cards with timestamps, agent identity, tool name, and result status
- per-agent run lanes - [ ] per-agent run lanes
- grouping of streaming deltas into one coherent answer step - [ ] grouping of streaming deltas into one coherent answer step
- jump-to-message and jump-to-agent actions - [ ] jump-to-message and jump-to-agent actions
### Replayable run traces ### Replayable run traces
- store a structured event log per run - [ ] store a structured event log per run
- replay a completed run step by step - [ ] replay a completed run step by step
- scrub through the run like a debugger timeline - [ ] scrub through the run like a debugger timeline
- inspect the exact sequence of agent activations and tool invocations - [ ] inspect the exact sequence of agent activations and tool invocations
- preserve environment metadata such as model, effort, project path, and pattern version - [ ] preserve environment metadata such as model, effort, project path, and pattern version
This would be a major differentiator. Most chat apps show outputs; very few make multi-agent execution truly inspectable. This would be a major differentiator. Most chat apps show outputs; very few make multi-agent execution truly inspectable.
### Pattern versioning ### Pattern versioning
- immutable versions for saved patterns - [ ] immutable versions for saved patterns
- "session used pattern v7" - [ ] "session used pattern v7"
- diff view for instructions, models, agents, and iteration counts - [ ] diff view for instructions, models, agents, and iteration counts
- rollback and duplicate-from-version - [ ] rollback and duplicate-from-version
- changelog notes for team-facing patterns - [ ] changelog notes for team-facing patterns
### Run comparison lab ### Run comparison lab
- run the same user prompt against multiple patterns or model mixes - [ ] run the same user prompt against multiple patterns or model mixes
- compare output quality, latency, handoff structure, and cost - [ ] compare output quality, latency, handoff structure, and cost
- save a winner as the new default - [ ] save a winner as the new default
- benchmark patterns against a reusable prompt set - [ ] benchmark patterns against a reusable prompt set
### Guardrails and policy controls ### Guardrails and policy controls
- max iterations per run - [ ] max iterations per run
- max tool calls per agent - [ ] max tool calls per agent
- time budget and cost budget - [ ] time budget and cost budget
- allowed tools per pattern - [ ] allowed tools per pattern
- allowed paths per project - [ ] allowed paths per project
- "tool use requires approval" mode - [ ] "tool use requires approval" mode
### Approval checkpoints ### Approval checkpoints
- pause before a tool call - [ ] pause before a tool call
- pause before handing off outside the original working set - [ ] pause before handing off outside the original working set
- pause before final answer publication - [ ] pause before final answer publication
- assign specific checkpoints to specific agents or pattern modes - [ ] assign specific checkpoints to specific agents or pattern modes
## 4. Advanced orchestration capabilities ## 4. Advanced orchestration capabilities
Once the control plane is solid, Kopaya should move from multi-agent chat to true workflow orchestration. Once the control plane is solid, Kopaya should move from multi-agent chat to true workflow orchestration.
| Priority | Initiative | Why it matters | Likely layers | | Priority | Initiative | Why it matters | Likely layers |
| --- | --- | --- | --- | | -------- | --------------------------------- | ----------------------------------------------------------------------------------------- | --------------------------------- |
| High | Planner-executor-evaluator loops | Strong default pattern for quality, validation, and self-correction. | Sidecar, pattern system, renderer | | High | Planner-executor-evaluator loops | Strong default pattern for quality, validation, and self-correction. | Sidecar, pattern system, renderer |
| High | Conditional routing and DAG flows | Real workflows need branching, retries, and conditional steps beyond today's fixed modes. | Pattern system, sidecar, renderer | | High | Conditional routing and DAG flows | Real workflows need branching, retries, and conditional steps beyond today's fixed modes. | Pattern system, sidecar, renderer |
| High | Background and long-running jobs | Users need runs that continue while they browse or switch sessions. | Main, sidecar, renderer | | High | Background and long-running jobs | Users need runs that continue while they browse or switch sessions. | Main, sidecar, renderer |
| Medium | Cross-project campaigns | Some workflows should coordinate across multiple repositories or workspaces. | Persistence, sidecar, renderer | | Medium | Cross-project campaigns | Some workflows should coordinate across multiple repositories or workspaces. | Persistence, sidecar, renderer |
| Medium | Memory layers | Users need structured memory beyond raw chat history. | Persistence, sidecar, renderer | | Medium | Memory layers | Users need structured memory beyond raw chat history. | Persistence, sidecar, renderer |
| Medium | Autonomy levels | Some runs should be advisory, some supervised, some semi-autonomous. | Renderer, sidecar, persistence | | Medium | Autonomy levels | Some runs should be advisory, some supervised, some semi-autonomous. | Renderer, sidecar, persistence |
| Backlog | Magentic mode support | Already reserved in the domain model and should activate when the runtime supports it. | Sidecar, shared domain, renderer | | Backlog | Magentic mode support | Already reserved in the domain model and should activate when the runtime supports it. | Sidecar, shared domain, renderer |
### Planner-executor-evaluator loops ### Planner-executor-evaluator loops
Add first-class support for patterns like: Add first-class support for patterns like:
- planner -> implementer -> reviewer - [ ] planner -> implementer -> reviewer
- researcher -> synthesizer -> critic - [ ] researcher -> synthesizer -> critic
- triage -> specialist -> verifier - [ ] triage -> specialist -> verifier
- generator -> judge -> repair loop until threshold - [ ] generator -> judge -> repair loop until threshold
This should be more than custom instructions. It should be a product concept with templates, visibility, and metrics. This should be more than custom instructions. It should be a product concept with templates, visibility, and metrics.
@@ -248,39 +250,39 @@ This should be more than custom instructions. It should be a product concept wit
Move beyond a fixed list of orchestration modes and introduce: Move beyond a fixed list of orchestration modes and introduce:
- conditional edges - [ ] conditional edges
- retries on low confidence - [ ] retries on low confidence
- fallback agent paths - [ ] fallback agent paths
- multi-branch flows that rejoin - [ ] multi-branch flows that rejoin
- "if tool X returns Y, route to specialist Z" - [ ] "if tool X returns Y, route to specialist Z"
A node-and-edge designer would make this much easier to understand than a purely form-based editor. A node-and-edge designer would make this much easier to understand than a purely form-based editor.
### Background and long-running jobs ### Background and long-running jobs
- queue runs for later - [ ] queue runs for later
- continue in the background while the user works elsewhere - [ ] continue in the background while the user works elsewhere
- desktop notifications when a run reaches a checkpoint or fails - [ ] desktop notifications when a run reaches a checkpoint or fails
- background research, repo audits, doc generation, or code review workflows - [ ] background research, repo audits, doc generation, or code review workflows
### Cross-project campaigns ### Cross-project campaigns
Examples: Examples:
- audit the same policy across multiple repositories - [ ] audit the same policy across multiple repositories
- generate migration plans across a workspace portfolio - [ ] generate migration plans across a workspace portfolio
- compare implementation patterns across projects - [ ] compare implementation patterns across projects
- run one planner over many project-specific executor sessions - [ ] run one planner over many project-specific executor sessions
### Memory layers ### Memory layers
Kopaya should eventually distinguish between: Kopaya should eventually distinguish between:
- session memory: just this thread - [ ] session memory: just this thread
- project memory: facts about a specific repository - [ ] project memory: facts about a specific repository
- pattern memory: lessons or defaults attached to a workflow - [ ] pattern memory: lessons or defaults attached to a workflow
- user preferences: tone, depth, risk tolerance, approval defaults - [ ] user preferences: tone, depth, risk tolerance, approval defaults
- team memory: shared conventions and approved instructions - [ ] team memory: shared conventions and approved instructions
Users should be able to inspect, edit, clear, and scope each memory layer. Users should be able to inspect, edit, clear, and scope each memory layer.
@@ -288,10 +290,10 @@ Users should be able to inspect, edit, clear, and scope each memory layer.
Introduce explicit run modes such as: Introduce explicit run modes such as:
- advisory only - [ ] advisory only
- supervised execution - [ ] supervised execution
- auto-run within guardrails - [ ] auto-run within guardrails
- background delegated task - [ ] background delegated task
This gives users a clearer mental model than burying autonomy inside pattern instructions. This gives users a clearer mental model than burying autonomy inside pattern instructions.
@@ -299,34 +301,34 @@ This gives users a clearer mental model than burying autonomy inside pattern ins
Single-user desktop value is important, but long-term adoption will benefit from team workflows. Single-user desktop value is important, but long-term adoption will benefit from team workflows.
| Priority | Initiative | Why users need it | Likely layers | | Priority | Initiative | Why users need it | Likely layers |
| --- | --- | --- | --- | | -------- | ---------------------------------- | -------------------------------------------------------------- | ----------------------------------- |
| High | Pattern import/export and registry | Teams need to share proven workflows. | Persistence, renderer | | High | Pattern import/export and registry | Teams need to share proven workflows. | Persistence, renderer |
| High | Shared run reports | Users need a clean way to send outcomes to teammates. | Renderer, persistence | | High | Shared run reports | Users need a clean way to send outcomes to teammates. | Renderer, persistence |
| Medium | Team workspaces | Shared projects, pattern libraries, and session visibility. | Persistence, backend services | | Medium | Team workspaces | Shared projects, pattern libraries, and session visibility. | Persistence, backend services |
| Medium | Comments and annotations | Humans need to discuss runs and approve or reject outputs. | Renderer, persistence | | Medium | Comments and annotations | Humans need to discuss runs and approve or reject outputs. | Renderer, persistence |
| Medium | Audit logs and secret governance | Important for enterprise or regulated use. | Main, persistence, backend services | | Medium | Audit logs and secret governance | Important for enterprise or regulated use. | Main, persistence, backend services |
| Medium | Roles and permissions | Useful once teams share patterns, credentials, and automation. | Backend services, renderer | | Medium | Roles and permissions | Useful once teams share patterns, credentials, and automation. | Backend services, renderer |
### Practical collaboration features ### Practical collaboration features
- comment on a pattern version - [ ] comment on a pattern version
- share a run summary instead of a raw transcript - [ ] share a run summary instead of a raw transcript
- mark a pattern as approved, experimental, or deprecated - [ ] mark a pattern as approved, experimental, or deprecated
- create a pattern library with tags like "coding", "docs", "triage", "research" - [ ] create a pattern library with tags like "coding", "docs", "triage", "research"
- import/export pattern bundles - [ ] import/export pattern bundles
## 6. Evaluation and continuous improvement ## 6. Evaluation and continuous improvement
If Kopaya is going to orchestrate important work, it needs a way to measure quality. If Kopaya is going to orchestrate important work, it needs a way to measure quality.
| Priority | Initiative | Why it matters | Likely layers | | Priority | Initiative | Why it matters | Likely layers |
| --- | --- | --- | --- | | -------- | -------------------------------- | -------------------------------------------------------------------- | ------------------------------ |
| High | Prompt and pattern eval suites | Users need a repeatable way to see if a pattern got better or worse. | Persistence, sidecar, renderer | | High | Prompt and pattern eval suites | Users need a repeatable way to see if a pattern got better or worse. | Persistence, sidecar, renderer |
| High | Regression testing for workflows | Teams need confidence before updating a widely used pattern. | Sidecar, persistence | | High | Regression testing for workflows | Teams need confidence before updating a widely used pattern. | Sidecar, persistence |
| Medium | Run quality scoring | Helpful for ranking candidate outputs and routing retries. | Sidecar, renderer | | Medium | Run quality scoring | Helpful for ranking candidate outputs and routing retries. | Sidecar, renderer |
| Medium | Cost/latency analytics | Users need to understand trade-offs between quality and speed. | Sidecar, persistence, renderer | | Medium | Cost/latency analytics | Users need to understand trade-offs between quality and speed. | Sidecar, persistence, renderer |
| Medium | Golden datasets for coding tasks | Useful for tuning workflows for a repository or team. | Persistence, tooling | | Medium | Golden datasets for coding tasks | Useful for tuning workflows for a repository or team. | Persistence, tooling |
### What this could unlock ### What this could unlock
@@ -345,12 +347,12 @@ These are the ideas with the best chance of making Kopaya feel distinct rather t
A proper debugger for AI runs: A proper debugger for AI runs:
- event timeline - [ ] event timeline
- step replay - [ ] step replay
- handoff graph - [ ] handoff graph
- tool call trace - [ ] tool call trace
- per-agent output inspection - [ ] per-agent output inspection
- final answer provenance - [ ] final answer provenance
This would make complex agent runs understandable in a way most products do not. This would make complex agent runs understandable in a way most products do not.
@@ -358,10 +360,10 @@ This would make complex agent runs understandable in a way most products do not.
Let a user turn an ad hoc scratchpad conversation into a reusable orchestration pattern: Let a user turn an ad hoc scratchpad conversation into a reusable orchestration pattern:
- detect the roles that emerged - [ ] detect the roles that emerged
- suggest agent breakdowns - [ ] suggest agent breakdowns
- convert a successful chat into a draft workflow - [ ] convert a successful chat into a draft workflow
- save the resulting pattern back into the library - [ ] save the resulting pattern back into the library
This would connect casual use and power-user workflow design. This would connect casual use and power-user workflow design.
@@ -369,11 +371,11 @@ This would connect casual use and power-user workflow design.
Instead of just comparing model outputs, compare: Instead of just comparing model outputs, compare:
- one agent vs multi-agent - [ ] one agent vs multi-agent
- sequential vs concurrent - [ ] sequential vs concurrent
- GPT-heavy vs Claude-heavy - [ ] GPT-heavy vs Claude-heavy
- high-effort vs medium-effort - [ ] high-effort vs medium-effort
- guarded vs unguarded runs - [ ] guarded vs unguarded runs
Kopaya should become the easiest place to answer, "which setup is actually better for this task?" Kopaya should become the easiest place to answer, "which setup is actually better for this task?"
@@ -381,21 +383,21 @@ Kopaya should become the easiest place to answer, "which setup is actually bette
Most agent tools either automate too much or stop at chat. Kopaya can own the middle ground: Most agent tools either automate too much or stop at chat. Kopaya can own the middle ground:
- route to human review at key points - [ ] route to human review at key points
- require approval before tool execution or publishing - [ ] require approval before tool execution or publishing
- allow humans to override, edit, or redirect handoffs - [ ] allow humans to override, edit, or redirect handoffs
### 5. Reproducible run snapshots ### 5. Reproducible run snapshots
Every important run should be reproducible with a snapshot of: Every important run should be reproducible with a snapshot of:
- project path or repo revision - [ ] project path or repo revision
- pattern version - [ ] pattern version
- agent list - [ ] agent list
- model selection - [ ] model selection
- reasoning effort - [ ] reasoning effort
- tool permissions - [ ] tool permissions
- event trace - [ ] event trace
This is especially valuable for engineering and enterprise use cases. This is especially valuable for engineering and enterprise use cases.