Compare commits

..
Author SHA1 Message Date
Gregory Schier 646f5a09b0 fix(models): let a snapshot race resolve to the version that won
Two sends of the same request can both miss the content-hash lookup and race to
insert. The unique index settles it; the loser wants exactly what the winner
wrote, not an error that costs its response a version link.
2026-09-05 21:24:41 -07:00
Gregory Schier e47088b507 docs: update the request versioning plan with what shipped 2026-09-05 21:23:39 -07:00
Gregory Schier 3ad6172bb1 test(send): cover the version link end to end
Drives the real HTTP pipeline against a stub executor and a real database, so
the assertion is the one the feature actually rests on: a stored send leaves a
response naming the request behind it, two sends of an unchanged request name
the same version, and editing afterwards is something the response pane can
detect.
2026-09-05 21:23:04 -07:00
Gregory Schier 7bd159b0b1 feat(client): offer the request a response was sent from
When the selected response's version no longer matches the live request, the
response header grows a "Request Changed" control with View Diff and Restore.
It follows the GraphQL editor's shape — a state-labelled dropdown button rather
than a new icon in the action row — because the label is the whole point: it
has to say *that* something is different before it offers to do anything about
it. Hidden when the two agree, and for responses recorded before versioning
existed, which have no version and never will.

The diff reuses the Git dialog's DiffViewer over YAML renderings of the two
documents, so a request diff reads like a commit diff.

Restore flushes pending edits first (the backend restores from the database, so
a debounced keystroke would otherwise land on top of it) and calls
wasUpdatedExternally afterwards, because the write came from this window and
the store's echo suppression would leave open editors showing the old content.

EditSessionTracker turns the boundaries only the frontend can see — switching
requests, losing focus, closing, falling idle for 60s — into snapshot calls. It
tracks no content of its own: content addressing already decides whether a
boundary had anything behind it, which is what keeps this a timer and two
assignments instead of a change-tracking system.
2026-09-05 21:20:20 -07:00
Gregory Schier b08b3277da feat(rpc): expose snapshot, compare and restore for request versions
Three commands, all host-independent so the browser build answers them from
the same model layer as the desktop:

- models_snapshot_request, for the edit-session boundaries only the frontend
  can see. It takes a reason and nothing else — the caller does not decide
  whether anything changed, because content addressing already has.
- models_request_version, which returns a version and the live request's
  content together so they are guaranteed comparable, plus the same
  content-hash verdict the backend uses rather than a second opinion formed in
  TypeScript.
- models_restore_request_version.

The browser host also snapshots before its own send, since its send pipeline
is in TypeScript rather than in the shared crate.

Bindings regenerated with CI's `cargo test --all --features
yaak-app-client/wry`, which also drops a stale ImportSourceResource from the
rpc-schema copy and adds a missing ImportSource to the plugins copy.
2026-09-05 21:16:29 -07:00
Gregory Schier 19a43e3785 feat(send): link every response to the request version that produced it
HTTP snapshots in resolve_send_inputs, the last point in the pipeline that
still holds the stored request — below it the request has been resolved against
its folder and workspace and then rendered, and neither is what a restore
should put back. Every host reaches sending through that function, so the
desktop, the CLI and plugin-triggered sends all get versions without each
knowing about them. gRPC and WebSocket connect do the same at their connection
upserts.

snapshot_request_for_send swallows its own errors: a send is not worth failing
over history that couldn't be written, and an ephemeral request has no id to
version. Either way the response just has no version to offer.
2026-09-05 21:05:34 -07:00
Gregory Schier 77fe1367a0 feat(models): add content-addressed request versions
One table versions HTTP, gRPC and WebSocket requests alike. A version stores
the model's editable content — the serialized model minus model/id/timestamps/
workspace/folder/sortPriority — and is addressed by the sha256 of that
document, with (model_id, content_hash) unique.

Content addressing is what makes the trigger side simple: snapshot_request can
be called by a send, a window blur and an idle timer on the same unedited
request and leave one row behind, so no caller has to reason about whether
anything changed. Dropping bookkeeping keys is what makes it stable — moving a
request between folders or re-sorting it rewrites those fields and nothing
else, and neither should mint a version or show up in a diff. The same rule
covers all three request types because they differ only in content fields.

Responses and gRPC/WebSocket connections gain a nullable version_id.

Versions are local history, so ModelVersion is deliberately absent from
AnyModel: no model-change rows, no frontend store bucket, no sync, no export.
Retention keeps anything a response points at, plus the newest 50 per request
within 30 days; request and workspace deletes cascade.
2026-09-05 21:01:14 -07:00
Gregory Schier 862fb6d65a docs: plan for request versioning 2026-09-05 20:55:59 -07:00
47 changed files with 1984 additions and 1069 deletions
Generated
-1
View File
@@ -11228,7 +11228,6 @@ dependencies = [
"md5 0.8.0",
"rusqlite",
"serde_json",
"sha2",
"tempfile",
"thiserror 2.0.17",
"tokio",
+101
View File
@@ -0,0 +1,101 @@
# Request versioning (IntelliJ Local History style)
Working plan for `feat/request-versioning`. Tracks
[save-request-data-for-response-history](https://yaak.app/feedback/posts/save-request-data-for-response-history).
Selecting an old response should be able to show, and restore, the request that produced it.
## Model
One table, `model_versions`, versions every request type:
| column | meaning |
| --- | --- |
| `id`, `model`, `created_at`, `updated_at` | usual model columns |
| `workspace_id` | owning workspace |
| `model_type` | `http_request` / `grpc_request` / `websocket_request` |
| `model_id` | the request the version belongs to |
| `content_hash` | sha256 of the canonical document |
| `document` | JSON of the request's editable content |
| `reason` | `send` / `switch` / `idle` / `restore` / `manual` |
`http_responses`, `grpc_connections` and `websocket_connections` each gain a nullable
`version_id`.
A version's `document` is the model's JSON with bookkeeping keys removed — `model`, `id`,
`createdAt`, `updatedAt`, `workspaceId`, `folderId`, `sortPriority`. One rule, applied the same
way to all three request types; the hash is taken over exactly what the document holds, so moving
a request between folders or re-sorting it never mints a version.
`(model_id, content_hash)` is unique, so dedup is the database's job rather than a code path that
can be forgotten. Sending an unchanged request ten times leaves one version and ten responses
pointing at it.
## Snapshot
One primitive, `ClientDb::snapshot_request(request, reason)`: build the document, hash it, return
the existing row for that hash or insert a new one, then prune. Everything calls it.
- **Sends.** `resolve_send_inputs` (HTTP, every host — desktop, CLI, plugin-triggered) snapshots
before the response row is created, and the resulting id rides down to the response.
gRPC and WebSocket connect paths do the same at their own connection upserts.
- **Edit-session boundaries the frontend can see**, all through one RPC: switching to another
request, window blur, app close, and a 60s idle timer after the last edit.
Over-triggering is free, so the trigger code stays dumb.
## Restore
`restore_request_version(version_id)`:
1. Snapshot the live request (reason `restore`), so anything newer than its last version is kept.
2. Merge the version's document over the live model, keeping bookkeeping fields.
3. Upsert. The written content's hash already exists, so no new version row appears.
The frontend calls `wasUpdatedExternally` afterwards so open editors reload.
## Retention
An unreferenced version survives only while it is among the newest 50 for its request *and* newer
than 30 days. A version referenced by a response lives as long as that response. Deleting a
request deletes its versions. Versions are local history: not synced to the filesystem, not in
Git, not exported.
## UI (v1, HTTP)
When the selected response's version differs from the live request, the response header grows a
state-labelled dropdown ("Request Changed", following the GraphQL editor's pattern) with **View
Diff** and **Restore**. The diff reuses the Git dialog's `DiffViewer` over YAML renderings of the
two documents. No versions timeline panel in v1; gRPC and WebSocket are wired on the backend from
day one and their UI can follow.
## Status
- [x] Migration + `ModelVersion` model + bindings
- [x] Hashing / document extraction, with tests
- [x] Queries: snapshot, prune, restore, cascade
- [x] Send pipelines: HTTP, gRPC, WebSocket (plus the browser host's own)
- [x] RPC commands + web/wasm host
- [x] Frontend: snapshot triggers, dropdown, diff dialog, restore
## Deliberately not in v1
- **No versions timeline panel.** The only entry point is a response, which is
what the feedback asked for. A "browse all versions of this request" view is a
second feature on the same data and can land later without a schema change.
- **No gRPC or WebSocket UI.** Both record versions from day one, so the history
is accumulating; only the indicator is HTTP-only.
- **No `manual` trigger.** The reason exists so that adding a "Save version now"
action later is a UI change and not a migration.
- **Folders, environments and workspaces are not versioned.** The response
timeline already records what a send inherited from them.
## Notes for later
- The version's `document` is the model minus bookkeeping keys, so a restore
merges over the live model and a field added after a version was captured
keeps its live value rather than being blanked.
- `content_hash` sorts object keys before hashing. Relying on
`serde_json::Map` being a `BTreeMap` is not safe: `preserve_order` is on in
some builds of this workspace and off in others, which is exactly the bug the
`key_order_does_not_change_the_hash` test caught.
@@ -30,6 +30,7 @@ import { EmptyStateText } from "./EmptyStateText";
import { ErrorBoundary } from "./ErrorBoundary";
import { HttpResponseTimeline } from "./HttpResponseTimeline";
import { RecentHttpResponsesDropdown } from "./RecentHttpResponsesDropdown";
import { RequestVersionDropdown } from "./RequestVersionDropdown";
import { RequestBodyViewer } from "./RequestBodyViewer";
import { ResponseCookies } from "./ResponseCookies";
import { ResponseHeaders } from "./ResponseHeaders";
@@ -263,13 +264,14 @@ export function HttpResponsePane({ style, className, activeRequestId }: Props) {
) : (
<span />
)}
<div className="justify-self-end shrink-0">
<HStack space={1} className="justify-self-end shrink-0">
<RequestVersionDropdown response={activeResponse} />
<RecentHttpResponsesDropdown
responses={responses}
activeResponse={activeResponse}
onPinnedResponseId={setPinnedResponseId}
/>
</div>
</HStack>
</div>
)}
</HStack>
@@ -261,20 +261,13 @@ function LoadedImportDataDialog({
const itemTree = useMemo(() => buildItemTree(items), [items]);
// A folder row's checkbox aggregates its subtree the way the git commit tree does: creates and
// updates toggle together, while removals only ever cascade beneath a removed folder. Checking
// anything also brings back the folders it needs to live in.
// updates toggle together, while removals only ever cascade beneath a removed folder.
const toggleNode = (node: CheckboxTreeNode<ImportPlanItem>, checked: boolean) => {
const targets = new Set(
collectItems(node)
.filter((i) => togglesWith(node.data, i))
.map((i) => i.modelId),
);
if (checked) {
const byId = new Map(items.map((i) => [i.modelId, i]));
for (const ancestor of ancestorsOf(node.data, byId)) {
if (ancestor.action === "not_imported") targets.add(ancestor.modelId);
}
}
setItems((prev) => prev.map((i) => (targets.has(i.modelId) ? { ...i, selected: checked } : i)));
};
@@ -290,17 +283,19 @@ function LoadedImportDataDialog({
const disabled = new Set<string>();
const byId = new Map(items.map((i) => [i.modelId, i]));
for (const item of items) {
for (const parent of ancestorsOf(item, byId)) {
if (parent.model !== "folder") break;
const missing =
(parent.action === "create" || parent.action === "not_imported") && !parent.selected;
// A not-imported row stays checkable: checking it brings its folders back with it
if (missing && item.action !== "delete" && item.action !== "not_imported") {
const seen = new Set<string>();
let parentId = item.parentId;
while (parentId != null && !seen.has(parentId)) {
seen.add(parentId);
const parent = byId.get(parentId);
if (parent == null || parent.model !== "folder") break;
if (parent.action === "create" && !parent.selected && item.action !== "delete") {
disabled.add(item.modelId);
}
if (parent.action === "delete" && parent.selected && item.action === "delete") {
disabled.add(item.modelId);
}
parentId = parent.parentId;
}
}
return disabled;
@@ -345,7 +340,6 @@ function LoadedImportDataDialog({
modelId: existing?.id ?? planned?.id ?? "workspace",
name: existing?.name ?? planned?.name ?? "New workspace",
selected: true,
changedFields: [],
},
children: itemTree,
};
@@ -364,7 +358,6 @@ function LoadedImportDataDialog({
checked={nodeCheckedStatus}
onCheck={toggleNode}
isCheckboxDisabled={(n) => disabledIds.has(n.key)}
isCollapsedByDefault={(n) => n.data.action === "not_imported"}
isRelevant={(n) => n.data.model === "workspace" || n.data.action !== "unchanged"}
renderRow={(n) => <ImportTreeRow item={n.data} onResolveConflict={resolveConflict} />}
/>
@@ -409,7 +402,7 @@ function LoadedImportDataDialog({
? "Importing"
: changeCount > 0
? `Apply ${changeCount} ${changeCount === 1 ? "Change" : "Changes"}`
: "Done"}
: "Apply"}
</Button>
</HStack>
</VStack>
@@ -572,13 +565,11 @@ function ImportTreeRow({
)}
<div className="truncate flex-1">{item.name}</div>
{item.action === "conflict" ? (
<div className="shrink-0">
<div className="shrink-0 flex items-center gap-1.5">
<SegmentedControl
name={`conflict-${item.modelId}`}
label={`Resolve conflict for ${item.name}`}
hideLabel
size="2xs"
help={actionHelp(item)}
value={item.resolution ?? "keep_mine"}
onChange={(v) => onResolveConflict(item.modelId, v)}
options={[
@@ -586,6 +577,7 @@ function ImportTreeRow({
{ value: "take_source", label: "Take source" },
]}
/>
<IconTooltip content={actionHelp(item)} iconSize="sm" />
</div>
) : (
actionLabel(item) && (
@@ -597,7 +589,6 @@ function ImportTreeRow({
item.action === "update" && "text-info",
item.action === "delete" && "text-danger",
item.action === "keep_local" && item.selected && "text-warning",
item.action === "not_imported" && "text-text-subtlest",
)}
>
{actionLabel(item)}
@@ -619,57 +610,28 @@ function actionLabel(item: ImportPlanItem): string | null {
return "removed";
case "keep_local":
return "edited";
case "not_imported":
return "not imported";
default:
return null;
}
}
function actionHelp(item: ImportPlanItem): string | null {
const help = (text: string) =>
item.changedFields.length > 0
? `${text} · ${item.changedFields.map(fieldLabel).join(", ")}`
: text;
switch (item.action) {
case "create":
return "Added since the last import";
case "update":
return help("Changed since the last import");
return "Changed since the last import";
case "delete":
return item.reason === "moved_into_not_imported_folder"
? "Moved into a folder that isn't imported. Import that folder instead to follow the move"
: "Deleted since the last import";
return "Deleted since the last import";
case "keep_local":
return help("Local edits made since the last import. Importing will revert them if checked");
return "Local edits made since the last import. Importing will revert them if checked";
case "conflict":
return help("Changed both here and in the file since the last import");
case "not_imported":
return "In the file, but not imported. Check it to import it";
return "Changed both here and in the file since the last import";
default:
return null;
}
}
function fieldLabel(field: string): string {
return field.replace(/([A-Z])/g, " $1").toLowerCase();
}
/** Every plan item above `item`, nearest first. */
function ancestorsOf(item: ImportPlanItem, byId: Map<string, ImportPlanItem>): ImportPlanItem[] {
const ancestors: ImportPlanItem[] = [];
const seen = new Set<string>();
let parentId = item.parentId;
while (parentId != null && !seen.has(parentId)) {
seen.add(parentId);
const parent = byId.get(parentId);
if (parent == null) break;
ancestors.push(parent);
parentId = parent.parentId;
}
return ancestors;
}
function buildItemTree(items: ImportPlanItem[]): CheckboxTreeNode<ImportPlanItem>[] {
const byId = new Map(items.map((i) => [i.modelId, i]));
const childrenOf = new Map<string, ImportPlanItem[]>();
@@ -716,7 +678,7 @@ function togglesWith(root: ImportPlanItem, item: ImportPlanItem): boolean {
if (item.action === "keep_local") {
return root.modelId === item.modelId && item.model !== "folder";
}
return item.action === "create" || item.action === "update" || item.action === "not_imported";
return item.action === "create" || item.action === "update";
}
function nodeCheckedStatus(
@@ -0,0 +1,79 @@
import type { HttpResponse, RequestVersionComparison } from "@yaakapp-internal/models";
import { Icon } from "@yaakapp-internal/ui";
import { stringify } from "yaml";
import { useRequestVersion } from "../hooks/useRequestVersion";
import { showDialog } from "../lib/dialog";
import { restoreRequestVersion } from "../lib/restoreRequestVersion";
import { Button } from "./core/Button";
import { DiffViewer } from "./core/Editor/DiffViewer";
import { Dropdown } from "./core/Dropdown";
interface Props {
response: Pick<HttpResponse, "requestId" | "versionId">;
}
/**
* Offers the request a response was sent from, when that is no longer the
* request you have.
*
* Hidden while the two agree, which is the overwhelmingly common case and the
* one where there is nothing to say. Responses recorded before versioning
* existed have no version and stay quiet forever.
*/
export function RequestVersionDropdown({ response }: Props) {
const { data: comparison } = useRequestVersion(response.versionId, response.requestId);
if (comparison == null || !comparison.differs) {
return null;
}
return (
<Dropdown
items={[
{
label: "View Diff",
leftSlot: <Icon icon="git_branch" />,
onSelect: () => showRequestVersionDiff(comparison),
},
{
label: "Restore This Version",
leftSlot: <Icon icon="history" />,
onSelect: () => restoreRequestVersion(comparison.version),
},
]}
>
<Button
size="2xs"
variant="border"
color="notice"
className="font-sans"
title="This request has changed since this response was sent"
forDropdown
>
Request Changed
</Button>
</Dropdown>
);
}
function showRequestVersionDiff(comparison: RequestVersionComparison) {
showDialog({
id: "request-version-diff",
title: "Request Changes Since This Response",
size: "full",
noPadding: true,
render: () => (
<div className="h-full flex flex-col px-4 pb-4">
<DiffViewer
original={toYaml(comparison.version.document)}
modified={toYaml(comparison.currentDocument)}
className="flex-1 min-h-0"
/>
</div>
),
});
}
/** Matches how the Git dialog renders a model for diffing. */
function toYaml(document: unknown): string {
return stringify(document, { indent: 2, lineWidth: 0 });
}
@@ -21,8 +21,6 @@ interface Props<T> {
isCheckboxDisabled?: (node: CheckboxTreeNode<T>) => boolean;
/** An irrelevant row is hidden unless one of its descendants is relevant */
isRelevant: (node: CheckboxTreeNode<T>) => boolean;
/** A node that starts collapsed, so a large subtree doesn't crowd out the rest */
isCollapsedByDefault?: (node: CheckboxTreeNode<T>) => boolean;
renderRow: (node: CheckboxTreeNode<T>) => ReactNode;
onSelectRow?: (node: CheckboxTreeNode<T>) => void;
canSelectRow?: (node: CheckboxTreeNode<T>) => boolean;
@@ -31,9 +29,7 @@ interface Props<T> {
export function CheckboxTree<T>(props: Props<T>) {
const { node, depth = 0 } = props;
const [collapsed, setCollapsed] = useState<boolean>(
() => props.isCollapsedByDefault?.(node) ?? false,
);
const [collapsed, setCollapsed] = useState<boolean>(false);
if (!hasRelevantNode(node, props.isRelevant)) return null;
const checked = props.checked(node);
@@ -6,7 +6,6 @@ import { useStateWithDeps } from "../../hooks/useStateWithDeps";
import { generateId } from "../../lib/generateId";
import { Button } from "./Button";
import { IconButton, type IconButtonProps } from "./IconButton";
import { IconTooltip } from "./IconTooltip";
import { Label } from "./Label";
interface Props<T extends string> {
@@ -37,15 +36,11 @@ export function SegmentedControl<T extends string>({
const containerRef = useRef<HTMLDivElement>(null);
const id = useRef(`input-${generateId()}`);
// A visually hidden label has nowhere to show the help, so the last option carries it
const inlineHelp =
hideLabel && help ? <IconTooltip tabIndex={-1} content={help} iconSize="xs" /> : null;
return (
<div className="w-full grid">
<Label
htmlFor={id.current}
help={hideLabel ? undefined : help}
help={help}
visuallyHidden={hideLabel}
className={classNames(labelClassName)}
>
@@ -83,10 +78,9 @@ export function SegmentedControl<T extends string>({
}
}}
>
{options.map((o, i) => {
{options.map((o) => {
const isSelected = selectedValue === o.value;
const isActive = value === o.value;
const rightSlot = i === options.length - 1 ? inlineHelp : null;
if (o.icon == null) {
return (
<Button
@@ -101,7 +95,6 @@ export function SegmentedControl<T extends string>({
isActive && "text-text!",
"focus:ring-1 focus:ring-border-focus",
)}
rightSlot={rightSlot}
onClick={() => onChange(o.value)}
>
{o.label}
@@ -124,7 +117,6 @@ export function SegmentedControl<T extends string>({
)}
title={o.label}
icon={o.icon}
rightSlot={rightSlot}
onClick={() => onChange(o.value)}
/>
);
@@ -0,0 +1,26 @@
import { useQuery } from "@tanstack/react-query";
import type { RequestVersionComparison } from "@yaakapp-internal/models";
import { useAtomValue } from "jotai";
import { allRequestsAtom } from "./useAllRequests";
import { rpc } from "../lib/rpc";
/**
* The request version a response was sent from, alongside the request as it
* stands now.
*
* Refetches when the live request is written, which is what keeps the "has this
* changed?" answer honest while someone edits. The comparison itself is the
* backend's — the frontend never hashes anything.
*/
export function useRequestVersion(versionId: string | null | undefined, requestId: string | null) {
const requests = useAtomValue(allRequestsAtom);
const liveUpdatedAt = requests.find((r) => r.id === requestId)?.updatedAt;
return useQuery({
placeholderData: (prev) => prev,
queryKey: ["request_version", versionId, liveUpdatedAt],
enabled: versionId != null,
queryFn: () =>
rpc<RequestVersionComparison>("models_request_version", { versionId: versionId! }),
});
}
@@ -0,0 +1,75 @@
import type { ModelVersionReason } from "@yaakapp-internal/models";
import { afterEach, beforeEach, describe, expect, test, vi } from "vitest";
import { EditSessionTracker } from "./editSessionTracker";
type Capture = [requestId: string, reason: ModelVersionReason];
function tracker(idleMs = 1000) {
const captured: Capture[] = [];
return {
captured,
tracker: new EditSessionTracker((id, reason) => captured.push([id, reason]), idleMs),
};
}
describe("EditSessionTracker", () => {
beforeEach(() => vi.useFakeTimers());
afterEach(() => vi.useRealTimers());
test("captures a request once it has been left alone", () => {
const { tracker: t, captured } = tracker();
t.noteEdit("rq_1");
vi.advanceTimersByTime(999);
expect(captured).toEqual([]);
vi.advanceTimersByTime(1);
expect(captured).toEqual([["rq_1", "idle"]]);
});
test("a burst of edits is one capture, not one per keystroke", () => {
const { tracker: t, captured } = tracker();
for (let i = 0; i < 10; i++) {
t.noteEdit("rq_1");
vi.advanceTimersByTime(500);
}
expect(captured).toEqual([]);
vi.advanceTimersByTime(1000);
expect(captured).toEqual([["rq_1", "idle"]]);
});
test("captures the request being left, not the one being opened", () => {
const { tracker: t, captured } = tracker();
t.noteActiveRequest("rq_1");
expect(captured).toEqual([]);
t.noteActiveRequest("rq_2");
expect(captured).toEqual([["rq_1", "switch"]]);
});
test("re-selecting the same request is not a boundary", () => {
const { tracker: t, captured } = tracker();
t.noteActiveRequest("rq_1");
t.noteActiveRequest("rq_1");
expect(captured).toEqual([]);
});
test("blur and close capture the request still on screen", () => {
const { tracker: t, captured } = tracker();
t.noteActiveRequest("rq_1");
t.noteBoundary();
t.noteBoundary();
expect(captured).toEqual([
["rq_1", "switch"],
["rq_1", "switch"],
]);
});
test("nothing is captured before a request is open", () => {
const { tracker: t, captured } = tracker();
t.noteBoundary();
t.noteActiveRequest(null);
expect(captured).toEqual([]);
});
});
@@ -0,0 +1,57 @@
import type { ModelVersionReason } from "@yaakapp-internal/models";
/**
* How long a request has to sit untouched before its edits become a version.
* Long enough that typing a URL is one version rather than forty, short enough
* that walking away from a half-finished edit still records it.
*/
export const IDLE_MS = 60_000;
type Snapshot = (requestId: string, reason: ModelVersionReason) => void;
/**
* When a request's editing session ends.
*
* The backend versions a request on every send, which covers "what produced
* this response". This covers the rest: an edit someone made and then walked
* away from, which no send would ever have captured.
*
* It deliberately knows nothing about *what* changed. Versions are
* content-addressed, so a boundary that turns out to have nothing behind it
* costs one query and creates nothing — which is what lets this stay a timer
* and two assignments instead of a change-tracking system.
*/
export class EditSessionTracker {
private timer: ReturnType<typeof setTimeout> | null = null;
private idleRequestId: string | null = null;
private activeRequestId: string | null = null;
constructor(
private readonly snapshot: Snapshot,
private readonly idleMs: number = IDLE_MS,
) {}
/** A request was written. Restarts its idle countdown. */
noteEdit(requestId: string) {
if (this.timer != null) clearTimeout(this.timer);
this.idleRequestId = requestId;
this.timer = setTimeout(() => {
this.timer = null;
const requestId = this.idleRequestId;
if (requestId != null) this.snapshot(requestId, "idle");
}, this.idleMs);
}
/** The user moved to a different request, so the one they left is finished. */
noteActiveRequest(requestId: string | null) {
if (requestId === this.activeRequestId) return;
const left = this.activeRequestId;
this.activeRequestId = requestId;
if (left != null) this.snapshot(left, "switch");
}
/** The window lost focus or is closing. */
noteBoundary() {
if (this.activeRequestId != null) this.snapshot(this.activeRequestId, "switch");
}
}
+50
View File
@@ -0,0 +1,50 @@
import { flushAllPendingPatches } from "@yaakapp-internal/models";
import type { ModelPayload, ModelVersion, ModelVersionReason } from "@yaakapp-internal/models";
import { platform } from "@yaakapp-internal/platform";
import { EditSessionTracker } from "./editSessionTracker";
import { activeRequestIdAtom } from "../hooks/useActiveRequestId";
import { jotaiStore } from "./jotai";
import { rpc } from "./rpc";
const REQUEST_MODELS = ["http_request", "grpc_request", "websocket_request"];
/**
* Ask the backend to capture a request's current content.
*
* Quiet by design: a version that fails to write is not worth a toast, because
* every caller below is reacting to the user leaving rather than asking for
* anything.
*/
export function snapshotRequestVersion(requestId: string, reason: ModelVersionReason) {
// Edits reach the database on a debounce, so flush before asking for a
// version of what is in it
flushAllPendingPatches();
rpc<ModelVersion>("models_snapshot_request", { requestId, reason }).catch((err: unknown) => {
console.warn("Failed to snapshot request version", err);
});
}
export function initRequestVersionSnapshots() {
const tracker = new EditSessionTracker(snapshotRequestVersion);
platform.listen<ModelPayload[]>("model_writes", (payloads) => {
for (const payload of payloads) {
if (payload.change.type !== "upsert") continue;
if (!REQUEST_MODELS.includes(payload.model.model)) continue;
tracker.noteEdit(payload.model.id);
}
});
tracker.noteActiveRequest(jotaiStore.get(activeRequestIdAtom));
jotaiStore.sub(activeRequestIdAtom, () => {
tracker.noteActiveRequest(jotaiStore.get(activeRequestIdAtom));
});
platform.window.onFocusChanged((focused) => {
if (!focused) tracker.noteBoundary();
});
// Closing is the last boundary there is. Nothing can be awaited here, but the
// write is already on its way and the backend outlives the window.
window.addEventListener("beforeunload", () => tracker.noteBoundary());
}
@@ -0,0 +1,35 @@
import { flushAllModelWrites } from "@yaakapp-internal/models";
import type { ModelVersion } from "@yaakapp-internal/models";
import { wasUpdatedExternally } from "../hooks/useRequestUpdateKey";
import { fireAndForget } from "./fireAndForget";
import { rpc } from "./rpc";
import { showToast } from "./toast";
/**
* Put an old version's content back into the live request.
*
* The backend captures whatever the request currently holds before
* overwriting it, so this is not a destructive action even when the last edit
* was never versioned — but the request is still rewritten under the user's
* cursor, so it is announced.
*/
export function restoreRequestVersion(version: ModelVersion) {
fireAndForget(
(async () => {
// The backend restores from the database, so anything still sitting in a
// debounce has to land first — otherwise it would overwrite the restore
await flushAllModelWrites();
const requestId = await rpc<string>("models_restore_request_version", {
versionId: version.id,
});
// The write came from this window, so the store's echo suppression would
// otherwise leave open editors showing what was there before
wasUpdatedExternally(requestId);
showToast({
id: "request-version-restored",
color: "success",
message: "Restored the request that produced this response",
});
})(),
);
}
+2
View File
@@ -8,6 +8,7 @@ import { createRoot } from "react-dom/client";
import { initGit } from "./init/git";
import { initSync } from "./init/sync";
import { initGlobalListeners } from "./lib/initGlobalListeners";
import { initRequestVersionSnapshots } from "./lib/requestVersions";
import { jotaiStore } from "./lib/jotai";
import { router } from "./lib/router";
@@ -36,6 +37,7 @@ initGit();
initSync();
initModelStore(jotaiStore);
initGlobalListeners();
initRequestVersionSnapshots();
await changeModelStoreWorkspace(null); // Load global models
console.log("Creating React root");
@@ -113,10 +113,6 @@ fn format_skipped(items: &[ImportPlanItem]) -> Option<String> {
if keep_local > 0 {
parts.push(format!("{keep_local} with local edits"));
}
let not_imported = count(ImportPlanAction::NotImported);
if not_imported > 0 {
parts.push(format!("{not_imported} previously not imported"));
}
let unchanged = count(ImportPlanAction::Unchanged);
if unchanged > 0 {
parts.push(format!("{unchanged} unchanged"));
@@ -4,7 +4,6 @@ use common::{cli_cmd, parse_created_id, query_manager, seed_request};
use predicates::str::contains;
use serde_json::Value;
use tempfile::TempDir;
use yaak_models::util::UpdateSource;
#[test]
fn export_writes_yaak_workspace_file() {
@@ -258,60 +257,3 @@ fn re_import_merges_into_linked_workspace() {
assert!(requests.iter().any(|r| r.name == "Request B"), "removal must not auto-apply");
assert!(requests.iter().any(|r| r.name == "Request C"));
}
#[test]
fn re_import_leaves_deleted_resources_alone() {
let temp_dir = TempDir::new().expect("Failed to create temp dir");
let data_dir = temp_dir.path();
let import_path = temp_dir.path().join("linked.json");
write_linked_fixture(
&import_path,
&[
("req_a", "Request A", "https://example.com/a"),
("req_b", "Request B", "https://example.com/b"),
],
);
cli_cmd(data_dir)
.args(["import", import_path.to_str().expect("import path is utf-8")])
.assert()
.success();
let workspace_id = {
let query_manager = query_manager(data_dir);
let db = query_manager.connect();
let workspace_id = db
.list_workspaces()
.expect("list workspaces")
.into_iter()
.find(|w| w.name == "Linked Workspace")
.expect("workspace imported")
.id;
let request_b = db
.list_http_requests(&workspace_id)
.expect("list requests")
.into_iter()
.find(|r| r.name == "Request B")
.expect("request B imported");
db.delete_http_request_by_id(&request_b.id, &UpdateSource::Sync)
.expect("delete request B");
workspace_id
};
cli_cmd(data_dir)
.args([
"import",
import_path.to_str().expect("import path is utf-8"),
"--workspace-id",
&workspace_id,
])
.assert()
.success()
.stdout(contains("Skipped 1 previously not imported"));
let query_manager = query_manager(data_dir);
let requests =
query_manager.connect().list_http_requests(&workspace_id).expect("list requests");
assert_eq!(requests.len(), 1, "a deleted request must not come back: {requests:?}");
assert_eq!(requests[0].name, "Request A");
}
+8
View File
@@ -40,6 +40,7 @@ use yaak_models::models::{
CookieJar, Environment, GrpcConnection, GrpcConnectionState, GrpcEvent,
GrpcEventType, HttpRequest, HttpResponse, HttpResponseState, Workspace,
};
use yaak_models::queries::any_request::AnyRequest;
use yaak_models::util::{BatchUpsertResult, ImportDestination, ImportPlan, UpdateSource};
use yaak_plugins::events::{
Color, ErrorResponse, FilterResponse, InternalEvent, InternalEventPayload, PluginContext,
@@ -330,6 +331,12 @@ async fn cmd_grpc_go<R: Runtime>(
let settings = app_handle.db().get_settings();
let client_cert = find_client_certificate(&request.url, &settings.client_certificates);
// Capture the stored request, not the rendered one: what a restore should
// put back is what the user typed
let version_id = app_handle
.db()
.snapshot_request_for_send(&AnyRequest::GrpcRequest(unrendered_request.clone()));
let conn = app_handle.db().upsert_grpc_connection(
&GrpcConnection {
workspace_id: request.workspace_id.clone(),
@@ -338,6 +345,7 @@ async fn cmd_grpc_go<R: Runtime>(
elapsed: 0,
state: GrpcConnectionState::Initialized,
url: request.url.clone(),
version_id,
..Default::default()
},
&UpdateSource::from_window_label(window.label()),
+14 -2
View File
@@ -37,8 +37,8 @@ use yaak_grpc::ServiceDefinition;
use yaak_models::blob_manager::BlobManager;
use yaak_models::models::{
GraphQlIntrospection, GrpcEvent, HttpRequest, HttpRequestHeader, HttpResponse,
HttpResponseEvent, ImportSource, Plugin, Settings, WebsocketConnection, WebsocketEvent,
WorkspaceMeta,
HttpResponseEvent, ImportSource, ModelVersion, Plugin, RequestVersionComparison, Settings,
WebsocketConnection, WebsocketEvent, WorkspaceMeta,
};
use yaak_models::query_manager::QueryManager;
use yaak_models::util::{BatchUpsertResult, ImportPlan};
@@ -653,6 +653,18 @@ async fn models_duplicate<R: Runtime>(ctx: ClientCtx<R>, req: ModelsDuplicateReq
Ok(yaak_commands::models::models_duplicate(ctx, req).await?)
}
async fn models_snapshot_request<R: Runtime>(ctx: ClientCtx<R>, req: ModelsSnapshotRequestReq) -> Result<ModelVersion> {
Ok(yaak_commands::models::models_snapshot_request(ctx, req).await?)
}
async fn models_request_version<R: Runtime>(ctx: ClientCtx<R>, req: ModelsRequestVersionReq) -> Result<RequestVersionComparison> {
Ok(yaak_commands::models::models_request_version(ctx, req).await?)
}
async fn models_restore_request_version<R: Runtime>(ctx: ClientCtx<R>, req: ModelsRestoreRequestVersionReq) -> Result<String> {
Ok(yaak_commands::models::models_restore_request_version(ctx, req).await?)
}
async fn models_websocket_events<R: Runtime>(ctx: ClientCtx<R>, req: ModelsWebsocketEventsReq) -> Result<Vec<WebsocketEvent>> {
Ok(yaak_commands::models::models_websocket_events(ctx, req).await?)
}
@@ -20,6 +20,7 @@ use yaak_models::models::{
HttpResponseHeader, WebsocketConnection, WebsocketConnectionState, WebsocketEvent,
WebsocketEventType,
};
use yaak_models::queries::any_request::AnyRequest;
use yaak_models::util::UpdateSource;
use yaak_plugins::events::{CallHttpAuthenticationRequest, HttpHeader, RenderPurpose};
use yaak_plugins::template_callback::PluginTemplateCallback;
@@ -169,10 +170,17 @@ pub async fn cmd_ws_connect<R: Runtime>(
)
.await?;
// Capture the stored request, not the rendered one: what a restore should
// put back is what the user typed
let version_id = app_handle
.db()
.snapshot_request_for_send(&AnyRequest::WebsocketRequest(unrendered_request.clone()));
let connection = app_handle.db().upsert_websocket_connection(
&WebsocketConnection {
workspace_id: request.workspace_id.clone(),
request_id: request_id.to_string(),
version_id,
..Default::default()
},
&UpdateSource::from_window_label(window.label()),
+51
View File
@@ -138,6 +138,10 @@ export type GrpcConnection = {
state: GrpcConnectionState;
trailers: { [key in string]?: string };
url: string;
/**
* The request version this connection was opened from, when one was captured.
*/
versionId: string | null;
};
export type GrpcConnectionState = "initialized" | "connected" | "closed";
@@ -242,6 +246,10 @@ export type HttpResponse = {
state: HttpResponseState;
url: string;
version: string | null;
/**
* The request version this response was sent from, when one was captured.
*/
versionId: string | null;
};
export type HttpResponseEvent = {
@@ -347,6 +355,28 @@ export type KeyValue = {
value: string;
};
export type ModelVersion = {
model: "model_version";
id: string;
createdAt: string;
updatedAt: string;
workspaceId: string;
/**
* The `model` field of the versioned model, eg. `http_request`.
*/
modelType: string;
modelId: string;
contentHash: string;
document: Record<string, any>;
reason: ModelVersionReason;
};
/**
* Why a version was captured. Not a UI label — the frontend decides how to
* phrase these — but it is what makes a history readable when debugging.
*/
export type ModelVersionReason = "send" | "switch" | "idle" | "restore" | "manual";
export type Plugin = {
model: "plugin";
id: string;
@@ -374,6 +404,23 @@ export type ProxySetting =
export type ProxySettingAuth = { user: string; password: string };
/**
* One version, next to the request as it stands now.
*
* Both halves come from the same place so they are guaranteed comparable: the
* frontend renders them side by side, and `differs` is the same content-hash
* comparison the backend uses everywhere else rather than a second opinion
* formed in TypeScript.
*/
export type RequestVersionComparison = {
version: ModelVersion;
/**
* The live request's editable content, in the same shape as the version's document.
*/
currentDocument: Record<string, any>;
differs: boolean;
};
export type Settings = {
model: "settings";
id: string;
@@ -438,6 +485,10 @@ export type WebsocketConnection = {
state: WebsocketConnectionState;
status: number;
url: string;
/**
* The request version this connection was opened from, when one was captured.
*/
versionId: string | null;
};
export type WebsocketConnectionState = "initialized" | "connected" | "closing" | "closed";
File diff suppressed because one or more lines are too long
+26 -80
View File
@@ -1,21 +1,7 @@
// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually.
import type {
Environment,
Folder,
GrpcRequest,
HttpRequest,
WebsocketRequest,
Workspace,
} from "./gen_models";
import type { Environment, Folder, GrpcRequest, HttpRequest, WebsocketRequest, Workspace } from "./gen_models";
export type BatchUpsertResult = {
workspaces: Array<Workspace>;
environments: Array<Environment>;
folders: Array<Folder>;
httpRequests: Array<HttpRequest>;
grpcRequests: Array<GrpcRequest>;
websocketRequests: Array<WebsocketRequest>;
};
export type BatchUpsertResult = { workspaces: Array<Workspace>, environments: Array<Environment>, folders: Array<Folder>, httpRequests: Array<HttpRequest>, grpcRequests: Array<GrpcRequest>, websocketRequests: Array<WebsocketRequest>, };
export type ImportConflictResolution = "keep_mine" | "take_source";
@@ -25,78 +11,38 @@ export type ImportConflictResolution = "keep_mine" | "take_source";
* The destination workspace and optional folder IDs are captured in the plan so the preview describes
* the exact destination that confirmation will use.
*/
export type ImportDestination =
| { type: "new_workspace" }
| { type: "existing_workspace"; workspaceId: string; folderId?: string };
export type ImportDestination = { "type": "new_workspace" } | { "type": "existing_workspace", workspaceId: string, folderId?: string, };
/**
* Where an import's contents came from, used to link the committed workspace back to it.
*/
export type ImportOrigin = {
/**
* The absolute file path or URL the contents were read from.
*/
origin: string;
label: string;
};
export type ImportPlan = {
importer: string;
destination: ImportDestination;
resources: BatchUpsertResult;
warnings: Array<ImportPlanWarning>;
/**
* Stable source key for every model in `resources`, keyed by its planned ID.
*/
sourceKeys: { [key in string]?: string };
/**
* One entry per plannable resource; commit applies only the selected ones.
*/
items: Array<ImportPlanItem>;
origin?: ImportOrigin;
};
export type ImportPlanAction =
| "create"
| "update"
| "delete"
| "unchanged"
| "keep_local"
| "conflict"
| "not_imported";
export type ImportPlanItem = {
action: ImportPlanAction;
model: ImportResourceType;
modelId: string;
name: string;
/**
* Planned parent folder ID for incoming resources; current parent for deletions.
*/
parentId?: string;
selected: boolean;
resolution?: ImportConflictResolution;
reason?: ImportPlanReason;
/**
* Fields where the source and the local copy disagree, so the preview can say why
*/
changedFields: Array<string>;
};
export type ImportOrigin = {
/**
* Extra context for an action that would otherwise be indistinguishable from its plain form.
* The absolute file path or URL the contents were read from.
*/
export type ImportPlanReason = "moved_into_not_imported_folder";
origin: string, label: string, };
export type ImportPlanWarning = { title: string; detail: string };
export type ImportPlan = { importer: string, destination: ImportDestination, resources: BatchUpsertResult, warnings: Array<ImportPlanWarning>,
/**
* Stable source key for every model in `resources`, keyed by its planned ID.
*/
sourceKeys: { [key in string]?: string },
/**
* One entry per plannable resource; commit applies only the selected ones.
*/
items: Array<ImportPlanItem>, origin?: ImportOrigin, };
export type ImportPlanAction = "create" | "update" | "delete" | "unchanged" | "keep_local" | "conflict";
export type ImportPlanItem = { action: ImportPlanAction, model: ImportResourceType, modelId: string, name: string,
/**
* Planned parent folder ID for incoming resources; current parent for deletions.
*/
parentId?: string, selected: boolean, resolution?: ImportConflictResolution, };
export type ImportPlanWarning = { title: string, detail: string, };
/**
* The model types an import plan can contain.
*/
export type ImportResourceType =
| "environment"
| "folder"
| "grpc_request"
| "http_request"
| "websocket_request"
| "workspace";
export type ImportResourceType = "environment" | "folder" | "grpc_request" | "http_request" | "websocket_request" | "workspace";
+27 -2
View File
@@ -21,8 +21,8 @@ use yaak_git::{
use yaak_grpc::ServiceDefinition;
use yaak_models::models::{
AnyModel, GraphQlIntrospection, GrpcEvent, HttpRequest, HttpRequestHeader, HttpResponse,
HttpResponseEvent, ImportSource, Plugin, Settings, WebsocketConnection, WebsocketEvent,
WorkspaceMeta,
HttpResponseEvent, ImportSource, ModelVersion, ModelVersionReason, Plugin,
RequestVersionComparison, Settings, WebsocketConnection, WebsocketEvent, WorkspaceMeta,
};
use yaak_models::util::{BatchUpsertResult, ImportDestination, ImportPlan};
use yaak_plugins::api::{PluginNameVersion, PluginSearchResponse, PluginUpdatesResponse};
@@ -534,6 +534,28 @@ pub struct ModelsDuplicateReq {
pub model_id: String,
}
#[derive(Debug, Deserialize, TS)]
#[serde(rename_all = "camelCase")]
#[ts(export, export_to = "gen_rpc.ts")]
pub struct ModelsSnapshotRequestReq {
pub request_id: String,
pub reason: ModelVersionReason,
}
#[derive(Debug, Deserialize, TS)]
#[serde(rename_all = "camelCase")]
#[ts(export, export_to = "gen_rpc.ts")]
pub struct ModelsRequestVersionReq {
pub version_id: String,
}
#[derive(Debug, Deserialize, TS)]
#[serde(rename_all = "camelCase")]
#[ts(export, export_to = "gen_rpc.ts")]
pub struct ModelsRestoreRequestVersionReq {
pub version_id: String,
}
#[derive(Debug, Deserialize, TS)]
#[serde(rename_all = "camelCase")]
#[ts(export, export_to = "gen_rpc.ts")]
@@ -981,6 +1003,9 @@ macro_rules! with_commands {
models_upsert(ModelsUpsertReq) -> String,
models_delete(ModelsDeleteReq) -> String,
models_duplicate(ModelsDuplicateReq) -> String,
models_snapshot_request(ModelsSnapshotRequestReq) -> ModelVersion,
models_request_version(ModelsRequestVersionReq) -> RequestVersionComparison,
models_restore_request_version(ModelsRestoreRequestVersionReq) -> String,
models_websocket_events(ModelsWebsocketEventsReq) -> Vec<WebsocketEvent>,
models_grpc_events(ModelsGrpcEventsReq) -> Vec<GrpcEvent>,
models_get_settings(ModelsGetSettingsReq) -> Settings,
+39 -2
View File
@@ -4,9 +4,10 @@
use crate::error::Result;
use crate::host::{Host, PluginHost};
use yaak_models::models::{
AnyModel, GraphQlIntrospection, GrpcEvent, HttpRequestHeader, Settings, WebsocketEvent,
WorkspaceMeta,
AnyModel, GraphQlIntrospection, GrpcEvent, HttpRequestHeader, ModelVersion,
RequestVersionComparison, Settings, WebsocketEvent, WorkspaceMeta,
};
use yaak_models::versions::version_document;
use yaak_models::queries::workspaces::default_headers;
use yaak_rpc_schema::*;
@@ -45,6 +46,42 @@ pub async fn models_duplicate<H: Host>(host: H, req: ModelsDuplicateReq) -> Resu
})?)
}
/// Capture the request's current content, from an edit-session boundary the
/// frontend can see: switching away, losing focus, closing, or falling idle.
///
/// The frontend does not track whether anything actually changed — versions are
/// content-addressed, so an unchanged request returns the version it already
/// had and the trigger code stays a one-liner.
pub async fn models_snapshot_request<H: Host>(
host: H,
req: ModelsSnapshotRequestReq,
) -> Result<ModelVersion> {
Ok(host.db().snapshot_request_by_id(&req.request_id, req.reason)?)
}
/// A version and the live request side by side, for the diff and for deciding
/// whether there is anything worth offering.
pub async fn models_request_version<H: Host>(
host: H,
req: ModelsRequestVersionReq,
) -> Result<RequestVersionComparison> {
let db = host.db();
let version = db.get_model_version(&req.version_id)?;
let current_document = version_document(&db.get_any_request(&version.model_id)?.to_value()?)?;
let differs = !db.request_matches_version(&version)?;
Ok(RequestVersionComparison { version, current_document, differs })
}
/// Returns the id of the request that was restored.
pub async fn models_restore_request_version<H: Host>(
host: H,
req: ModelsRestoreRequestVersionReq,
) -> Result<String> {
let source = host.update_source();
let restored = host.db().restore_request_version(&req.version_id, &source)?;
Ok(restored.id().to_string())
}
pub async fn models_websocket_events<H: Host>(
host: H,
req: ModelsWebsocketEventsReq,
+53 -8
View File
@@ -139,6 +139,10 @@ export type GrpcConnection = {
state: GrpcConnectionState;
trailers: { [key in string]?: string };
url: string;
/**
* The request version this connection was opened from, when one was captured.
*/
versionId: string | null;
};
export type GrpcConnectionState = "initialized" | "connected" | "closed";
@@ -243,6 +247,10 @@ export type HttpResponse = {
state: HttpResponseState;
url: string;
version: string | null;
/**
* The request version this response was sent from, when one was captured.
*/
versionId: string | null;
};
export type HttpResponseEvent = {
@@ -356,14 +364,8 @@ export type ImportSourceResource = {
importSourceId: string;
sourceKey: string;
modelType: string;
/**
* `None` once the user has decided not to import this key
*/
modelId?: string;
/**
* Hash of the resource as last applied or decided from the source, if one was recorded
*/
contentHash?: string;
modelId: string;
snapshot: string;
};
export type InheritedBoolSetting = { enabled?: boolean; value: boolean };
@@ -388,6 +390,28 @@ export type ModelPayload = {
change: ModelChangeEvent;
};
export type ModelVersion = {
model: "model_version";
id: string;
createdAt: string;
updatedAt: string;
workspaceId: string;
/**
* The `model` field of the versioned model, eg. `http_request`.
*/
modelType: string;
modelId: string;
contentHash: string;
document: Record<string, any>;
reason: ModelVersionReason;
};
/**
* Why a version was captured. Not a UI label — the frontend decides how to
* phrase these — but it is what makes a history readable when debugging.
*/
export type ModelVersionReason = "send" | "switch" | "idle" | "restore" | "manual";
export type ParentAuthentication = {
authentication: Record<string, any>;
authenticationType: string | null;
@@ -431,6 +455,23 @@ export type ProxySetting =
export type ProxySettingAuth = { user: string; password: string };
/**
* One version, next to the request as it stands now.
*
* Both halves come from the same place so they are guaranteed comparable: the
* frontend renders them side by side, and `differs` is the same content-hash
* comparison the backend uses everywhere else rather than a second opinion
* formed in TypeScript.
*/
export type RequestVersionComparison = {
version: ModelVersion;
/**
* The live request's editable content, in the same shape as the version's document.
*/
currentDocument: Record<string, any>;
differs: boolean;
};
export type Settings = {
model: "settings";
id: string;
@@ -494,6 +535,10 @@ export type WebsocketConnection = {
state: WebsocketConnectionState;
status: number;
url: string;
/**
* The request version this connection was opened from, when one was captured.
*/
versionId: string | null;
};
export type WebsocketConnectionState = "initialized" | "connected" | "closing" | "closed";
+26 -80
View File
@@ -1,21 +1,7 @@
// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually.
import type {
Environment,
Folder,
GrpcRequest,
HttpRequest,
WebsocketRequest,
Workspace,
} from "./gen_models";
import type { Environment, Folder, GrpcRequest, HttpRequest, WebsocketRequest, Workspace } from "./gen_models";
export type BatchUpsertResult = {
workspaces: Array<Workspace>;
environments: Array<Environment>;
folders: Array<Folder>;
httpRequests: Array<HttpRequest>;
grpcRequests: Array<GrpcRequest>;
websocketRequests: Array<WebsocketRequest>;
};
export type BatchUpsertResult = { workspaces: Array<Workspace>, environments: Array<Environment>, folders: Array<Folder>, httpRequests: Array<HttpRequest>, grpcRequests: Array<GrpcRequest>, websocketRequests: Array<WebsocketRequest>, };
export type ImportConflictResolution = "keep_mine" | "take_source";
@@ -25,78 +11,38 @@ export type ImportConflictResolution = "keep_mine" | "take_source";
* The destination workspace and optional folder IDs are captured in the plan so the preview describes
* the exact destination that confirmation will use.
*/
export type ImportDestination =
| { type: "new_workspace" }
| { type: "existing_workspace"; workspaceId: string; folderId?: string };
export type ImportDestination = { "type": "new_workspace" } | { "type": "existing_workspace", workspaceId: string, folderId?: string, };
/**
* Where an import's contents came from, used to link the committed workspace back to it.
*/
export type ImportOrigin = {
/**
* The absolute file path or URL the contents were read from.
*/
origin: string;
label: string;
};
export type ImportPlan = {
importer: string;
destination: ImportDestination;
resources: BatchUpsertResult;
warnings: Array<ImportPlanWarning>;
/**
* Stable source key for every model in `resources`, keyed by its planned ID.
*/
sourceKeys: { [key in string]?: string };
/**
* One entry per plannable resource; commit applies only the selected ones.
*/
items: Array<ImportPlanItem>;
origin?: ImportOrigin;
};
export type ImportPlanAction =
| "create"
| "update"
| "delete"
| "unchanged"
| "keep_local"
| "conflict"
| "not_imported";
export type ImportPlanItem = {
action: ImportPlanAction;
model: ImportResourceType;
modelId: string;
name: string;
/**
* Planned parent folder ID for incoming resources; current parent for deletions.
*/
parentId?: string;
selected: boolean;
resolution?: ImportConflictResolution;
reason?: ImportPlanReason;
/**
* Fields where the source and the local copy disagree, so the preview can say why
*/
changedFields: Array<string>;
};
export type ImportOrigin = {
/**
* Extra context for an action that would otherwise be indistinguishable from its plain form.
* The absolute file path or URL the contents were read from.
*/
export type ImportPlanReason = "moved_into_not_imported_folder";
origin: string, label: string, };
export type ImportPlanWarning = { title: string; detail: string };
export type ImportPlan = { importer: string, destination: ImportDestination, resources: BatchUpsertResult, warnings: Array<ImportPlanWarning>,
/**
* Stable source key for every model in `resources`, keyed by its planned ID.
*/
sourceKeys: { [key in string]?: string },
/**
* One entry per plannable resource; commit applies only the selected ones.
*/
items: Array<ImportPlanItem>, origin?: ImportOrigin, };
export type ImportPlanAction = "create" | "update" | "delete" | "unchanged" | "keep_local" | "conflict";
export type ImportPlanItem = { action: ImportPlanAction, model: ImportResourceType, modelId: string, name: string,
/**
* Planned parent folder ID for incoming resources; current parent for deletions.
*/
parentId?: string, selected: boolean, resolution?: ImportConflictResolution, };
export type ImportPlanWarning = { title: string, detail: string, };
/**
* The model types an import plan can contain.
*/
export type ImportResourceType =
| "environment"
| "folder"
| "grpc_request"
| "http_request"
| "websocket_request"
| "workspace";
export type ImportResourceType = "environment" | "folder" | "grpc_request" | "http_request" | "websocket_request" | "workspace";
@@ -1,24 +0,0 @@
-- Replace the per-resource snapshot with a content hash, and let a row exist without a model
-- so a resource the user chose not to import can be remembered.
CREATE TABLE import_source_resources_new
(
model TEXT DEFAULT 'import_source_resource' NOT NULL,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP NOT NULL,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP NOT NULL,
import_source_id TEXT NOT NULL,
source_key TEXT NOT NULL,
model_type TEXT NOT NULL,
model_id TEXT,
content_hash TEXT,
PRIMARY KEY (import_source_id, source_key)
);
INSERT INTO import_source_resources_new (model, created_at, updated_at, import_source_id,
source_key, model_type, model_id, content_hash)
SELECT model, created_at, updated_at, import_source_id, source_key, model_type, model_id, NULL
FROM import_source_resources;
DROP TABLE import_source_resources;
ALTER TABLE import_source_resources_new
RENAME TO import_source_resources;
@@ -0,0 +1,20 @@
CREATE TABLE model_versions
(
id TEXT NOT NULL PRIMARY KEY,
model TEXT DEFAULT 'model_version' NOT NULL,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP NOT NULL,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP NOT NULL,
workspace_id TEXT NOT NULL,
model_type TEXT NOT NULL,
model_id TEXT NOT NULL,
content_hash TEXT NOT NULL,
document TEXT NOT NULL,
reason TEXT NOT NULL
);
-- Content addressing, enforced by the database rather than by every caller.
CREATE UNIQUE INDEX model_versions_content ON model_versions (model_id, content_hash);
ALTER TABLE http_responses ADD COLUMN version_id TEXT;
ALTER TABLE grpc_connections ADD COLUMN version_id TEXT;
ALTER TABLE websocket_connections ADD COLUMN version_id TEXT;
+14
View File
@@ -118,6 +118,20 @@ impl<'a> ClientDb<'a> {
Ok(m.clone())
}
/// Upsert a model WITHOUT recording a model change or emitting an event.
///
/// Only for rows that are nobody's business but this process's — model
/// versions, whose whole point is that they are local history. Anything the
/// frontend, sync or another window should learn about goes through
/// [`Self::upsert`].
pub(crate) fn upsert_untracked<M>(&self, model: &M) -> Result<M>
where
M: UpsertModelInfo + Clone,
{
let (m, _created) = self.ctx.upsert(model, &UpdateSource::Background.to_db())?;
Ok(m)
}
fn record_model_change(&self, payload: &ModelPayload) -> Result<()> {
let payload_json = serde_json::to_string(payload)?;
let source_json = serde_json::to_string(&payload.update_source)?;
+1
View File
@@ -21,6 +21,7 @@ pub mod queries;
pub mod query_manager;
pub mod render;
pub mod util;
pub mod versions;
/// Per-connection setup, applied by every pool on every connection it opens.
fn init_connection(conn: &rusqlite::Connection) -> rusqlite::Result<()> {
+166 -7
View File
@@ -1539,6 +1539,8 @@ pub struct WebsocketConnection {
pub state: WebsocketConnectionState,
pub status: i32,
pub url: String,
/// The request version this connection was opened from, when one was captured.
pub version_id: Option<String>,
}
impl UpsertModelInfo for WebsocketConnection {
@@ -1578,6 +1580,7 @@ impl UpsertModelInfo for WebsocketConnection {
(State, serde_json::to_value(&self.state)?.as_str().into()),
(Status, self.status.into()),
(Url, self.url.into()),
(VersionId, self.version_id.into()),
])
}
@@ -1590,6 +1593,7 @@ impl UpsertModelInfo for WebsocketConnection {
WebsocketConnectionIden::State,
WebsocketConnectionIden::Status,
WebsocketConnectionIden::Url,
WebsocketConnectionIden::VersionId,
]
}
@@ -1612,6 +1616,7 @@ impl UpsertModelInfo for WebsocketConnection {
error: row.get("error")?,
state: serde_json::from_str(format!(r#""{state}""#).as_str()).unwrap(),
status: row.get("status")?,
version_id: row.get("version_id").unwrap_or_default(),
})
}
}
@@ -1965,6 +1970,8 @@ pub struct HttpResponse {
pub state: HttpResponseState,
pub url: String,
pub version: Option<String>,
/// The request version this response was sent from, when one was captured.
pub version_id: Option<String>,
}
impl UpsertModelInfo for HttpResponse {
@@ -2014,6 +2021,7 @@ impl UpsertModelInfo for HttpResponse {
(Url, self.url.into()),
(Version, self.version.into()),
(RequestContentLength, self.request_content_length.into()),
(VersionId, self.version_id.into()),
])
}
@@ -2036,6 +2044,7 @@ impl UpsertModelInfo for HttpResponse {
HttpResponseIden::StatusReason,
HttpResponseIden::Url,
HttpResponseIden::Version,
HttpResponseIden::VersionId,
]
}
@@ -2071,6 +2080,7 @@ impl UpsertModelInfo for HttpResponse {
r.get::<_, String>("request_headers").unwrap_or_default().as_str(),
)
.unwrap_or_default(),
version_id: r.get("version_id").unwrap_or_default(),
})
}
}
@@ -2516,6 +2526,8 @@ pub struct GrpcConnection {
pub state: GrpcConnectionState,
pub trailers: BTreeMap<String, String>,
pub url: String,
/// The request version this connection was opened from, when one was captured.
pub version_id: Option<String>,
}
impl UpsertModelInfo for GrpcConnection {
@@ -2557,6 +2569,7 @@ impl UpsertModelInfo for GrpcConnection {
(Error, self.error.as_ref().map(|s| s.as_str()).into()),
(Trailers, serde_json::to_string(&self.trailers)?.into()),
(Url, self.url.into()),
(VersionId, self.version_id.into()),
])
}
@@ -2571,6 +2584,7 @@ impl UpsertModelInfo for GrpcConnection {
GrpcConnectionIden::Error,
GrpcConnectionIden::Trailers,
GrpcConnectionIden::Url,
GrpcConnectionIden::VersionId,
]
}
@@ -2595,6 +2609,7 @@ impl UpsertModelInfo for GrpcConnection {
url: row.get("url")?,
error: row.get("error")?,
trailers: serde_json::from_str(trailers.as_str()).unwrap_or_default(),
version_id: row.get("version_id").unwrap_or_default(),
})
}
}
@@ -3118,12 +3133,8 @@ pub struct ImportSourceResource {
pub import_source_id: String,
pub source_key: String,
pub model_type: String,
/// `None` once the user has decided not to import this key
#[ts(optional)]
pub model_id: Option<String>,
/// Hash of the resource as last applied or decided from the source, if one was recorded
#[ts(optional)]
pub content_hash: Option<String>,
pub model_id: String,
pub snapshot: String,
}
impl<'s> TryFrom<&Row<'s>> for ImportSourceResource {
@@ -3138,11 +3149,159 @@ impl<'s> TryFrom<&Row<'s>> for ImportSourceResource {
source_key: r.get("source_key")?,
model_type: r.get("model_type")?,
model_id: r.get("model_id")?,
content_hash: r.get("content_hash")?,
snapshot: r.get("snapshot")?,
})
}
}
/// Why a version was captured. Not a UI label — the frontend decides how to
/// phrase these — but it is what makes a history readable when debugging.
#[derive(Debug, Clone, Copy, PartialEq, Serialize, Deserialize, TS)]
#[serde(rename_all = "snake_case")]
#[ts(export, export_to = "gen_models.ts")]
pub enum ModelVersionReason {
Send,
Switch,
Idle,
Restore,
/// Reserved: an explicit "save a version now" action, which has no UI yet.
Manual,
}
impl Default for ModelVersionReason {
fn default() -> Self {
Self::Manual
}
}
/// A point-in-time copy of one request's editable content.
///
/// Versions are content-addressed: `content_hash` covers exactly what
/// `document` holds, and `(model_id, content_hash)` is unique, so capturing the
/// same content twice returns the row that already exists. That is what lets
/// every send snapshot unconditionally without growing the table.
///
/// Deliberately absent from [`AnyModel`]: versions are local history. They are
/// not synced, not exported, and not mirrored into the frontend's model store —
/// the frontend asks for the one version it needs to show.
impl Default for ModelVersion {
fn default() -> Self {
Self {
model: "model_version".to_string(),
id: String::new(),
created_at: NaiveDateTime::default(),
updated_at: NaiveDateTime::default(),
workspace_id: String::new(),
model_type: String::new(),
model_id: String::new(),
content_hash: String::new(),
document: Value::Object(Default::default()),
reason: ModelVersionReason::default(),
}
}
}
#[derive(Debug, Clone, Serialize, Deserialize, TS)]
#[serde(default, rename_all = "camelCase")]
#[ts(export, export_to = "gen_models.ts")]
#[enum_def(table_name = "model_versions")]
pub struct ModelVersion {
#[ts(type = "\"model_version\"")]
pub model: String,
pub id: String,
pub created_at: NaiveDateTime,
pub updated_at: NaiveDateTime,
pub workspace_id: String,
/// The `model` field of the versioned model, eg. `http_request`.
pub model_type: String,
pub model_id: String,
pub content_hash: String,
#[ts(type = "Record<string, any>")]
pub document: Value,
pub reason: ModelVersionReason,
}
impl UpsertModelInfo for ModelVersion {
fn table_name() -> impl IntoTableRef + IntoIden {
ModelVersionIden::Table
}
fn id_column() -> impl IntoIden + Eq + Clone {
ModelVersionIden::Id
}
fn generate_id() -> String {
generate_prefixed_id("mv")
}
fn order_by() -> (impl IntoColumnRef, Order) {
(ModelVersionIden::CreatedAt, Desc)
}
fn get_id(&self) -> String {
self.id.clone()
}
fn insert_values(
self,
source: &UpdateSource,
) -> DbResult<Vec<(impl IntoIden + Eq, impl Into<SimpleExpr>)>> {
use ModelVersionIden::*;
Ok(vec![
(CreatedAt, upsert_date(source, self.created_at)),
(UpdatedAt, upsert_date(source, self.updated_at)),
(WorkspaceId, self.workspace_id.into()),
(ModelType, self.model_type.into()),
(ModelId, self.model_id.into()),
(ContentHash, self.content_hash.into()),
(Document, serde_json::to_string(&self.document)?.into()),
(Reason, serde_json::to_value(self.reason)?.as_str().into()),
])
}
fn update_columns() -> Vec<impl IntoIden> {
vec![ModelVersionIden::UpdatedAt]
}
fn from_row(row: &Row) -> rusqlite::Result<Self>
where
Self: Sized,
{
let document: String = row.get("document")?;
let reason: String = row.get("reason")?;
Ok(Self {
id: row.get("id")?,
model: row.get("model")?,
created_at: row.get("created_at")?,
updated_at: row.get("updated_at")?,
workspace_id: row.get("workspace_id")?,
model_type: row.get("model_type")?,
model_id: row.get("model_id")?,
content_hash: row.get("content_hash")?,
document: serde_json::from_str(&document).unwrap_or_default(),
reason: serde_json::from_str(format!(r#""{reason}""#).as_str()).unwrap_or_default(),
})
}
}
/// One version, next to the request as it stands now.
///
/// Both halves come from the same place so they are guaranteed comparable: the
/// frontend renders them side by side, and `differs` is the same content-hash
/// comparison the backend uses everywhere else rather than a second opinion
/// formed in TypeScript.
#[derive(Debug, Clone, Serialize, Deserialize, TS)]
#[serde(rename_all = "camelCase")]
#[ts(export, export_to = "gen_models.ts")]
pub struct RequestVersionComparison {
pub version: ModelVersion,
/// The live request's editable content, in the same shape as the version's document.
#[ts(type = "Record<string, any>")]
pub current_document: Value,
pub differs: bool,
}
/// Only used as a `from_row` fallback for an unparseable settings column. The
/// value a *new* model gets comes from that model's `Default` impl.
fn default_request_message_size_setting() -> InheritedIntSetting {
@@ -1,6 +1,7 @@
use crate::client_db::ClientDb;
use crate::error::Result;
use crate::models::{GrpcRequest, HttpRequest, WebsocketRequest};
use serde_json::Value;
pub enum AnyRequest {
HttpRequest(HttpRequest),
@@ -8,6 +9,36 @@ pub enum AnyRequest {
WebsocketRequest(WebsocketRequest),
}
/// Run an expression against whichever request this is, bound as `$request`.
macro_rules! with_request {
($self:expr, |$request:ident| $body:expr) => {
match $self {
AnyRequest::HttpRequest($request) => $body,
AnyRequest::GrpcRequest($request) => $body,
AnyRequest::WebsocketRequest($request) => $body,
}
};
}
impl AnyRequest {
pub fn id(&self) -> &str {
with_request!(self, |request| &request.id)
}
pub fn workspace_id(&self) -> &str {
with_request!(self, |request| &request.workspace_id)
}
/// The model name, eg. `http_request`.
pub fn model_type(&self) -> &str {
with_request!(self, |request| &request.model)
}
pub fn to_value(&self) -> Result<Value> {
Ok(with_request!(self, |request| serde_json::to_value(request)?))
}
}
impl<'a> ClientDb<'a> {
pub fn get_any_request(&self, id: &str) -> Result<AnyRequest> {
if let Ok(http_request) = self.get_http_request(id) {
@@ -38,6 +38,7 @@ impl<'a> ClientDb<'a> {
source: &UpdateSource,
) -> Result<GrpcRequest> {
self.delete_all_grpc_connections_for_request(m.id.as_str(), source)?;
self.delete_model_versions_for_model(m.id.as_str())?;
self.delete(m, source)
}
@@ -24,6 +24,7 @@ impl<'a> ClientDb<'a> {
source: &UpdateSource,
) -> Result<HttpRequest> {
self.delete_all_http_responses_for_request(m.id.as_str(), source)?;
self.delete_model_versions_for_model(m.id.as_str())?;
self.delete(m, source)
}
@@ -34,7 +34,7 @@ impl<'a> ClientDb<'a> {
ImportSourceResourceIden::SourceKey,
ImportSourceResourceIden::ModelType,
ImportSourceResourceIden::ModelId,
ImportSourceResourceIden::ContentHash,
ImportSourceResourceIden::Snapshot,
])
.values_panic([
CurrentTimestamp.into(),
@@ -42,8 +42,8 @@ impl<'a> ClientDb<'a> {
resource.import_source_id.as_str().into(),
resource.source_key.as_str().into(),
resource.model_type.as_str().into(),
resource.model_id.clone().into(),
resource.content_hash.clone().into(),
resource.model_id.as_str().into(),
resource.snapshot.as_str().into(),
])
.on_conflict(
OnConflict::columns([
@@ -54,7 +54,7 @@ impl<'a> ClientDb<'a> {
ImportSourceResourceIden::UpdatedAt,
ImportSourceResourceIden::ModelType,
ImportSourceResourceIden::ModelId,
ImportSourceResourceIden::ContentHash,
ImportSourceResourceIden::Snapshot,
])
.to_owned(),
)
+1
View File
@@ -15,6 +15,7 @@ mod import_source_resources;
mod import_sources;
mod key_values;
mod model_changes;
mod model_versions;
mod plugin_key_values;
mod plugins;
mod settings;
@@ -0,0 +1,455 @@
use crate::client_db::ClientDb;
use crate::error::Result;
use crate::models::{
GrpcRequest, HttpRequest, ModelVersion, ModelVersionIden, ModelVersionReason, UpsertModelInfo,
WebsocketRequest,
};
use crate::queries::any_request::AnyRequest;
use crate::util::UpdateSource;
use crate::versions::{apply_version_document, content_hash, version_document};
use log::warn;
use sea_query::{Expr, ExprTrait, Query, SqliteQueryBuilder};
use sea_query_rusqlite::RusqliteBinder;
/// Unreferenced versions older than this are dropped.
const RETENTION_DAYS: i64 = 30;
/// How many unreferenced versions a request keeps, newest first.
const RETENTION_COUNT: i64 = 50;
impl<'a> ClientDb<'a> {
pub fn get_model_version(&self, id: &str) -> Result<ModelVersion> {
self.find_one(ModelVersionIden::Id, id)
}
/// Every version of one model, newest first.
pub fn list_model_versions(&self, model_id: &str) -> Result<Vec<ModelVersion>> {
self.find_many(ModelVersionIden::ModelId, model_id, None)
}
/// Capture a request's current content, or return the version that already
/// holds it.
///
/// The single entry point for creating versions. Callers do not check
/// whether anything changed first — that is what content addressing is for,
/// and it is why a send, a window blur and an idle timer can all call this
/// on the same unedited request and leave one row behind.
pub fn snapshot_request(
&self,
request: &AnyRequest,
reason: ModelVersionReason,
) -> Result<ModelVersion> {
let document = version_document(&request.to_value()?)?;
let content_hash = content_hash(&document)?;
if let Some(existing) = self.find_version_by_hash(request.id(), &content_hash) {
return Ok(existing);
}
let version = self.upsert_untracked(&ModelVersion {
workspace_id: request.workspace_id().to_string(),
model_type: request.model_type().to_string(),
model_id: request.id().to_string(),
content_hash: content_hash.clone(),
document,
reason,
..Default::default()
});
let version = match version {
Ok(version) => version,
// Two sends of the same request can both miss the lookup above and
// race to insert. The unique index settles it, and the loser wants
// exactly what the winner wrote.
Err(err) => match self.find_version_by_hash(request.id(), &content_hash) {
Some(existing) => return Ok(existing),
None => return Err(err),
},
};
self.prune_model_versions(request.id())?;
Ok(version)
}
pub fn snapshot_request_by_id(
&self,
request_id: &str,
reason: ModelVersionReason,
) -> Result<ModelVersion> {
self.snapshot_request(&self.get_any_request(request_id)?, reason)
}
/// What every send calls: capture the request, and don't make a fuss.
///
/// A send is not worth failing over history that couldn't be written, and
/// a request with no id is ephemeral and has nothing to version. Either way
/// the response just has no version to offer.
pub fn snapshot_request_for_send(&self, request: &AnyRequest) -> Option<String> {
if request.id().is_empty() {
return None;
}
match self.snapshot_request(request, ModelVersionReason::Send) {
Ok(version) => Some(version.id),
Err(err) => {
warn!("Failed to snapshot request before send: {err}");
None
}
}
}
/// Write a version's content back over the live request.
///
/// Anything the live request has picked up since its last version is
/// captured first, so a restore is never the thing that loses an edit. The
/// content being written already has a version — the one being restored —
/// so this leaves no new row behind.
pub fn restore_request_version(
&self,
version_id: &str,
source: &UpdateSource,
) -> Result<AnyRequest> {
let version = self.get_model_version(version_id)?;
let live = self.get_any_request(&version.model_id)?;
self.snapshot_request(&live, ModelVersionReason::Restore)?;
let restored = apply_version_document(&live.to_value()?, &version.document);
Ok(match live {
AnyRequest::HttpRequest(_) => AnyRequest::HttpRequest(
self.upsert_http_request(&serde_json::from_value::<HttpRequest>(restored)?, source)?,
),
AnyRequest::GrpcRequest(_) => AnyRequest::GrpcRequest(
self.upsert_grpc_request(&serde_json::from_value::<GrpcRequest>(restored)?, source)?,
),
AnyRequest::WebsocketRequest(_) => AnyRequest::WebsocketRequest(
self.upsert_websocket_request(
&serde_json::from_value::<WebsocketRequest>(restored)?,
source,
)?,
),
})
}
/// Whether a request's content has moved on from a given version.
pub fn request_matches_version(&self, version: &ModelVersion) -> Result<bool> {
let live = self.get_any_request(&version.model_id)?;
let hash = content_hash(&version_document(&live.to_value()?)?)?;
Ok(hash == version.content_hash)
}
pub fn delete_model_versions_for_model(&self, model_id: &str) -> Result<usize> {
self.delete_many_untracked::<ModelVersion>(ModelVersionIden::ModelId, model_id)
}
/// Drop the versions a request no longer needs.
///
/// A version referenced by a response outlives retention entirely — the
/// point of the feature is that an old response can still show what sent
/// it. Everything else is history the user has not asked to keep, and
/// survives only while it is both recent and among the newest few.
pub fn prune_model_versions(&self, model_id: &str) -> Result<usize> {
let cutoff = format!("-{RETENTION_DAYS} days");
let sql = r#"
DELETE FROM model_versions
WHERE model_id = ?1
AND id NOT IN (
SELECT version_id FROM http_responses WHERE request_id = ?1 AND version_id IS NOT NULL
UNION
SELECT version_id FROM grpc_connections WHERE request_id = ?1 AND version_id IS NOT NULL
UNION
SELECT version_id FROM websocket_connections WHERE request_id = ?1 AND version_id IS NOT NULL
)
AND (
created_at < datetime('now', ?2)
OR id NOT IN (
SELECT id FROM model_versions WHERE model_id = ?1
ORDER BY created_at DESC, rowid DESC LIMIT ?3
)
)
"#;
Ok(self.conn().execute(sql, rusqlite::params![model_id, cutoff, RETENTION_COUNT])?)
}
fn find_version_by_hash(&self, model_id: &str, content_hash: &str) -> Option<ModelVersion> {
let (sql, params) = Query::select()
.from(ModelVersionIden::Table)
.column(sea_query::Asterisk)
.cond_where(
Expr::col(ModelVersionIden::ModelId)
.eq(model_id)
.and(Expr::col(ModelVersionIden::ContentHash).eq(content_hash)),
)
.build_rusqlite(SqliteQueryBuilder);
let mut stmt = self.conn().prepare(sql.as_str()).ok()?;
stmt.query_row(&*params.as_params(), ModelVersion::from_row).ok()
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::client_db::ClientDb;
use crate::init_in_memory;
use crate::models::{HttpRequest, HttpResponse, Workspace};
fn source() -> UpdateSource {
UpdateSource::Background
}
fn seed(db: &ClientDb) -> (Workspace, HttpRequest) {
let workspace = db
.upsert_workspace(&Workspace { name: "Versions".to_string(), ..Default::default() }, &source())
.expect("Failed to upsert workspace");
let request = db
.upsert_http_request(
&HttpRequest {
workspace_id: workspace.id.clone(),
name: "Original".to_string(),
url: "https://example.com/one".to_string(),
..Default::default()
},
&source(),
)
.expect("Failed to upsert request");
(workspace, request)
}
fn snapshot(db: &ClientDb, request_id: &str, reason: ModelVersionReason) -> ModelVersion {
db.snapshot_request_by_id(request_id, reason).expect("Failed to snapshot")
}
#[test]
fn snapshotting_unchanged_content_reuses_the_same_version() {
let (query_manager, _blobs, _rx) = init_in_memory().expect("Failed to init DB");
let db = query_manager.connect();
let (_workspace, request) = seed(&db);
let first = snapshot(&db, &request.id, ModelVersionReason::Send);
let second = snapshot(&db, &request.id, ModelVersionReason::Idle);
let third = snapshot(&db, &request.id, ModelVersionReason::Switch);
assert_eq!(first.id, second.id);
assert_eq!(first.id, third.id);
// The first capture's reason is the one that sticks; a version is its content
assert_eq!(second.reason, ModelVersionReason::Send);
assert_eq!(db.list_model_versions(&request.id).unwrap().len(), 1);
}
#[test]
fn bookkeeping_writes_do_not_mint_a_version() {
let (query_manager, _blobs, _rx) = init_in_memory().expect("Failed to init DB");
let db = query_manager.connect();
let (_workspace, request) = seed(&db);
let first = snapshot(&db, &request.id, ModelVersionReason::Send);
let folder = db
.upsert_folder(
&crate::models::Folder {
workspace_id: request.workspace_id.clone(),
..Default::default()
},
&source(),
)
.unwrap();
db.upsert_http_request(
&HttpRequest {
folder_id: Some(folder.id),
sort_priority: 42.0,
..db.get_http_request(&request.id).unwrap()
},
&source(),
)
.unwrap();
assert_eq!(snapshot(&db, &request.id, ModelVersionReason::Idle).id, first.id);
assert_eq!(db.list_model_versions(&request.id).unwrap().len(), 1);
}
#[test]
fn editing_content_mints_a_version() {
let (query_manager, _blobs, _rx) = init_in_memory().expect("Failed to init DB");
let db = query_manager.connect();
let (_workspace, request) = seed(&db);
snapshot(&db, &request.id, ModelVersionReason::Send);
db.upsert_http_request(
&HttpRequest { url: "https://example.com/two".to_string(), ..request.clone() },
&source(),
)
.unwrap();
snapshot(&db, &request.id, ModelVersionReason::Idle);
assert_eq!(db.list_model_versions(&request.id).unwrap().len(), 2);
}
#[test]
fn restoring_writes_the_old_content_back_without_a_new_version() {
let (query_manager, _blobs, _rx) = init_in_memory().expect("Failed to init DB");
let db = query_manager.connect();
let (_workspace, request) = seed(&db);
let original = snapshot(&db, &request.id, ModelVersionReason::Send);
db.upsert_http_request(
&HttpRequest {
url: "https://example.com/two".to_string(),
name: "Edited".to_string(),
..request.clone()
},
&source(),
)
.unwrap();
let edited = snapshot(&db, &request.id, ModelVersionReason::Idle);
db.restore_request_version(&original.id, &source()).expect("Failed to restore");
let live = db.get_http_request(&request.id).unwrap();
assert_eq!(live.url, "https://example.com/one");
assert_eq!(live.name, "Original");
assert_eq!(live.id, request.id);
// The restored content already had a version, and the edit it replaced
// still has its own, so nothing new appears
let versions = db.list_model_versions(&request.id).unwrap();
assert_eq!(versions.len(), 2);
assert!(versions.iter().any(|v| v.id == original.id));
assert!(versions.iter().any(|v| v.id == edited.id));
}
/// The case restore exists to be safe for: an edit that was never captured.
#[test]
fn restoring_captures_uncaptured_edits_first() {
let (query_manager, _blobs, _rx) = init_in_memory().expect("Failed to init DB");
let db = query_manager.connect();
let (_workspace, request) = seed(&db);
let original = snapshot(&db, &request.id, ModelVersionReason::Send);
db.upsert_http_request(
&HttpRequest { url: "https://example.com/unsaved".to_string(), ..request.clone() },
&source(),
)
.unwrap();
db.restore_request_version(&original.id, &source()).expect("Failed to restore");
let versions = db.list_model_versions(&request.id).unwrap();
assert_eq!(versions.len(), 2);
let rescued = versions.iter().find(|v| v.id != original.id).unwrap();
assert_eq!(rescued.reason, ModelVersionReason::Restore);
assert_eq!(rescued.document.get("url").unwrap(), "https://example.com/unsaved");
}
#[test]
fn request_matches_version_tracks_the_live_content() {
let (query_manager, _blobs, _rx) = init_in_memory().expect("Failed to init DB");
let db = query_manager.connect();
let (_workspace, request) = seed(&db);
let version = snapshot(&db, &request.id, ModelVersionReason::Send);
assert!(db.request_matches_version(&version).unwrap());
db.upsert_http_request(
&HttpRequest { url: "https://example.com/two".to_string(), ..request.clone() },
&source(),
)
.unwrap();
assert!(!db.request_matches_version(&version).unwrap());
}
/// Write `count` distinct versions by walking the request's URL forward.
fn make_versions(db: &ClientDb, request: &HttpRequest, count: usize) -> Vec<ModelVersion> {
(0..count)
.map(|i| {
db.upsert_http_request(
&HttpRequest { url: format!("https://example.com/{i}"), ..request.clone() },
&source(),
)
.unwrap();
snapshot(db, &request.id, ModelVersionReason::Idle)
})
.collect()
}
#[test]
fn unreferenced_versions_are_pruned_to_the_newest_fifty() {
let (query_manager, _blobs, _rx) = init_in_memory().expect("Failed to init DB");
let db = query_manager.connect();
let (_workspace, request) = seed(&db);
let versions = make_versions(&db, &request, RETENTION_COUNT as usize + 10);
let kept = db.list_model_versions(&request.id).unwrap();
assert_eq!(kept.len(), RETENTION_COUNT as usize);
// The oldest went first
assert!(!kept.iter().any(|v| v.id == versions[0].id));
assert!(kept.iter().any(|v| v.id == versions.last().unwrap().id));
}
#[test]
fn a_referenced_version_survives_retention() {
let (query_manager, blobs, _rx) = init_in_memory().expect("Failed to init DB");
let db = query_manager.connect();
let (workspace, request) = seed(&db);
let pinned = snapshot(&db, &request.id, ModelVersionReason::Send);
db.upsert_http_response(
&HttpResponse {
request_id: request.id.clone(),
workspace_id: workspace.id.clone(),
version_id: Some(pinned.id.clone()),
..Default::default()
},
&source(),
&blobs,
)
.unwrap();
make_versions(&db, &request, RETENTION_COUNT as usize + 10);
let kept = db.list_model_versions(&request.id).unwrap();
assert!(
kept.iter().any(|v| v.id == pinned.id),
"a version a response points at must outlive retention",
);
}
#[test]
fn unreferenced_versions_expire_after_thirty_days() {
let (query_manager, _blobs, _rx) = init_in_memory().expect("Failed to init DB");
let db = query_manager.connect();
let (_workspace, request) = seed(&db);
let old = snapshot(&db, &request.id, ModelVersionReason::Send);
db.conn()
.execute(
"UPDATE model_versions SET created_at = datetime('now', '-31 days') WHERE id = ?1",
rusqlite::params![old.id],
)
.unwrap();
// Any later capture prunes
db.upsert_http_request(
&HttpRequest { url: "https://example.com/two".to_string(), ..request.clone() },
&source(),
)
.unwrap();
let fresh = snapshot(&db, &request.id, ModelVersionReason::Idle);
let kept = db.list_model_versions(&request.id).unwrap();
assert_eq!(kept.len(), 1);
assert_eq!(kept[0].id, fresh.id);
}
#[test]
fn deleting_a_request_deletes_its_versions() {
let (query_manager, _blobs, _rx) = init_in_memory().expect("Failed to init DB");
let db = query_manager.connect();
let (_workspace, request) = seed(&db);
make_versions(&db, &request, 3);
assert!(!db.list_model_versions(&request.id).unwrap().is_empty());
db.delete_http_request_by_id(&request.id, &source()).unwrap();
assert!(db.list_model_versions(&request.id).unwrap().is_empty());
}
}
@@ -40,6 +40,7 @@ impl<'a> ClientDb<'a> {
source: &UpdateSource,
) -> Result<WebsocketRequest> {
self.delete_all_websocket_connections_for_request(websocket_request.id.as_str(), source)?;
self.delete_model_versions_for_model(websocket_request.id.as_str())?;
self.delete(websocket_request, source)
}
+2 -1
View File
@@ -7,7 +7,7 @@ use crate::models::{
GraphQlIntrospection, GraphQlIntrospectionIden, GrpcConnection, GrpcConnectionIden, GrpcEvent,
GrpcEventIden, GrpcRequest, GrpcRequestIden, HttpRequest, HttpRequestHeader, HttpRequestIden,
HttpResponse, HttpResponseEvent, HttpResponseEventIden, HttpResponseIden, ImportSource,
ImportSourceIden, ResolvedHttpRequestSettings, ResolvedSetting, SyncState, SyncStateIden,
ImportSourceIden, ModelVersion, ModelVersionIden, ResolvedHttpRequestSettings, ResolvedSetting, SyncState, SyncStateIden,
WebsocketConnection,
WebsocketConnectionIden, WebsocketEvent, WebsocketEventIden, WebsocketRequest,
WebsocketRequestIden, Workspace, WorkspaceIden, WorkspaceMeta, WorkspaceMetaIden,
@@ -90,6 +90,7 @@ impl<'a> ClientDb<'a> {
self.delete_import_source_resources(&import_source.id)?;
}
self.delete_many_untracked::<ImportSource>(ImportSourceIden::WorkspaceId, wid)?;
self.delete_many_untracked::<ModelVersion>(ModelVersionIden::WorkspaceId, wid)?;
self.delete_many_untracked::<SyncState>(SyncStateIden::WorkspaceId, wid)?;
self.delete_many_untracked::<WorkspaceMeta>(WorkspaceMetaIden::WorkspaceId, wid)?;
self.delete(workspace, source)
-15
View File
@@ -169,16 +169,6 @@ pub enum ImportPlanAction {
Unchanged,
KeepLocal,
Conflict,
/// Present in the source but previously turned down; selecting it imports it again
NotImported,
}
/// Extra context for an action that would otherwise be indistinguishable from its plain form.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize, TS)]
#[serde(rename_all = "snake_case")]
#[ts(export, export_to = "gen_util.ts")]
pub enum ImportPlanReason {
MovedIntoNotImportedFolder,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize, TS)]
@@ -203,11 +193,6 @@ pub struct ImportPlanItem {
pub selected: bool,
#[ts(optional)]
pub resolution: Option<ImportConflictResolution>,
#[ts(optional)]
pub reason: Option<ImportPlanReason>,
/// Fields where the source and the local copy disagree, so the preview can say why
#[serde(default)]
pub changed_fields: Vec<String>,
}
#[derive(Debug, Deserialize, Serialize, TS)]
+240
View File
@@ -0,0 +1,240 @@
//! Content addressing for model versions.
//!
//! A version's identity is its *content*, so the two functions here — what
//! counts as content, and how content becomes a hash — are the whole of it.
//! Everything else about versioning (when to capture, what to keep, how to
//! restore) is built on top and stays in `queries::model_versions`.
use crate::error::Result;
use serde::Serialize;
use serde_json::{Map, Value};
use sha2::{Digest, Sha256};
/// Keys that describe a model's place in the workspace rather than what the
/// user typed into it.
///
/// Dropping them is what makes a version stable: moving a request into a
/// folder, dragging it up the sidebar, or simply saving it again all rewrite
/// these and nothing else, and none of them should mint a version or show up
/// in a diff. It is also why one rule covers HTTP, gRPC and WebSocket — the
/// three differ only in the content fields, which are all kept.
const BOOKKEEPING_KEYS: &[&str] =
&["model", "id", "createdAt", "updatedAt", "workspaceId", "folderId", "sortPriority"];
/// The editable content of a model, as the object a version stores.
pub fn version_document<T: Serialize>(model: &T) -> Result<Value> {
let mut value = serde_json::to_value(model)?;
if let Some(object) = value.as_object_mut() {
for key in BOOKKEEPING_KEYS {
object.remove(*key);
}
}
Ok(value)
}
/// The hash a version is addressed by.
pub fn content_hash(document: &Value) -> Result<String> {
let mut canonical = String::new();
write_canonical(document, &mut canonical);
Ok(hex::encode(Sha256::digest(canonical.as_bytes())))
}
/// Serialize with object keys in sorted order.
///
/// Plain `to_string` would not do: whether `serde_json::Map` preserves
/// insertion order or sorts is a workspace-wide feature decision, and a
/// document read back from SQLite has whatever order it was written in. Sorting
/// here makes the hash depend on the content and nothing else, in every build.
fn write_canonical(value: &Value, out: &mut String) {
match value {
Value::Object(map) => {
let mut keys = map.keys().collect::<Vec<_>>();
keys.sort_unstable();
out.push('{');
for (i, key) in keys.into_iter().enumerate() {
if i > 0 {
out.push(',');
}
write_canonical(&Value::String(key.clone()), out);
out.push(':');
write_canonical(&map[key], out);
}
out.push('}');
}
Value::Array(items) => {
out.push('[');
for (i, item) in items.iter().enumerate() {
if i > 0 {
out.push(',');
}
write_canonical(item, out);
}
out.push(']');
}
scalar => out.push_str(&scalar.to_string()),
}
}
/// Lay a version's document back over a live model.
///
/// Keys the document carries win; keys it doesn't mention keep whatever the
/// live model has. That covers both halves of a restore: bookkeeping (id,
/// folder, sort order) survives because the document never held it, and a field
/// added to the model after the version was captured survives because the
/// version predates it.
pub fn apply_version_document(live: &Value, document: &Value) -> Value {
let mut merged = live.as_object().cloned().unwrap_or_else(Map::new);
if let Some(document) = document.as_object() {
for (key, value) in document {
merged.insert(key.clone(), value.clone());
}
}
Value::Object(merged)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::models::{HttpRequest, HttpRequestHeader};
use chrono::Utc;
fn request() -> HttpRequest {
HttpRequest {
id: "rq_1".to_string(),
workspace_id: "wk_1".to_string(),
folder_id: Some("fl_1".to_string()),
name: "Get user".to_string(),
url: "https://example.com/users/1".to_string(),
method: "GET".to_string(),
sort_priority: 1.0,
headers: vec![HttpRequestHeader {
name: "Accept".to_string(),
value: "application/json".to_string(),
..Default::default()
}],
..Default::default()
}
}
fn hash_of(request: &HttpRequest) -> String {
content_hash(&version_document(request).unwrap()).unwrap()
}
#[test]
fn document_holds_content_and_drops_bookkeeping() {
let document = version_document(&request()).unwrap();
let object = document.as_object().unwrap();
for key in BOOKKEEPING_KEYS {
assert!(!object.contains_key(*key), "document should not carry {key}");
}
assert_eq!(object.get("url").unwrap(), "https://example.com/users/1");
assert_eq!(object.get("name").unwrap(), "Get user");
assert_eq!(object.get("method").unwrap(), "GET");
assert!(object.contains_key("headers"));
assert!(object.contains_key("body"));
assert!(object.contains_key("authentication"));
assert!(object.contains_key("description"));
assert!(object.contains_key("settingFollowRedirects"));
}
#[test]
fn bookkeeping_never_changes_the_hash() {
let base = hash_of(&request());
let moved = HttpRequest { folder_id: Some("fl_2".to_string()), ..request() };
assert_eq!(hash_of(&moved), base, "folder");
let resorted = HttpRequest { sort_priority: 99.5, ..request() };
assert_eq!(hash_of(&resorted), base, "sort priority");
let touched =
HttpRequest { updated_at: Utc::now().naive_utc(), created_at: Utc::now().naive_utc(), ..request() };
assert_eq!(hash_of(&touched), base, "timestamps");
let renamed_id = HttpRequest { id: "rq_2".to_string(), ..request() };
assert_eq!(hash_of(&renamed_id), base, "id");
let moved_workspace = HttpRequest { workspace_id: "wk_2".to_string(), ..request() };
assert_eq!(hash_of(&moved_workspace), base, "workspace");
}
#[test]
fn editable_content_changes_the_hash() {
let base = hash_of(&request());
assert_ne!(hash_of(&HttpRequest { url: "https://example.com/users/2".into(), ..request() }), base);
assert_ne!(hash_of(&HttpRequest { method: "POST".into(), ..request() }), base);
assert_ne!(hash_of(&HttpRequest { name: "Get other user".into(), ..request() }), base);
assert_ne!(hash_of(&HttpRequest { description: "Notes".into(), ..request() }), base);
assert_ne!(hash_of(&HttpRequest { headers: vec![], ..request() }), base);
assert_ne!(
hash_of(&HttpRequest { body_type: Some("application/json".into()), ..request() }),
base
);
}
/// The hash has to survive a round trip through SQLite, which stores the
/// document as text and hands back whatever order it was written in. It
/// also has to survive `serde_json`'s `preserve_order` feature being on in
/// one build of the workspace and off in another.
#[test]
fn key_order_does_not_change_the_hash() {
let a: Value = serde_json::from_str(r#"{"url":"a","method":"GET"}"#).unwrap();
let b: Value = serde_json::from_str(r#"{"method":"GET","url":"a"}"#).unwrap();
assert_eq!(content_hash(&a).unwrap(), content_hash(&b).unwrap());
}
#[test]
fn key_order_does_not_change_the_hash_when_nested() {
let a: Value =
serde_json::from_str(r#"{"body":{"text":"x","type":"json"},"headers":[{"a":1,"b":2}]}"#)
.unwrap();
let b: Value =
serde_json::from_str(r#"{"headers":[{"b":2,"a":1}],"body":{"type":"json","text":"x"}}"#)
.unwrap();
assert_eq!(content_hash(&a).unwrap(), content_hash(&b).unwrap());
}
/// Sorting keys must not make different documents collide.
#[test]
fn array_order_still_changes_the_hash() {
let a: Value = serde_json::from_str(r#"{"headers":[{"n":"a"},{"n":"b"}]}"#).unwrap();
let b: Value = serde_json::from_str(r#"{"headers":[{"n":"b"},{"n":"a"}]}"#).unwrap();
assert_ne!(content_hash(&a).unwrap(), content_hash(&b).unwrap());
}
#[test]
fn applying_a_document_keeps_the_live_model_identity() {
let live = serde_json::to_value(request()).unwrap();
let document = version_document(&HttpRequest {
url: "https://example.com/users/2".to_string(),
..request()
})
.unwrap();
let merged = apply_version_document(&live, &document);
let object = merged.as_object().unwrap();
assert_eq!(object.get("url").unwrap(), "https://example.com/users/2");
assert_eq!(object.get("id").unwrap(), "rq_1");
assert_eq!(object.get("folderId").unwrap(), "fl_1");
assert_eq!(object.get("sortPriority").unwrap(), 1.0);
assert_eq!(object.get("model").unwrap(), "http_request");
}
/// A version captured before a field existed must not blank that field out.
#[test]
fn applying_an_older_document_leaves_unknown_fields_alone() {
let live = serde_json::to_value(request()).unwrap();
let document = serde_json::json!({ "url": "https://example.com/old" });
let merged = apply_version_document(&live, &document);
let object = merged.as_object().unwrap();
assert_eq!(object.get("url").unwrap(), "https://example.com/old");
assert_eq!(object.get("method").unwrap(), "GET");
assert_eq!(object.get("name").unwrap(), "Get user");
}
}
+12
View File
@@ -138,6 +138,10 @@ export type GrpcConnection = {
state: GrpcConnectionState;
trailers: { [key in string]?: string };
url: string;
/**
* The request version this connection was opened from, when one was captured.
*/
versionId: string | null;
};
export type GrpcConnectionState = "initialized" | "connected" | "closed";
@@ -242,6 +246,10 @@ export type HttpResponse = {
state: HttpResponseState;
url: string;
version: string | null;
/**
* The request version this response was sent from, when one was captured.
*/
versionId: string | null;
};
export type HttpResponseEvent = {
@@ -430,6 +438,10 @@ export type WebsocketConnection = {
state: WebsocketConnectionState;
status: number;
url: string;
/**
* The request version this connection was opened from, when one was captured.
*/
versionId: string | null;
};
export type WebsocketConnectionState = "initialized" | "connected" | "closing" | "closed";
+46 -1
View File
@@ -32,12 +32,13 @@ use yaak_models::blob_manager::{BlobManager, BodyChunk};
use yaak_models::cookies::apply_cookie_changes;
use yaak_models::models::{
AnyModel, Cookie, CookieJar, HttpRequest, HttpResponseEvent, HttpResponseEventData,
HttpSendSettings,
HttpSendSettings, ModelVersionReason, RequestVersionComparison,
};
use yaak_models::models_ops;
use yaak_models::query_manager::QueryManager;
use yaak_models::render::render_http_request;
use yaak_models::util::{ModelPayload, UpdateSource};
use yaak_models::versions::version_document;
use yaak_templates::{RenderOptions, TemplateCallback};
/// Names inside the VFS, not paths on any disk. Two files because the desktop
@@ -218,6 +219,19 @@ struct UpsertIntrospectionReq {
content: Option<String>,
}
#[derive(Deserialize)]
#[serde(rename_all = "camelCase")]
struct SnapshotRequestReq {
request_id: String,
reason: ModelVersionReason,
}
#[derive(Deserialize)]
#[serde(rename_all = "camelCase")]
struct VersionIdReq {
version_id: String,
}
#[derive(Deserialize)]
#[serde(rename_all = "camelCase")]
struct ResponseIdReq {
@@ -319,6 +333,37 @@ fn dispatch(
to_json(id)
}
"models_snapshot_request" => {
let req: SnapshotRequestReq = from_js(payload)?;
to_json(
host.queries
.connect()
.snapshot_request_by_id(&req.request_id, req.reason)
.map_err(js_error)?,
)
}
"models_request_version" => {
let req: VersionIdReq = from_js(payload)?;
let db = host.queries.connect();
let version = db.get_model_version(&req.version_id).map_err(js_error)?;
let request = db.get_any_request(&version.model_id).map_err(js_error)?;
let current_document =
version_document(&request.to_value().map_err(js_error)?).map_err(js_error)?;
let differs = !db.request_matches_version(&version).map_err(js_error)?;
to_json(RequestVersionComparison { version, current_document, differs })
}
"models_restore_request_version" => {
let req: VersionIdReq = from_js(payload)?;
let restored = host
.queries
.connect()
.restore_request_version(&req.version_id, source)
.map_err(js_error)?;
to_json(restored.id().to_string())
}
"models_get_settings" => to_json(host.queries.connect().get_settings()),
"models_get_graphql_introspection" => {
-1
View File
@@ -9,7 +9,6 @@ async-trait = "0.1"
base64 = "0.22.1" # For carrying body chunks over a text-only plugin transport
log = { workspace = true }
md5 = "0.8.0"
sha2 = { workspace = true }
chrono = { workspace = true }
serde_json = { workspace = true }
thiserror = { workspace = true }
+109 -694
View File
File diff suppressed because it is too large Load Diff
+104 -3
View File
@@ -24,9 +24,10 @@ use yaak_http::types::{
use yaak_models::blob_manager::{BlobManager, BodyChunk};
use yaak_models::models::{
ClientCertificate, Cookie, CookieJar, DnsOverride, Environment, HttpRequest, HttpResponse,
HttpResponseEvent, HttpResponseEventData, HttpResponseHeader, HttpResponseState, ProxySetting,
ProxySettingAuth, ResolvedHttpRequestSettings,
HttpResponseEvent, HttpResponseEventData, HttpResponseHeader, HttpResponseState,
ProxySetting, ProxySettingAuth, ResolvedHttpRequestSettings,
};
use yaak_models::queries::any_request::AnyRequest;
use yaak_models::query_manager::QueryManager;
use yaak_models::render::render_http_request;
use yaak_models::util::{UpdateSource, generate_prefixed_id};
@@ -283,6 +284,9 @@ pub struct HttpSendInputs {
/// Cookies the send starts with. The store is shared, so reading it back after the send
/// returns (or fails) yields the cookies the transaction collected.
pub cookie_store: Option<CookieStore>,
/// The version holding the request's content as it was when this send was resolved,
/// which the response will point at. `None` for an ephemeral request with no id.
pub version_id: Option<String>,
}
/// Where a send writes its response. Without it, the send keeps everything in memory: no
@@ -434,6 +438,13 @@ pub fn resolve_send_inputs(
client_certificates: settings.client_certificates,
},
cookie_store: cookies.map(CookieStore::from_cookies),
// Captured here rather than deeper in the send because this is the last place that
// still holds the *stored* request: further down it has been resolved against its
// folder and workspace and then rendered, and neither of those is what a restore
// should put back. Every host reaches sending through this function — the desktop,
// the CLI, plugin-triggered sends — so every response gets a version without each
// of them remembering to ask for one.
version_id: db.snapshot_request_for_send(&AnyRequest::HttpRequest(request.clone())),
})
}
@@ -581,7 +592,8 @@ pub async fn send_http_request_by_id<T: TemplateCallback>(
pub async fn send_http_request<T: TemplateCallback>(
params: SendHttpRequestParams<'_, T>,
) -> Result<SendHttpRequestResult> {
let HttpSendInputs { request, environment_chain, runtime_config, cookie_store } = params.inputs;
let HttpSendInputs { request, environment_chain, runtime_config, cookie_store, version_id } =
params.inputs;
let (request, auth_context_id) = request.into_parts();
let storage = params.storage;
let send_options = runtime_config.send_options();
@@ -619,6 +631,7 @@ pub async fn send_http_request<T: TemplateCallback>(
let mut response = params.existing_response.unwrap_or_default();
response.request_id = request.id.clone();
response.workspace_id = request.workspace_id.clone();
response.version_id = version_id;
response.request_content_length = request_content_length;
response.request_headers = sendable_request
.headers
@@ -1345,6 +1358,7 @@ mod tests {
client_certificates: Vec::new(),
},
cookie_store: Some(CookieStore::new()),
version_id: None,
},
template_callback: &NoopTemplateCallback,
storage: None,
@@ -1414,6 +1428,7 @@ mod tests {
client_certificates: Vec::new(),
},
cookie_store: Some(CookieStore::new()),
version_id: None,
},
template_callback: &NoopTemplateCallback,
storage: None,
@@ -1466,6 +1481,92 @@ mod tests {
(query_manager, cookie_jar, temp_dir)
}
/// The whole point of the feature, end to end: a stored send must leave a
/// response that can name the request behind it, and repeated sends of an
/// unchanged request must all name the same one.
#[tokio::test]
async fn a_stored_send_links_the_request_version_that_produced_it() {
let (query_manager, blob_manager, temp_dir) = seed_send_storage();
let request = query_manager
.connect()
.upsert_http_request(
&HttpRequest {
workspace_id: "wk_test".to_string(),
url: "http://localhost/test".to_string(),
name: "Original".to_string(),
..Default::default()
},
&UpdateSource::Sync,
)
.expect("Failed to seed request");
let first = stored_send(&query_manager, &blob_manager, temp_dir.path(), &request).await;
let second = stored_send(&query_manager, &blob_manager, temp_dir.path(), &request).await;
let version_id = first.version_id.clone().expect("a stored send must record a version");
assert_eq!(second.version_id, Some(version_id.clone()), "an unchanged request is one version");
let db = query_manager.connect();
let version = db.get_model_version(&version_id).expect("Failed to load version");
assert_eq!(version.model_id, request.id);
assert_eq!(version.document.get("url").unwrap(), "http://localhost/test");
assert_eq!(db.list_model_versions(&request.id).unwrap().len(), 1);
// Editing after the fact is what the response pane has to be able to notice
query_manager
.connect()
.upsert_http_request(&HttpRequest { name: "Edited".to_string(), ..request.clone() }, &UpdateSource::Sync)
.expect("Failed to edit request");
assert!(!db.request_matches_version(&version).expect("Failed to compare"));
}
async fn stored_send(
query_manager: &QueryManager,
blob_manager: &BlobManager,
response_dir: &std::path::Path,
request: &HttpRequest,
) -> HttpResponse {
let executor = StubExecutor { body: b"hello world" };
let inputs = resolve_send_inputs(query_manager, request, None, None)
.expect("Failed to resolve send inputs");
send_http_request(SendHttpRequestParams {
inputs,
template_callback: &NoopTemplateCallback,
storage: Some(ResponseStorage {
query_manager,
blob_manager,
update_source: UpdateSource::Sync,
response_dir,
}),
emit_events_to: None,
emit_response_body_chunks_to: None,
cancelled_rx: None,
existing_response: None,
prepare_sendable_request: None,
executor: &executor,
})
.await
.expect("send should succeed")
.response
}
fn seed_send_storage() -> (QueryManager, BlobManager, TempDir) {
let temp_dir = TempDir::new().expect("Failed to create temp dir");
let (query_manager, blob_manager, _rx) = yaak_models::init_standalone(
&temp_dir.path().join("db.sqlite"),
&temp_dir.path().join("blobs.sqlite"),
)
.expect("Failed to initialize DB");
query_manager
.connect()
.upsert_workspace(
&Workspace { id: "wk_test".to_string(), ..Default::default() },
&UpdateSource::Sync,
)
.expect("Failed to seed workspace");
(query_manager, blob_manager, temp_dir)
}
fn cookie(name: &str) -> Cookie {
Cookie {
name: name.to_string(),
+26 -5
View File
@@ -63,6 +63,10 @@ const HANDLERS: Partial<Record<AppCmd, Handler>> = {
db.rpc("models_get_graphql_introspection", payload),
models_upsert_graphql_introspection: (payload, db) =>
db.rpc("models_upsert_graphql_introspection", payload),
models_snapshot_request: (payload, db) => db.rpc("models_snapshot_request", payload),
models_request_version: (payload, db) => db.rpc("models_request_version", payload),
models_restore_request_version: (payload, db) =>
db.rpc("models_restore_request_version", payload),
models_grpc_events: (payload, db) => db.rpc("models_grpc_events", payload),
models_websocket_events: (payload, db) => db.rpc("models_websocket_events", payload),
cmd_get_workspace_meta: (payload, db) => db.rpc("cmd_get_workspace_meta", payload),
@@ -76,7 +80,12 @@ const HANDLERS: Partial<Record<AppCmd, Handler>> = {
cmd_send_http_request: (payload, db) => {
const requestId = str(payload, "requestId");
if (requestId == null) throw new Error("cmd_send_http_request needs a requestId");
return sendHttpRequest(db, requestId, str(payload, "environmentId"), str(payload, "cookieJarId"));
return sendHttpRequest(
db,
requestId,
str(payload, "environmentId"),
str(payload, "cookieJarId"),
);
},
/* -------------------------------- app ---------------------------------- */
@@ -262,10 +271,16 @@ const DECLINED: Partial<Record<AppCmd, [reason: string, capability: CapabilityNa
cmd_ws_connect: ["WebSocket requests aren't available in the browser yet", "websocket"],
cmd_ws_send: ["WebSocket requests aren't available in the browser yet", "websocket"],
cmd_ws_close: ["WebSocket requests aren't available in the browser yet", "websocket"],
cmd_ws_delete_connections: ["WebSocket requests aren't available in the browser yet", "websocket"],
cmd_ws_delete_connections: [
"WebSocket requests aren't available in the browser yet",
"websocket",
],
// Anything that needs files the page can't reach.
cmd_import_data: ["Importing from a file needs a filesystem, which a browser tab has no", "localFiles"],
cmd_import_data: [
"Importing from a file needs a filesystem, which a browser tab has no",
"localFiles",
],
cmd_import_url: ["Importing from a URL needs the Yaak server, which isn't available yet", null],
cmd_commit_import: ["Importing needs a plugin, which this host doesn't run", null],
cmd_list_import_sources: ["Importing isn't available in the browser yet", null],
@@ -298,8 +313,14 @@ const DECLINED: Partial<Record<AppCmd, [reason: string, capability: CapabilityNa
cmd_plugins_uninstall: ["Plugins aren't available in the browser yet", "plugins"],
cmd_plugins_updates: ["Plugins aren't available in the browser yet", "plugins"],
cmd_plugins_update_all: ["Plugins aren't available in the browser yet", "plugins"],
cmd_template_function_config: ["Template functions come from plugins, which this host doesn't run", "plugins"],
cmd_template_tokens_to_string: ["Template functions come from plugins, which this host doesn't run", "plugins"],
cmd_template_function_config: [
"Template functions come from plugins, which this host doesn't run",
"plugins",
],
cmd_template_tokens_to_string: [
"Template functions come from plugins, which this host doesn't run",
"plugins",
],
cmd_call_http_request_action: ["Plugins aren't available in the browser yet", "plugins"],
cmd_call_websocket_request_action: ["Plugins aren't available in the browser yet", "plugins"],
cmd_call_grpc_request_action: ["Plugins aren't available in the browser yet", "plugins"],
+41 -3
View File
@@ -30,6 +30,7 @@ import type {
HttpResponse,
HttpResponseEventData,
HttpSendSettings,
ModelVersion,
} from "@yaakapp-internal/models";
import type { Frame, SendRequest } from "@yaakapp-internal/web";
import type { WorkerConnection } from "./connection";
@@ -71,7 +72,13 @@ export async function sendHttpRequest(
// a failure to render or to reach the server lands in the response pane as
// that response's error rather than as a toast that names no request.
const workspaceId = await workspaceIdOfRequest(db, requestId);
const response = new ResponseWriter(db, { model: "http_response", requestId, workspaceId });
const versionId = await snapshotRequestVersion(db, requestId);
const response = new ResponseWriter(db, {
model: "http_response",
requestId,
workspaceId,
versionId,
});
await response.create();
const cancel = new AbortController();
@@ -88,6 +95,29 @@ export async function sendHttpRequest(
return response.current();
}
/**
* Capture what is about to be sent, so the response can offer it back later.
* The desktop does this inside its send pipeline; this host's pipeline is here,
* so this is where it goes. Versions are content-addressed, so repeated sends
* of an unchanged request all point at the same one.
*/
async function snapshotRequestVersion(
db: WorkerConnection,
requestId: string,
): Promise<string | undefined> {
try {
const version = await db.rpc<ModelVersion>("models_snapshot_request", {
requestId,
reason: "send",
});
return version.id;
} catch (err) {
// History is not worth failing a send over
console.warn("Failed to snapshot request version", err);
return undefined;
}
}
async function runSend(
db: WorkerConnection,
response: ResponseWriter,
@@ -315,8 +345,16 @@ class TimelineWriter {
* yaak-models), so an edit made while the send was in flight survives rather
* than being written over by the send's stale snapshot.
*/
async function persistCookies(db: WorkerConnection, jar: CookieJar, cookies: Cookie[]): Promise<void> {
await db.rpc("web_persist_send_cookies", { cookieJarId: jar.id, before: jar.cookies, after: cookies });
async function persistCookies(
db: WorkerConnection,
jar: CookieJar,
cookies: Cookie[],
): Promise<void> {
await db.rpc("web_persist_send_cookies", {
cookieJarId: jar.id,
before: jar.cookies,
after: cookies,
});
}
/**