cynarAI 63f96b6edd fix(ci): restore the web client and the failing product checks (#83)
* fix(ci): restore browser preview module and onboarding e2e

The web bundle imported a browser-preview store that was never added, so production builds and every Playwright signup failed before the shell rendered. Mobile check failed because Expo packages were a patch behind the SDK.

Restore the store, type the optional desktop bridge, and walk whichever onboarding step scripted CI actually opens.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(ci): stop the blank web client and the send 500

The core barrel pulled git checkout into the browser, so node:child_process crashed the page before sign-up. Serve that module from a node export instead.

Add the missing run tool-call budget columns that threads/send writes. Guard an indexed shell flag for typecheck, and expect the files key the desktop preload already exposes.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(ci): isolate pnpm setup and the takeover membership

The Team Computer takeover journey inserted a space member the
default-space trigger had already created, so shared Postgres rejected
the unique pair. Self-hosted jobs now keep a per-job pnpm home and
store, and artifact uploads no longer fail a job that already passed.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(ci): clear typecheck, lint, and unit failures

Inbound channel rosters can mark a participant list as authoritative.
Answered channel asks stay off the group, and clearing a thread waits
until delegated returns land. Lint, unit, and web e2e contracts follow
the current shell, composer, and settings.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): align e2e with the shipped shell and onboarding

Web E2E still expected older focus buttons, nested project rows, and
settings that never opened MCP. Persist the signed-in space, list
non-default spaces under Projekte, and keep the quote control on screen.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): open group reply from a revealed hover rail

The first user row covered its Reply control until the rail accepted
pointer events. The group transcript now reveals that rail, then replies.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): stop the shell render loop and settle computer and mention UI

A working bot rewrote preview state on every render, the computer menu never stayed still, and the mention picker did not expose its expanded state or option name.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(ci): stop idle probes hanging and accept a follow-up send

Background probes walked every process fd and timed out on the shared runner. Linux now tracks the marker with flock and /proc/locks. A new composer draft can send while an earlier send is still refreshing, and group members stay mentionable.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(adapters): detect background work when flock ids are padded

The idle probe compared /proc/locks with an unpadded device key, so majors
or minors under 16 never matched. It deleted the live marker and reported
idle while the command still held the lock. Probe by taking the lock, and
print the cancel key the way the kernel does.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(adapters): stabilize background-work probe tests on self-hosted CI

Clear stale marker files before each probe scenario and allow longer
polling so launch can acquire flock on busy persistent runners.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): do not block onboarding on spaces list

Render the first onboarding step from me and integration setup alone.
Resolve the optional skip-to-app redirect after that so Playwright
signups are not stuck on the loading shell when spaces.list is slow.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(ci): restore green gates after second main rebase

Re-apply biome formatting on Shell/SortableGroup and strict TypeScript
assertions in computer-idle tests dropped when skipping the duplicate
conflict-resolution commit.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): wait for onboarding bootstrap before E2E proceeds

Expose data-onboarding-step on the onboarding shell and have Playwright
wait until loading finishes. Ignore ad-hoc screenshot scripts on
self-hosted runners so they do not fail lint.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): fall back when onboarding bootstrap stalls

Race me/integration setup against a timeout and promote loading to the
emoji step after 20s so Playwright is not blocked on a hung RPC.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): render onboarding immediately for E2E signups

Start on the emoji step instead of a loading shell that waits on RPC.
Bootstrap can still redirect once me and integration setup resolve.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(ci): stabilize web e2e on self-hosted runners

Per-job pnpm stores broke node_modules symlinks when setup-node restored
cached installs, so Vite could not resolve react during dependency
optimization. Keep a runner-scoped store, clear stale Vite caches, and pin
Playwright to one worker in CI to reduce onboarding RPC contention.

Add Vite resolve aliases for react/react-dom, wait for an interactive
onboarding step in E2E helpers, and include the React hook smoke spec from
main.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): resolve React aliases through pnpm package layout

Root node_modules/react does not exist under pnpm, so Vite aliases pointed
at missing paths and broke production builds and the Playwright webServer.
Resolve react and react-dom from the web package and map jsx-runtime to the
actual entry files.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(core): type the lazy child_process spawn cache

Main's lazy require left spawn typed as null after assignment, which broke
Typecheck on the rebased branch.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(db): mock git-checkout export path in unit tests

Vitest resolves vi.mock against package exports; the .js suffix is not
declared for @rakazo/core/git-checkout.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): stabilize E2E after main sidebar and onboarding changes

Use first() for duplicate transcript lines, wait for interactive onboarding
before model-required assertions, and allow project sidebar to settle after
create. Extend the Playwright job timeout so the single-worker suite can finish.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): restore Projekte sidebar and integrations MCP entry

Use the default space for Unterhaltungen/Projekte separation instead of the
active space header, persist the bootstrapped space in localStorage, wire
SettingsOverlay to MCP management, and compare hover tones on inactive rows.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): satisfy biome format on homeSpaceId line

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): align sidebar E2E with nested Projekte shell layout

Scope Unterhaltungen assertions to conversation rows only, target project-space
Chief actions under .sidebar-projekte, and stabilize hover-tone polling.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix: satisfy lint after main merge (format and unused var)

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): show project-space 1:1 bots in Projekte sidebar

Include lone bots in projectSidebarGroups, render bot rows in Projekte,
and stabilize E2E for nested sidebar layout and duplicate group entries.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): disambiguate duplicate group sidebar buttons in E2E

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): align spaces and group-chats E2E with Projekte sidebar IA

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): dedupe Projekte sidebar rows and stabilize E2E specs

- Deduplicate project space and group-chat entries in projectSidebarGroups
- Scope group-chats/spaces E2E clicks to first Projekte match after IA split
- Wait for shell ready before mention-picker keyboard test; restore main locators
- Fix PrivacySettingsOverlay lint and align biome schema with CLI 2.5.13

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): restrict project drag handle and stabilize space E2E

- Move SortableGroup drag listeners to ProjectDragGrip only so sidebar clicks work
- Scope space delete actions to project-space rows; wait for menu in E2E
- Switch group settings E2E via direct navigation after unsaved draft check
- Merge main (DataTrustPanel) and fix Shell import order for CI lint

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): stabilize message hover Escape assertion in E2E

Assert the More menu closes and the hover rail stays visible instead of
requiring focus restoration, which is flaky across Chromium builds.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* chore: format apps/api/src/app.ts after main merge

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): stabilize team-computer E2E send after bot switch

Wait for shell ready and use the Send button instead of Enter after
switching bots and closing the computer overlay. The prior run timed
out on threads/send with only an apps/api format commit in between.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>
2026-09-23 22:09:49 +02:00

829 lines
32 KiB
TypeScript

import { spawnSync } from "node:child_process";
import http from "node:http";
import net from "node:net";
import { resolveSupervisorToken } from "@rakazo/core";
import { describe, expect, it } from "vitest";
import {
MAX_SUPERVISOR_FILE_REQUEST_BYTES,
MAX_SUPERVISOR_REQUEST_BYTES,
resolveDockerSocketPath,
supervisorApp,
supervisorRequestBodyLimit,
waitForScreenReady,
} from "./index.js";
import {
assertRequestIdentity,
attemptComputerControl,
browserProfilePathForScreen,
ComputerControlUnavailableError,
clearComputerScreenRegistry,
completeReleasedScreen,
computerControlTimeoutMs,
containerActionStep,
containerActionSteps,
DOCKER_BROWSER_ALIASES,
demuxDockerStream,
ensureScreenCommand,
hasComputerIdentity,
hasValidBearerToken,
interactiveScreenCommand,
isComputerControlUnavailable,
nextScreenIndex,
normalizeWorkspaceRelative,
parseObservation,
preferComputerControl,
releaseAssignedScreen,
resetManagedScreensCommand,
type ScreenAssignment,
sandboxCommandTimedOut,
sandboxTimeoutCommand,
screenReleaseStopCommand,
shouldReplayComputerActions,
stopExtraScreenCommand,
teardownReleasedScreen,
withKeyedLock,
} from "./supervisor-logic.js";
const token = resolveSupervisorToken(process.env);
describe("computer screen readiness", () => {
it("waits for the server to actually answer HTTP requests before succeeding", async () => {
const server = http.createServer((_req, res) => res.end("ok"));
await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
const address = server.address();
if (!address || typeof address === "string") throw new Error("expected a TCP address");
try {
await expect(waitForScreenReady("127.0.0.1", address.port, 2_000)).resolves.toBe(true);
} finally {
server.close();
}
});
it("does not treat an open TCP port as ready when nothing is serving HTTP on it yet", async () => {
// Regression test: a bare TCP accept (e.g. the Docker port mapping coming
// up before websockify inside the container does) must not be mistaken
// for the screen being ready — that gap is exactly what caused
// "socket hang up" in the browser.
const server = net.createServer((socket) => socket.destroy());
await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
const address = server.address();
if (!address || typeof address === "string") throw new Error("expected a TCP address");
try {
await expect(waitForScreenReady("127.0.0.1", address.port, 700)).resolves.toBe(false);
} finally {
server.close();
}
});
it("does not treat HTTP error responses as ready", async () => {
const server = http.createServer((_req, res) => {
res.statusCode = 503;
res.end("starting");
});
await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
const address = server.address();
if (!address || typeof address === "string") throw new Error("expected a TCP address");
try {
await expect(waitForScreenReady("127.0.0.1", address.port, 700)).resolves.toBe(false);
} finally {
server.close();
}
});
it("does not treat redirect responses as ready", async () => {
const server = http.createServer((_req, res) => {
res.statusCode = 302;
res.setHeader("Location", "/vnc.html");
res.end();
});
await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
const address = server.address();
if (!address || typeof address === "string") throw new Error("expected a TCP address");
try {
await expect(waitForScreenReady("127.0.0.1", address.port, 700)).resolves.toBe(false);
} finally {
server.close();
}
});
it("times out instead of hanging when nothing is listening", async () => {
const server = net.createServer();
await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
const address = server.address();
if (!address || typeof address === "string") throw new Error("expected a TCP address");
const closedPort = address.port;
await new Promise<void>((resolve) => server.close(() => resolve()));
await expect(waitForScreenReady("127.0.0.1", closedPort, 700)).resolves.toBe(false);
});
});
describe("sandbox supervisor Docker endpoint", () => {
it("respects Docker host and socket overrides before platform defaults", () => {
expect(resolveDockerSocketPath({ DOCKER_HOST: "tcp://docker.test:2375" }, "win32")).toBe(
undefined,
);
expect(resolveDockerSocketPath({ DOCKER_SOCKET: "/tmp/docker.sock" }, "win32")).toBe(
"/tmp/docker.sock",
);
expect(resolveDockerSocketPath({}, "win32")).toBe("//./pipe/docker_engine");
expect(resolveDockerSocketPath({}, "linux")).toBe("/var/run/docker.sock");
});
});
describe("sandbox supervisor HTTP boundary", () => {
it("keeps health public while every computer route requires the service token", async () => {
const health = await supervisorApp.request("/health");
expect(health.status).toBe(200);
await expect(health.json()).resolves.toMatchObject({ ok: true });
const protectedRequests: Array<[string, string]> = [
["POST", "/computers"],
["GET", "/computers/id"],
["POST", "/computers/id/exec"],
["POST", "/computers/id/observe"],
["POST", "/computers/id/actions"],
["POST", "/computers/id/browser"],
["GET", "/computers/id/files"],
["POST", "/computers/id/files"],
["GET", "/computers/id/screen"],
["POST", "/computers/id/screen-mode"],
["DELETE", "/computers/id/screen"],
["POST", "/computers/id/input"],
["POST", "/computers/id/stop"],
["DELETE", "/computers/id"],
];
for (const [method, pathname] of protectedRequests) {
const response = await supervisorApp.request(pathname, { method });
expect(response.status, `${method} ${pathname}`).toBe(401);
await expect(response.json()).resolves.toEqual({ error: "unauthorized" });
}
});
it("rejects malformed and incorrect bearer credentials", () => {
expect(hasValidBearerToken(undefined, token)).toBe(false);
expect(hasValidBearerToken(token, token)).toBe(false);
expect(hasValidBearerToken("Basic credentials", token)).toBe(false);
expect(hasValidBearerToken(`Bearer ${"x".repeat(token.length)}`, token)).toBe(false);
expect(hasValidBearerToken(`Bearer ${token}`, token)).toBe(true);
});
it("bounds authenticated control request bodies before JSON parsing", async () => {
const headers = {
authorization: `Bearer ${token}`,
"content-type": "application/json",
};
const declared = await supervisorApp.request("/computers/id/actions", {
method: "POST",
headers: {
...headers,
"content-length": String(MAX_SUPERVISOR_REQUEST_BYTES + 1),
},
body: "{}",
});
const streamed = await supervisorApp.request("/computers/id/actions", {
method: "POST",
headers,
body: JSON.stringify({ padding: "x".repeat(MAX_SUPERVISOR_REQUEST_BYTES) }),
});
for (const response of [declared, streamed]) {
expect(response.status).toBe(413);
await expect(response.json()).resolves.toEqual({ error: "Request body is too large." });
}
});
it("keeps the larger file-write allowance scoped to that exact POST route", () => {
expect(supervisorRequestBodyLimit("POST", "/computers/id/files")).toBe(
MAX_SUPERVISOR_FILE_REQUEST_BYTES,
);
expect(supervisorRequestBodyLimit("POST", "/computers/id/files/")).toBe(
MAX_SUPERVISOR_FILE_REQUEST_BYTES,
);
expect(supervisorRequestBodyLimit("PUT", "/computers/id/files")).toBe(
MAX_SUPERVISOR_REQUEST_BYTES,
);
expect(supervisorRequestBodyLimit("POST", "/computers/id/actions")).toBe(
MAX_SUPERVISOR_REQUEST_BYTES,
);
expect(supervisorRequestBodyLimit("POST", "/computers/files")).toBe(
MAX_SUPERVISOR_REQUEST_BYTES,
);
expect(supervisorRequestBodyLimit("POST", "/computers/id/files/extra")).toBe(
MAX_SUPERVISOR_REQUEST_BYTES,
);
});
it("rejects a provision request whose identity headers do not match its body", async () => {
const response = await supervisorApp.request("/computers", {
method: "POST",
headers: {
authorization: `Bearer ${token}`,
"content-type": "application/json",
"x-rakazo-bot-id": "other-bot",
"x-rakazo-space-id": "workspace",
},
body: JSON.stringify({
botId: "bot",
spaceId: "workspace",
homePath: "/tmp/never-used",
}),
});
expect(response.status).toBe(500);
await expect(response.json()).resolves.toEqual({ error: "computer identity mismatch" });
});
});
describe("sandbox supervisor input containment", () => {
it("normalizes portable paths and rejects lexical traversal", () => {
expect(normalizeWorkspaceRelative("/notes\\result.txt")).toBe("notes/result.txt");
expect(normalizeWorkspaceRelative("//notes///result.txt")).toBe("notes/result.txt");
expect(() => normalizeWorkspaceRelative("../outside")).toThrow(/escapes/);
// "." is the current directory and stays inside the workspace.
expect(normalizeWorkspaceRelative("notes/./result.txt")).toBe("notes/result.txt");
expect(() => normalizeWorkspaceRelative("notes/../outside")).toThrow(/escapes/);
});
it("requires both bot and workspace identities to match", () => {
expect(() =>
assertRequestIdentity("bot", "workspace", { botId: "bot", spaceId: "workspace" }),
).not.toThrow();
expect(() =>
assertRequestIdentity(undefined, "workspace", { botId: "bot", spaceId: "workspace" }),
).toThrow(/identity mismatch/);
expect(() =>
assertRequestIdentity("bot", "other", { botId: "bot", spaceId: "workspace" }),
).toThrow(/identity mismatch/);
});
it("accepts the legacy workspace label without weakening container identity", () => {
expect(
hasComputerIdentity({ "rakazo.botId": "bot", "rakazo.workspaceId": "space" }, "bot", "space"),
).toBe(true);
expect(
hasComputerIdentity(
{ "rakazo.botId": "bot", "rakazo.workspaceId": "other-space" },
"bot",
"space",
),
).toBe(false);
expect(
hasComputerIdentity(
{
"rakazo.botId": "bot",
"rakazo.spaceId": "space",
"rakazo.workspaceId": "other-space",
},
"bot",
"space",
),
).toBe(true);
});
it("bounds scroll and wait actions before sending them to the computer", () => {
expect(containerActionStep({ kind: "wait", ms: 99_999 })).toEqual({ waitMs: 5_000 });
expect(containerActionStep({ kind: "wait", ms: -1 })).toEqual({ waitMs: 0 });
expect(containerActionStep({ kind: "scroll", direction: "up", amount: 99 })).toEqual({
argv: ["env", "DISPLAY=:1", "xdotool", "click", "--repeat", "20", "4"],
});
});
it("routes Docker browser aliases through the safe wrapper on every display", () => {
for (const application of DOCKER_BROWSER_ALIASES) {
expect(
containerActionStep({ kind: "launch", application, uri: "https://example.com" }, ":2"),
).toEqual({
argv: ["env", "DISPLAY=:2", "rakazo-browser", "https://example.com"],
});
}
expect(containerActionStep({ kind: "launch", application: "xterm" }, ":3")).toEqual({
argv: ["env", "DISPLAY=:3", "xterm"],
});
expect(containerActionStep({ kind: "open", path: "https://example.com" }, ":3")).toEqual({
argv: ["env", "DISPLAY=:3", "xdg-open", "https://example.com"],
});
});
it("routes mixed-case Docker browser aliases through the safe wrapper", () => {
for (const application of ["Chrome", "Firefox", "Chromium", "Google-Chrome"]) {
expect(
containerActionStep({ kind: "launch", application, uri: "https://example.com" }, ":2"),
).toEqual({
argv: ["env", "DISPLAY=:2", "rakazo-browser", "https://example.com"],
});
}
expect(containerActionStep({ kind: "launch", application: "XTerm" }, ":3")).toEqual({
argv: ["env", "DISPLAY=:3", "XTerm"],
});
});
it("keeps browser routing argv identical for control and Docker exec fallback", () => {
const action = { kind: "launch" as const, application: "chromium", uri: "https://example.com" };
const profile = browserProfilePathForScreen("writer");
expect(containerActionSteps([action], ":2", profile)).toEqual([
{
argv: [
"env",
"DISPLAY=:2",
`RAKAZO_BROWSER_PROFILE=${profile}`,
"rakazo-browser",
"https://example.com",
],
},
]);
expect(
containerActionSteps([{ kind: "open", path: "https://example.com" }], ":2", profile),
).toEqual([
{
argv: [
"env",
"DISPLAY=:2",
`RAKAZO_BROWSER_PROFILE=${profile}`,
"xdg-open",
"https://example.com",
],
},
]);
});
it("falls back to docker-exec when computer control fails", async () => {
await expect(
preferComputerControl(
async () => {
throw new Error("connection refused");
},
async () => "docker-exec",
),
).resolves.toBe("docker-exec");
await expect(preferComputerControl(undefined, async () => "docker-exec")).resolves.toBe(
"docker-exec",
);
await expect(
preferComputerControl(
async () => "fast-path",
async () => "docker-exec",
),
).resolves.toBe("fast-path");
});
it("replays actions only when control was never reached", async () => {
await expect(attemptComputerControl(undefined)).resolves.toEqual({ status: "unavailable" });
await expect(
attemptComputerControl(async () => {
throw new ComputerControlUnavailableError("fetch failed");
}),
).resolves.toEqual({ status: "unavailable" });
await expect(
attemptComputerControl(async () => {
throw new Error("computer action failed");
}),
).resolves.toMatchObject({ status: "failed" });
const timeout = Object.assign(new Error("The operation was aborted due to timeout"), {
name: "TimeoutError",
});
await expect(
attemptComputerControl(async () => {
throw timeout;
}),
).resolves.toMatchObject({ status: "failed" });
expect(isComputerControlUnavailable(new TypeError("fetch failed"))).toBe(false);
expect(
isComputerControlUnavailable(
Object.assign(new TypeError("fetch failed"), {
cause: new Error("connect ECONNREFUSED 127.0.0.1:7070"),
}),
),
).toBe(true);
expect(
isComputerControlUnavailable(
Object.assign(new TypeError("fetch failed"), {
cause: new Error("connect ENETUNREACH 172.18.0.4:7070"),
}),
),
).toBe(true);
expect(
isComputerControlUnavailable(
Object.assign(new TypeError("fetch failed"), {
cause: new Error("read ECONNRESET"),
}),
),
).toBe(false);
expect(isComputerControlUnavailable(timeout)).toBe(false);
await expect(attemptComputerControl(async () => ({ completed: 2 }))).resolves.toEqual({
status: "ok",
value: { completed: 2 },
});
});
it("falls back on connection refused but does not replay after a request-sent failure", async () => {
const refused = await attemptComputerControl(async () => {
throw Object.assign(new TypeError("fetch failed"), {
cause: new Error("connect ECONNREFUSED 172.18.0.4:7070"),
});
});
expect(refused).toEqual({ status: "unavailable" });
expect(shouldReplayComputerActions(refused)).toBe(true);
const afterWrite = await attemptComputerControl(async () => {
throw new Error("computer control failed");
});
expect(afterWrite).toMatchObject({ status: "failed" });
expect(shouldReplayComputerActions(afterWrite)).toBe(false);
const timedOut = await attemptComputerControl(async () => {
throw Object.assign(new Error("The operation was aborted due to timeout"), {
name: "TimeoutError",
});
});
expect(timedOut).toMatchObject({ status: "failed" });
expect(shouldReplayComputerActions(timedOut)).toBe(false);
const reset = await attemptComputerControl(async () => {
throw Object.assign(new TypeError("fetch failed"), {
cause: new Error("read ECONNRESET"),
});
});
expect(reset).toMatchObject({ status: "failed" });
expect(shouldReplayComputerActions(reset)).toBe(false);
});
it("extends the computer control deadline for mapped waits", () => {
expect(computerControlTimeoutMs([])).toBe(15_000);
expect(computerControlTimeoutMs([{ kind: "wait", ms: 5_000 }], 5_000)).toBe(25_000);
expect(
computerControlTimeoutMs(
[
{ kind: "wait", ms: 5_000 },
{ kind: "wait", ms: 5_000 },
{ kind: "wait", ms: 5_000 },
{ kind: "wait", ms: 5_000 },
{ kind: "wait", ms: 5_000 },
{ kind: "wait", ms: 5_000 },
{ kind: "wait", ms: 5_000 },
{ kind: "wait", ms: 5_000 },
{ kind: "wait", ms: 5_000 },
{ kind: "wait", ms: 5_000 },
],
5_000,
),
).toBe(60_000);
});
it("wraps sandbox commands in a process-tree timeout", () => {
expect(sandboxTimeoutCommand(["bash", "-lc", "sleep 10"], 2_500, "/tmp/completed-124")).toEqual(
[
"timeout",
"--kill-after=1s",
"2.5s",
"sh",
"-c",
'"$@"; status=$?; if [ "$status" -eq 124 ]; then : > "$0"; fi; exit "$status"',
"/tmp/completed-124",
"bash",
"-lc",
"sleep 10",
],
);
expect(sandboxCommandTimedOut(124, false)).toBe(true);
expect(sandboxCommandTimedOut(124, true)).toBe(false);
expect(sandboxCommandTimedOut(1, false)).toBe(false);
});
it("keeps the viewer read-only and uses a separate process for takeover control", () => {
expect(interactiveScreenCommand(false)).toMatch(/pkill .*sockets\/control-1-/);
expect(interactiveScreenCommand(false)).not.toMatch(/x11vnc -display/);
expect(interactiveScreenCommand(true, "lease-new")).toMatch(
/x11vnc -display .* -rfbport 0 -unixsock .*control-1-/,
);
expect(interactiveScreenCommand(true, "lease-new")).toMatch(/6080/);
expect(interactiveScreenCommand(true, "lease-new")).not.toContain("sockets/view-1-");
expect(interactiveScreenCommand(false, "lease-old")).toContain("= 'lease-old'");
expect(interactiveScreenCommand(false, "lease-old")).toContain("RAKAZO_CONTROL_RELEASED");
});
it("assigns distinct screen indexes per Team bot and starts extra displays", () => {
const assigned = new Map<string, ScreenAssignment>();
expect(nextScreenIndex(assigned, "writer")).toBe(0);
expect(nextScreenIndex(assigned, "researcher")).toBe(1);
expect(nextScreenIndex(assigned, "writer")).toBe(0);
expect(ensureScreenCommand(0, "writer", "view-token")).toContain("-display :1");
expect(ensureScreenCommand(0, "writer", "view-token")).toContain("seq 1 100");
expect(ensureScreenCommand(1, "researcher", "view-token")).toContain("Xvfb :2");
expect(ensureScreenCommand(1, "researcher", "view-token")).toContain("sockets/view-2-");
expect(ensureScreenCommand(1, "researcher", "view-token")).toContain("0.0.0.0:6080");
expect(() => nextScreenIndex(assigned, "overflow", undefined, 1)).toThrow(
/cannot allocate another screen/,
);
});
it("generates syntactically valid shell to start an extra display", () => {
for (const command of [
ensureScreenCommand(0, "writer", "view-token"),
ensureScreenCommand(1, "researcher", "view-token"),
stopExtraScreenCommand(0, "writer"),
stopExtraScreenCommand(1, "researcher"),
interactiveScreenCommand(false, "lease-old"),
interactiveScreenCommand(true, "lease-new"),
resetManagedScreensCommand(),
]) {
const result = spawnSync("bash", ["-n"], { input: command });
expect(result.status).toBe(0);
expect(result.stderr.toString()).toBe("");
}
});
it("serializes screen lifecycle operations by container", async () => {
const locks = new Map<string, Promise<void>>();
const events: string[] = [];
let releaseFirst!: () => void;
const firstBlocked = new Promise<void>((resolve) => {
releaseFirst = resolve;
});
const first = withKeyedLock(locks, "container-1", async () => {
events.push("ensure:start");
await firstBlocked;
events.push("ensure:end");
});
const release = withKeyedLock(locks, "container-1", async () => {
events.push("release");
});
await new Promise<void>((resolve) => setImmediate(resolve));
expect(events).toEqual(["ensure:start"]);
releaseFirst();
await Promise.all([first, release]);
expect(events).toEqual(["ensure:start", "ensure:end", "release"]);
expect(locks.size).toBe(0);
});
it("resets stale managed screens without killing unrelated container jobs", () => {
const command = resetManagedScreensCommand();
expect(command).toContain("chromium-bot-*");
expect(command).toContain("for marker in /tmp/rakazo/browser-profile-*");
expect(command).toContain("/tmp/rakazo/browser-pid-*");
expect(command).not.toContain("pkill -9 -1");
});
it("keeps a persistent independent profile per bot across display slots", () => {
const writer = browserProfilePathForScreen("writer");
const researcher = browserProfilePathForScreen("researcher");
const command = ensureScreenCommand(0, "writer", "view-token");
expect(writer).not.toBe(researcher);
expect(command).toContain(writer);
expect(ensureScreenCommand(3, "writer", "view-token")).toContain(writer);
expect(ensureScreenCommand(0, "researcher", "view-token")).toContain(researcher);
expect(command).not.toContain("/home/rakazo/.browser-profiles/chromium/.");
expect(command).not.toContain(".rakazo-base-generation");
expect(command).toContain("browser-pid-");
expect(command).toContain("tr '\\0' '\\n' <\"/proc/$pid/cmdline\"");
expect(browserProfilePathForScreen("../../writer")).toMatch(
/^\/home\/rakazo\/\.browser-profiles\/chromium-bot-[0-9a-f]+$/,
);
});
it("allocates a thousand bots without a configured cap", () => {
const assigned = new Map<string, ScreenAssignment>();
for (let index = 0; index < 1000; index++)
expect(nextScreenIndex(assigned, `bot-${index}`)).toBe(index);
});
it("frees a released screen slot so a ninth Team bot can reuse it", () => {
const assigned = new Map<string, ScreenAssignment>();
for (let index = 0; index < 8; index += 1) {
expect(nextScreenIndex(assigned, `bot-${index}`)).toBe(index);
}
expect(() => nextScreenIndex(assigned, "bot-8", undefined, 8)).toThrow(
/cannot allocate another screen/,
);
expect(releaseAssignedScreen(assigned, "bot-3")).toBe(3);
expect(assigned.get("bot-0")?.index).toBe(0);
expect(assigned.get("bot-3")?.releasing).toBe(true);
expect(() => nextScreenIndex(assigned, "bot-3")).toThrow(/still being released/);
expect(() => nextScreenIndex(assigned, "bot-8", undefined, 8)).toThrow(
/cannot allocate another screen/,
);
completeReleasedScreen(assigned, "bot-3", 3);
expect(assigned.get("bot-3")).toBeUndefined();
expect(nextScreenIndex(assigned, "bot-8")).toBe(3);
expect(nextScreenIndex(assigned, "bot-0")).toBe(0);
expect(releaseAssignedScreen(assigned, "missing")).toBeUndefined();
expect(nextScreenIndex(assigned, "bot-9")).toBe(8);
});
it("retains a screen slot when teardown fails", async () => {
const assigned = new Map<string, ScreenAssignment>();
expect(nextScreenIndex(assigned, "writer")).toBe(0);
expect(releaseAssignedScreen(assigned, "writer")).toBe(0);
await expect(
teardownReleasedScreen(assigned, "writer", 0, async () => ({
code: 1,
stderr: "browser still running",
})),
).rejects.toThrow("browser still running");
expect(assigned.get("writer")).toEqual({ index: 0, releasing: true });
expect(() => nextScreenIndex(assigned, "writer")).toThrow(/still being released/);
expect(nextScreenIndex(assigned, "researcher")).toBe(1);
expect(releaseAssignedScreen(assigned, "writer")).toBe(0);
});
it("clears all screen assignments when a container stops so slots can be reused", () => {
const registry = new Map<string, Map<string, ScreenAssignment>>();
const containerId = "container-1";
const assigned = new Map<string, ScreenAssignment>();
registry.set(containerId, assigned);
for (let index = 0; index < 8; index += 1) {
nextScreenIndex(assigned, `bot-${index}`);
}
expect(() => nextScreenIndex(assigned, "bot-8", undefined, 8)).toThrow(
/cannot allocate another screen/,
);
clearComputerScreenRegistry(registry, containerId);
expect(registry.has(containerId)).toBe(false);
const fresh = new Map<string, ScreenAssignment>();
registry.set(containerId, fresh);
for (let index = 0; index < 8; index += 1) {
expect(nextScreenIndex(fresh, `bot-fresh-${index}`)).toBe(index);
}
});
it("does not release a screen reclaimed by a newer execution fence", () => {
const assigned = new Map<string, ScreenAssignment>();
expect(nextScreenIndex(assigned, "writer", "run-1:1")).toBe(0);
expect(nextScreenIndex(assigned, "writer", "run-2:2")).toBe(0);
expect(releaseAssignedScreen(assigned, "writer", "run-1:1")).toBeUndefined();
expect(nextScreenIndex(assigned, "researcher")).toBe(1);
expect(releaseAssignedScreen(assigned, "writer", "run-2:2")).toBe(0);
completeReleasedScreen(assigned, "writer", 0);
});
it("releases a retained screen after the same run resumes", () => {
const assigned = new Map<string, ScreenAssignment>();
expect(nextScreenIndex(assigned, "writer", "run-1:1")).toBe(0);
expect(releaseAssignedScreen(assigned, "writer", "run-1:8")).toBe(0);
completeReleasedScreen(assigned, "writer", 0);
expect(nextScreenIndex(assigned, "researcher")).toBe(0);
});
it("does not let a delayed request restore an older lease", () => {
const assigned = new Map<string, ScreenAssignment>();
expect(nextScreenIndex(assigned, "writer", "run-2:2")).toBe(0);
expect(() => nextScreenIndex(assigned, "writer", "run-1:1")).toThrow(
/owned by a newer execution/,
);
expect(releaseAssignedScreen(assigned, "writer", "run-1:1")).toBeUndefined();
expect(releaseAssignedScreen(assigned, "writer", "run-2:2")).toBe(0);
});
it("does not let a closing viewer release a screen claimed by a run", () => {
const assigned = new Map<string, ScreenAssignment>();
expect(nextScreenIndex(assigned, "writer", "screen-view-writer:0")).toBe(0);
expect(nextScreenIndex(assigned, "writer", "run-1:1")).toBe(0);
expect(releaseAssignedScreen(assigned, "writer", "screen-view-writer:0")).toBeUndefined();
expect(releaseAssignedScreen(assigned, "writer", "run-1:1")).toBe(0);
});
it("stops the released bot's browser without tearing down the primary desktop", () => {
const primary = stopExtraScreenCommand(0, "writer");
expect(primary).toContain(`--user-data-dir=${browserProfilePathForScreen("writer")}`);
expect(primary).toContain("kill -KILL");
expect(primary).not.toMatch(/Xvfb :1 /);
expect(primary).not.toContain("websockify");
expect(primary).toContain("sockets/view-1-");
expect(primary).toContain("sockets/control-1-");
expect(primary).toContain("rm -f /tmp/rakazo/control-token-1");
expect(primary).toContain("transport failed to stop");
const extra = stopExtraScreenCommand(1, "researcher");
expect(extra).toContain("[X]vfb :2 -screen");
expect(extra).toContain("[f]luxbox -rc /tmp/fluxbox-home-2/.fluxbox/init");
expect(extra).toContain("sockets/view-2-");
expect(extra).not.toContain("websockify");
expect(extra).toContain(`--user-data-dir=${browserProfilePathForScreen("researcher")}`);
});
it("on cancel, stops only the matching bot's primary-display Chromium", () => {
const stop = stopExtraScreenCommand(0, "writer");
expect(stop).toContain(`--user-data-dir=${browserProfilePathForScreen("writer")}`);
expect(stop).not.toContain(`--user-data-dir=${browserProfilePathForScreen("researcher")}`);
expect(stop).not.toContain("Xvfb");
expect(stop).not.toContain("fluxbox");
});
it("does not stop the primary browser when a present registry rejects release", () => {
expect(
screenReleaseStopCommand(undefined, {
hasRegistry: true,
cancelRunWork: true,
screenId: "writer",
}),
).toBe("");
});
it("still stops the matching bot browser on cancel when the screen registry is missing", () => {
// After a supervisor restart, in-memory assignments are gone; cancel must still
// tear down the bot's orphaned Chromium without falling back on a rejected lease.
const orphanCancelStop = screenReleaseStopCommand(undefined, {
hasRegistry: false,
cancelRunWork: true,
screenId: "writer",
});
expect(orphanCancelStop).toContain(`--user-data-dir=${browserProfilePathForScreen("writer")}`);
expect(orphanCancelStop).not.toContain("Xvfb");
});
it("parses a captured frame without trusting optional desktop metadata", () => {
expect(
parseObservation(
[
"GEOM 1280 800",
"CURSOR X=12 Y=34 SCREEN=0 WINDOW=99",
"WINDOW 99",
"TITLE Browser",
"IMAGE AQID",
].join("\n"),
),
).toEqual({
image: "AQID",
mimeType: "image/png",
width: 1280,
height: 800,
cursor: { x: 12, y: 34 },
activeWindow: { id: "99", title: "Browser" },
});
expect(() => parseObservation("GEOM 1280 800\nIMAGE ")).toThrow(/no image/);
});
});
describe("docker exec stream demux", () => {
const frame = (type: number, text: string) => {
const payload = Buffer.from(text, "utf8");
const header = Buffer.alloc(8);
header[0] = type;
header.writeUInt32BE(payload.length, 4);
return Buffer.concat([header, payload]);
};
it("keeps stderr frames out of stdout", () => {
const stream = Buffer.concat([
frame(1, "aGVsbG8="),
frame(2, "python: DeprecationWarning\n"),
frame(1, "\n"),
]);
expect(demuxDockerStream(stream)).toEqual({
stdout: "aGVsbG8=\n",
stderr: "python: DeprecationWarning\n",
});
});
it("treats a raw tty stream as stdout", () => {
expect(demuxDockerStream(Buffer.from("plain output\n"))).toEqual({
stdout: "plain output\n",
stderr: "",
});
expect(demuxDockerStream(Buffer.alloc(0))).toEqual({ stdout: "", stderr: "" });
});
it("falls back to raw stdout when the stream is not a complete multiplexed sequence", () => {
const cut = Buffer.concat([frame(1, "kept"), frame(2, "truncated stderr")]).subarray(
0,
12 + 8 + 6,
);
expect(demuxDockerStream(cut)).toEqual({ stdout: cut.toString("utf8"), stderr: "" });
const dangling = Buffer.concat([frame(1, "ok"), Buffer.from([1, 0, 0])]);
expect(demuxDockerStream(dangling)).toEqual({
stdout: dangling.toString("utf8"),
stderr: "",
});
});
it("treats raw payloads that begin with 0x01 or 0x02 as stdout when not valid frames", () => {
// size 0xffffffff does not fit remaining bytes → not a complete multiplexed stream
const raw01 = Buffer.from([0x01, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0x41, 0x42]);
expect(demuxDockerStream(raw01)).toEqual({
stdout: raw01.toString("utf8"),
stderr: "",
});
const raw02 = Buffer.from([0x02, 0x00, 0x00, 0x00, 0xff, 0xff, 0xff, 0xff, 0x43, 0x44]);
expect(demuxDockerStream(raw02)).toEqual({
stdout: raw02.toString("utf8"),
stderr: "",
});
});
it("rejects frames with nonzero reserved header padding as raw stdout", () => {
// type 1, nonzero padding, size 0 — would look like an empty stdout frame without the check
const padded = Buffer.from([0x01, 0xff, 0xff, 0xff, 0x00, 0x00, 0x00, 0x00]);
expect(demuxDockerStream(padded)).toEqual({
stdout: padded.toString("utf8"),
stderr: "",
});
});
});