evOo-local/apps/web/e2e/golden.spec.ts
cynarAI 63f96b6edd fix(ci): restore the web client and the failing product checks (#83)
* fix(ci): restore browser preview module and onboarding e2e

The web bundle imported a browser-preview store that was never added, so production builds and every Playwright signup failed before the shell rendered. Mobile check failed because Expo packages were a patch behind the SDK.

Restore the store, type the optional desktop bridge, and walk whichever onboarding step scripted CI actually opens.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(ci): stop the blank web client and the send 500

The core barrel pulled git checkout into the browser, so node:child_process crashed the page before sign-up. Serve that module from a node export instead.

Add the missing run tool-call budget columns that threads/send writes. Guard an indexed shell flag for typecheck, and expect the files key the desktop preload already exposes.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(ci): isolate pnpm setup and the takeover membership

The Team Computer takeover journey inserted a space member the
default-space trigger had already created, so shared Postgres rejected
the unique pair. Self-hosted jobs now keep a per-job pnpm home and
store, and artifact uploads no longer fail a job that already passed.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(ci): clear typecheck, lint, and unit failures

Inbound channel rosters can mark a participant list as authoritative.
Answered channel asks stay off the group, and clearing a thread waits
until delegated returns land. Lint, unit, and web e2e contracts follow
the current shell, composer, and settings.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): align e2e with the shipped shell and onboarding

Web E2E still expected older focus buttons, nested project rows, and
settings that never opened MCP. Persist the signed-in space, list
non-default spaces under Projekte, and keep the quote control on screen.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): open group reply from a revealed hover rail

The first user row covered its Reply control until the rail accepted
pointer events. The group transcript now reveals that rail, then replies.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): stop the shell render loop and settle computer and mention UI

A working bot rewrote preview state on every render, the computer menu never stayed still, and the mention picker did not expose its expanded state or option name.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(ci): stop idle probes hanging and accept a follow-up send

Background probes walked every process fd and timed out on the shared runner. Linux now tracks the marker with flock and /proc/locks. A new composer draft can send while an earlier send is still refreshing, and group members stay mentionable.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(adapters): detect background work when flock ids are padded

The idle probe compared /proc/locks with an unpadded device key, so majors
or minors under 16 never matched. It deleted the live marker and reported
idle while the command still held the lock. Probe by taking the lock, and
print the cancel key the way the kernel does.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(adapters): stabilize background-work probe tests on self-hosted CI

Clear stale marker files before each probe scenario and allow longer
polling so launch can acquire flock on busy persistent runners.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): do not block onboarding on spaces list

Render the first onboarding step from me and integration setup alone.
Resolve the optional skip-to-app redirect after that so Playwright
signups are not stuck on the loading shell when spaces.list is slow.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(ci): restore green gates after second main rebase

Re-apply biome formatting on Shell/SortableGroup and strict TypeScript
assertions in computer-idle tests dropped when skipping the duplicate
conflict-resolution commit.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): wait for onboarding bootstrap before E2E proceeds

Expose data-onboarding-step on the onboarding shell and have Playwright
wait until loading finishes. Ignore ad-hoc screenshot scripts on
self-hosted runners so they do not fail lint.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): fall back when onboarding bootstrap stalls

Race me/integration setup against a timeout and promote loading to the
emoji step after 20s so Playwright is not blocked on a hung RPC.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): render onboarding immediately for E2E signups

Start on the emoji step instead of a loading shell that waits on RPC.
Bootstrap can still redirect once me and integration setup resolve.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(ci): stabilize web e2e on self-hosted runners

Per-job pnpm stores broke node_modules symlinks when setup-node restored
cached installs, so Vite could not resolve react during dependency
optimization. Keep a runner-scoped store, clear stale Vite caches, and pin
Playwright to one worker in CI to reduce onboarding RPC contention.

Add Vite resolve aliases for react/react-dom, wait for an interactive
onboarding step in E2E helpers, and include the React hook smoke spec from
main.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): resolve React aliases through pnpm package layout

Root node_modules/react does not exist under pnpm, so Vite aliases pointed
at missing paths and broke production builds and the Playwright webServer.
Resolve react and react-dom from the web package and map jsx-runtime to the
actual entry files.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(core): type the lazy child_process spawn cache

Main's lazy require left spawn typed as null after assignment, which broke
Typecheck on the rebased branch.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(db): mock git-checkout export path in unit tests

Vitest resolves vi.mock against package exports; the .js suffix is not
declared for @rakazo/core/git-checkout.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): stabilize E2E after main sidebar and onboarding changes

Use first() for duplicate transcript lines, wait for interactive onboarding
before model-required assertions, and allow project sidebar to settle after
create. Extend the Playwright job timeout so the single-worker suite can finish.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): restore Projekte sidebar and integrations MCP entry

Use the default space for Unterhaltungen/Projekte separation instead of the
active space header, persist the bootstrapped space in localStorage, wire
SettingsOverlay to MCP management, and compare hover tones on inactive rows.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): satisfy biome format on homeSpaceId line

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): align sidebar E2E with nested Projekte shell layout

Scope Unterhaltungen assertions to conversation rows only, target project-space
Chief actions under .sidebar-projekte, and stabilize hover-tone polling.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix: satisfy lint after main merge (format and unused var)

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): show project-space 1:1 bots in Projekte sidebar

Include lone bots in projectSidebarGroups, render bot rows in Projekte,
and stabilize E2E for nested sidebar layout and duplicate group entries.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): disambiguate duplicate group sidebar buttons in E2E

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): align spaces and group-chats E2E with Projekte sidebar IA

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): dedupe Projekte sidebar rows and stabilize E2E specs

- Deduplicate project space and group-chat entries in projectSidebarGroups
- Scope group-chats/spaces E2E clicks to first Projekte match after IA split
- Wait for shell ready before mention-picker keyboard test; restore main locators
- Fix PrivacySettingsOverlay lint and align biome schema with CLI 2.5.13

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): restrict project drag handle and stabilize space E2E

- Move SortableGroup drag listeners to ProjectDragGrip only so sidebar clicks work
- Scope space delete actions to project-space rows; wait for menu in E2E
- Switch group settings E2E via direct navigation after unsaved draft check
- Merge main (DataTrustPanel) and fix Shell import order for CI lint

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): stabilize message hover Escape assertion in E2E

Assert the More menu closes and the hover rail stays visible instead of
requiring focus restoration, which is flaky across Chromium builds.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* chore: format apps/api/src/app.ts after main merge

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

* fix(web): stabilize team-computer E2E send after bot switch

Wait for shell ready and use the Send button instead of Enter after
switching bots and closing the computer overlay. The prior run timed
out on threads/send with only an apps/api format commit in between.

Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>
2026-09-23 22:09:49 +02:00

503 lines
25 KiB
TypeScript
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

import { expect, type Page, test } from "@playwright/test";
import {
activeBotId,
captureScreenshot,
completeOnboarding,
openUserSettings,
realSandboxTimeout,
rpc,
signup,
} from "./helpers";
function sidebarBotButton(page: Page, name: RegExp | string) {
return page.locator("[data-sidebar-group] [data-roster-bot-id]").filter({
has: page.locator("[data-roster-bot-name]").filter({ hasText: name }),
});
}
test.describe.configure({ mode: "serial" });
test("two users are isolated and a bot completes durable work", async ({ browser }, testInfo) => {
const a = await browser.newContext();
const b = await browser.newContext();
const pageA = await a.newPage();
const pageB = await b.newPage();
const stamp = Date.now();
await signup(pageA, `ada-${stamp}@rakazo.test`, "password12", "Ada", testInfo);
await completeOnboarding(pageA, testInfo);
await expect(pageA.getByText("Chief").first()).toBeVisible();
await signup(pageB, `bob-${stamp}@rakazo.test`, "password12", "Bob");
await completeOnboarding(pageB);
await expect(pageB.getByText("Chief").first()).toBeVisible();
await expect(pageB.getByText("Ada", { exact: true })).toHaveCount(0);
const composer = pageA.getByPlaceholder(/Message/);
await composer.fill("write a file in your home called notes/result.txt that says isolation-ok");
await pageA.keyboard.press("Enter");
await expect(
pageA.getByText(/writing that into my home|isolation-ok|handled/i).first(),
).toBeVisible({
timeout: 30_000,
});
await pageA.reload();
await expect(pageA.getByText(/isolation-ok|writing that into my home/i).first()).toBeVisible();
await captureScreenshot(pageA, testInfo, "07-durable-bot-work");
await a.close();
await b.close();
});
test("takeover, routine, plugins, and export are reachable", async ({ page }, testInfo) => {
const stamp = Date.now();
await signup(page, `flow-${stamp}@rakazo.test`, "password12", "Flow");
await completeOnboarding(page);
const composer = page.getByPlaceholder(/Message/);
await composer.fill("install the gsc cli and sign in");
await page.keyboard.press("Enter");
await expect(
page.getByText(/handing you the computer|sign in to continue|protected input/i).first(),
).toBeVisible({ timeout: realSandboxTimeout(90_000, 30_000) });
await expect
.poll(() => threadRunStatus(page), {
timeout: realSandboxTimeout(90_000, 30_000),
message: "the protected-input run must be ready for takeover",
})
.toBe("waiting_takeover");
await captureScreenshot(page, testInfo, "08-protected-input-request");
await page.getByTitle("Agent computer").click();
const sidePanel = page.getByTestId("side-panel");
await expect(sidePanel).toHaveCSS("width", "384px");
const [mainBox, panelBox] = await Promise.all([
page.locator("main").boundingBox(),
sidePanel.boundingBox(),
]);
expect(mainBox).not.toBeNull();
expect(panelBox).not.toBeNull();
expect((mainBox?.x ?? 0) + (mainBox?.width ?? 0)).toBeLessThanOrEqual(panelBox?.x ?? 0);
await sidePanel.getByTestId("computer-preview").hover();
await sidePanel.getByTestId("computer-preview-open").click();
await expect(page.getByRole("button", { name: "Close computer" })).toBeVisible();
await expect(page.getByRole("button", { name: "Skip", exact: true }).last()).toBeVisible();
await expect(page.getByRole("button", { name: "I’m done", exact: true }).last()).toBeVisible();
if (process.env.SANDBOX_PROVIDER === "box") await waitForBoxFramebuffer(page);
await captureScreenshot(page, testInfo, "09-computer-takeover-outcomes");
await page.getByRole("button", { name: "I’m done", exact: true }).last().click();
await expect(page.getByRole("button", { name: "Close computer" })).toBeHidden();
await expect(page.getByText(/signed in|session stays/i).first()).toBeVisible({
timeout: realSandboxTimeout(90_000, 30_000),
});
await composer.fill("sign in again so I can skip this time");
await page.keyboard.press("Enter");
await expect
.poll(() => threadRunStatus(page), {
timeout: realSandboxTimeout(90_000, 30_000),
message: "the second protected-input run must be ready for takeover",
})
.toBe("waiting_takeover");
// Agent computer toggles the panel. Re-open when closed so Open can refresh computer status.
if ((await sidePanel.getAttribute("data-panel")) === "computer") {
await page.getByTitle("Agent computer").click();
}
await page.getByTitle("Agent computer").click();
await expect(sidePanel).toHaveAttribute("data-panel", "computer");
await sidePanel.getByTestId("computer-preview").hover();
const openComputer = sidePanel.getByTestId("computer-preview-open");
await expect(openComputer).toBeVisible({ timeout: 30_000 });
await openComputer.click();
await expect(page.getByRole("button", { name: "Close computer" })).toBeVisible();
await page.getByRole("button", { name: "Skip", exact: true }).last().click();
await expect(page.getByRole("button", { name: "Close computer" })).toBeHidden();
await expect(page.getByText(/login was skipped/i).last()).toBeVisible({
timeout: realSandboxTimeout(90_000, 30_000),
});
await captureScreenshot(page, testInfo, "09a-computer-takeover-skipped");
await page.getByRole("button", { name: "Create Routine" }).click();
await page.locator("label:has-text('Name') input").fill("Monday briefing");
await page
.locator("label:has-text('Instruction') textarea")
.fill("write a file in your home called notes/result.txt that says routine-ok");
await page.getByRole("button", { name: "Add trigger" }).click();
await page.getByRole("menuitem", { name: "On a schedule" }).hover();
await page.getByRole("menuitem", { name: "Every day", exact: true }).click();
const savedRoutine = page.waitForResponse(
(response) => response.url().includes("/rpc/routines/create") && response.ok(),
);
await page.getByRole("button", { name: "Save" }).click();
await savedRoutine;
await expect(page.getByRole("button", { name: "Save" })).toBeEnabled();
await page.getByRole("button", { name: "Back" }).click();
await expect(page.getByText("Monday briefing")).toBeVisible();
await captureScreenshot(page, testInfo, "10-routine-created");
await openUserSettings(page, "integrations");
await expect(page.getByPlaceholder("Search apps")).toBeVisible();
const featured = page.getByTestId("featured-connectors");
await expect(featured).toContainText(
/Gmail[\s\S]*Google Calendar[\s\S]*Google Drive[\s\S]*Slack[\s\S]*Notion/,
);
await expect(page.getByText("GitHub", { exact: true })).toBeVisible();
await expect(page.getByRole("button", { name: "Add Executor", exact: true })).toBeHidden();
await expect(page.getByRole("button", { name: "Add Treg", exact: true })).toBeHidden();
await expect(page.getByRole("button", { name: "Add MCP server", exact: true })).toBeHidden();
await expect(page.getByRole("button", { name: "Add OpenAPI", exact: true })).toBeHidden();
await expect(page.getByRole("button", { name: "Add GraphQL", exact: true })).toBeHidden();
await expect(page.getByText("Tool sources", { exact: true })).toBeHidden();
await expect(
page.getByText("Connect apps or add Treg, MCP, and OpenAPI tool sources.", { exact: true }),
).toBeHidden();
await captureScreenshot(page, testInfo, "11-plugins-catalog");
// Nearest ancestor with an Add/Added control (featured tile or catalog row).
const gmailRow = featured
.getByText("Gmail", { exact: true })
.locator("xpath=ancestor::*[.//button][1]");
await gmailRow.getByRole("button", { name: "Add", exact: true }).click();
await expect(gmailRow.getByRole("button", { name: "Added", exact: true })).toBeVisible();
await expect(page.getByTestId("connection-tile-gmail")).toBeVisible();
await captureScreenshot(page, testInfo, "11a-connected-plugins");
await gmailRow.getByRole("button", { name: "Added", exact: true }).click();
const detail = page.getByTestId("connection-detail");
await expect(detail).toBeVisible();
await expect(page.getByTestId("connection-accounts")).toBeVisible();
await expect(page.getByTestId("connection-tools")).toBeVisible();
await detail.getByRole("button", { name: "Add another", exact: true }).click();
await expect(detail.getByLabel("Account label")).toHaveCount(2);
await detail.getByLabel("Account label").nth(1).fill("Work");
const renamed = page.waitForResponse(
(response) => response.url().includes("connections/rename") && response.ok(),
);
await detail.getByLabel("Account label").nth(1).blur();
await renamed;
await expect(detail.getByLabel("Account label").nth(1)).toHaveValue("Work");
await captureScreenshot(page, testInfo, "11a2-multi-account-plugins");
await page.getByRole("button", { name: "Close integrations settings" }).click();
await openUserSettings(page, "integrations");
await expect(page.getByPlaceholder("Search apps")).toBeVisible();
const gmailTileAgain = page.getByTestId("connection-tile-gmail");
await expect(gmailTileAgain.getByRole("button", { name: "Added", exact: true })).toBeVisible();
await gmailTileAgain.click();
const detailAgain = page.getByTestId("connection-detail");
await expect(detailAgain.getByLabel("Account label")).toHaveCount(2);
await expect(detailAgain.getByLabel("Account label").nth(1)).toHaveValue("Work");
await detailAgain.getByRole("button", { name: "Remove", exact: true }).last().click();
await expect(detailAgain.getByLabel("Account label")).toHaveCount(1);
await detailAgain.getByRole("button", { name: "Uninstall", exact: true }).click();
await expect(page.getByTestId("connection-detail")).toHaveCount(0);
const gmailRowEmpty = page
.getByTestId("featured-connectors")
.getByText("Gmail", { exact: true })
.locator("xpath=ancestor::*[.//button][1]");
await expect(gmailRowEmpty.getByRole("button", { name: "Add", exact: true })).toBeVisible();
await captureScreenshot(page, testInfo, "11b-connected-plugins-empty");
const linearRow = page
.getByText("Linear", { exact: true })
.locator("xpath=ancestor::*[.//button][1]");
const connectPopup = page.waitForEvent("popup");
await linearRow.getByRole("button", { name: "Add", exact: true }).click();
const popup = await connectPopup;
await popup.close();
await expect(linearRow.getByRole("button", { name: "Added", exact: true })).toBeVisible();
await linearRow.getByRole("button", { name: "Added", exact: true }).click();
const linearDetail = page.getByTestId("connection-detail");
await expect(linearDetail).toBeVisible();
await linearDetail.getByRole("button", { name: "Uninstall", exact: true }).click();
await expect(page.getByTestId("connection-detail")).toHaveCount(0);
await expect(linearRow.getByRole("button", { name: "Add", exact: true })).toBeVisible();
const advanced = page.getByTestId("integrations-advanced");
await advanced.evaluate((element) => {
(element as HTMLDetailsElement).open = true;
});
await expect(page.getByRole("button", { name: "Manage MCP servers", exact: true })).toBeVisible();
await expect(page.getByRole("button", { name: "Add MCP server", exact: true })).toBeVisible();
await expect(page.getByRole("button", { name: "Add OpenAPI", exact: true })).toBeVisible();
await expect(page.getByRole("button", { name: "Add GraphQL", exact: true })).toBeVisible();
await expect(page.getByRole("button", { name: "Add Executor", exact: true })).toBeVisible();
await expect(page.getByRole("button", { name: "Add Treg", exact: true })).toBeVisible();
await expect(page.getByText("Tool sources", { exact: true })).toBeVisible();
await expect(page.getByRole("button", { name: "MCP servers", exact: true })).toBeHidden();
// Thin Advanced smoke only. GraphQL install and order screenshots live in graphql-integrations.spec.ts.
await expect(page.getByTestId("integrations-catalog-feed")).toBeVisible();
// Catalog Search is optional chrome; add buttons stay MCP → OpenAPI → GraphQL → Executor → Treg.
const advancedActions = advanced.getByTestId("integrations-advanced-add").locator("button");
await expect(advancedActions).toHaveCount(5);
await expect(advancedActions.nth(0)).toHaveText("Add MCP server");
await expect(advancedActions.nth(1)).toHaveText("Add OpenAPI");
await expect(advancedActions.nth(2)).toHaveText("Add GraphQL");
await expect(advancedActions.nth(3)).toHaveText("Add Executor");
await expect(advancedActions.nth(4)).toHaveText("Add Treg");
const feed = page.getByTestId("integrations-catalog-feed");
await feed.getByRole("textbox", { name: "Integration domain" }).fill("github.com");
await feed.getByRole("button", { name: "Search", exact: true }).click();
await expect(feed.getByText("GitHub", { exact: true })).toBeVisible();
await expect(feed.getByRole("button", { name: "MCP · Add", exact: true })).toBeVisible();
await expect(feed.getByRole("button", { name: "OPENAPI · Add", exact: true })).toBeVisible();
await expect(feed.getByRole("button", { name: "GRAPHQL · Manual", exact: true })).toBeDisabled();
await expect(feed.getByRole("button", { name: "CLI · Manual", exact: true })).toBeDisabled();
// Editing the query must drop prior results so a stale Add cannot run for the old domain.
await feed.getByRole("textbox", { name: "Integration domain" }).fill("gitlab.com");
await expect(feed.getByText("GitHub", { exact: true })).toBeHidden();
await feed.getByRole("textbox", { name: "Integration domain" }).fill("github.com");
await feed.getByRole("button", { name: "Search", exact: true }).click();
await expect(feed.getByText("GitHub", { exact: true })).toBeVisible();
await captureScreenshot(page, testInfo, "11c-catalog-feed");
await feed.getByRole("button", { name: "MCP · Add", exact: true }).click();
await expect(page.getByPlaceholder("Display name")).toHaveValue("GitHub");
await expect(page.getByPlaceholder("https://example.com/mcp")).toHaveValue(
"https://mcp.example.test/mcp",
);
await expect(page.locator("select")).toHaveValue("bearer");
await page.getByRole("button", { name: "Cancel", exact: true }).click();
await page.getByRole("button", { name: "Add Treg", exact: true }).click();
await page.getByPlaceholder("Treg token").fill("fake-treg-browser-credential");
await page.getByRole("button", { name: "Verify and add", exact: true }).click();
await expect(page.getByText(/MCP · https:\/\/treg\.to\/mcp\/ · credential saved/)).toBeVisible();
await page.getByRole("button", { name: "Add MCP server", exact: true }).click();
await page.getByPlaceholder("Display name").fill("Browser MCP");
await page.getByPlaceholder("https://example.com/mcp").fill("https://mcp.example.test/mcp");
await page.getByRole("button", { name: "Verify and add", exact: true }).click();
await expect(page.getByText(/MCP · https:\/\/mcp\.example\.test\/mcp · no auth/)).toBeVisible();
await page.getByRole("button", { name: "Add OpenAPI", exact: true }).click();
await page.getByPlaceholder("Display name").fill("Browser API");
await page
.getByPlaceholder("https://example.com/openapi.json")
.fill("https://api.example.test/openapi.json");
await page.locator("select").selectOption("bearer");
await page.getByPlaceholder("Credential").fill("fake-openapi-browser-credential");
await page.getByRole("button", { name: "Verify and add", exact: true }).click();
await expect(
page.getByText(/API · https:\/\/api\.example\.test\/v1 · credential saved/),
).toBeVisible();
await page.getByRole("button", { name: "Add Executor", exact: true }).click();
await expect(page.getByPlaceholder("Display name")).toHaveValue("Executor");
await page
.getByPlaceholder("https://executor.example/mcp")
.fill("https://executor.example.test/mcp");
await page.getByPlaceholder("Executor token").fill("fake-executor-browser-credential");
await page.getByRole("button", { name: "Verify and add", exact: true }).click();
await expect(
page.getByText(/MCP · https:\/\/executor\.example\.test\/mcp · credential saved/),
).toBeVisible();
await captureScreenshot(page, testInfo, "11d-provider-emulators");
await page.getByRole("button", { name: "Close integrations settings" }).click();
await page.getByText("Chief").first().click();
const gear = page.getByRole("button", { name: "Show settings" });
if (!(await gear.isVisible().catch(() => false))) {
await page.getByTitle("Agent computer").click();
}
await gear.click();
const downloadPromise = page.waitForEvent("download");
await page.getByRole("button", { name: "Export" }).click();
const download = await downloadPromise;
expect(download.suggestedFilename()).toMatch(/chief-export\.json/i);
const settings = page.getByTestId("bot-settings");
await expect(settings.getByRole("button", { name: "Archive bot" })).toHaveCount(0);
await expect(settings.getByRole("button", { name: "Delete bot" })).toHaveCount(0);
await page.getByRole("button", { name: "Close panel" }).click();
await page.locator("aside").first().getByRole("button", { name: /Chief/ }).first().click({
button: "right",
});
const botMenu = page.getByRole("menu", { name: "Actions for Chief" });
await expect(botMenu.getByRole("menuitem", { name: "Archive" })).toBeVisible();
await botMenu.getByRole("menuitem", { name: "Delete" }).click();
await expect(page.getByRole("radio", { name: /Keep memories/ })).toBeChecked();
await expect(page.getByRole("radio", { name: /Delete memories too/ })).toBeVisible();
await page.getByRole("button", { name: "Cancel" }).click();
await captureScreenshot(page, testInfo, "12-bot-settings");
});
test("sign-in, spawn, and stop work in the shell", async ({ page }, testInfo) => {
const browserErrors: string[] = [];
const failedRequests: string[] = [];
page.on("pageerror", (error) => browserErrors.push(error.message));
page.on("console", (message) => {
if (message.type() === "error") browserErrors.push(message.text());
});
page.on("requestfailed", (request) => {
const errorText = request.failure()?.errorText ?? "";
// React StrictMode and route changes cancel in-flight loads. That is not a failed response.
if (errorText.includes("ERR_ABORTED")) return;
failedRequests.push(`${request.method()} ${request.url()} ${errorText}`);
});
const stamp = Date.now();
const email = `shell-${stamp}@rakazo.test`;
await signup(page, email, "password12", "Shell");
await completeOnboarding(page);
await page.evaluate(() => {
Object.defineProperty(globalThis.crypto, "randomUUID", {
value: undefined,
configurable: true,
});
});
const composer = page.locator('textarea[name="chat-message"]');
await composer.fill("spawn a bot named Scout to research venues");
await page.keyboard.press("Enter");
await expect(sidebarBotButton(page, /Scout/)).toBeVisible({
timeout: 30_000,
});
await captureScreenshot(page, testInfo, "13-spawned-bot");
await page
.locator("[data-sidebar-group]")
.getByRole("button", { name: /^Chief/ })
.click();
await composer.fill("keep working until I stop you");
await page.keyboard.press("Enter");
await expect(page.getByRole("button", { name: "Stop", exact: true })).toBeVisible({
timeout: 30_000,
});
await expect(page.getByTestId("composer-steering-status")).toHaveCount(0);
await expect(page.getByText("Messages sent now guide the next turn.")).toHaveCount(0);
await expect(page.getByText(/^Steer /)).toHaveCount(0);
await expect(composer).toHaveAttribute("placeholder", "Message…");
await expect(page.getByRole("button", { name: "Send", exact: true })).toBeVisible();
await composer.fill("Use the newer report and keep the answer short.");
await page.keyboard.press("Tab");
await expect(
page.getByTestId("composer-bar").getByRole("button", { name: "Voice", exact: true }),
).toBeFocused();
await page.keyboard.press("Tab");
await expect(page.getByRole("button", { name: "Send", exact: true })).toBeFocused();
await page.keyboard.press("Enter");
await expect(
page.getByTestId("transcript").getByText("Use the newer report and keep the answer short."),
).toBeVisible();
await expect(page.getByRole("button", { name: "Stop", exact: true })).toBeVisible();
await captureScreenshot(page, testInfo, "14-active-bot-work");
await page.setViewportSize({ width: 390, height: 844 });
await expect(page.getByRole("button", { name: "Send", exact: true })).toBeVisible();
await expect(page.getByRole("button", { name: "Stop", exact: true })).toBeVisible();
await expect(composer).toHaveAttribute("placeholder", "Message…");
await captureScreenshot(page, testInfo, "14-active-bot-work-mobile");
await page.setViewportSize({ width: 1280, height: 720 });
expect(browserErrors).toEqual([]);
expect(failedRequests).toEqual([]);
let releaseStopRequest: () => void = () => undefined;
let markStopRequestStarted: () => void = () => undefined;
const stopRequestStarted = new Promise<void>((resolve) => {
markStopRequestStarted = resolve;
});
await page.route("**/rpc/threads/stop", async (route) => {
markStopRequestStarted();
await new Promise<void>((release) => {
releaseStopRequest = release;
});
await route.continue();
});
await page.getByRole("button", { name: "Stop", exact: true }).click();
await stopRequestStarted;
await expect(page.getByRole("button", { name: "Send", exact: true })).toBeDisabled();
await expect(page.getByRole("button", { name: "Stop", exact: true })).toBeDisabled();
releaseStopRequest();
// Idle Send stays disabled with an empty draft; wait for Stop to leave instead.
await expect(page.getByRole("button", { name: "Stop", exact: true })).toHaveCount(0, {
timeout: 30_000,
});
await expect(page.getByRole("button", { name: "Send", exact: true })).toBeVisible();
await page.context().clearCookies();
await page.goto("/sign-in");
await page.getByPlaceholder("Your email address").fill(email);
await page.getByPlaceholder("Password").fill("password12");
await page.getByRole("button", { name: "Continue with email" }).click();
await page.waitForURL(/\/app/, { timeout: 20_000 });
await expect(sidebarBotButton(page, /^Chief/)).toBeVisible();
await expect(sidebarBotButton(page, /Scout/)).toBeVisible();
await captureScreenshot(page, testInfo, "15-restored-session");
});
test("bot context menu pins, duplicates, edits, and confirms deletion", async ({
page,
}, testInfo) => {
const stamp = Date.now();
await signup(page, `menu-${stamp}@rakazo.test`, "password12", "Menu");
await completeOnboarding(page);
const chief = page.getByRole("button", { name: /Chief/ }).first();
await chief.click({ button: "right" });
await expect(page.getByRole("menu", { name: "Actions for Chief" })).toBeVisible();
await expect(page.getByRole("menuitem", { name: "Edit Profile" })).toBeVisible();
await expect(page.getByRole("menuitem", { name: "Duplicate" })).toBeVisible();
await expect(page.getByRole("menuitem", { name: "Delete" })).toBeVisible();
await captureScreenshot(page, testInfo, "16-bot-context-menu");
await page.getByRole("menuitem", { name: "Mark as Unread" }).click();
// Chief is the open bot, so the auto-read on window focus must not undo the manual mark.
await page.evaluate(() => window.dispatchEvent(new Event("focus")));
await chief.click({ button: "right" });
await expect(page.getByRole("menuitem", { name: "Mark as Read" })).toBeVisible();
await page.getByRole("menuitem", { name: "Mark as Read" }).click();
await chief.click({ button: "right" });
await page.getByRole("menuitem", { name: "Pin", exact: true }).click();
await chief.click({ button: "right" });
await expect(page.getByRole("menuitem", { name: "Unpin", exact: true })).toBeVisible();
await page.getByRole("menuitem", { name: "Duplicate" }).click();
await expect(page.getByText("Chief copy").first()).toBeVisible();
await captureScreenshot(page, testInfo, "17-pinned-and-duplicated-bot");
const copy = page.getByRole("button", { name: /Chief copy/ }).first();
await copy.click({ button: "right" });
await page.getByRole("menuitem", { name: "Delete" }).click();
await expect(page.getByRole("alertdialog", { name: "Delete Chief copy?" })).toBeVisible();
await captureScreenshot(page, testInfo, "18-delete-confirmation");
await page.getByRole("button", { name: "Cancel" }).click();
await chief.click({ button: "right" });
await page.getByRole("menuitem", { name: "Edit Profile" }).click();
await expect(page.locator("label:has-text('Name') input")).toHaveValue("Chief");
await captureScreenshot(page, testInfo, "19-edit-profile");
});
async function threadRunStatus(page: Page) {
const result = await rpc<{ run?: { status?: string } | null }>(page, "threads/get", {
botId: activeBotId(page),
});
return result.run?.status ?? "idle";
}
async function waitForBoxFramebuffer(page: Page) {
await expect
.poll(
async () => {
for (const frame of page.frames()) {
if (frame === page.mainFrame()) continue;
const canvas = frame.locator("canvas").first();
if ((await canvas.count()) === 0) continue;
const ready = await canvas
.evaluate((element) => {
const framebuffer = element as HTMLCanvasElement;
return framebuffer.width > 0 && framebuffer.height > 0;
})
.catch(() => false);
if (ready) return true;
}
return false;
},
{ timeout: 60_000, message: "the Box noVNC framebuffer must be ready" },
)
.toBe(true);
}