evOo-local/PROJECT.md
cynarAI ca869561ac
Voice-first streaming, non-blocking follow-ups and chat UX (#205)
* media: check ffmpeg filter support before subtitle operations

* sandbox: handle synchronous spawn errors safely in desktop sandbox

* Voice-first streaming, non-blocking follow-ups, quiet routines and chat UX

Why: questions blocked the chat, follow-up messages had no acknowledgement, message actions sat beside the bubble, and voice turn-taking added latency.

What: waiting runs accept steering messages and the bot marks each claimed follow-up as seen; unclaimed follow-ups show as queued. Reaction, copy and reply controls sit under each message. Streaming voice (VAD endpointing, barge-in that stops the run, clause-based TTS), an optimistic input queue, procedural earcons and delta-based quiet routines were added. Onboarding can be skipped straight to the main conversation. Dual-surface layout and project pickers are included. Status colors use semantic tokens.

Tested: unit suites for core, adapters, api and web pass, typecheck and Biome are clean. Browser end-to-end tests were not run locally.

* Update translation catalogs

Re-extracted catalogs for the new and moved strings; the catalog sync test passes.

* Add project plan and test readiness notes

Notes from the milestone work: feature inventory, test infrastructure and test readiness. No code changes.

* fix(ci): align scale guards, space deletion, computer toggle, and mobile expo deps

* fix(ci): restore space deletion guard, browser toggle, and run-steps contracts

* fix(ci): restore model resolution errors, group stage, and onboarding flow

* fix(ci): restore in-app-browser close button, disable stop button while sending, and sync lingui catalogs

* fix(ci): address web e2e failures for bot-crud, choice cards, message quote, and voice onboarding

* test(web): wait for avatar and rig visibility in oo-avatar motion spec

* test(web): wait for persisted message IDs in message-hover-actions pagination spec

* Onboarding: always drop the local voice when the voice step is skipped

Skipping waited only for the latest voice load and disconnected only
when the client saw the connect succeed. An earlier or client-failed
connect could still persist the local credential, so a user who
skipped voice was left with one. Wait for every load and always
disconnect.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy1md9QE1ayJbDCwJ4vAQs

* Shell: hide the optimistic copy once the transcript shows the send

A message sent while the bot works streamed into the transcript before
its send request resolved, so the same text showed twice until the
optimistic bubble was removed. Drop an optimistic message as soon as a
persisted user message with the same text appears.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy1md9QE1ayJbDCwJ4vAQs

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-10-15 14:08:31 +02:00

7.7 KiB

Project: Dual-Surface-View in evOo (apps/web)

Architecture

evOo (apps/web) organizes the workspace into two primary surfaces:

  1. Mission Stream: The chat, agent transcript (data-testid="transcript"), run steps (data-testid="run-steps"), and composer inside <main>.
  2. Living Canvas: The computer surface (data-testid="computer-preview"), workspace files explorer (WorkspaceFilesView), code inspector, in-app browser, and artifact viewer inside <aside data-testid="side-panel">.

In desktop viewports (>= 768px md), these two surfaces operate as equal, first-class split surfaces within Shell.tsx:

  • Positioned side-by-side without modal dialog occlusion.
  • Separated by an interactive DualSurfaceDivider supporting fluid pointer drag-resizing, double-click snap to 50/50 balance, and keyboard arrow adjustments.
  • Controlled via quick-switch header preset controls: Focus Stream (100/0), Dual-Surface Split (50/50), Focus Canvas (0/100), and a side-swapping toggle (Stream-first vs. Canvas-first).
  • Configured by persisted preferences (localStorage via dual-surface-pref.ts) with safe defaults and bounded numeric clamping.
  • Responsive degradation (< 768px md): Automatically transitions to stacked/drawer/single-surface mode without mutating desktop split ratio preferences.
  • Monochrome design system adhering to @rakazo/ui-tokens with zero hardcoded hex or arbitrary CSS values.
  • Internationalized using LinguiJS with 100% German (de) translation coverage.
+-----------------------------------------------------------------------------------+
| Top Navigation Bar: [Bots Toggle] ... [Preset: 100/0 | 50/50 | 0/100] [Swap] ...   |
+----------------------------------------------------+---+--------------------------+
| Surface A (e.g. Mission Stream / Chat)             | D | Surface B (Living Canvas)|
|                                                    | I |                          |
| - Transcript (`data-testid="transcript"`)         | V | - Computer Preview       |
| - Run steps (`data-testid="run-steps"`)           | I | - Workspace Explorer     |
| - Composer                                         | D | - Code Inspector         |
|                                                    | E | (`data-testid="side-panel")|
|                                                    | R |                          |
+----------------------------------------------------+---+--------------------------+

Feature Inventory

# Feature Description Milestone Source
1 Preferences Storage & Schema dual-surface-pref.ts managing splitRatio (0.20-0.80, default 0.50), layoutMode (split, stream, canvas), orientation (stream-first, canvas-first) with localStorage read/write and bounds clamping M1 ORIGINAL_REQUEST §R3
2 Layout Hook & State Coordination useDualSurfaceLayout.ts React hook handling layout state, preset toggles, orientation flip, and media query synchronization M1 ORIGINAL_REQUEST §R1, R3
3 Fluid Drag-Resize & Snap Divider DualSurfaceDivider.tsx + useDualSurfaceResize.ts with pointer capture, hover/active semantic tokens, bounds clamping, double-click 50/50 snap, and ARIA separator attributes M2 ORIGINAL_REQUEST §R2, R4
4 Header Preset Controls & Orientation Swap DualSurfaceHeaderControls.tsx with quick-switch presets (100/0, 50/50, 0/100) and Swap Orientation toggle, styled with semantic tokens and full accessibility M2 ORIGINAL_REQUEST §R1, R2
5 Shell Flex Integration & Surface Swap Coordinated flex container in Shell.tsx hosting <main> and <aside data-testid="side-panel">, supporting dynamic order swapping while preserving existing test IDs M3 ORIGINAL_REQUEST §R1, R2, R4
6 Responsive Degradation (< md) Seamless fallback on viewports < 768px to stacked/single-surface navigation without corrupting persisted desktop preferences M3 ORIGINAL_REQUEST §R3
7 Design System & Token Compliance Exclusively semantic tokens from @rakazo/ui-tokens (bg-border, hover:bg-accent, active:bg-primary, ring-ring), monochrome styling, no hardcoded hex/arbitrary css M3 ORIGINAL_REQUEST §R4
8 Localization (German Translations) Lingui message extraction, full German translations in messages.po, clean intl:extract and intl:compile M3 ORIGINAL_REQUEST §Acceptance Criteria
9 E2E Testing & Adversarial Hardening Comprehensive test suite (Tiers 1-4) and adversarial hardening (Tier 5), 100% green tests (78/78 suites baseline + new tests) and 0 TS errors M4 ORIGINAL_REQUEST §Acceptance Criteria

Milestones

# Name Scope Dependencies Status
M1 Preferences, Layout Hook & State Management dual-surface-pref.ts, useDualSurfaceLayout.ts, unit tests none DONE
M2 Interactive Divider & Header Preset Controls DualSurfaceDivider.tsx, useDualSurfaceResize.ts, dual-surface-header-controls.tsx, unit/DOM tests M1 DONE
M3 Shell Integration, Responsive Degradation & Localization Shell.tsx layout refactoring, responsive fallback, German translations in messages.po, preserve test IDs M2 DONE
M4 E2E Test Suite (Tiers 1-4) & Adversarial Hardening (Tier 5) Comprehensive E2E test suite execution, 100% pass, forensic integrity audit M3, E2E Track IN_PROGRESS

Interface Contracts

dual-surface-pref.ts

export type DualSurfaceLayoutMode = "split" | "stream" | "canvas";
export type DualSurfaceOrientation = "stream-first" | "canvas-first";

export interface DualSurfacePreferences {
  splitRatio: number; // 0.20 to 0.80, default 0.50
  layoutMode: DualSurfaceLayoutMode;
  orientation: DualSurfaceOrientation;
}

export function readDualSurfacePreferences(): DualSurfacePreferences;
export function writeDualSurfacePreferences(prefs: Partial<DualSurfacePreferences>): void;
export function clampSplitRatio(ratio: number): number;

useDualSurfaceLayout.ts

export interface DualSurfaceLayoutState {
  splitRatio: number;
  layoutMode: DualSurfaceLayoutMode;
  orientation: DualSurfaceOrientation;
  isMobile: boolean; // < 768px (md)
  setSplitRatio: (ratio: number) => void;
  setLayoutMode: (mode: DualSurfaceLayoutMode) => void;
  setOrientation: (orientation: DualSurfaceOrientation) => void;
  toggleOrientation: () => void;
  resetSplit: () => void; // Snaps to 0.50
}

DualSurfaceDivider.tsx

export interface DualSurfaceDividerProps {
  splitRatio: number;
  onResize: (newRatio: number) => void;
  onReset: () => void; // Double-click snap
  disabled?: boolean;
}

DualSurfaceHeaderControls.tsx

export interface DualSurfaceHeaderControlsProps {
  layoutMode: DualSurfaceLayoutMode;
  orientation: DualSurfaceOrientation;
  onSelectMode: (mode: DualSurfaceLayoutMode) => void;
  onToggleOrientation: () => void;
  disabled?: boolean;
}

Code Layout

  • apps/web/src/lib/dual-surface-pref.ts: Preference reading, writing, and boundary clamping.
  • apps/web/src/lib/dual-surface-pref.test.ts: Unit tests for storage and mathematical boundaries.
  • apps/web/src/hooks/useDualSurfaceLayout.ts: Layout state and responsive lifecycle management.
  • apps/web/src/hooks/useDualSurfaceResize.ts: Pointer capture and keyboard navigation hook for divider.
  • apps/web/src/components/DualSurfaceDivider.tsx: Accessible interactive divider component.
  • apps/web/src/pages/shell/dual-surface-header-controls.tsx: Quick-switch header presets and swap buttons.
  • apps/web/src/pages/Shell.tsx: Integration into application root layout.
  • apps/web/src/locales/de/messages.po: German translation catalog entries.
  • apps/web/src/tests/dual-surface-view.test.tsx: Component integration and DOM interaction tests.