* media: check ffmpeg filter support before subtitle operations * sandbox: handle synchronous spawn errors safely in desktop sandbox * Voice-first streaming, non-blocking follow-ups, quiet routines and chat UX Why: questions blocked the chat, follow-up messages had no acknowledgement, message actions sat beside the bubble, and voice turn-taking added latency. What: waiting runs accept steering messages and the bot marks each claimed follow-up as seen; unclaimed follow-ups show as queued. Reaction, copy and reply controls sit under each message. Streaming voice (VAD endpointing, barge-in that stops the run, clause-based TTS), an optimistic input queue, procedural earcons and delta-based quiet routines were added. Onboarding can be skipped straight to the main conversation. Dual-surface layout and project pickers are included. Status colors use semantic tokens. Tested: unit suites for core, adapters, api and web pass, typecheck and Biome are clean. Browser end-to-end tests were not run locally. * Update translation catalogs Re-extracted catalogs for the new and moved strings; the catalog sync test passes. * Add project plan and test readiness notes Notes from the milestone work: feature inventory, test infrastructure and test readiness. No code changes. * fix(ci): align scale guards, space deletion, computer toggle, and mobile expo deps * fix(ci): restore space deletion guard, browser toggle, and run-steps contracts * fix(ci): restore model resolution errors, group stage, and onboarding flow * fix(ci): restore in-app-browser close button, disable stop button while sending, and sync lingui catalogs * fix(ci): address web e2e failures for bot-crud, choice cards, message quote, and voice onboarding * test(web): wait for avatar and rig visibility in oo-avatar motion spec * test(web): wait for persisted message IDs in message-hover-actions pagination spec * Onboarding: always drop the local voice when the voice step is skipped Skipping waited only for the latest voice load and disconnected only when the client saw the connect succeed. An earlier or client-failed connect could still persist the local credential, so a user who skipped voice was left with one. Wait for every load and always disconnect. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy1md9QE1ayJbDCwJ4vAQs * Shell: hide the optimistic copy once the transcript shows the send A message sent while the bot works streamed into the transcript before its send request resolved, so the same text showed twice until the optimistic bubble was removed. Drop an optimistic message as soon as a persisted user message with the same text appears. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy1md9QE1ayJbDCwJ4vAQs --------- Co-authored-by: Claude <noreply@anthropic.com>
15 KiB
15 KiB
TEST_READY: evOo Voice-First Autonomous Agent Operating System
Status
- Test Suite Status: READY
- Author:
test_writer_r2_1(Dual-Track E2E Testing Specialist) - Timestamp: 2026-10-12T13:05:00Z
- Target Milestones: M1, M2, M3, M4, M5 (Features F1–F12 across Requirements R1, R2, R3)
- Test Framework: Vitest 4.1.11 with
// @vitest-environment jsdom& Pure React 19 harnesses
1. Authored Test Suites Summary
| # | Test File Path | Scope & Feature Focus | Tiers Covered | Total Test Cases | Passing Baseline |
|---|---|---|---|---|---|
| 1 | apps/web/src/tests/e2e-voice-first-foundation.test.ts |
Streaming Audio Wire (<120ms framing, PCM16, VAD), Full-Duplex Audio & Barge-In Interruption, Incremental Utterance Streamer (<300ms TTFS, invite strip), Tactile Monochrome Earcons | Tier 1, Tier 2 (F1–F4) | 40 | ✅ 40 / 40 Passed |
| 2 | apps/web/src/tests/e2e-autonomous-agency-queues.test.ts |
Optimistic Input Queue (Zero UI Freeze, unblocked stop), Mid-Step Tool Cancellation & Active Steering, Dynamic Action Queue State Machine, Quiet Background Routines & Delta Evaluation (on_change) |
Tier 1, Tier 2 (F5–F8) | 40 | ✅ 40 / 40 Passed |
| 3 | apps/web/src/tests/e2e-minimalist-tactile-ux.test.tsx |
Tactile Minimalist Step Execution (strip JobProgressCard bloat, folding # step), 30-Second Rapid Onboarding (zero questionnaires, 3 action sparks), Monochrome Token Hygiene (@rakazo/ui-tokens), Living Canvas Progressive Disclosure |
Tier 1, Tier 2 (F9–F12) | 40 | ✅ 40 / 40 Passed |
| 4 | apps/web/src/tests/e2e-cross-feature-workflows.test.ts |
12 Pairwise Cross-Feature Interactions & 7 Realistic End-to-End Workflows (Hands-free meeting, voice barge-in tool abort, high-frequency prompt burst, quiet background routines, 30s onboarding, progressive disclosure, voice network resilience) | Tier 3, Tier 4 | 19 | ✅ 19 / 19 Passed |
| Total | 4 Comprehensive Test Suites + 3 Fixtures | Full Feature Inventory (F1 to F12) Coverage | Tiers 1–4 | 139 tests | ✅ 139 / 139 (100% Pass) |
2. Requirement & Feature Coverage Matrix
| Feature | Source Requirement | Primary Test File | Tiers | Pass / Fail |
|---|---|---|---|---|
| F1: Streaming Audio Wire & Low-Latency VAD | ORIGINAL_REQUEST §R1 | e2e-voice-first-foundation.test.ts |
T1, T2, T3, T4 | ✅ PASS (10/10) |
| F2: Full-Duplex Audio & Barge-In Interruption | ORIGINAL_REQUEST §R1 | e2e-voice-first-foundation.test.ts |
T1, T2, T3, T4 | ✅ PASS (10/10) |
| F3: Incremental Utterance Streamer & Invite Strip | ORIGINAL_REQUEST §R1 | e2e-voice-first-foundation.test.ts |
T1, T2, T3, T4 | ✅ PASS (10/10) |
| F4: Cross-Platform Earcons & Mobile Audio | ORIGINAL_REQUEST §R1 | e2e-voice-first-foundation.test.ts |
T1, T2, T3, T4 | ✅ PASS (10/10) |
| F5: Optimistic Input Queue & Zero UI Freeze | ORIGINAL_REQUEST §R2 | e2e-autonomous-agency-queues.test.ts |
T1, T2, T3, T4 | ✅ PASS (10/10) |
| F6: Mid-Step Tool Cancellation & Active Steering | ORIGINAL_REQUEST §R2 | e2e-autonomous-agency-queues.test.ts |
T1, T2, T3, T4 | ✅ PASS (10/10) |
| F7: Outcome-Driven Action Queue State Machine | ORIGINAL_REQUEST §R2 | e2e-autonomous-agency-queues.test.ts |
T1, T2, T3, T4 | ✅ PASS (10/10) |
| F8: Quiet Background Routines & Delta Evaluation | ORIGINAL_REQUEST §R2 | e2e-autonomous-agency-queues.test.ts |
T1, T2, T3, T4 | ✅ PASS (10/10) |
| F9: Tactile Minimalist Step Execution | ORIGINAL_REQUEST §R3 | e2e-minimalist-tactile-ux.test.tsx |
T1, T2, T3, T4 | ✅ PASS (10/10) |
| F10: 30-Second Rapid Onboarding | ORIGINAL_REQUEST §R3 | e2e-minimalist-tactile-ux.test.tsx |
T1, T2, T3, T4 | ✅ PASS (10/10) |
| F11: Monochrome Semantic Token Hygiene | ORIGINAL_REQUEST §R3 | e2e-minimalist-tactile-ux.test.tsx |
T1, T2, T3, T4 | ✅ PASS (10/10) |
| F12: Living Canvas Progressive Disclosure | ORIGINAL_REQUEST §R3 | e2e-minimalist-tactile-ux.test.tsx |
T1, T2, T3, T4 | ✅ PASS (10/10) |
| T3: Pairwise Cross-Feature Interactions | Cross-Module Interactions | e2e-cross-feature-workflows.test.ts |
Tier 3 | ✅ PASS (12/12) |
| T4: Real-World Application Scenarios | Operational Workflows | e2e-cross-feature-workflows.test.ts |
Tier 4 | ✅ PASS (7/7) |
3. Tier Breakdown & Test Cases
Tier 1: Feature Coverage (60 Tests)
- F1: 80ms PCM16 frame chunking (1280 samples at 16kHz); VAD normalized RMS energy; <120ms cutoff threshold eliminating 850ms silence floor; monotonic timestamp increment; streaming client transcript & playback chunk callbacks.
- F2: Continuous microphone capture with acoustic echo cancellation (
echoCancellation: true); user speech energy detection while bot speaking triggers barge-in; immediate playback audio mute; backend abort propagation (abort('barge_in')); state transition from speaking to listening. - F3: Incremental clause segmentation in 4–6 word chunks; time-to-first-sound pipelining (<300ms TTFS target); stripping robotic invite prompts ("What would you like to do next?"); code fence description transformation per
speakable(); natural punctuation boundary clause splits. - F4: Core tactile earcons (
listening-start,listening-stop,speaking-start,speaking-done); Web Audio procedural oscillator profiles; master audio bus ducking (0.28 ducking ratio); cross-platform mobile earcon mapping; state machine audio cue triggers (interact,attention). - F5: Submitting message while another in-flight enqueues without UI freeze; temporary optimistic ID assignment (
temp-...) and timestamp; composer text clearing immediately; Stop button unblocked and clickable; sequential FIFO drainage. - F6: Ingesting steering during active tool call dispatches mid-step cancellation signal; tool abort controller receives
abort('steering_redirection'); aborted tool output discarded from history; agent pivots immediately to steering intent; steering message marked claimed. - F7:
DynamicActionQueueinitialization; multi-step prompt synthesis intoActionQueueStepitems; queue advancement starts pending step withstartedAt; completing step setscompletedAtand advances;cancelActive()marks step cancelled. - F8:
RoutineNotificationPolicywith"on_change"evaluates output delta hash; identical output hash suppresses notification (quiet run); distinct output hash triggers notification; policy"always"notifies unconditionally; policy"never"suppresses all notifications. - F9: Minimalist folding step button renders with
data-testid="run-steps"; displays clean# stepcounter without progress track bloat; clicking fold button togglesaria-expanded; step tool name and summary rendered when expanded; keyboard Enter/Space toggles fold. - F10: 1-click bootstrap creates initial agent workspace without 11-step questionnaire; auto-detects system timezone via
Intl.DateTimeFormat; auto-detects system language vianavigator.language; surfaces 3 tactile starter sparks; selecting spark starts session (<30s value). - F11: Theme tokens definition from
@rakazo/ui-tokens; zero hardcoded product hex colors; zero arbitrary Tailwind brackets[var(--...)]; accessible focus rings (focus-visible:ring-ring); status indicators use semantic tokens (text-destructive,text-muted-foreground). - F12: Living Canvas pane renders with
data-testid="side-panel"; Mission Stream renders withdata-testid="mission-stream-pane"anddata-testid="transcript"; preserves critical test IDs (computer-preview,dual-surface-divider,dual-surface-header-controls); progressive disclosure on artifact creation; tab switching between Files, Sandbox, Browser.
Tier 2: Boundary & Corner Cases (60 Tests)
- F1: Empty audio buffer (
Int16Array(0)) handled with RMS 0 and no NaN errors; pure silence buffer maintains energy 0 and triggers silence cutoff without hang; clipping values clamped to Int16 limits (-32768 to 32767); out-of-order timestamp detection and jitter buffer tolerance; network drop triggers client reconnect and state preservation. - F2: Microphone permission denied handled with graceful error state; interruption when audio playback is already idle is safe no-op; micro-burst transients (<50ms noise pop) filtered by VAD debounce; rapid alternating speaker turns (ping-pong barge-in); concurrent interrupt and incoming playback chunk race condition resolves in favor of user interrupt.
- F3: Single-word answers streamed without minimum character delay stalling; code-only block output sanitized without synthetic audio artifacts; emoji-only message stream strips emojis and does not stall TTS engine; micro-fragment token arrivals buffered into complete words; unexpected LLM stream termination cleanly finalizes active clause.
- F4: Rapid consecutive sound cue triggers (<10ms) queue smoothly without audio glitches; muted/silent mode suppresses oscillator output while executing state transition; AudioContext suspended state auto-resumes on first user gesture; unknown sound cue name falls back gracefully without throwing exceptions; cross-platform volume normalization preserves peak headroom.
- F5: Rapid typing burst (10 rapid inputs within 100ms) enqueues all 10 without dropping; bounded queue size enforces max pending limit with graceful error; network failure while message in flight triggers failed state with retry option without losing text; empty/whitespace-only messages rejected before enqueueing; duplicate messages submitted in rapid succession deduplicated if flagged.
- F6: Race condition: tool finishes at the exact moment abort signal fires resolves cleanly; tool execution timeout triggers automated mid-step cancellation with
timeoutreason; multiple rapid contradictory steering messages resolves to most recent intent; tool that throws exception upon abort caught gracefully without crashing runtime; external system side-effect fence prevents state pollution on aborted tools. - F7: Synthesizing empty or meaningless input results in clean no-op;
advance()called on an empty queue or completed queue is safe no-op; failed step status prevents automatic advance unless retry policy allows; step status state machine strictly validates allowable transitions; queue serialization and deserialization preserves complete step fidelity. - F8: First execution (
prevOutputHash === '') with"on_change"triggers initial notification; policy"on_failure"triggers notification only when routine status is"failed"; whitespace/formatting variations evaluated cleanly against normalized hash; high frequency routine executions do not spam notifications if output remains constant; routine execution failure self-heals without crashing scheduler. - F9: Empty steps list renders cleanly without error or broken layout; extreme step count (50+ tool calls) remains folded by default without layout overflow; failed step renders clean status badge without noisy error stacks unless expanded; absence of chrome bloat (removal of animated tracks, live timers
· 1:24, uppercase calloutsAUFGABE); rapid folding/unfolding toggles do not oscillate or corrupt DOM state. - F10: Missing or invalid timezone fallback safely resolves to UTC without throwing error; missing or invalid locale fallback safely resolves to
en; re-entering onboarding while workspace already exists redirects to active space; starter spark with special characters sanitized safely; offline onboarding gracefully saves initial draft locally without blocking UI. - F11: Semantic contrast in monochrome mode preserves readable text tokens; disabled component states apply semantic opacity
disabled:opacity-50rather than arbitrary grey; high contrast mode support: borders preserve visibility via semanticborder-border; dynamic theme switching updates token class mappings deterministically; interactive hover states use semantic hover tokens (hover:bg-accent). - F12: Closing Living Canvas collapses aside pane without unmounting workspace state; rapid switching between canvas tabs maintains individual surface state; resizing viewport below
mdbreakpoint gracefully transitions Dual-Surface to responsive layout; expanding viewport back abovemdrestores Dual-Surface split without reset; concurrent artifact creation from multiple sub-agents surfaces items progressively without flicker.
Tier 3: Cross-Feature Interactions (12 Tests)
- Pair 1: Voice barge-in interrupt while action queue is actively running step.
- Pair 2: Optimistic queue burst while streaming audio TTS is playing.
- Pair 3: Mid-step tool cancellation triggered via voice interruption event.
- Pair 4: Quiet background routine fires during active voice session.
- Pair 5: 30-second rapid onboarding directly initiates full-duplex voice stage.
- Pair 6: Action queue synthesis dynamically surfaces Living Canvas for artifact inspection.
- Pair 7: Tactile folding step reflects mid-step tool cancellation without UI glitch.
- Pair 8: Voice streaming client reconnects after temporary network drop while action queue advances.
- Pair 9: Audio earcon triggers on optimistic queue drain and background delta detection.
- Pair 10: Living Canvas orientation swap while action queue runs background steps.
- Pair 11: Voice interrupt resets incremental utterance clause pipelining.
- Pair 12: Delta routine evaluation suppresses notification while user is typing in optimistic composer.
Tier 4: Real-World Application Scenarios (7 Comprehensive Workflows)
- Scenario 1: Ambient Hands-Free Meeting & Live Action Queue.
- Scenario 2: Conversational Barge-In & Mid-Step Tool Cancellation.
- Scenario 3: High-Frequency Multi-Prompt Burst & Optimistic Ordering.
- Scenario 4: Quiet Background Routine with Delta Detection & No Nagging.
- Scenario 5: Rapid 30-Second Zero-Questionnaire Onboarding to Live Execution.
- Scenario 6: Tactile Dual-Surface Progressive Disclosure & Step Folding Workflow.
- Scenario 7: Network Loss & Reconnection Resilience during Full-Duplex Voice Session.
4. Verification Method
Run the following command in the repository root to execute the complete E2E test suite:
pnpm vitest run apps/web/src/tests/e2e-voice-first-foundation.test.ts apps/web/src/tests/e2e-autonomous-agency-queues.test.ts apps/web/src/tests/e2e-minimalist-tactile-ux.test.tsx apps/web/src/tests/e2e-cross-feature-workflows.test.ts
Output:
Test Files 4 passed (4)
Tests 139 passed (139)
Duration ~3.0s
TypeScript Check:
pnpm --filter @rakazo/web check
Biome Lint & Format Check:
pnpm biome check apps/web/src/tests/e2e-*.ts* apps/web/src/tests/fixtures/*.ts