evOo-local/TEST_READY.md
cynarAI ca869561ac
Voice-first streaming, non-blocking follow-ups and chat UX (#205)
* media: check ffmpeg filter support before subtitle operations

* sandbox: handle synchronous spawn errors safely in desktop sandbox

* Voice-first streaming, non-blocking follow-ups, quiet routines and chat UX

Why: questions blocked the chat, follow-up messages had no acknowledgement, message actions sat beside the bubble, and voice turn-taking added latency.

What: waiting runs accept steering messages and the bot marks each claimed follow-up as seen; unclaimed follow-ups show as queued. Reaction, copy and reply controls sit under each message. Streaming voice (VAD endpointing, barge-in that stops the run, clause-based TTS), an optimistic input queue, procedural earcons and delta-based quiet routines were added. Onboarding can be skipped straight to the main conversation. Dual-surface layout and project pickers are included. Status colors use semantic tokens.

Tested: unit suites for core, adapters, api and web pass, typecheck and Biome are clean. Browser end-to-end tests were not run locally.

* Update translation catalogs

Re-extracted catalogs for the new and moved strings; the catalog sync test passes.

* Add project plan and test readiness notes

Notes from the milestone work: feature inventory, test infrastructure and test readiness. No code changes.

* fix(ci): align scale guards, space deletion, computer toggle, and mobile expo deps

* fix(ci): restore space deletion guard, browser toggle, and run-steps contracts

* fix(ci): restore model resolution errors, group stage, and onboarding flow

* fix(ci): restore in-app-browser close button, disable stop button while sending, and sync lingui catalogs

* fix(ci): address web e2e failures for bot-crud, choice cards, message quote, and voice onboarding

* test(web): wait for avatar and rig visibility in oo-avatar motion spec

* test(web): wait for persisted message IDs in message-hover-actions pagination spec

* Onboarding: always drop the local voice when the voice step is skipped

Skipping waited only for the latest voice load and disconnected only
when the client saw the connect succeed. An earlier or client-failed
connect could still persist the local credential, so a user who
skipped voice was left with one. Wait for every load and always
disconnect.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy1md9QE1ayJbDCwJ4vAQs

* Shell: hide the optimistic copy once the transcript shows the send

A message sent while the bot works streamed into the transcript before
its send request resolved, so the same text showed twice until the
optimistic bubble was removed. Drop an optimistic message as soon as a
persisted user message with the same text appears.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wy1md9QE1ayJbDCwJ4vAQs

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-10-15 14:08:31 +02:00

15 KiB
Raw Permalink Blame History

TEST_READY: evOo Voice-First Autonomous Agent Operating System

Status

  • Test Suite Status: READY
  • Author: test_writer_r2_1 (Dual-Track E2E Testing Specialist)
  • Timestamp: 2026-10-12T13:05:00Z
  • Target Milestones: M1, M2, M3, M4, M5 (Features F1–F12 across Requirements R1, R2, R3)
  • Test Framework: Vitest 4.1.11 with // @vitest-environment jsdom & Pure React 19 harnesses

1. Authored Test Suites Summary

# Test File Path Scope & Feature Focus Tiers Covered Total Test Cases Passing Baseline
1 apps/web/src/tests/e2e-voice-first-foundation.test.ts Streaming Audio Wire (<120ms framing, PCM16, VAD), Full-Duplex Audio & Barge-In Interruption, Incremental Utterance Streamer (<300ms TTFS, invite strip), Tactile Monochrome Earcons Tier 1, Tier 2 (F1–F4) 40 ✅ 40 / 40 Passed
2 apps/web/src/tests/e2e-autonomous-agency-queues.test.ts Optimistic Input Queue (Zero UI Freeze, unblocked stop), Mid-Step Tool Cancellation & Active Steering, Dynamic Action Queue State Machine, Quiet Background Routines & Delta Evaluation (on_change) Tier 1, Tier 2 (F5–F8) 40 ✅ 40 / 40 Passed
3 apps/web/src/tests/e2e-minimalist-tactile-ux.test.tsx Tactile Minimalist Step Execution (strip JobProgressCard bloat, folding # step), 30-Second Rapid Onboarding (zero questionnaires, 3 action sparks), Monochrome Token Hygiene (@rakazo/ui-tokens), Living Canvas Progressive Disclosure Tier 1, Tier 2 (F9–F12) 40 ✅ 40 / 40 Passed
4 apps/web/src/tests/e2e-cross-feature-workflows.test.ts 12 Pairwise Cross-Feature Interactions & 7 Realistic End-to-End Workflows (Hands-free meeting, voice barge-in tool abort, high-frequency prompt burst, quiet background routines, 30s onboarding, progressive disclosure, voice network resilience) Tier 3, Tier 4 19 ✅ 19 / 19 Passed
Total 4 Comprehensive Test Suites + 3 Fixtures Full Feature Inventory (F1 to F12) Coverage Tiers 1–4 139 tests ✅ 139 / 139 (100% Pass)

2. Requirement & Feature Coverage Matrix

Feature Source Requirement Primary Test File Tiers Pass / Fail
F1: Streaming Audio Wire & Low-Latency VAD ORIGINAL_REQUEST §R1 e2e-voice-first-foundation.test.ts T1, T2, T3, T4 ✅ PASS (10/10)
F2: Full-Duplex Audio & Barge-In Interruption ORIGINAL_REQUEST §R1 e2e-voice-first-foundation.test.ts T1, T2, T3, T4 ✅ PASS (10/10)
F3: Incremental Utterance Streamer & Invite Strip ORIGINAL_REQUEST §R1 e2e-voice-first-foundation.test.ts T1, T2, T3, T4 ✅ PASS (10/10)
F4: Cross-Platform Earcons & Mobile Audio ORIGINAL_REQUEST §R1 e2e-voice-first-foundation.test.ts T1, T2, T3, T4 ✅ PASS (10/10)
F5: Optimistic Input Queue & Zero UI Freeze ORIGINAL_REQUEST §R2 e2e-autonomous-agency-queues.test.ts T1, T2, T3, T4 ✅ PASS (10/10)
F6: Mid-Step Tool Cancellation & Active Steering ORIGINAL_REQUEST §R2 e2e-autonomous-agency-queues.test.ts T1, T2, T3, T4 ✅ PASS (10/10)
F7: Outcome-Driven Action Queue State Machine ORIGINAL_REQUEST §R2 e2e-autonomous-agency-queues.test.ts T1, T2, T3, T4 ✅ PASS (10/10)
F8: Quiet Background Routines & Delta Evaluation ORIGINAL_REQUEST §R2 e2e-autonomous-agency-queues.test.ts T1, T2, T3, T4 ✅ PASS (10/10)
F9: Tactile Minimalist Step Execution ORIGINAL_REQUEST §R3 e2e-minimalist-tactile-ux.test.tsx T1, T2, T3, T4 ✅ PASS (10/10)
F10: 30-Second Rapid Onboarding ORIGINAL_REQUEST §R3 e2e-minimalist-tactile-ux.test.tsx T1, T2, T3, T4 ✅ PASS (10/10)
F11: Monochrome Semantic Token Hygiene ORIGINAL_REQUEST §R3 e2e-minimalist-tactile-ux.test.tsx T1, T2, T3, T4 ✅ PASS (10/10)
F12: Living Canvas Progressive Disclosure ORIGINAL_REQUEST §R3 e2e-minimalist-tactile-ux.test.tsx T1, T2, T3, T4 ✅ PASS (10/10)
T3: Pairwise Cross-Feature Interactions Cross-Module Interactions e2e-cross-feature-workflows.test.ts Tier 3 ✅ PASS (12/12)
T4: Real-World Application Scenarios Operational Workflows e2e-cross-feature-workflows.test.ts Tier 4 ✅ PASS (7/7)

3. Tier Breakdown & Test Cases

Tier 1: Feature Coverage (60 Tests)

  • F1: 80ms PCM16 frame chunking (1280 samples at 16kHz); VAD normalized RMS energy; <120ms cutoff threshold eliminating 850ms silence floor; monotonic timestamp increment; streaming client transcript & playback chunk callbacks.
  • F2: Continuous microphone capture with acoustic echo cancellation (echoCancellation: true); user speech energy detection while bot speaking triggers barge-in; immediate playback audio mute; backend abort propagation (abort('barge_in')); state transition from speaking to listening.
  • F3: Incremental clause segmentation in 4–6 word chunks; time-to-first-sound pipelining (<300ms TTFS target); stripping robotic invite prompts ("What would you like to do next?"); code fence description transformation per speakable(); natural punctuation boundary clause splits.
  • F4: Core tactile earcons (listening-start, listening-stop, speaking-start, speaking-done); Web Audio procedural oscillator profiles; master audio bus ducking (0.28 ducking ratio); cross-platform mobile earcon mapping; state machine audio cue triggers (interact, attention).
  • F5: Submitting message while another in-flight enqueues without UI freeze; temporary optimistic ID assignment (temp-...) and timestamp; composer text clearing immediately; Stop button unblocked and clickable; sequential FIFO drainage.
  • F6: Ingesting steering during active tool call dispatches mid-step cancellation signal; tool abort controller receives abort('steering_redirection'); aborted tool output discarded from history; agent pivots immediately to steering intent; steering message marked claimed.
  • F7: DynamicActionQueue initialization; multi-step prompt synthesis into ActionQueueStep items; queue advancement starts pending step with startedAt; completing step sets completedAt and advances; cancelActive() marks step cancelled.
  • F8: RoutineNotificationPolicy with "on_change" evaluates output delta hash; identical output hash suppresses notification (quiet run); distinct output hash triggers notification; policy "always" notifies unconditionally; policy "never" suppresses all notifications.
  • F9: Minimalist folding step button renders with data-testid="run-steps"; displays clean # step counter without progress track bloat; clicking fold button toggles aria-expanded; step tool name and summary rendered when expanded; keyboard Enter/Space toggles fold.
  • F10: 1-click bootstrap creates initial agent workspace without 11-step questionnaire; auto-detects system timezone via Intl.DateTimeFormat; auto-detects system language via navigator.language; surfaces 3 tactile starter sparks; selecting spark starts session (<30s value).
  • F11: Theme tokens definition from @rakazo/ui-tokens; zero hardcoded product hex colors; zero arbitrary Tailwind brackets [var(--...)]; accessible focus rings (focus-visible:ring-ring); status indicators use semantic tokens (text-destructive, text-muted-foreground).
  • F12: Living Canvas pane renders with data-testid="side-panel"; Mission Stream renders with data-testid="mission-stream-pane" and data-testid="transcript"; preserves critical test IDs (computer-preview, dual-surface-divider, dual-surface-header-controls); progressive disclosure on artifact creation; tab switching between Files, Sandbox, Browser.

Tier 2: Boundary & Corner Cases (60 Tests)

  • F1: Empty audio buffer (Int16Array(0)) handled with RMS 0 and no NaN errors; pure silence buffer maintains energy 0 and triggers silence cutoff without hang; clipping values clamped to Int16 limits (-32768 to 32767); out-of-order timestamp detection and jitter buffer tolerance; network drop triggers client reconnect and state preservation.
  • F2: Microphone permission denied handled with graceful error state; interruption when audio playback is already idle is safe no-op; micro-burst transients (<50ms noise pop) filtered by VAD debounce; rapid alternating speaker turns (ping-pong barge-in); concurrent interrupt and incoming playback chunk race condition resolves in favor of user interrupt.
  • F3: Single-word answers streamed without minimum character delay stalling; code-only block output sanitized without synthetic audio artifacts; emoji-only message stream strips emojis and does not stall TTS engine; micro-fragment token arrivals buffered into complete words; unexpected LLM stream termination cleanly finalizes active clause.
  • F4: Rapid consecutive sound cue triggers (<10ms) queue smoothly without audio glitches; muted/silent mode suppresses oscillator output while executing state transition; AudioContext suspended state auto-resumes on first user gesture; unknown sound cue name falls back gracefully without throwing exceptions; cross-platform volume normalization preserves peak headroom.
  • F5: Rapid typing burst (10 rapid inputs within 100ms) enqueues all 10 without dropping; bounded queue size enforces max pending limit with graceful error; network failure while message in flight triggers failed state with retry option without losing text; empty/whitespace-only messages rejected before enqueueing; duplicate messages submitted in rapid succession deduplicated if flagged.
  • F6: Race condition: tool finishes at the exact moment abort signal fires resolves cleanly; tool execution timeout triggers automated mid-step cancellation with timeout reason; multiple rapid contradictory steering messages resolves to most recent intent; tool that throws exception upon abort caught gracefully without crashing runtime; external system side-effect fence prevents state pollution on aborted tools.
  • F7: Synthesizing empty or meaningless input results in clean no-op; advance() called on an empty queue or completed queue is safe no-op; failed step status prevents automatic advance unless retry policy allows; step status state machine strictly validates allowable transitions; queue serialization and deserialization preserves complete step fidelity.
  • F8: First execution (prevOutputHash === '') with "on_change" triggers initial notification; policy "on_failure" triggers notification only when routine status is "failed"; whitespace/formatting variations evaluated cleanly against normalized hash; high frequency routine executions do not spam notifications if output remains constant; routine execution failure self-heals without crashing scheduler.
  • F9: Empty steps list renders cleanly without error or broken layout; extreme step count (50+ tool calls) remains folded by default without layout overflow; failed step renders clean status badge without noisy error stacks unless expanded; absence of chrome bloat (removal of animated tracks, live timers · 1:24, uppercase callouts AUFGABE); rapid folding/unfolding toggles do not oscillate or corrupt DOM state.
  • F10: Missing or invalid timezone fallback safely resolves to UTC without throwing error; missing or invalid locale fallback safely resolves to en; re-entering onboarding while workspace already exists redirects to active space; starter spark with special characters sanitized safely; offline onboarding gracefully saves initial draft locally without blocking UI.
  • F11: Semantic contrast in monochrome mode preserves readable text tokens; disabled component states apply semantic opacity disabled:opacity-50 rather than arbitrary grey; high contrast mode support: borders preserve visibility via semantic border-border; dynamic theme switching updates token class mappings deterministically; interactive hover states use semantic hover tokens (hover:bg-accent).
  • F12: Closing Living Canvas collapses aside pane without unmounting workspace state; rapid switching between canvas tabs maintains individual surface state; resizing viewport below md breakpoint gracefully transitions Dual-Surface to responsive layout; expanding viewport back above md restores Dual-Surface split without reset; concurrent artifact creation from multiple sub-agents surfaces items progressively without flicker.

Tier 3: Cross-Feature Interactions (12 Tests)

  • Pair 1: Voice barge-in interrupt while action queue is actively running step.
  • Pair 2: Optimistic queue burst while streaming audio TTS is playing.
  • Pair 3: Mid-step tool cancellation triggered via voice interruption event.
  • Pair 4: Quiet background routine fires during active voice session.
  • Pair 5: 30-second rapid onboarding directly initiates full-duplex voice stage.
  • Pair 6: Action queue synthesis dynamically surfaces Living Canvas for artifact inspection.
  • Pair 7: Tactile folding step reflects mid-step tool cancellation without UI glitch.
  • Pair 8: Voice streaming client reconnects after temporary network drop while action queue advances.
  • Pair 9: Audio earcon triggers on optimistic queue drain and background delta detection.
  • Pair 10: Living Canvas orientation swap while action queue runs background steps.
  • Pair 11: Voice interrupt resets incremental utterance clause pipelining.
  • Pair 12: Delta routine evaluation suppresses notification while user is typing in optimistic composer.

Tier 4: Real-World Application Scenarios (7 Comprehensive Workflows)

  • Scenario 1: Ambient Hands-Free Meeting & Live Action Queue.
  • Scenario 2: Conversational Barge-In & Mid-Step Tool Cancellation.
  • Scenario 3: High-Frequency Multi-Prompt Burst & Optimistic Ordering.
  • Scenario 4: Quiet Background Routine with Delta Detection & No Nagging.
  • Scenario 5: Rapid 30-Second Zero-Questionnaire Onboarding to Live Execution.
  • Scenario 6: Tactile Dual-Surface Progressive Disclosure & Step Folding Workflow.
  • Scenario 7: Network Loss & Reconnection Resilience during Full-Duplex Voice Session.

4. Verification Method

Run the following command in the repository root to execute the complete E2E test suite:

pnpm vitest run apps/web/src/tests/e2e-voice-first-foundation.test.ts apps/web/src/tests/e2e-autonomous-agency-queues.test.ts apps/web/src/tests/e2e-minimalist-tactile-ux.test.tsx apps/web/src/tests/e2e-cross-feature-workflows.test.ts

Output:

Test Files  4 passed (4)
Tests       139 passed (139)
Duration    ~3.0s

TypeScript Check:

pnpm --filter @rakazo/web check

Biome Lint & Format Check:

pnpm biome check apps/web/src/tests/e2e-*.ts* apps/web/src/tests/fixtures/*.ts