# E2E Test Infrastructure: evOo Voice-First Autonomous Agent OS (Dual Track)

## 1. Test Philosophy & Principles
- **Opaque-Box & Requirement-Driven**: Derived strictly from user specifications in `ORIGINAL_REQUEST.md` (§R1, §R2, §R3) and architecture specifications in `orchestrator_2/PROJECT.md` (§F1–§F12).
- **Interface Contract Verification**: Validates concrete contracts for `StreamingVoiceFrame`, `StreamingVoiceClient`, `ActionQueueStep`, `DynamicActionQueue`, and `RoutineNotificationPolicy`.
- **Offline & Deterministic Execution**: Zero external cloud or hosted vendor dependencies; uses simulated Web Audio, deterministic PCM frame generators, and pure React 19 testing harnesses.
- **Strict Linting & Design System Compliance**: Conforms strictly to `@rakazo/ui-tokens` monochrome tokens, zero arbitrary hex colors, top-level `import type`, and clean Biome formatting.

---

## 2. Test Architecture & Runner Commands

### Execution Commands
1. **Run All E2E OS Test Suites (139 tests)**:
   ```bash
   pnpm vitest run apps/web/src/tests/e2e-voice-first-foundation.test.ts apps/web/src/tests/e2e-autonomous-agency-queues.test.ts apps/web/src/tests/e2e-minimalist-tactile-ux.test.tsx apps/web/src/tests/e2e-cross-feature-workflows.test.ts
   ```
2. **Run Individual Feature Suites**:
   - Voice Foundation (F1–F4):
     ```bash
     pnpm vitest run apps/web/src/tests/e2e-voice-first-foundation.test.ts
     ```
   - Autonomous Agency & Queues (F5–F8):
     ```bash
     pnpm vitest run apps/web/src/tests/e2e-autonomous-agency-queues.test.ts
     ```
   - Minimalist Tactile UX & Onboarding (F9–F12):
     ```bash
     pnpm vitest run apps/web/src/tests/e2e-minimalist-tactile-ux.test.tsx
     ```
   - Cross-Feature & Real-World Workflows (T3 & T4):
     ```bash
     pnpm vitest run apps/web/src/tests/e2e-cross-feature-workflows.test.ts
     ```
3. **TypeScript Compilation Check**:
   ```bash
   pnpm --filter @rakazo/web check
   ```
4. **Code Quality & Linter**:
   ```bash
   pnpm biome check apps/web/src/tests/e2e-*.ts* apps/web/src/tests/fixtures/*.ts
   ```

---

## 3. Feature Inventory & Test Coverage Matrix

| # | Feature | Scope & Requirement | Test File | T1 | T2 | T3 | T4 | Total Tests |
|---|---------|---------------------|-----------|:--:|:--:|:--:|:--:|:-----------:|
| **F1** | Streaming Audio Wire & Low-Latency VAD | ORIGINAL_REQUEST §R1 (<120ms framing, PCM16, VAD) | `e2e-voice-first-foundation.test.ts` | 5 | 5 | ✓ | ✓ | 10+ |
| **F2** | Full-Duplex Audio & Barge-In Interruption | ORIGINAL_REQUEST §R1 (Continuous AEC, barge-in mute) | `e2e-voice-first-foundation.test.ts` | 5 | 5 | ✓ | ✓ | 10+ |
| **F3** | Incremental Utterance Streamer & Invite Strip | ORIGINAL_REQUEST §R1 (Clause pipelining, no invite delays) | `e2e-voice-first-foundation.test.ts` | 5 | 5 | ✓ | ✓ | 10+ |
| **F4** | Cross-Platform Earcons & Mobile Audio | ORIGINAL_REQUEST §R1 (Tactile Web Audio & mobile cues) | `e2e-voice-first-foundation.test.ts` | 5 | 5 | ✓ | ✓ | 10+ |
| **F5** | Optimistic Input Queue & Zero UI Freeze | ORIGINAL_REQUEST §R2 (Non-blocking composer, optimistic FIFO) | `e2e-autonomous-agency-queues.test.ts` | 5 | 5 | ✓ | ✓ | 10+ |
| **F6** | Mid-Step Tool Cancellation & Active Steering | ORIGINAL_REQUEST §R2 (Mid-execution abort, active steering) | `e2e-autonomous-agency-queues.test.ts` | 5 | 5 | ✓ | ✓ | 10+ |
| **F7** | Outcome-Driven Action Queue State Machine | ORIGINAL_REQUEST §R2 (Dynamic action queue, outcomes) | `e2e-autonomous-agency-queues.test.ts` | 5 | 5 | ✓ | ✓ | 10+ |
| **F8** | Quiet Background Routines & Delta Evaluation | ORIGINAL_REQUEST §R2 (on_change delta hash gating, no nag) | `e2e-autonomous-agency-queues.test.ts` | 5 | 5 | ✓ | ✓ | 10+ |
| **F9** | Tactile Minimalist Step Execution | ORIGINAL_REQUEST §R3 (Strip chrome bloat, folding step) | `e2e-minimalist-tactile-ux.test.tsx` | 5 | 5 | ✓ | ✓ | 10+ |
| **F10** | 30-Second Rapid Onboarding | ORIGINAL_REQUEST §R3 (1-click bootstrap, 3 action sparks) | `e2e-minimalist-tactile-ux.test.tsx` | 5 | 5 | ✓ | ✓ | 10+ |
| **F11** | Monochrome Semantic Token Hygiene | ORIGINAL_REQUEST §R3 (@rakazo/ui-tokens, zero rogue hex) | `e2e-minimalist-tactile-ux.test.tsx` | 5 | 5 | ✓ | ✓ | 10+ |
| **F12** | Living Canvas Progressive Disclosure | ORIGINAL_REQUEST §R3 (Dynamic canvas, preserve test IDs) | `e2e-minimalist-tactile-ux.test.tsx` | 5 | 5 | ✓ | ✓ | 10+ |
| **T3** | Cross-Feature Interactions | Pairwise combinations across R1, R2, R3 | `e2e-cross-feature-workflows.test.ts` | - | - | 12 | - | 12 |
| **T4** | Real-World Application Workflows | Realistic end-to-end operational scenarios | `e2e-cross-feature-workflows.test.ts` | - | - | - | 7 | 7 |
| **Total** | **All Features F1–F12 across T1–T4** | **100% Comprehensive Coverage** | **4 Suites + 3 Fixtures** | **60** | **60** | **12** | **7** | **139** |

---

## 4. Test Tiers Breakdown

### Tier 1: Feature Coverage (60 Tests: 5 Tests × 12 Features)
- **F1**: 80ms PCM16 frame chunking (1280 samples at 16kHz); VAD normalized RMS energy; <120ms cutoff threshold eliminating 850ms silence floor; monotonic timestamp increment; streaming client transcript & playback chunk callbacks.
- **F2**: Continuous microphone capture with acoustic echo cancellation (`echoCancellation: true`); user speech energy detection while bot speaking triggers barge-in; immediate playback audio mute; backend abort propagation (`abort('barge_in')`); state transition from speaking to listening.
- **F3**: Incremental clause segmentation in 4–6 word chunks; time-to-first-sound pipelining (<300ms TTFS target); stripping robotic invite prompts ("What would you like to do next?"); code fence description transformation per `speakable()`; natural punctuation boundary clause splits.
- **F4**: Core tactile earcons (`listening-start`, `listening-stop`, `speaking-start`, `speaking-done`); Web Audio procedural oscillator profiles; master audio bus ducking (0.28 ducking ratio); cross-platform mobile earcon mapping; state machine audio cue triggers (`interact`, `attention`).
- **F5**: Submitting message while another in-flight enqueues without UI freeze; temporary optimistic ID assignment (`temp-...`) and timestamp; composer text clearing immediately; Stop button unblocked and clickable; sequential FIFO drainage.
- **F6**: Ingesting steering during active tool call dispatches mid-step cancellation signal; tool abort controller receives `abort('steering_redirection')`; aborted tool output discarded from history; agent pivots immediately to steering intent; steering message marked claimed.
- **F7**: `DynamicActionQueue` initialization; multi-step prompt synthesis into `ActionQueueStep` items; queue advancement starts pending step with `startedAt`; completing step sets `completedAt` and advances; `cancelActive()` marks step cancelled.
- **F8**: `RoutineNotificationPolicy` with `"on_change"` evaluates output delta hash; identical output hash suppresses notification (quiet run); distinct output hash triggers notification; policy `"always"` notifies unconditionally; policy `"never"` suppresses all notifications.
- **F9**: Minimalist folding step button renders with `data-testid="run-steps"`; displays clean `# step` counter without progress track bloat; clicking fold button toggles `aria-expanded`; step tool name and summary rendered when expanded; keyboard Enter/Space toggles fold.
- **F10**: 1-click bootstrap creates initial agent workspace without 11-step questionnaire; auto-detects system timezone via `Intl.DateTimeFormat`; auto-detects system language via `navigator.language`; surfaces 3 tactile starter sparks; selecting spark starts session (<30s value).
- **F11**: Theme tokens definition from `@rakazo/ui-tokens`; zero hardcoded product hex colors; zero arbitrary Tailwind brackets `[var(--...)]`; accessible focus rings (`focus-visible:ring-ring`); status indicators use semantic tokens (`text-destructive`, `text-muted-foreground`).
- **F12**: Living Canvas pane renders with `data-testid="side-panel"`; Mission Stream renders with `data-testid="mission-stream-pane"` and `data-testid="transcript"`; preserves critical test IDs (`computer-preview`, `dual-surface-divider`, `dual-surface-header-controls`); progressive disclosure on artifact creation; tab switching between Files, Sandbox, Browser.

### Tier 2: Boundary & Corner Cases (60 Tests: 5 Tests × 12 Features)
- **F1**: Empty audio buffer (`Int16Array(0)`) handled with RMS 0 and no NaN errors; pure silence buffer maintains energy 0 and triggers silence cutoff without hang; clipping values clamped to Int16 limits (-32768 to 32767); out-of-order timestamp detection and jitter buffer tolerance; network drop triggers client reconnect and state preservation.
- **F2**: Microphone permission denied handled with graceful error state; interruption when audio playback is already idle is safe no-op; micro-burst transients (<50ms noise pop) filtered by VAD debounce; rapid alternating speaker turns (ping-pong barge-in); concurrent interrupt and incoming playback chunk race condition resolves in favor of user interrupt.
- **F3**: Single-word answers streamed without minimum character delay stalling; code-only block output sanitized without synthetic audio artifacts; emoji-only message stream strips emojis and does not stall TTS engine; micro-fragment token arrivals buffered into complete words; unexpected LLM stream termination cleanly finalizes active clause.
- **F4**: Rapid consecutive sound cue triggers (<10ms) queue smoothly without audio glitches; muted/silent mode suppresses oscillator output while executing state transition; AudioContext suspended state auto-resumes on first user gesture; unknown sound cue name falls back gracefully without throwing exceptions; cross-platform volume normalization preserves peak headroom.
- **F5**: Rapid typing burst (10 rapid inputs within 100ms) enqueues all 10 without dropping; bounded queue size enforces max pending limit with graceful error; network failure while message in flight triggers failed state with retry option without losing text; empty/whitespace-only messages rejected before enqueueing; duplicate messages submitted in rapid succession deduplicated if flagged.
- **F6**: Race condition: tool finishes at the exact moment abort signal fires resolves cleanly; tool execution timeout triggers automated mid-step cancellation with `timeout` reason; multiple rapid contradictory steering messages resolves to most recent intent; tool that throws exception upon abort caught gracefully without crashing runtime; external system side-effect fence prevents state pollution on aborted tools.
- **F7**: Synthesizing empty or meaningless input results in clean no-op; `advance()` called on an empty queue or completed queue is safe no-op; failed step status prevents automatic advance unless retry policy allows; step status state machine strictly validates allowable transitions; queue serialization and deserialization preserves complete step fidelity.
- **F8**: First execution (`prevOutputHash === ''`) with `"on_change"` triggers initial notification; policy `"on_failure"` triggers notification only when routine status is `"failed"`; whitespace/formatting variations evaluated cleanly against normalized hash; high frequency routine executions do not spam notifications if output remains constant; routine execution failure self-heals without crashing scheduler.
- **F9**: Empty steps list renders cleanly without error or broken layout; extreme step count (50+ tool calls) remains folded by default without layout overflow; failed step renders clean status badge without noisy error stacks unless expanded; absence of chrome bloat (removal of animated tracks, live timers `· 1:24`, uppercase callouts `AUFGABE`); rapid folding/unfolding toggles do not oscillate or corrupt DOM state.
- **F10**: Missing or invalid timezone fallback safely resolves to UTC without throwing error; missing or invalid locale fallback safely resolves to `en`; re-entering onboarding while workspace already exists redirects to active space; starter spark with special characters sanitized safely; offline onboarding gracefully saves initial draft locally without blocking UI.
- **F11**: Semantic contrast in monochrome mode preserves readable text tokens; disabled component states apply semantic opacity `disabled:opacity-50` rather than arbitrary grey; high contrast mode support: borders preserve visibility via semantic `border-border`; dynamic theme switching updates token class mappings deterministically; interactive hover states use semantic hover tokens (`hover:bg-accent`).
- **F12**: Closing Living Canvas collapses aside pane without unmounting workspace state; rapid switching between canvas tabs maintains individual surface state; resizing viewport below `md` breakpoint gracefully transitions Dual-Surface to responsive layout; expanding viewport back above `md` restores Dual-Surface split without reset; concurrent artifact creation from multiple sub-agents surfaces items progressively without flicker.

### Tier 3: Cross-Feature Interactions (12 Tests: Pairwise Coverage)
- **Pair 1**: Voice barge-in interrupt while action queue is actively running step (aborts active step, pauses execution, pivots queue).
- **Pair 2**: Optimistic queue burst while streaming audio TTS is playing (synthesizes follow-ups without audio dropout).
- **Pair 3**: Mid-step tool cancellation triggered via voice interruption event (voice audio energy spike propagates to in-flight tool abort controller).
- **Pair 4**: Quiet background routine fires during active voice session (audio ducking remains unaffected; output unchanged suppresses chat/audio notification).
- **Pair 5**: 30-second rapid onboarding directly initiates full-duplex voice stage (first-time user goes from 1-click bootstrap straight to hands-free voice dialogue).
- **Pair 6**: Action queue synthesis dynamically surfaces Living Canvas for artifact inspection (multi-step queue opens canvas on artifact creation).
- **Pair 7**: Tactile folding step reflects mid-step tool cancellation without UI glitch (shows `[cancelled]` step folded cleanly).
- **Pair 8**: Voice streaming client reconnects after temporary network drop while action queue advances to next step.
- **Pair 9**: Audio earcon triggers on optimistic queue drain and background delta detection (`interact` on drain, `attention` on delta).
- **Pair 10**: Living Canvas orientation swap while action queue runs background steps (DOM flex order reverses while steps advance uninterrupted).
- **Pair 11**: Voice interrupt resets incremental utterance clause pipelining (discards queued clauses immediately).
- **Pair 12**: Delta routine evaluation suppresses notification while user is typing in optimistic composer (zero disruption to active composer focus).

### Tier 4: Real-World Application Scenarios (7 Comprehensive Workflows)
- **Scenario 1: Ambient Hands-Free Meeting & Live Action Queue**: User dictates three multi-step action items in voice mode -> VAD frames audio -> parsed into Dynamic Action Queue -> steps advance deterministically -> earcon signals completion.
- **Scenario 2: Conversational Barge-In & Mid-Step Tool Cancellation**: Agent executing long command -> user speaks barge-in redirect -> playback instantly muted -> in-flight tool aborted -> queue pivoted to new intent.
- **Scenario 3: High-Frequency Multi-Prompt Burst & Optimistic Ordering**: User types 3 follow-up messages rapidly during elevated latency -> optimistic queue renders all 3 immediately with temporary IDs -> requests drain in order -> zero UI freeze.
- **Scenario 4: Quiet Background Routine with Delta Detection & No Nagging**: Scheduled routine runs periodic cycles -> hash unchanged -> zero notifications -> on cycle with detected anomaly, delta triggers notification -> user alerted only when critical.
- **Scenario 5: Rapid 30-Second Zero-Questionnaire Onboarding to Live Execution**: New user lands -> system auto-detects timezone and locale -> user clicks starter spark "Research competitor architecture" -> workspace boots instantly -> agent begins execution within 30 seconds.
- **Scenario 6: Tactile Dual-Surface Progressive Disclosure & Step Folding Workflow**: User asks agent to inspect repo -> agent executes 12 tool steps -> steps fold into clean `# steps` button -> agent writes file -> Living Canvas smoothly unfolds without modal occlusion.
- **Scenario 7: Network Loss & Reconnection Resilience during Full-Duplex Voice Session**: Voice stream active -> network packet loss occurs -> streaming client detects connection drop -> buffers outgoing frames -> reconnects -> flushes buffer and resumes conversation without data loss.

---

## 5. Artifact Index & Directory Layout

```
apps/web/src/tests/
├── e2e-voice-first-foundation.test.ts        # 40 tests (Features F1-F4, Tiers 1-2)
├── e2e-autonomous-agency-queues.test.ts      # 40 tests (Features F5-F8, Tiers 1-2)
├── e2e-minimalist-tactile-ux.test.tsx        # 40 tests (Features F9-F12, Tiers 1-2)
├── e2e-cross-feature-workflows.test.ts       # 19 tests (Tier 3 Pairwise & Tier 4 Scenarios)
└── fixtures/
    ├── voice-contracts.ts                    # F1-F4 contracts & reference VAD/framing models
    ├── agency-contracts.ts                   # F5-F8 contracts, Action Queue & Routine policy models
    └── ux-contracts.ts                       # F9-F12 contracts, environment detection & starter sparks
```