0703d79b93
Replace the full-bleed cyan/violet/emerald bloom field with one dark glass panel: the Oo, a live caption whose new words fade in, and a compact control row (phase label, transcript, interrupt, hang up). An iridescent aurora rises from the panel's bottom edge, follows mic energy while listening, drifts while thinking and blooms while speaking. Its spectrum is the shared bot palette rotated so the active Oo's color sits in the middle, so no new hex values. Voice entry buttons drop the hardcoded cyan/amber styling for neutral tokens plus a thin aurora rim when voice is ready. Dead voice CSS is removed and the voice e2e now captures the stage. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012G3dUMVDCDL3PwQE4WTQxL
169 lines
9.4 KiB
Markdown
169 lines
9.4 KiB
Markdown
# EvOo Voice — Interaction Model
|
||
|
||
## Gesamterfahrung (encounter)
|
||
|
||
A conversation with an Oo is a **shared encounter**, not a thread with a tiny avatar:
|
||
|
||
| Layer | Role |
|
||
|-------|------|
|
||
| **Oo Stage** | Primary — large `BotAvatar` / persona presence, runtime pose, awareness strip |
|
||
| **Transcript** | Secondary protocol — compact caption trail, scrollable record |
|
||
| **Sound** | Calm procedural cues wired to real events (work finished, voice phases, handoffs) |
|
||
|
||
Text mode (`OoStage` in Shell) and Voice mode (`VoiceStage` overlay) share the same metaphor: eyes/presence + space + sound. Brand lock (asymmetric Kulleraugen) is unchanged.
|
||
|
||
### Product model — conversation-centric (Grok-like)
|
||
|
||
**Oos (agents) exist**, but the product is organized around **conversations (Unterhaltungen)**, not “pick an agent row = the whole app”:
|
||
|
||
| Layer | What it is |
|
||
|-------|------------|
|
||
| **Team** | Roster of your Oos — who you can talk to |
|
||
| **1:1 Unterhaltung** | One Oo, one shared encounter (`OoStage`) |
|
||
| **Gruppenunterhaltung** | Multiple Oos (+ humans later) in one encounter (`OoGroupStage`) |
|
||
| **Projekte** | Group conversations organized in **Ordner** (`BotSection` folders in sidebar) |
|
||
|
||
| Mode | Shell route | Stage |
|
||
|------|-------------|-------|
|
||
| **Unterhaltung** (1:1) | `/app/:botId` | `OoStage` — one Oo hero |
|
||
| **Gruppenunterhaltung** | `/app/g/:groupId?s=live` | `OoGroupStage` — active-speaker hero + ensemble row; **Riffing** when peers spark (`docs/GROUP-CONVERSATIONS.md`) |
|
||
| **Project sessions** | `/app/g/:groupId` (timeline) | No stage — session list (`ProjectSessionTimeline`) |
|
||
|
||
Clicking an Oo in **Team** opens a **1:1 conversation**, not “the agent product”. Clicking a project opens a **group conversation** or its session timeline. **Gesamterfahrung** (stage + sound + transcript record) applies to both conversation types.
|
||
|
||
**Emergent ensemble:** *Das Ganze ist mehr als die Summe seiner Teile* — power from distinct specialists + chemistry in groups, not one mega-agent (`docs/PRODUCT.md`).
|
||
|
||
Voice is the **immersive primary mode** for 1:1 bot conversations. Text chat uses the same stage-first layout; the transcript stays readable but no longer carries identity chrome.
|
||
|
||
## Entry
|
||
|
||
| Surface | Action |
|
||
|---------|--------|
|
||
| Composer **Voice** pill | Opens Voice Stage when a provider is connected; otherwise opens Voice settings |
|
||
| Transcript “Speak this reply” | Reads one message aloud (text-first affordance, unchanged) |
|
||
|
||
The composer and stage Voice controls use `h-control-lg` in neutral tokens; when voice is ready an iridescent aurora rim (`VoiceAuroraRim`) runs along the bottom edge.
|
||
|
||
## Voice Stage
|
||
|
||
Full-screen dark layer with one glass panel (`VoiceStage`): the Oo, the live caption, and a control row. An iridescent aurora (`VoiceAurora`) rises from the panel's bottom edge. Its spectrum is the bot palette (`botColors` from `@rakazo/ui-tokens`) rotated so the active Oo's own color sits in the center.
|
||
|
||
### Persona & consciousness layer
|
||
|
||
Persona is **derived at runtime** from existing bot fields (`name`, `title`, `description`) — aligned with `templates/*` archetypes, not a separate marketing field.
|
||
|
||
| System | Path |
|
||
|--------|------|
|
||
| Archetype detection | `apps/web/src/lib/bot-persona.ts` |
|
||
| i18n status copy | `apps/web/src/lib/voice-persona-copy.ts` |
|
||
| Awareness strip | `VoiceAwarenessStrip` — subagent, handoff, delegation, peer reply |
|
||
| Visual identity | `BotAvatar` — persona shape + unified `EvooMascot` soul (eyes/mouth/hands) driven by voice phase |
|
||
| Voice delivery | `speaker.speak(..., { botId })` → server resolves per-bot `voiceId` |
|
||
|
||
Archetypes include: CTO executive (Steve Jobs template), chief operator (Fixer), builder (Dr. Eggbot), curator (Webby), work/life inboxes, home steward, quant, engineer, **specialist** (narrow fallback — no Wunderwuzzi). See `docs/PRODUCT.md` and `docs/PERSONA-SELECTION.md`.
|
||
|
||
**Companion tone:** geek-peer (fachsimpeln when wanted) or quiet-doer (execute without condescension) — inferred per user, not one voice for everyone.
|
||
|
||
**Accessibility of ambition:** grandma-simple entry (Thermomix to hard math — no power-user gate); expert-grade ceiling per specialty. Same product, different crews; never a mediocre everything-bot.
|
||
|
||
Phase labels, idle prompts, and thinking lines are **character-specific** (DE/EN via Lingui). Collaboration narration uses persona-aware phrasing (“casting”, “routing”, “handing off”) instead of generic tool labels.
|
||
|
||
### States
|
||
|
||
| Phase | Visual | Audio |
|
||
|-------|--------|-------|
|
||
| **Listening** | Aurora rises and brightens with mic energy, breathing | Dictation active; silence endpoint sends |
|
||
| **Thinking** | Aurora settles low and drifts side to side | TTS idle; awaiting bot reply |
|
||
| **Speaking** | Full aurora bloom, conductor mascot pose | TTS playback; caption shown live |
|
||
|
||
Aurora height and opacity follow `dictation.energy` (0–1) while listening (`voiceAuroraBlobs` in `apps/web/src/lib/voice-aurora.ts`). The caption (`VoiceLiveText`) fades each new word in; words already on screen stay put while an utterance grows and the caption restarts for a new one. Mascot uses `EvooMascot` poses: idle / thinking / wave.
|
||
|
||
### Controls
|
||
|
||
The control row holds the persona phase label (live region), Transcript, Interrupt (mic), and Hang up (×).
|
||
|
||
| Input | Action |
|
||
|-------|--------|
|
||
| **Space** | Interrupt (stop TTS or cancel dictation, resume listening) |
|
||
| **Esc** | Hang up — exit Voice Stage |
|
||
| **T** | Toggle transcript drawer (secondary record) |
|
||
|
||
### Transcript
|
||
|
||
Collapsible panel shows the last messages from the thread plus the live utterance. Chat scroll behind the stage is dimmed but unchanged; voice does not replace thread history in the database.
|
||
|
||
## Accessibility
|
||
|
||
- `prefers-reduced-motion: reduce` disables aurora breathing/drift, the panel entrance, and word fade-in; the static aurora remains.
|
||
- Dialog semantics: `role="dialog"`, `aria-modal`, labelled bot name.
|
||
- Hang up exposes `aria-label="Hang up"`.
|
||
- Shortcuts are exposed through `aria-keyshortcuts` on the matching controls instead of on-screen hint text.
|
||
|
||
## Architecture
|
||
|
||
```
|
||
Shell (composer Voice)
|
||
└─ CallView (orchestration: dictation + speaker + thread send)
|
||
└─ VoiceStage (panel, controls, transcript drawer)
|
||
├─ VoiceAurora (bottom-edge glow, phase + energy)
|
||
└─ VoiceLiveText (word-by-word caption)
|
||
```
|
||
|
||
Shared singletons: `dictation` (`apps/web/src/lib/dictation.ts`), `speaker` (`apps/web/src/lib/tts.ts`).
|
||
|
||
## Design tokens
|
||
|
||
Voice chrome uses `text-ui-*`, `size-icon-*`, and `h-control-*` / `size-control-lg` — no sub-13px labels on primary controls. See `DESIGN.md` for the full scale.
|
||
|
||
## Sound language
|
||
|
||
Procedural Web Audio (`oo-sound.ts`, `oo-vocals.ts`, `oo-audio-bus.ts`) — no sample packs. Full motion/SFX spec: `packages/ui-web/MOTION-EVOO-CHARACTER.md`.
|
||
|
||
| Cue | Trigger |
|
||
|-----|---------|
|
||
| `app-start` | Cold open — mint→violet shimmer |
|
||
| `interact` | Primary nav / roster taps |
|
||
| `success-resolve` | Run finished — mint→gold |
|
||
| `need-you` | `waiting_input` / takeover — with `ouh` vocal |
|
||
| `work-complete` | Alias of `success-resolve` |
|
||
| `listening-start` / `listening-stop` | Voice phase |
|
||
| `speaking-start` / `speaking-done` | TTS lifecycle |
|
||
| `attention` | Run error in composer |
|
||
| `message-sent` | Composer send |
|
||
| `handoff` / `group-spark` | Thread collaboration |
|
||
|
||
Reaction vocals (`mh` / `Mhh` nod, `ouh`, `hm` on success): state-change only, 8–12 s cooldown, **−18…−12 LUFS** via shared bus, **duck under TTS/PTT**.
|
||
|
||
Toggle: **Settings → Voice → Sound feedback**. `prefers-reduced-motion: reduce` → no vocals/SFX loops; character visual state only.
|
||
|
||
## Current slice (this PR)
|
||
|
||
**Stage + sound + onboarding reassurance** — shared encounter without requiring the user to sit in front of the screen. Voice Stage is opt-in per session; text mode uses the same Oo Stage metaphor.
|
||
|
||
## Ambient & video presence (future roadmap — do not ship without consent UX)
|
||
|
||
Ingo capability vision — **document only** until explicit opt-in flows exist. Not part of the current Gesamterfahrung slice.
|
||
|
||
| Idea | Product rule |
|
||
|------|--------------|
|
||
| **Video calls optional** | Only when the user wants; a simpler audio-only path stays first-class for users who prefer it |
|
||
| **"It's just an Oo"** | User does not need to sit in front constantly — practical help can be ambient (notifications, background runs, crew handoffs) |
|
||
| **Ambient presence** | An Oo can carry a lot of practical presence without demanding attention — aligned with quiet-doer mode |
|
||
| **Long-term metaphors (opt-in only)** | Could *eventually* stand in for an alarm system or a home camera — **strictly user-opt-in**, never default, never surveillance-as-a-feature |
|
||
|
||
### Hard gates (aligned with `docs/OO-RELATIONSHIP.md`)
|
||
|
||
- **No always-on camera or mic** without a dedicated, revocable consent UX — not buried in ToS
|
||
- **Never creepy** — warm competence, not uncanny stalking; earned context only
|
||
- **Not yes-men** — push back on unsafe or over-broad monitoring requests
|
||
- **No Wunderwuzzi** — home/security lanes are specialist Oos with narrow scope, not one omniscient watcher
|
||
|
||
When video or ambient sensing ships: same encounter grammar (Stage + sound + transcript), plus a visible **presence mode** indicator the user controls.
|
||
|
||
## Out of scope (later)
|
||
|
||
- Dedicated `/voice` route
|
||
- Hold-to-talk in composer
|
||
- WebGL / real audio analyser on TTS output
|
||
- Always-on video / ambient camera without explicit consent UX (see roadmap above)
|