b81f7d18f6
The repository still carried the pre-evOo prototype: a Python orchestrator, the old HTML "evoo-platform" cockpit (whose test ran in the root unit suite), an apex-trading package, the hot-reload deploy.sh for it, a permissions manifest, an architecture note with local file links, agent handoff notes, a Steve Jobs persona template and about ninety one-off scripts that probe a developer database with hard-coded local URLs. None of it is used by the product, CI or the real deploy in ops/evo-deploy. Delete those files, drop the evoo-platform test glob from the root vitest config, the Cockpit section from .env.example and the dead biome and gitignore entries, and update the docs that listed the removed tools. scripts/start-tmux.sh no longer starts the removed sentinel and resolves the checkout from its own location instead of a machine-specific path; scripts/seed-ollama.ts requires DATABASE_URL instead of falling back to a local URL with a username. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
169 lines
9.4 KiB
Markdown
169 lines
9.4 KiB
Markdown
# EvOo Voice — Interaction Model
|
||
|
||
## Gesamterfahrung (encounter)
|
||
|
||
A conversation with an Oo is a **shared encounter**, not a thread with a tiny avatar:
|
||
|
||
| Layer | Role |
|
||
|-------|------|
|
||
| **Oo Stage** | Primary — large `BotAvatar` / persona presence, runtime pose, awareness strip |
|
||
| **Transcript** | Secondary protocol — compact caption trail, scrollable record |
|
||
| **Sound** | Calm procedural cues wired to real events (work finished, voice phases, handoffs) |
|
||
|
||
Text mode (`OoStage` in Shell) and Voice mode (`VoiceStage` overlay) share the same metaphor: eyes/presence + space + sound. Brand lock (asymmetric Kulleraugen) is unchanged.
|
||
|
||
### Product model — conversation-centric (Grok-like)
|
||
|
||
**Oos (agents) exist**, but the product is organized around **conversations (Unterhaltungen)**, not “pick an agent row = the whole app”:
|
||
|
||
| Layer | What it is |
|
||
|-------|------------|
|
||
| **Team** | Roster of your Oos — who you can talk to |
|
||
| **1:1 Unterhaltung** | One Oo, one shared encounter (`OoStage`) |
|
||
| **Gruppenunterhaltung** | Multiple Oos (+ humans later) in one encounter (`OoGroupStage`) |
|
||
| **Projekte** | Group conversations organized in **Ordner** (`BotSection` folders in sidebar) |
|
||
|
||
| Mode | Shell route | Stage |
|
||
|------|-------------|-------|
|
||
| **Unterhaltung** (1:1) | `/app/:botId` | `OoStage` — one Oo hero |
|
||
| **Gruppenunterhaltung** | `/app/g/:groupId?s=live` | `OoGroupStage` — active-speaker hero + ensemble row; **Riffing** when peers spark (`docs/GROUP-CONVERSATIONS.md`) |
|
||
| **Project sessions** | `/app/g/:groupId` (timeline) | No stage — session list (`ProjectSessionTimeline`) |
|
||
|
||
Clicking an Oo in **Team** opens a **1:1 conversation**, not “the agent product”. Clicking a project opens a **group conversation** or its session timeline. **Gesamterfahrung** (stage + sound + transcript record) applies to both conversation types.
|
||
|
||
**Emergent ensemble:** *Das Ganze ist mehr als die Summe seiner Teile* — power from distinct specialists + chemistry in groups, not one mega-agent (`docs/PRODUCT.md`).
|
||
|
||
Voice is the **immersive primary mode** for 1:1 bot conversations. Text chat uses the same stage-first layout; the transcript stays readable but no longer carries identity chrome.
|
||
|
||
## Entry
|
||
|
||
| Surface | Action |
|
||
|---------|--------|
|
||
| Composer **Voice** pill | Opens Voice Stage when a provider is connected; otherwise opens Voice settings |
|
||
| Transcript “Speak this reply” | Reads one message aloud (text-first affordance, unchanged) |
|
||
|
||
The composer and stage Voice controls use `h-control-lg` in neutral tokens; when voice is ready an iridescent aurora rim (`VoiceAuroraRim`) runs along the bottom edge.
|
||
|
||
## Voice Stage
|
||
|
||
Full-screen dark layer with one glass panel (`VoiceStage`): the Oo, the live caption, and a control row. An iridescent aurora (`VoiceAurora`) rises from the panel's bottom edge. Its spectrum is the bot palette (`botColors` from `@rakazo/ui-tokens`) rotated so the active Oo's own color sits in the center.
|
||
|
||
### Persona & consciousness layer
|
||
|
||
Persona is **derived at runtime** from existing bot fields (`name`, `title`, `description`) — aligned with `templates/*` archetypes, not a separate marketing field.
|
||
|
||
| System | Path |
|
||
|--------|------|
|
||
| Archetype detection | `apps/web/src/lib/bot-persona.ts` |
|
||
| i18n status copy | `apps/web/src/lib/voice-persona-copy.ts` |
|
||
| Awareness strip | `VoiceAwarenessStrip` — subagent, handoff, delegation, peer reply |
|
||
| Visual identity | `BotAvatar` — persona shape + unified `EvooMascot` soul (eyes/mouth/hands) driven by voice phase |
|
||
| Voice delivery | `speaker.speak(..., { botId })` → server resolves per-bot `voiceId` |
|
||
|
||
Archetypes include: CTO executive, chief operator (Fixer), builder (Dr. Eggbot), curator (Webby), work/life inboxes, home steward, quant, engineer, **specialist** (narrow fallback — no Wunderwuzzi). See `docs/PRODUCT.md` and `docs/PERSONA-SELECTION.md`.
|
||
|
||
**Companion tone:** geek-peer (fachsimpeln when wanted) or quiet-doer (execute without condescension) — inferred per user, not one voice for everyone.
|
||
|
||
**Accessibility of ambition:** grandma-simple entry (Thermomix to hard math — no power-user gate); expert-grade ceiling per specialty. Same product, different crews; never a mediocre everything-bot.
|
||
|
||
Phase labels, idle prompts, and thinking lines are **character-specific** (DE/EN via Lingui). Collaboration narration uses persona-aware phrasing (“casting”, “routing”, “handing off”) instead of generic tool labels.
|
||
|
||
### States
|
||
|
||
| Phase | Visual | Audio |
|
||
|-------|--------|-------|
|
||
| **Listening** | Aurora rises and brightens with mic energy, breathing | Dictation active; silence endpoint sends |
|
||
| **Thinking** | Aurora settles low and drifts side to side | TTS idle; awaiting bot reply |
|
||
| **Speaking** | Full aurora bloom, conductor mascot pose | TTS playback; caption shown live |
|
||
|
||
Aurora height and opacity follow `dictation.energy` (0–1) while listening (`voiceAuroraBlobs` in `apps/web/src/lib/voice-aurora.ts`). The caption (`VoiceLiveText`) fades each new word in; words already on screen stay put while an utterance grows and the caption restarts for a new one. Mascot uses `EvooMascot` poses: idle / thinking / wave.
|
||
|
||
### Controls
|
||
|
||
The control row holds the persona phase label (live region), Transcript, Interrupt (mic), and Hang up (×).
|
||
|
||
| Input | Action |
|
||
|-------|--------|
|
||
| **Space** | Interrupt (stop TTS or cancel dictation, resume listening) |
|
||
| **Esc** | Hang up — exit Voice Stage |
|
||
| **T** | Toggle transcript drawer (secondary record) |
|
||
|
||
### Transcript
|
||
|
||
Collapsible panel shows the last messages from the thread plus the live utterance. Chat scroll behind the stage is dimmed but unchanged; voice does not replace thread history in the database.
|
||
|
||
## Accessibility
|
||
|
||
- `prefers-reduced-motion: reduce` disables aurora breathing/drift, the panel entrance, and word fade-in; the static aurora remains.
|
||
- Dialog semantics: `role="dialog"`, `aria-modal`, labelled bot name.
|
||
- Hang up exposes `aria-label="Hang up"`.
|
||
- Shortcuts are exposed through `aria-keyshortcuts` on the matching controls instead of on-screen hint text.
|
||
|
||
## Architecture
|
||
|
||
```
|
||
Shell (composer Voice)
|
||
└─ CallView (orchestration: dictation + speaker + thread send)
|
||
└─ VoiceStage (panel, controls, transcript drawer)
|
||
├─ VoiceAurora (bottom-edge glow, phase + energy)
|
||
└─ VoiceLiveText (word-by-word caption)
|
||
```
|
||
|
||
Shared singletons: `dictation` (`apps/web/src/lib/dictation.ts`), `speaker` (`apps/web/src/lib/tts.ts`).
|
||
|
||
## Design tokens
|
||
|
||
Voice chrome uses `text-ui-*`, `size-icon-*`, and `h-control-*` / `size-control-lg` — no sub-13px labels on primary controls. See `DESIGN.md` for the full scale.
|
||
|
||
## Sound language
|
||
|
||
Procedural Web Audio (`oo-sound.ts`, `oo-vocals.ts`, `oo-audio-bus.ts`) — no sample packs. Full motion/SFX spec: `packages/ui-web/MOTION-EVOO-CHARACTER.md`.
|
||
|
||
| Cue | Trigger |
|
||
|-----|---------|
|
||
| `app-start` | Cold open — mint→violet shimmer |
|
||
| `interact` | Primary nav / roster taps |
|
||
| `success-resolve` | Run finished — mint→gold |
|
||
| `need-you` | `waiting_input` / takeover — with `ouh` vocal |
|
||
| `work-complete` | Alias of `success-resolve` |
|
||
| `listening-start` / `listening-stop` | Voice phase |
|
||
| `speaking-start` / `speaking-done` | TTS lifecycle |
|
||
| `attention` | Run error in composer |
|
||
| `message-sent` | Composer send |
|
||
| `handoff` / `group-spark` | Thread collaboration |
|
||
|
||
Reaction vocals (`mh` / `Mhh` nod, `ouh`, `hm` on success): state-change only, 8–12 s cooldown, **−18…−12 LUFS** via shared bus, **duck under TTS/PTT**.
|
||
|
||
Toggle: **Settings → Voice → Sound feedback**. `prefers-reduced-motion: reduce` → no vocals/SFX loops; character visual state only.
|
||
|
||
## Current slice (this PR)
|
||
|
||
**Stage + sound + onboarding reassurance** — shared encounter without requiring the user to sit in front of the screen. Voice Stage is opt-in per session; text mode uses the same Oo Stage metaphor.
|
||
|
||
## Ambient & video presence (future roadmap — do not ship without consent UX)
|
||
|
||
Ingo capability vision — **document only** until explicit opt-in flows exist. Not part of the current Gesamterfahrung slice.
|
||
|
||
| Idea | Product rule |
|
||
|------|--------------|
|
||
| **Video calls optional** | Only when the user wants; a simpler audio-only path stays first-class for users who prefer it |
|
||
| **"It's just an Oo"** | User does not need to sit in front constantly — practical help can be ambient (notifications, background runs, crew handoffs) |
|
||
| **Ambient presence** | An Oo can carry a lot of practical presence without demanding attention — aligned with quiet-doer mode |
|
||
| **Long-term metaphors (opt-in only)** | Could *eventually* stand in for an alarm system or a home camera — **strictly user-opt-in**, never default, never surveillance-as-a-feature |
|
||
|
||
### Hard gates (aligned with `docs/OO-RELATIONSHIP.md`)
|
||
|
||
- **No always-on camera or mic** without a dedicated, revocable consent UX — not buried in ToS
|
||
- **Never creepy** — warm competence, not uncanny stalking; earned context only
|
||
- **Not yes-men** — push back on unsafe or over-broad monitoring requests
|
||
- **No Wunderwuzzi** — home/security lanes are specialist Oos with narrow scope, not one omniscient watcher
|
||
|
||
When video or ambient sensing ships: same encounter grammar (Stage + sound + transcript), plus a visible **presence mode** indicator the user controls.
|
||
|
||
## Out of scope (later)
|
||
|
||
- Dedicated `/voice` route
|
||
- Hold-to-talk in composer
|
||
- WebGL / real audio analyser on TTS output
|
||
- Always-on video / ambient camera without explicit consent UX (see roadmap above)
|