5ae8f15931
* feat(web): Ingo UX — proactivity signals, Oo face richness, soft audio Priority S: Heute Brief buckets (Needs you ≤3 / Doing / Done), roster doing-verbs and Braucht-dich chips, sidebar badge on Heute, need-you ping on waiting states. Priority A/B: Web Audio cues (app-start shimmer, interact tick, success-resolve, need-you ouh), reaction vocals with cooldown, optional eOo cold-open voice mark. Oo eyes gain iris/specular detail and asymmetric blink; hands reduced to rest except punctuation poses; speaking mouth animates. Co-authored-by: cynarAI <cynarAI@users.noreply.github.com> * fix(web): biome format and unused import in HeuteBrief Co-authored-by: cynarAI <cynarAI@users.noreply.github.com> * fix(web): a11y and import order for Heute badge Co-authored-by: cynarAI <cynarAI@users.noreply.github.com> * docs+motion: Motion God craft — pose stagger, audio bus, ducking Add MOTION-EVOO-CHARACTER.md (Brand Soul brief): eyes lead, mouth ~80%, hands punctuation, 40–80 ms eye→mouth lag, 220–320 ms springs, SFX LUFS and duck-under-TTS/PTT, nod mh/Mhh, reduced-motion rules. Implement motion-tokens, staggered mouthPose in EvooMascot, shared oo-audio-bus with voice-lane ducking, mint→gold success, hm nod vocal. Co-authored-by: cynarAI <cynarAI@users.noreply.github.com> * fix(web): align conversation roster with bot-only sidebar groups after rebase Co-authored-by: cynarAI <cynarAI@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: cynarAI <cynarAI@users.noreply.github.com>
166 lines
8.6 KiB
Markdown
166 lines
8.6 KiB
Markdown
# EvOo Voice — Interaction Model
|
||
|
||
## Gesamterfahrung (encounter)
|
||
|
||
A conversation with an Oo is a **shared encounter**, not a thread with a tiny avatar:
|
||
|
||
| Layer | Role |
|
||
|-------|------|
|
||
| **Oo Stage** | Primary — large `BotAvatar` / persona presence, runtime pose, awareness strip |
|
||
| **Transcript** | Secondary protocol — compact caption trail, scrollable record |
|
||
| **Sound** | Calm procedural cues wired to real events (work finished, voice phases, handoffs) |
|
||
|
||
Text mode (`OoStage` in Shell) and Voice mode (`VoiceStage` overlay) share the same metaphor: eyes/presence + space + sound. Brand lock (asymmetric Kulleraugen) is unchanged.
|
||
|
||
### Product model — conversation-centric (Grok-like)
|
||
|
||
**Oos (agents) exist**, but the product is organized around **conversations (Unterhaltungen)**, not “pick an agent row = the whole app”:
|
||
|
||
| Layer | What it is |
|
||
|-------|------------|
|
||
| **Team** | Roster of your Oos — who you can talk to |
|
||
| **1:1 Unterhaltung** | One Oo, one shared encounter (`OoStage`) |
|
||
| **Gruppenunterhaltung** | Multiple Oos (+ humans later) in one encounter (`OoGroupStage`) |
|
||
| **Projekte** | Group conversations organized in **Ordner** (`BotSection` folders in sidebar) |
|
||
|
||
| Mode | Shell route | Stage |
|
||
|------|-------------|-------|
|
||
| **Unterhaltung** (1:1) | `/app/:botId` | `OoStage` — one Oo hero |
|
||
| **Gruppenunterhaltung** | `/app/g/:groupId?s=live` | `OoGroupStage` — active-speaker hero + ensemble row; **Riffing** when peers spark (`docs/GROUP-CONVERSATIONS.md`) |
|
||
| **Project sessions** | `/app/g/:groupId` (timeline) | No stage — session list (`ProjectSessionTimeline`) |
|
||
|
||
Clicking an Oo in **Team** opens a **1:1 conversation**, not “the agent product”. Clicking a project opens a **group conversation** or its session timeline. **Gesamterfahrung** (stage + sound + transcript record) applies to both conversation types.
|
||
|
||
**Emergent ensemble:** *Das Ganze ist mehr als die Summe seiner Teile* — power from distinct specialists + chemistry in groups, not one mega-agent (`docs/PRODUCT.md`).
|
||
|
||
Voice is the **immersive primary mode** for 1:1 bot conversations. Text chat uses the same stage-first layout; the transcript stays readable but no longer carries identity chrome.
|
||
|
||
## Entry
|
||
|
||
| Surface | Action |
|
||
|---------|--------|
|
||
| Composer **Voice** pill | Opens Voice Stage when a provider is connected; otherwise opens Voice settings |
|
||
| Transcript “Speak this reply” | Reads one message aloud (text-first affordance, unchanged) |
|
||
|
||
The composer Voice control uses `h-control-lg`, cyan bloom when ready, and amber styling when setup is needed.
|
||
|
||
## Voice Stage
|
||
|
||
Full-bleed immersive layer (`VoiceStage` + `VoiceCanvas`) replaces the modal call dialog.
|
||
|
||
### Persona & consciousness layer
|
||
|
||
Persona is **derived at runtime** from existing bot fields (`name`, `title`, `description`) — aligned with `templates/*` archetypes, not a separate marketing field.
|
||
|
||
| System | Path |
|
||
|--------|------|
|
||
| Archetype detection | `apps/web/src/lib/bot-persona.ts` |
|
||
| i18n status copy | `apps/web/src/lib/voice-persona-copy.ts` |
|
||
| Awareness strip | `VoiceAwarenessStrip` — subagent, handoff, delegation, peer reply |
|
||
| Visual identity | `BotAvatar` — persona shape + unified `EvooMascot` soul (eyes/mouth/hands) driven by voice phase |
|
||
| Voice delivery | `speaker.speak(..., { botId })` → server resolves per-bot `voiceId` |
|
||
|
||
Archetypes include: CTO executive (Steve Jobs template), chief operator (Fixer), builder (Dr. Eggbot), curator (Webby), work/life inboxes, home steward, quant, engineer, **specialist** (narrow fallback — no Wunderwuzzi). See `docs/PRODUCT.md` and `docs/PERSONA-SELECTION.md`.
|
||
|
||
**Companion tone:** geek-peer (fachsimpeln when wanted) or quiet-doer (execute without condescension) — inferred per user, not one voice for everyone.
|
||
|
||
**Accessibility of ambition:** grandma-simple entry (Thermomix to hard math — no power-user gate); expert-grade ceiling per specialty. Same product, different crews; never a mediocre everything-bot.
|
||
|
||
Phase labels, idle prompts, and thinking lines are **character-specific** (DE/EN via Lingui). Collaboration narration uses persona-aware phrasing (“casting”, “routing”, “handing off”) instead of generic tool labels.
|
||
|
||
### States
|
||
|
||
| Phase | Visual | Audio |
|
||
|-------|--------|-------|
|
||
| **Listening** | Cyan dual-mind bloom, mic-reactive energy bars | Dictation active; silence endpoint sends |
|
||
| **Thinking** | Violet orbital ring, calm bars | TTS idle; awaiting bot reply |
|
||
| **Speaking** | Emerald bloom, conductor mascot pose | TTS playback; caption shown live |
|
||
|
||
Presence visuals sync to `dictation.energy` (0–1) while listening. Mascot uses `EvooMascot` poses: idle / thinking / wave.
|
||
|
||
### Controls
|
||
|
||
| Input | Action |
|
||
|-------|--------|
|
||
| **Space** | Interrupt (stop TTS or cancel dictation, resume listening) |
|
||
| **Esc** | Hang up — exit Voice Stage |
|
||
| **T** | Toggle transcript drawer (secondary record) |
|
||
|
||
### Transcript
|
||
|
||
Collapsible panel shows the last messages from the thread plus the live utterance. Chat scroll behind the stage is dimmed but unchanged; voice does not replace thread history in the database.
|
||
|
||
## Accessibility
|
||
|
||
- `prefers-reduced-motion: reduce` disables bloom drift, ring spin, and composer pulse; static gradients remain.
|
||
- Dialog semantics: `role="dialog"`, `aria-modal`, labelled bot name.
|
||
- Hang up exposes `aria-label="Hang up"`.
|
||
- Keyboard exit paths documented in-stage.
|
||
|
||
## Architecture
|
||
|
||
```
|
||
Shell (composer Voice)
|
||
└─ CallView (orchestration: dictation + speaker + thread send)
|
||
└─ VoiceStage (layout, controls, transcript drawer)
|
||
└─ VoiceCanvas (ambient bloom field)
|
||
```
|
||
|
||
Shared singletons: `dictation` (`apps/web/src/lib/dictation.ts`), `speaker` (`apps/web/src/lib/tts.ts`).
|
||
|
||
## Design tokens
|
||
|
||
Voice chrome uses `text-ui-*`, `size-icon-*`, and `h-control-*` / `size-control-lg` — no sub-13px labels on primary controls. See `DESIGN.md` for the full scale.
|
||
|
||
## Sound language
|
||
|
||
Procedural Web Audio (`oo-sound.ts`, `oo-vocals.ts`, `oo-audio-bus.ts`) — no sample packs. Full motion/SFX spec: `packages/ui-web/MOTION-EVOO-CHARACTER.md`.
|
||
|
||
| Cue | Trigger |
|
||
|-----|---------|
|
||
| `app-start` | Cold open — mint→violet shimmer |
|
||
| `interact` | Primary nav / roster taps |
|
||
| `success-resolve` | Run finished — mint→gold |
|
||
| `need-you` | `waiting_input` / takeover — with `ouh` vocal |
|
||
| `work-complete` | Alias of `success-resolve` |
|
||
| `listening-start` / `listening-stop` | Voice phase |
|
||
| `speaking-start` / `speaking-done` | TTS lifecycle |
|
||
| `attention` | Run error in composer |
|
||
| `message-sent` | Composer send |
|
||
| `handoff` / `group-spark` | Thread collaboration |
|
||
|
||
Reaction vocals (`mh` / `Mhh` nod, `ouh`, `hm` on success): state-change only, 8–12 s cooldown, **−18…−12 LUFS** via shared bus, **duck under TTS/PTT**.
|
||
|
||
Toggle: **Settings → Voice → Sound feedback**. `prefers-reduced-motion: reduce` → no vocals/SFX loops; character visual state only.
|
||
|
||
## Current slice (this PR)
|
||
|
||
**Stage + sound + onboarding reassurance** — shared encounter without requiring the user to sit in front of the screen. Voice Stage is opt-in per session; text mode uses the same Oo Stage metaphor.
|
||
|
||
## Ambient & video presence (future roadmap — do not ship without consent UX)
|
||
|
||
Ingo capability vision — **document only** until explicit opt-in flows exist. Not part of the current Gesamterfahrung slice.
|
||
|
||
| Idea | Product rule |
|
||
|------|--------------|
|
||
| **Video calls optional** | Only when the user wants; a simpler audio-only path stays first-class for users who prefer it |
|
||
| **"It's just an Oo"** | User does not need to sit in front constantly — practical help can be ambient (notifications, background runs, crew handoffs) |
|
||
| **Ambient presence** | An Oo can carry a lot of practical presence without demanding attention — aligned with quiet-doer mode |
|
||
| **Long-term metaphors (opt-in only)** | Could *eventually* stand in for an alarm system or a home camera — **strictly user-opt-in**, never default, never surveillance-as-a-feature |
|
||
|
||
### Hard gates (aligned with `docs/OO-RELATIONSHIP.md`)
|
||
|
||
- **No always-on camera or mic** without a dedicated, revocable consent UX — not buried in ToS
|
||
- **Never creepy** — warm competence, not uncanny stalking; earned context only
|
||
- **Not yes-men** — push back on unsafe or over-broad monitoring requests
|
||
- **No Wunderwuzzi** — home/security lanes are specialist Oos with narrow scope, not one omniscient watcher
|
||
|
||
When video or ambient sensing ships: same encounter grammar (Stage + sound + transcript), plus a visible **presence mode** indicator the user controls.
|
||
|
||
## Out of scope (later)
|
||
|
||
- Dedicated `/voice` route
|
||
- Hold-to-talk in composer
|
||
- WebGL / real audio analyser on TTS output
|
||
- Always-on video / ambient camera without explicit consent UX (see roadmap above)
|