evOo-local/VOICE.md

EvOo Voice — Interaction Model

Gesamterfahrung (encounter)

A conversation with an Oo is a shared encounter, not a thread with a tiny avatar:

Layer	Role
Oo Stage	Primary — large BotAvatar / persona presence, runtime pose, awareness strip
Transcript	Secondary protocol — compact caption trail, scrollable record
Sound	Calm procedural cues wired to real events (work finished, voice phases, handoffs)

Text mode (OoStage in Shell) and Voice mode (VoiceStage overlay) share the same metaphor: eyes/presence + space + sound. Brand lock (asymmetric Kulleraugen) is unchanged.

Product model — conversation-centric (Grok-like)

Oos (agents) exist, but the product is organized around conversations (Unterhaltungen), not “pick an agent row = the whole app”:

Layer	What it is
Team	Roster of your Oos — who you can talk to
1:1 Unterhaltung	One Oo, one shared encounter (OoStage)
Gruppenunterhaltung	Multiple Oos (+ humans later) in one encounter (OoGroupStage)
Projekte	Group conversations organized in Ordner (BotSection folders in sidebar)

Mode	Shell route	Stage
Unterhaltung (1:1)	/app/:botId	OoStage — one Oo hero
Gruppenunterhaltung	/app/g/:groupId?s=live	OoGroupStage — active-speaker hero + ensemble row; Riffing when peers spark (docs/GROUP-CONVERSATIONS.md)
Project sessions	/app/g/:groupId (timeline)	No stage — session list (ProjectSessionTimeline)

Clicking an Oo in Team opens a 1:1 conversation, not “the agent product”. Clicking a project opens a group conversation or its session timeline. Gesamterfahrung (stage + sound + transcript record) applies to both conversation types.

Emergent ensemble: Das Ganze ist mehr als die Summe seiner Teile — power from distinct specialists + chemistry in groups, not one mega-agent (docs/PRODUCT.md).

Voice is the immersive primary mode for 1:1 bot conversations. Text chat uses the same stage-first layout; the transcript stays readable but no longer carries identity chrome.

Entry

Surface	Action
Composer Voice pill	Opens Voice Stage when a provider is connected; otherwise opens Voice settings
Transcript “Speak this reply”	Reads one message aloud (text-first affordance, unchanged)

The composer and stage Voice controls use h-control-lg in neutral tokens; when voice is ready an iridescent aurora rim (VoiceAuroraRim) runs along the bottom edge.

Voice Stage

Full-screen dark layer with one glass panel (VoiceStage): the Oo, the live caption, and a control row. An iridescent aurora (VoiceAurora) rises from the panel's bottom edge. Its spectrum is the bot palette (botColors from @rakazo/ui-tokens) rotated so the active Oo's own color sits in the center.

Persona & consciousness layer

Persona is derived at runtime from existing bot fields (name, title, description) — aligned with templates/* archetypes, not a separate marketing field.

System	Path
Archetype detection	apps/web/src/lib/bot-persona.ts
i18n status copy	apps/web/src/lib/voice-persona-copy.ts
Awareness strip	VoiceAwarenessStrip — subagent, handoff, delegation, peer reply
Visual identity	BotAvatar — persona shape + unified EvooMascot soul (eyes/mouth/hands) driven by voice phase
Voice delivery	speaker.speak(..., { botId }) → server resolves per-bot voiceId

Archetypes include: CTO executive (Steve Jobs template), chief operator (Fixer), builder (Dr. Eggbot), curator (Webby), work/life inboxes, home steward, quant, engineer, specialist (narrow fallback — no Wunderwuzzi). See docs/PRODUCT.md and docs/PERSONA-SELECTION.md.

Companion tone: geek-peer (fachsimpeln when wanted) or quiet-doer (execute without condescension) — inferred per user, not one voice for everyone.

Accessibility of ambition: grandma-simple entry (Thermomix to hard math — no power-user gate); expert-grade ceiling per specialty. Same product, different crews; never a mediocre everything-bot.

Phase labels, idle prompts, and thinking lines are character-specific (DE/EN via Lingui). Collaboration narration uses persona-aware phrasing (“casting”, “routing”, “handing off”) instead of generic tool labels.

States

Phase	Visual	Audio
Listening	Aurora rises and brightens with mic energy, breathing	Dictation active; silence endpoint sends
Thinking	Aurora settles low and drifts side to side	TTS idle; awaiting bot reply
Speaking	Full aurora bloom, conductor mascot pose	TTS playback; caption shown live

Aurora height and opacity follow dictation.energy (0–1) while listening (voiceAuroraBlobs in apps/web/src/lib/voice-aurora.ts). The caption (VoiceLiveText) fades each new word in; words already on screen stay put while an utterance grows and the caption restarts for a new one. Mascot uses EvooMascot poses: idle / thinking / wave.

Controls

The control row holds the persona phase label (live region), Transcript, Interrupt (mic), and Hang up (×).

Input	Action
Space	Interrupt (stop TTS or cancel dictation, resume listening)
Esc	Hang up — exit Voice Stage
T	Toggle transcript drawer (secondary record)

Transcript

Collapsible panel shows the last messages from the thread plus the live utterance. Chat scroll behind the stage is dimmed but unchanged; voice does not replace thread history in the database.

Accessibility

• prefers-reduced-motion: reduce disables aurora breathing/drift, the panel entrance, and word fade-in; the static aurora remains.
• Dialog semantics: role="dialog", aria-modal, labelled bot name.
• Hang up exposes aria-label="Hang up".
• Shortcuts are exposed through aria-keyshortcuts on the matching controls instead of on-screen hint text.

Architecture

Shell (composer Voice)
  └─ CallView (orchestration: dictation + speaker + thread send)
       └─ VoiceStage (panel, controls, transcript drawer)
            ├─ VoiceAurora (bottom-edge glow, phase + energy)
            └─ VoiceLiveText (word-by-word caption)

Shared singletons: dictation (apps/web/src/lib/dictation.ts), speaker (apps/web/src/lib/tts.ts).

Design tokens

Voice chrome uses text-ui-*, size-icon-*, and h-control-* / size-control-lg — no sub-13px labels on primary controls. See DESIGN.md for the full scale.

Sound language

Procedural Web Audio (oo-sound.ts, oo-vocals.ts, oo-audio-bus.ts) — no sample packs. Full motion/SFX spec: packages/ui-web/MOTION-EVOO-CHARACTER.md.

Cue	Trigger
app-start	Cold open — mint→violet shimmer
interact	Primary nav / roster taps
success-resolve	Run finished — mint→gold
need-you	waiting_input / takeover — with ouh vocal
work-complete	Alias of success-resolve
listening-start / listening-stop	Voice phase
speaking-start / speaking-done	TTS lifecycle
attention	Run error in composer
message-sent	Composer send
handoff / group-spark	Thread collaboration

Reaction vocals (mh / Mhh nod, ouh, hm on success): state-change only, 8–12 s cooldown, −18…−12 LUFS via shared bus, duck under TTS/PTT.

Toggle: Settings → Voice → Sound feedback. prefers-reduced-motion: reduce → no vocals/SFX loops; character visual state only.

Current slice (this PR)

Stage + sound + onboarding reassurance — shared encounter without requiring the user to sit in front of the screen. Voice Stage is opt-in per session; text mode uses the same Oo Stage metaphor.

Ambient & video presence (future roadmap — do not ship without consent UX)

Ingo capability vision — document only until explicit opt-in flows exist. Not part of the current Gesamterfahrung slice.

Idea	Product rule
Video calls optional	Only when the user wants; a simpler audio-only path stays first-class for users who prefer it
"It's just an Oo"	User does not need to sit in front constantly — practical help can be ambient (notifications, background runs, crew handoffs)
Ambient presence	An Oo can carry a lot of practical presence without demanding attention — aligned with quiet-doer mode
Long-term metaphors (opt-in only)	Could eventually stand in for an alarm system or a home camera — strictly user-opt-in, never default, never surveillance-as-a-feature

Hard gates (aligned with docs/OO-RELATIONSHIP.md)

• No always-on camera or mic without a dedicated, revocable consent UX — not buried in ToS
• Never creepy — warm competence, not uncanny stalking; earned context only
• Not yes-men — push back on unsafe or over-broad monitoring requests
• No Wunderwuzzi — home/security lanes are specialist Oos with narrow scope, not one omniscient watcher

When video or ambient sensing ships: same encounter grammar (Stage + sound + transcript), plus a visible presence mode indicator the user controls.

Out of scope (later)

• Dedicated /voice route
• Hold-to-talk in composer
• WebGL / real audio analyser on TTS output
• Always-on video / ambient camera without explicit consent UX (see roadmap above)