0b3ce11df8
* media: check ffmpeg filter support before subtitle operations * sandbox: handle synchronous spawn errors safely in desktop sandbox * Voice-first streaming, non-blocking follow-ups, quiet routines and chat UX Why: questions blocked the chat, follow-up messages had no acknowledgement, message actions sat beside the bubble, and voice turn-taking added latency. What: waiting runs accept steering messages and the bot marks each claimed follow-up as seen; unclaimed follow-ups show as queued. Reaction, copy and reply controls sit under each message. Streaming voice (VAD endpointing, barge-in that stops the run, clause-based TTS), an optimistic input queue, procedural earcons and delta-based quiet routines were added. Onboarding can be skipped straight to the main conversation. Dual-surface layout and project pickers are included. Status colors use semantic tokens. Tested: unit suites for core, adapters, api and web pass, typecheck and Biome are clean. Browser end-to-end tests were not run locally. * Update translation catalogs Re-extracted catalogs for the new and moved strings; the catalog sync test passes. * Add project plan and test readiness notes Notes from the milestone work: feature inventory, test infrastructure and test readiness. No code changes. * fix(ci): align scale guards, space deletion, computer toggle, and mobile expo deps * fix(ci): restore space deletion guard, browser toggle, and run-steps contracts * fix(ci): restore model resolution errors, group stage, and onboarding flow * fix(ci): restore in-app-browser close button, disable stop button while sending, and sync lingui catalogs * fix(ci): address web e2e failures for bot-crud, choice cards, message quote, and voice onboarding * test(web): wait for avatar and rig visibility in oo-avatar motion spec * test(web): wait for persisted message IDs in message-hover-actions pagination spec * feat: recruit library agents into project conversations and scan project folders * test(web): expect repo description in GithubRepoPicker onChange * Onboarding: always drop the local voice when the voice step is skipped Skipping waited only for the latest voice load and disconnected only when the client saw the connect succeed. An earlier or client-failed connect could still persist the local credential, so a user who skipped voice was left with one. Wait for every load and always disconnect. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy1md9QE1ayJbDCwJ4vAQs * Shell: hide the optimistic copy once the transcript shows the send A message sent while the bot works streamed into the transcript before its send request resolved, so the same text showed twice until the optimistic bubble was removed. Drop an optimistic message as soon as a persisted user message with the same text appears. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wy1md9QE1ayJbDCwJ4vAQs --------- Co-authored-by: Claude <noreply@anthropic.com>
176 lines
9.1 KiB
Markdown
176 lines
9.1 KiB
Markdown
# evOo — Product rules (Gesamterfahrung)
|
|
|
|
## Design mantra (apply ruthlessly)
|
|
|
|
**EINFACHHEIT und Durchgängigkeit. Dennoch super flexibel. Weniger ist Mehr.**
|
|
|
|
| Rule | Product |
|
|
|------|---------|
|
|
| **One model** | Unterhaltungen (1:1 / Gruppe), Ordner/Projekte, specialist Oos |
|
|
| **One language** | Same stage, sound, and transcript grammar everywhere — no one-off chrome |
|
|
| **Hide power** | Depth in specialists and settings — not every control on first screen |
|
|
| **Cut clutter** | Fewer, stronger affordances; stub/TODO lists stay in docs, not UI |
|
|
|
|
When a slice gets feature-heavy: prune to the simplest coherent path that still delivers Stage + sound + calm onboarding.
|
|
|
|
## Conversations, not agent rows
|
|
|
|
The product is organized as **Unterhaltungen** (see `VOICE.md`). **Team** lists your Oos; opening one starts a **conversation**, not “the agent app”.
|
|
|
|
| Type | Who |
|
|
|------|-----|
|
|
| 1:1 Unterhaltung | One Oo |
|
|
| Gruppenunterhaltung | Multiple Oos (+ humans later) — **fun factor essential** (see `docs/GROUP-CONVERSATIONS.md`) |
|
|
| Projekte | One group conversation per project, led by its **Projektkoordinator** |
|
|
|
|
### Projects recruit from the library
|
|
|
|
- Oos are created once, in the home space: that is the **library**. Each has one job.
|
|
- A project is a space with a fixed coordinator (`Bot.kind="main"`) and one main group conversation. A linked folder's README or manifest, or the GitHub repo's description, tells the coordinator what the project is (`PROJECT_GOAL.md`) when the user gave no goal.
|
|
- The coordinator recruits the Oos a request needs with `delegate_task`, briefing them on the task card. A recruited Oo gets a project member (`spawnKey` `recruit:<libraryBotId>`, `packages/db/src/project-recruits.ts`) that joins the conversation. The member keeps the Oo's persona and picks up later edits to it. It works with the project's files, memory, computer and connections. The Oo's home memory stays home.
|
|
|
|
## Deepest personalization
|
|
|
|
evOo must adapt to each user **more individually than any other app**. Persona/crew fit is **lifestyle-agnostic** — office worker, club/scene person, hardcore nerd: same mechanism, simplest path to what *they* want.
|
|
|
|
| Companion tone | When |
|
|
|----------------|------|
|
|
| **Geek-peer / Fachsimpeln** | User wants depth — match enthusiasm, stay in lane |
|
|
| **Quiet-doer** | User wants outcomes — never act like we know better |
|
|
|
|
Onboarding harvest infers **help & vibe** (`user-help-vibe.ts` → `MEMORY.md`), then spawn/select specialist Oos and Unterhaltungen — **not one personality for everyone**. See `docs/PERSONA-SELECTION.md`.
|
|
|
|
## Accessibility of ambition
|
|
|
|
Power-user literacy must **not** be required. Same product, radically different crews/Unterhaltungen — **domain excellence** in each case.
|
|
|
|
| User | Gets |
|
|
|------|------|
|
|
| Mom, Thermomix | That — simply, excellently |
|
|
| Hard math | Specialist Oo at **very high** level |
|
|
| Software development | Specialist Oo at **very high** level |
|
|
|
|
| Axis | Rule |
|
|
|------|------|
|
|
| **Skill floor** | Grandma-simple entry — no jargon gate, no setup maze |
|
|
| **Skill ceiling** | Expert-grade **per specialty** — depth when wanted |
|
|
| **Anti-pattern** | Wunderwuzzi mediocre at everything |
|
|
|
|
Simple entry, deep ceiling. UI and onboarding copy stay calm and plain; specialist lanes carry the depth.
|
|
|
|
## Emergent ensemble
|
|
|
|
**Das Ganze ist mehr als die Summe seiner Teile.**
|
|
|
|
Power comes from **more individualists and differences** — a crew of distinct specialists with chemistry — not from one mega-agent. Capability **emerges** when diverse Oos meet in Unterhaltungen and Gruppen.
|
|
|
|
| Pillar | Why |
|
|
|--------|-----|
|
|
| **No Wunderwuzzi** | Depth per lane beats one mediocre generalist |
|
|
| **Specialty ceiling** | Each Oo expert-grade in its domain |
|
|
| **Fun friction** | Banter and pushback between personalities — useful spark |
|
|
| **Ensemble > solo** | Group stage + handoffs make the *crew* feel stronger than any single Oo |
|
|
|
|
Chief coordinates; **the crew delivers**. Product, prompts, and stage should never imply one bot could replace the group.
|
|
|
|
## No Wunderwuzzi
|
|
|
|
Every Oo has a **clear specialty**. There is no do-everything generalist personality.
|
|
|
|
- Persona archetypes map to templates (CTO, Fixer, quant, engineer, …).
|
|
- Unmatched bots fall back to **`specialist`**, not “companion/helper”.
|
|
- **Chief** (first onboarding bot) is seeded as **Chief of Staff** — coordinates the crew, does not replace specialists.
|
|
- Group encounters show **active speaker + ensemble**; collaboration is the model — **banter & chemistry**, not corporate-dull committee.
|
|
|
|
When adding templates or copy: narrow scope, explicit role line, defer out-of-lane work to other Oos.
|
|
|
|
## First-run emotional north star
|
|
|
|
The app **sets itself up**. It discovers what it can (profile, tools, crew) and shows the user:
|
|
|
|
> *Don't stress. We're here now. Trust us — there are many of us. We've got this.*
|
|
|
|
| Principle | Product |
|
|
|-----------|---------|
|
|
| **Calm collective crew** | Reassurance copy + discovery progress (`OnboardingDiscoveryShell`), not a long wizard |
|
|
| **Honest discovery** | Progress bar + one active step in onboarding — future sources stay in docs, not UI laundry lists |
|
|
| **Specialists** | Chief of Staff + crew framing; no Wunderwuzzi |
|
|
|
|
Surfaces: `Welcome.tsx` (spirit), `Onboarding.tsx` (progress + minimal model/tools steps).
|
|
|
|
## Onboarding → user profile
|
|
|
|
During onboarding, harvest **respectful local signals** into the existing **user-scoped `MEMORY.md`** (space memory — not a parallel brain):
|
|
|
|
| Signal | Source |
|
|
|--------|--------|
|
|
| Name, email | `rpc.me()` |
|
|
| Timezone | `Intl` / `localTimezone()` |
|
|
| Locale, UI locale | `navigator.language`, saved UI pref |
|
|
| Languages, platform | `navigator` |
|
|
| Default model, computer host | bootstrap `Me` |
|
|
|
|
Implementation:
|
|
|
|
| Layer | Path |
|
|
|-------|------|
|
|
| Profile harvest | `apps/web/src/lib/user-profile-harvest.ts` |
|
|
| Help & vibe inference | `apps/web/src/lib/user-help-vibe.ts` |
|
|
| Persona/crew selection notes | `docs/PERSONA-SELECTION.md` |
|
|
| Discovery plan + progress | `apps/web/src/lib/onboarding-discovery.ts` |
|
|
| Reassurance UI | `apps/web/src/components/onboarding/OnboardingDiscoveryShell.tsx` |
|
|
|
|
Called from `Onboarding.tsx` after first bot ensure.
|
|
|
|
## Oo relationship over time
|
|
|
|
| Principle | Meaning |
|
|
|-----------|---------|
|
|
| **Always on** | Presence when needed — 24/7 feeling without theater |
|
|
| **Compounding** | Stronger/more attuned the more the user works with their Oos (memory + threads) |
|
|
| **Never creepy** | Warm competence; no surveillance vibes |
|
|
| **Not yes-men** | Respectful pushback — don't just echo |
|
|
|
|
Seeded in user `MEMORY.md` (`user-relationship-contract.ts`). Persona copy and prompts follow. Details: `docs/OO-RELATIONSHIP.md`.
|
|
|
|
## Memory and the user model
|
|
|
|
| Store | Holds | Written by |
|
|
|-------|-------|------------|
|
|
| `UserProfile` + `UserFact` (Settings → About you) | The user model: who the user is, across spaces; a fact can be limited to chosen spaces | `update_user_profile`, onboarding, chat consolidation, the user |
|
|
| User-scoped `MEMORY.md` (Settings → Memory) | Space notes shared by the bots of one space | The user, onboarding harvest |
|
|
| Bot-scoped `MEMORY.md` (bot → Knowledge) | One bot's notes in one space | `remember` appends dated notes; `mode: "replace"` rewrites |
|
|
|
|
Space notes are not migrated into facts: facts are short, consented statements about the user,
|
|
space notes are free-form context for one space. A memory document holds at most 16 KB, so a
|
|
bot's notes and the space notes both reach the prompt; when it is full, `remember` asks the bot
|
|
to condense it.
|
|
|
|
## Capability vision — ambient & video (future roadmap)
|
|
|
|
**This PR does not build surveillance.** Current Gesamterfahrung stays **Stage + sound + calm onboarding** (`VOICE.md`). The following is Ingo's long-term direction — optional layers only.
|
|
|
|
| Capability | Rule |
|
|
|------------|------|
|
|
| **Video calls** | Optional — user-initiated; audio-only remains the simpler default |
|
|
| **Ambient help** | Oo handles practical presence in the background; user need not sit in front of the app |
|
|
| **"It's just an Oo"** | Presence without performative attention-seeking — compounding via use, not constant face time |
|
|
| **Home metaphors (far future)** | Could replace alarm system or surveillance camera **only** with strict opt-in — never creepy, never default-on |
|
|
|
|
### Consent & anti-creep gates
|
|
|
|
- No always-on camera/mic without **explicit, revocable consent UX**
|
|
- Any sensing feature must pass *earned or creepy?* (`docs/OO-RELATIONSHIP.md`)
|
|
- Specialist lanes for home/security — not one mega-agent watching everything
|
|
- Respectful pushback when a setup would be over-broad or unsafe
|
|
|
|
Details and encounter grammar: `VOICE.md` § Ambient & video presence.
|
|
|
|
### TODO (future passes)
|
|
|
|
- Merge focus choice from `onboarding.chooseFocus` into help/vibe (`mergeHelpVibeHints` — mapping exists, wire on answer)
|
|
- Merge connector authorizations into profile
|
|
- Wire **projects-on-disk** / **habits-routines** / **pain-points** discovery sources (currently shown as planned stubs)
|
|
- Suggested Unterhaltungen + specialist Oos + project folders from discovery results
|
|
- Optional explicit name confirmation step in onboarding UI
|