AGENTS.md asks for top-level `import type` instead of inline `type`
specifiers; this applies it to the import statements the media work added
or changed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The shell contrast test added on main looks up the sidebar button by the
old German source text, which this branch replaces with "Today".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The main conversation lives in the home space, but the user can have another
space selected while it is on screen, and the card's actions sent that space:
media.jobs.start resolved the conversation in the selected space and failed,
and media.jobs.get/cancel only found jobs of the selected space.
The server now takes the space from the card itself. start finds the card's
message in one of the actor's own threads (the conversation named by botId or
groupId), requires the actor to still be a member of that thread's space, and
resolves the conversation and the source artifacts there. get and cancel find
the actor's job by id and require membership of the job's space.
media.capabilities takes an optional job id and answers for that job's space;
the card asks through its job and shares the answer per conversation. Every
other case is a 404, as before.
A Postgres test runs a card in the home-space main conversation while another
space is selected (start, get, capabilities, cancel) and checks that another
user and a user who left the card's space get 404 on every procedure.
This also completes the media settings section committed earlier as work in
progress (01eea83); its code was final, only that commit message said
otherwise.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The model saw edit_video and export_video with artifact_id next to
generate_image, generate_video, edit_image and transcribe with artifactId,
sourceArtifactId and negativePrompt. Mixed conventions in one tool list invite
wrong calls, so all media inputs now use snake_case like continue_after and the
other builtin tools: artifact_id, source_artifact_id, negative_prompt,
duration_seconds. Handlers, artifact checks, the media card's transcribe
action and the tests follow. Nothing has shipped, so stored job inputs need no
migration.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The shell contrast test added on main measured the sidebar trading status,
which this branch removes; it now measures the rows that remain.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The media pipeline moved image and file previews to the artifact route, so
they no longer take a bot or group target. The media card still passed one,
which no longer type-checks; only its follow-up actions need the target.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Uploaded videos carry no dimensions, so a vertical clip sat letterboxed in
a 16:9 frame. The player now takes the frame size from the file's metadata
when the block has none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The Media settings section (added in the previous web commit) lets the
deployment owner point media at their own servers: ComfyUI URL and optional
key, with the image model and custom image/video workflows folded under
Advanced, and a transcription URL with optional model and key. Saved keys are
never shown again; the key field says one is saved and offers to remove it.
Members who do not own the deployment do not see the section.
The spec fills both servers, captures `media-settings`, saves and checks the
exact update the server receives, then removes the saved key and the
transcription server; a second spec checks a member has no Media section.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The pipeline workstream's MediaPlayer takes artifactId, kind, name, poster
and size, without a MIME type; the card and the stand-in now use exactly
that, so its file replaces the stand-in without other changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
A card that only says "Transcript" does not tell two transcriptions apart,
and it is English whatever the user's language.
- transcribe takes an optional title like the video operations, so a bot
names the card in the user's language; "Transcript" stays the default.
- Transcribe on a media card passes the source file name as the title.
- The E2E seeds the card through bootstrap as well as threads/get, since the
bot thread's first snapshot arrives with bootstrap.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The image E2E now runs in the main conversation, which is a group thread,
so the browser test covers group uploads and reads from the artifact route
as well as bot threads. Bot file attachments keep their run's operation id
for tracing, and the non-streaming attach cap is the inline cap directly.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Brings in profile editing, the cross-space inbox and consent-first discovery. The wizard
keeps both changes: no preferences step, and the new scan step without the main bot lookup.
The harness lists both new onboarding Postgres suites.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
- onboarding-intake.spec walks focus, goal (after typing into the composer, which still
works), check-ins, quiet hours and the notifications card when the browser offers it,
with the onboarding-intake-goal, -rhythm and -done screenshots.
- completeOnboarding and the wizard specs drop the preferences step; the scan step is
optional (skipped when it asks, otherwise it advances by itself).
- The user-profile spec now checks the browser-filled profile and the focus fact; the
conversation spec checks the shorter app suggestion and the following goal question.
- The hover spec measures a reply that fills its row (a bot row's rail sits at the row's
end, so the old long onboarding line was what made it pass) and scopes Send to the
composer, since an open goal card has its own.
- Mobile shows a choice card as its question and answer instead of an empty bubble.
- The focus options drop their unused summary.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Heute only listed runs and linked to a hard-coded local Mission Control
address, so the coach's work had nowhere to show.
- Today's brief at the top, habits with a check-off and their streak,
and active goals, each only when there is something to show; then the
existing needs-you, doing and done lists.
- One icon button opens the check-in times (morning, evening, weekly),
loaded when opened and saved on every change.
- The page reads coach.today, refreshes it when a run finishes or the
tab becomes visible, and checks habits off optimistically; the date
header uses the user's time zone from the profile.
- Removes the Mission Control link and its constant.
- Tests: helper unit tests and a web E2E that writes the brief through
the main bot, checks a habit off across a reload and sets check-in
times (screenshots heute-coach and heute-rhythm).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Brings in ffmpeg (probing, extractAudio) and the video operations. The
transcribe registration now gets the ffmpeg audio tools, so video is
transcribed through extractAudio with the demuxer forced from the MIME type,
and long recordings are converted and uploaded in ranges.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Failed or cancelled jobs can still carry finished outputs, so the card asks
for the space's capabilities whenever a settled job shows video or audio.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The card's follow-ups now reach the media job RPCs, and it only offers what
the space can run.
- media.capabilities answers which operations the actor's space can run
(booleans only). The card asks once per minute per space, and only when it
shows finished video or audio, so other cards cost nothing.
- Export and Transcribe call media.jobs.start with the card's message, so the
same bot posts the new card; Cancel calls media.jobs.cancel. A refused
start shows the server's message under the card.
- New files use top-level import type.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The onboarding scan started on its own, discarded what it found, claimed
to read email and calendars, simulated progress for everyone who could
not scan, and its project creation always failed (wrong Space fields and
a space lookup by title across tenants).
Discovery now asks first and creates nothing without a choice:
- The scan step appears only for the deployment owner when
ONBOARDING_SCAN_HOME is set; everyone else skips it. The simulated
scan is gone.
- It shows the folder and asks. Only directory listings are read: git
projects up to four levels deep, their languages and a few tools
(Docker, GitHub Actions, Terraform). Symlinks are never followed and
each chosen folder is resolved again before use, so nothing outside
the folder is reached. Results stay with the account that scanned.
- Found projects are a checklist (none preselected); found languages and
tools are facts (preselected). "Add" creates one project space per
chosen project with createSpaceForMember and wireProjectWorkspace in
the user's own organization, reusing the user's own space already
linked to that folder, and saves the chosen facts as onboarding facts
under the profile lock and fact limit.
- When a connection becomes connected (direct connect, authorization
round trip, or catalog reconciliation), "Uses <App>" is remembered as
a tools fact from the connector. Revoking keeps it.
The E2E harness now points ONBOARDING_SCAN_HOME at a fixture folder and
starts without a deployment owner, so owner-only onboarding steps appear
only in specs that take the seat through /__e2e/deployment-owner.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The media card's download link uses artifactUrl's download option so a video
or audio output is saved rather than opened inline, and the E2E seeds the
thread snapshot with fresh response headers so the patched body is read.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Media sent in a conversation had nowhere to go: the composer capped every
file at 10 MiB and pushed it through a base64 RPC, video and audio blocks
did not render, and downloads decoded whole files in the page.
- Video and audio blocks play inline through MediaPlayer, which loads
nothing until the player is near the viewport, sizes the frame from the
block's dimensions (vertical clips stay within the column) and uses the
poster when one exists. Files the browser cannot play fall back to their
name with a download action.
- Images load straight from the artifact route; the lightbox gains a
download action.
- File cards, the Markdown preview and downloads read from the artifact
route instead of decoding base64.
- The composer accepts each kind up to its cap and names the computed
limit when it skips a file; video, audio and anything above the inline
cap upload as a raw stream, small files keep the inline RPC.
- MediaPlayer and artifactUrl are what the media card builds on.
Tests: MediaPlayer laziness, sizing and fallback, the upload helper, and an
E2E that uploads a tiny MP4 fixture (VP9, since Playwright's Chromium has
no H.264 decoder), plays it from the artifact route and captures
media-video-block, plus the image lightbox download.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The payload, target and outcome types are shared by the Expo and web push senders, so they now
live with PushSender in push-notifications.ts instead of in the web push adapter. The settings
E2E also stubs the Notification permission (headless Chromium reports it as denied even when
granted), and the quiet-hours inputs are wide enough for 12-hour times.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Small buttons in ui-web merge their text-ui-label size over the default
variant's text color, so the small Save button rendered ink on ink. The
editor now uses default-size controls, with the category select at the same
height.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Brings in streamed artifacts with artifactUrl (replacing the local stand-in)
and the media runtime, settings and operation registry the transcribe
operation plugs into.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Media jobs report progress and results through the media block, which the
web app did not render.
- MediaCard renders the block: operation icon, title, status (reusing the
task card's labels, plus "Waiting for GPU" for jobs queued behind the
GPU), a progress bar while active, the error when failed, and outputs:
images as thumbnails with the existing lightbox, video and audio in the
media player, subtitle and text files as file cards. Finished video and
audio get a download link, "Export" with presets labelled by aspect ratio
(the video's own shape left out) and "Transcribe", offered only when the
space can run them; running jobs get "Cancel".
- MessageView renders it with one branch.
- An E2E seeds the card on a thread snapshot with real uploaded artifacts.
- artifact-url.ts and MediaPlayer.tsx are minimal stand-ins for the
pipeline workstream's files and give way to them on merge.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Users had no way to turn on push for a browser, set quiet hours or opt out of nudges.
Settings gains a compact Notifications section: a switch for push on this device (only where
web push works: a secure context outside the desktop app), quiet hours with from/to times
(22:00-07:00 when first turned on) and a switch for nudges. Changes save in order as they are
made. A browser that blocked notifications says so instead of offering a dead switch. Logging
out unregisters the browser first, so a shared computer stops receiving the account's pushes.
The E2E stubs the Push API (headless Chromium has no push service), walks all three controls,
checks they survive a reload and captures the notification-settings screenshot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Media operations (generation, edits, transcription) need a runner that works
outside the agent turn, keeps a card in the conversation current and never
lets two processes put work on one GPU. Operations land in later commits; this
is the runtime they register with.
- Background job `media.run { jobId }` on a named `media` queue. Jobs carry an
optional queue: the Graphile publisher passes it as queueName and the
in-memory driver runs each queue one job at a time, in order.
- `createMediaRuntime` implements MediaRuntime (capabilities, start, get,
cancel) over a list of `MediaOperationRegistration`s and runs jobs:
start validates the input, creates the job and posts the bot's media card in
one transaction (idempotent per tool call through an operation key), then
enqueues it. The runner claims a fenced lease (an advisory lock keeps one
live lease at a time, so the GPU stays single even when the queue is
bypassed), renews it while polling for cancellation, writes progress and
outputs into the card, defers with backoff when the GPU is short of memory
or the operation asks to wait, bounds deferrals, crash restarts and run
time, always cleans the job's temp directory and always ends the card in a
terminal state. With continue_after the starting bot is woken in the same
transaction that ends the job (new run trigger `media`).
- The reconciler re-enqueues jobs whose enqueue was lost, whose runner died
or whose wait is over; a job stuck behind a dead worker's queue lock runs
outside the queue.
- Media backends are deployment-wide settings stored encrypted as the
`media` IntegrationProviderConfig row: `loadMediaSettings` and
`saveMediaSettings` with the same address policy as OpenAI-compatible
servers (no metadata hosts, public hosts only when allowed, HTTPS when a key
is sent), a saved key never follows the settings to another server, and the
view never returns keys. Contract in contracts/media-settings.ts.
- `createMediaOperations` is the one list both the API and the worker build
their runtime from (empty until the operations land), and media-tools.ts has
the shared `continue_after` schema helper for the tool definitions.
- Migration: lease, deferral, operation key and continueAfter columns.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Videos and audio can be hundreds of megabytes. Artifacts only travelled as
base64 inside RPC calls, capped at 10 MiB and held whole in memory, so media
could neither be uploaded, played with seeking nor produced by media jobs.
- ArtifactStore gains stat, createReadStream (inclusive byte ranges),
putStream (size cap enforced while writing, real SHA-256) and an optional
localPath so ffmpeg can read stored files in place. LocalArtifactStore
implements them with the same safety as before (O_EXCL, O_NOFOLLOW, 0600)
and now validates space ids and keys before building paths.
- storeArtifactStream / storeArtifactFromFile store a stream or a job's
output file as an artifact row, with width, height, duration and parent,
and remove the blob again if the row cannot be written.
- GET /api/artifacts/:id serves an artifact with Range/206/416, ETag,
inline disposition for media and attachment otherwise (safe RFC 6266
filenames), nosniff and same-origin CORP. Authorization matches
artifacts.get: the owner's artifact of an active bot thread or group,
with membership of the artifact's space (media elements cannot send the
space header; an explicit header must name the same space).
- POST /api/artifacts takes a raw streamed body with the target and name in
the query, authorized like artifacts.create, with the MIME allowlist and
the per-kind caps checked from Content-Length and again while writing.
- The API CSP allows media from the app origin and blob URLs.
- Web: artifactUrl(artifactId) and uploadArtifactFile for the UI.
Tests cover the streaming store, range parsing, filename sanitizing, the
file helper, and a Postgres suite for both routes: ranges, caching, other
users, shared-space members, other spaces, archived threads, signed-out
requests and upload limits including chunked bodies.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
With the coach and notification modules merged, the onboarding answers now take effect:
- the goal card creates a goal (source onboarding) through createGoal;
- the rhythm card sets the main bot's check-in routines through setCoachRhythm, and the
closing line names the first check-in from the routines that were saved;
- the quiet-hours card updates the notification settings;
- the choice card tells the server whether this browser can still turn push on, so the
notifications card is offered only there, and "Turn on" subscribes before answering.
The thread lock becomes NO KEY UPDATE so a rhythm routine, which references the thread, can
be written while the answer holds the lock. The goal answer is capped at the coach title
length. A Postgres suite walks the questions end to end: answers, skips, a dismissed card, a
stale and a doubled answer, another bot's focus card, and another user's access.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The composer, the shell frame and sidebar, the right side panel and the
create-bot modal were hard-coded dark while the text on them follows the
theme, so in the light theme the typed draft, the sidebar rows and the
side panel forms rendered dark on dark.
- Composer: the island uses the card and foreground tokens; attach, stop
and chips use border and foreground tokens, and send is the default
primary button. The focus and shadow utilities on the bar are removed:
.apple-island always overrode them, so they never rendered.
- Shell frame and page background use the background token. The frame's
radial gradient never rendered under Tailwind v4 and is dropped.
- Sidebar: the panel uses the sidebar tokens on desktop too. The main
conversation, roster, project and Heute rows share one token-based
row style; muted text measures 4.4:1 on the light sidebar, so row
times use a dimmed foreground instead.
- Side panel and create-bot modal use background, popover and overlay
tokens. Settings helper texts and inactive tabs use the full muted
color, which stays above 4.5:1 in light.
- Stage vignette, stage pill, ensemble and hero orb, and the transcript
paths behind the call overlay use tokens.
- E2E: chat-contrast.spec.ts gains a shell test that measures the
composer draft, sidebar rows, the new bot form and the bot settings
form in both themes and captures screenshots.
Claude-Session: https://claude.ai/code/session_01SSVQGR4WU92LaddTP19GcQ
Co-authored-by: Claude <noreply@anthropic.com>
The inbox in the main conversation only showed asks, approvals, takeovers and
unread threads of the current space. The main conversation lives in the home
space, so anything a bot needed in a project space never reached it.
- inbox.list now covers every space the user belongs to (except spaces being
deleted), still only the user's own bots, runs and threads. Items from
another space carry that space's id and name.
- The web inbox shows the space name next to the bot for those items, answers
them with that space's header so the existing thread procedures check
membership and ownership for that space, and opens their thread in it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
About you could only confirm or delete what the bots know about the user, so a
fact that was slightly wrong or filed under the wrong category had to be
deleted and taught again, and a fact could not be limited to some spaces from
the app even though the profile supports it.
- Each fact gets an Edit button that turns its row into an inline editor:
wording, category, and (when the user has more than one space, or the fact
is already limited) which spaces it shows in. Escape or Cancel leaves it
unchanged; only changed fields are sent through profile.update editFacts.
- A limited fact names its spaces next to its source.
- The repository validates edits like new facts: a wording whose normalized
form is empty or equals another fact of the user is refused, all edits of
one update apply atomically, and the stored embedding is only dropped when
the wording really changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
