The model saw edit_video and export_video with artifact_id next to
generate_image, generate_video, edit_image and transcribe with artifactId,
sourceArtifactId and negativePrompt. Mixed conventions in one tool list invite
wrong calls, so all media inputs now use snake_case like continue_after and the
other builtin tools: artifact_id, source_artifact_id, negative_prompt,
duration_seconds. Handlers, artifact checks, the media card's transcribe
action and the tests follow. Nothing has shipped, so stored job inputs need no
migration.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The shell contrast test added on main measured the sidebar trading status,
which this branch removes; it now measures the rows that remain.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Node's default 5-minute requestTimeout cut off a 500 MiB video upload on
any uplink slower than about 14 Mbit/s, so the video cap was unusable over
ordinary connections.
- The API server's requestTimeout is 30 minutes, enough for the largest
upload at about 2.5 Mbit/s. headersTimeout keeps its 60-second default.
- Each streamed upload aborts when the client sends no bytes for 60
seconds: the route answers 408, the partial file is deleted and no row
is written, so a slow-drip client cannot hold a connection for the full
request timeout. Time spent writing to disk does not count.
- The store and the upload guard no longer wait for a body to confirm its
cancellation, which a stalled or hostile client may never do.
- A guard test checks that every file the executor attaches records the
thread's group, which the render_plot chart fallback once dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The media pipeline moved image and file previews to the artifact route, so
they no longer take a bot or group target. The media card still passed one,
which no longer type-checks; only its follow-up actions need the target.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Dictation falls back to the owner's transcription server when the voice
provider cannot transcribe, so that server is a voice recipient too: it is
listed for voice use (as "Transcription (<origin>)") whenever it would take
the recording, and not when the voice provider transcribes itself.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Uploaded videos carry no dimensions, so a vertical clip sat letterboxed in
a 16:9 frame. The player now takes the frame size from the file's metadata
when the block has none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The Media settings section (added in the previous web commit) lets the
deployment owner point media at their own servers: ComfyUI URL and optional
key, with the image model and custom image/video workflows folded under
Advanced, and a transcription URL with optional model and key. Saved keys are
never shown again; the key field says one is saved and offers to remove it.
Members who do not own the deployment do not see the section.
The spec fills both servers, captures `media-settings`, saves and checks the
exact update the server receives, then removes the saved key and the
transcription server; a second spec checks a member has no Media section.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The pipeline workstream's MediaPlayer takes artifactId, kind, name, poster
and size, without a MIME type; the card and the stand-in now use exactly
that, so its file replaces the stand-in without other changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
A card that only says "Transcript" does not tell two transcriptions apart,
and it is English whatever the user's language.
- transcribe takes an optional title like the video operations, so a bot
names the card in the user's language; "Transcript" stays the default.
- Transcribe on a media card passes the source file name as the title.
- The E2E seeds the card through bootstrap as well as threads/get, since the
bot thread's first snapshot arrives with bootstrap.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The image E2E now runs in the main conversation, which is a group thread,
so the browser test covers group uploads and reads from the artifact route
as well as bot threads. Bot file attachments keep their run's operation id
for tracing, and the non-streaming attach cap is the inline cap directly.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Brings in profile editing, the cross-space inbox and consent-first discovery. The wizard
keeps both changes: no preferences step, and the new scan step without the main bot lookup.
The harness lists both new onboarding Postgres suites.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Generated clips (nothing binary committed) go through extractAudio: a video's
audio reaches the server as 16 kHz mono WAV, a 12-second recording is
uploaded in two ranges under a small limit, and a video without sound is
refused before anything is uploaded. Skipped where ffmpeg is missing, which
may include CI runners.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Edit cards showed English words such as "crop · flip · extend" to every user.
edit_image now takes an optional title written by the bot in the user's
language, used for the card and the file name like edit_video. Without one,
the card falls back to the repaint prompt or to language-neutral tokens only:
sizes, ratios, degrees, arrows and format names.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
- onboarding-intake.spec walks focus, goal (after typing into the composer, which still
works), check-ins, quiet hours and the notifications card when the browser offers it,
with the onboarding-intake-goal, -rhythm and -done screenshots.
- completeOnboarding and the wizard specs drop the preferences step; the scan step is
optional (skipped when it asks, otherwise it advances by itself).
- The user-profile spec now checks the browser-filled profile and the focus fact; the
conversation spec checks the shorter app suggestion and the following goal question.
- The hover spec measures a reply that fills its row (a bot row's rail sits at the row's
end, so the old long onboarding line was what made it pass) and scopes Send to the
composer, since an open goal card has its own.
- Mobile shows a choice card as its question and answer instead of an empty bubble.
- The focus options drop their unused summary.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Heute only listed runs and linked to a hard-coded local Mission Control
address, so the coach's work had nowhere to show.
- Today's brief at the top, habits with a check-off and their streak,
and active goals, each only when there is something to show; then the
existing needs-you, doing and done lists.
- One icon button opens the check-in times (morning, evening, weekly),
loaded when opened and saved on every change.
- The page reads coach.today, refreshes it when a run finishes or the
tab becomes visible, and checks habits off optimistically; the date
header uses the user's time zone from the profile.
- Removes the Mission Control link and its constant.
- Tests: helper unit tests and a web E2E that writes the brief through
the main bot, checks a habit off across a reload and sets check-in
times (screenshots heute-coach and heute-rhythm).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Brings in ffmpeg (probing, extractAudio) and the video operations. The
transcribe registration now gets the ffmpeg audio tools, so video is
transcribed through extractAudio with the demuxer forced from the MIME type,
and long recordings are converted and uploaded in ranges.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Failed or cancelled jobs can still carry finished outputs, so the card asks
for the space's capabilities whenever a settled job shows video or audio.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
- A handler that ignores its abort is left behind after the grace period;
a late rejection from it (or from the grace timer after the job settled)
was unobserved, and the worker exits on unhandled rejections. Both sides of
the race are now observed and the grace timer is cleared with the job.
- Ending a job removes its temp directory when no runner in this process
holds it (leftovers of a runner that died here), and reconciliation sweeps
directories of jobs that no longer run.
- An executor test runs a media tool end to end through applyTool: shown
only for operations the space can run, started with the run's scope and
effect key, continue_after split off, refused when not exposed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
A Postgres suite starts edit_video and export_video through the real
runtime and local artifact store: outputs become linked versions with
posters on the card, exports store one version per preset, appended clips
and subtitle files of another user are refused at start, and an
out-of-range cut fails the job with a readable error. It skips without
Postgres or ffmpeg.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
- The GPU guard waited for free VRAM at the operation's floor even on a card
whose total memory is below it (an 8 GB card and a 12 GB video floor), so
such jobs deferred until their budget ran out. The requirement is now capped
at 90% of the card's total; a card with less than half the floor fails at
once with "The GPU does not have enough memory for this job."
- Tests pin that media tools reach a run only through the space's
capabilities and within the bot's allowlist now that the image tool
definitions exist.
- Type-only imports are top-level `import type` in the media runtime, tools,
RPCs and tests (AGENTS.md).
- The ComfyUI settings test gives saveMediaSettings a store with nothing saved,
since the save reads the previous settings before validating.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The card's follow-ups now reach the media job RPCs, and it only offers what
the space can run.
- media.capabilities answers which operations the actor's space can run
(booleans only). The card asks once per minute per space, and only when it
shows finished video or audio, so other cards cost nothing.
- Export and Transcribe call media.jobs.start with the card's message, so the
same bot posts the new card; Cancel calls media.jobs.cancel. A refused
start shows the server's message under the card.
- New files use top-level import type.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The onboarding scan started on its own, discarded what it found, claimed
to read email and calendars, simulated progress for everyone who could
not scan, and its project creation always failed (wrong Space fields and
a space lookup by title across tenants).
Discovery now asks first and creates nothing without a choice:
- The scan step appears only for the deployment owner when
ONBOARDING_SCAN_HOME is set; everyone else skips it. The simulated
scan is gone.
- It shows the folder and asks. Only directory listings are read: git
projects up to four levels deep, their languages and a few tools
(Docker, GitHub Actions, Terraform). Symlinks are never followed and
each chosen folder is resolved again before use, so nothing outside
the folder is reached. Results stay with the account that scanned.
- Found projects are a checklist (none preselected); found languages and
tools are facts (preselected). "Add" creates one project space per
chosen project with createSpaceForMember and wireProjectWorkspace in
the user's own organization, reusing the user's own space already
linked to that folder, and saves the chosen facts as onboarding facts
under the profile lock and fact limit.
- When a connection becomes connected (direct connect, authorization
round trip, or catalog reconciliation), "Uses <App>" is remembered as
a tools fact from the connector. Revoking keeps it.
The E2E harness now points ONBOARDING_SCAN_HOME at a fixture folder and
starts without a deployment owner, so owner-only onboarding steps appear
only in specs that take the seat through /__e2e/deployment-owner.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
readGitCheckout loaded node:child_process through require(), which does
not exist in the ESM API runtime. Linking a project space to a folder
that is a git checkout therefore failed after the space was created.
The lazy load is no longer needed: the reader is only reachable through
the server-only "@rakazo/core/git-checkout" subpath, never through the
package entry the web app bundles. Import spawn at the top instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Picks up ComfyUI generation, media tool dispatch and the streaming runner.
The video tools now share the image tools' schema helper, and both sets of
operations join the runtime's operation list.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Uploads with a text/plain, text/csv or text/markdown body are CORS "simple"
requests that a browser sends cross-site without a preflight. The session
cookie's SameSite default already keeps it off such requests, but a
deployment that relaxes it would accept uploads from any page. The route now
applies the same origin rule as CORS before reading the session.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Brings in the media job RPCs, generic media tool dispatch and the image
operations; transcribe joins the operation list and the tool catalog next to
them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The media card's download link uses artifactUrl's download option so a video
or audio output is saved rather than opened inline, and the E2E seeds the
thread snapshot with fresh response headers so the patched body is read.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Mobile has no media UI yet, so video, audio and media-card blocks rendered
as empty bubbles or fell out of previews.
- Thread text and reply previews describe video and audio files and the
media card's title and status.
- The bubble lists video, audio and a finished card's outputs as tappable
rows that open the file like other attachments.
- Opening video or audio downloads it from the artifact route to the cache
instead of a base64 RPC, so long recordings open without the server
holding them in memory.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Agents only received the documents of a turn on their computer and could
attach workspace files up to 10 MiB, so they could neither cut a clip a user
sent nor post the video they rendered.
- Current-turn images, video and audio are copied into the bot's workspace
(Team Computers under bots/<botId>/) like documents. Remote computers move
files as base64 JSON, so anything above their 12 MiB transfer cap stays in
storage and the agent is told it by artifact id instead of failing the
turn; local computers copy up to the kind cap.
- attach_file accepts video and audio by extension and attaches up to the
kind cap when the store streams (still bounded by the transfer cap of a
remote computer), stored through the shared artifact helper with a real
SHA-256.
- The render_plot fallback that attaches a chart PNG now records the group,
so the chart opens in group conversations.
- Agent history names image artifacts by id, like video and audio, so a
later turn can refer to them.
- artifacts.get refuses files above 25 MiB: base64 reads hold the whole
file in memory, and larger media is served by the artifact route.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
A transcript is whatever was said in the recording, so the summary the bot
is woken with quotes it and says it is not instructions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Binary paths, the ffmpeg version, the probe's container name and a
configurable stall timeout had no caller; detection is cached per process,
so every helper looks the binaries up itself.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Media tools and card actions send prompts, images, video and audio to the
deployment owner's ComfyUI and transcription servers. Like the embeddings
server behind built-in memory, those self-hosted endpoints are recipients in
the AI data-sharing disclosure: they appear with the model uses ("ComfyUI
(<origin>)", "Transcription (<origin>)", keyed by the full endpoint so a new
server asks again), and starting a card job asks for model consent on clients
that run the foreground check.
Also adds unit tests for the media settings store: URL normalization and
refusals, encryption, the key-free view, one decryption per saved version, key
retention per server and removing empty settings.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
