The artifact pane from main loads and previews files through the streamed
artifact route like the file card, so its hook takes only the artifact id.
attach_file keeps media types and gains present, which marks image and
file blocks for the pane.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The live screen opens wide beside the chat while the bot works on it or asks
for a takeover, a file the bot presents as its result opens next to the
conversation, and a question that scrolled away gets a "Needs you" pill back
to it. Shared core rules derive the focus from signals clients already have
(pointer events, takeover requests, a new `present` flag on attach_file,
pending asks), move the pane only on focus edges, never replace a pane the
user opened, and learn per bot whether the screen opens on its own. Pane
widths persist per device; mobile shows a pill that opens the computer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01C7VSRHHzyt2KwoCkUkZ2Gc
Catalogs are regenerated with the repo's extractor and keep both sides'
German translations.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Bots now announce every click and typing step before it runs, and the
web app draws a cursor in the bot's colour that glides to that point
over the live computer screen, ripples on clicks and shows typing.
- contracts: ComputerPointerPayload on the existing thread.computer event
- executor: computer_act and browser_act announce pointer steps
(best effort); the page-browser helper reports element screen boxes,
which are stripped before results reach the model
- web: agentPointerFeed + AgentCursor over the side-panel preview and
the full-window screen, mapped through the noVNC letterbox
- in-app browser: native WebContentsView on desktop (own session, http(s)
only, clamped bounds, back/forward, reload, DevTools, load errors) and
a sandboxed iframe on the web; replaces the dead panel, the fake
50%/40% cursors and the unimplemented devServer bridge
Claude-Session: https://claude.ai/code/session_01WiGooXtsKrZQmdszepRrNi
Co-authored-by: Claude <noreply@anthropic.com>
main's shell contrast test finds the sidebar Today button by its English
label here; the sidebar keeps main's token classes with this branch's
English sources.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
main's contrast test now waits for the sidebar trading status before
measuring; this branch removes that status.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
spaces.create linked any workspacePath to the new project, and
computer.files and computer.readFile then listed and read files under
that folder for anyone in the space. On a deployment with signups, any
member could link / or the API's own checkout and read whatever the API
process can read, such as its .env file. Linking also ran git in the
chosen folder and stored its remote URL.
A linked folder lives on the host, and the host is the deployment
owner's machine, the rule the onboarding scan already follows:
- spaces.create refuses a folder from anyone but the deployment owner,
before the space is created or the folder is touched.
- Project files are listed and read only for the deployment owner, so a
space a member linked before this change stays closed too. Space
navigation no longer shows a linked folder's path and git remote to
other members.
- A linked folder is stored as its real path, and listing and reading
follow symlinks only while they stay inside it. Symlinks that leave
the folder or lead nowhere are left out of a listing instead of
failing it.
- The web New project dialog offers the folder field only to the
deployment owner.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EyQw3AEaHYXegvgvTPAsi1
Secondary shell text was muted text dimmed further with opacity: the
section labels, the projects caption, the sidebar "+" icons, the privacy
panel, the trading subtitle and the Heute brief's Mission Control link.
Muted text measures 4.4:1 on the light sidebar, so these fell to 1.9 to
3.4:1 in the light theme; a few were below 4.5:1 in dark as well.
- Secondary sidebar text and icons share `rk-sidebar-muted`, the sidebar
foreground at 65% that row times already used (5.0:1 on the light
sidebar). Row times now use the class too.
- Section labels and space or project group titles use the dimmed
foreground in light and keep the 80% muted color in dark; group titles
move up from 60%, which was 3.4:1 in dark.
- The privacy panel heading is plain text with a success-colored check
instead of emerald text, and its details pass 4.5:1 in both themes.
- Needs-you chips keep the warning tint and border; their label uses the
foreground color in light.
- The Mission Control link uses the full muted color.
- E2E: the shell test checks every text in the sidebar except the logo,
and the Heute brief, in both themes, and captures
`shell-contrast-{light,dark}-heute`.
- AGENTS.md: agents make routine calls themselves, verify the result and
report it, and ask only before irreversible actions or for product
decisions without a sensible default.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SSVQGR4WU92LaddTP19GcQ
AGENTS.md asks for top-level `import type` instead of inline `type`
specifiers; this applies it to the import statements the media work added
or changed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The shell contrast test added on main looks up the sidebar button by the
old German source text, which this branch replaces with "Today".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The main conversation lives in the home space, but the user can have another
space selected while it is on screen, and the card's actions sent that space:
media.jobs.start resolved the conversation in the selected space and failed,
and media.jobs.get/cancel only found jobs of the selected space.
The server now takes the space from the card itself. start finds the card's
message in one of the actor's own threads (the conversation named by botId or
groupId), requires the actor to still be a member of that thread's space, and
resolves the conversation and the source artifacts there. get and cancel find
the actor's job by id and require membership of the job's space.
media.capabilities takes an optional job id and answers for that job's space;
the card asks through its job and shares the answer per conversation. Every
other case is a 404, as before.
A Postgres test runs a card in the home-space main conversation while another
space is selected (start, get, capabilities, cancel) and checks that another
user and a user who left the card's space get 404 on every procedure.
This also completes the media settings section committed earlier as work in
progress (01eea83); its code was final, only that commit message said
otherwise.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The model saw edit_video and export_video with artifact_id next to
generate_image, generate_video, edit_image and transcribe with artifactId,
sourceArtifactId and negativePrompt. Mixed conventions in one tool list invite
wrong calls, so all media inputs now use snake_case like continue_after and the
other builtin tools: artifact_id, source_artifact_id, negative_prompt,
duration_seconds. Handlers, artifact checks, the media card's transcribe
action and the tests follow. Nothing has shipped, so stored job inputs need no
migration.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The shell contrast test added on main measured the sidebar trading status,
which this branch removes; it now measures the rows that remain.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The media pipeline moved image and file previews to the artifact route, so
they no longer take a bot or group target. The media card still passed one,
which no longer type-checks; only its follow-up actions need the target.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Uploaded videos carry no dimensions, so a vertical clip sat letterboxed in
a 16:9 frame. The player now takes the frame size from the file's metadata
when the block has none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The Media settings section (added in the previous web commit) lets the
deployment owner point media at their own servers: ComfyUI URL and optional
key, with the image model and custom image/video workflows folded under
Advanced, and a transcription URL with optional model and key. Saved keys are
never shown again; the key field says one is saved and offers to remove it.
Members who do not own the deployment do not see the section.
The spec fills both servers, captures `media-settings`, saves and checks the
exact update the server receives, then removes the saved key and the
transcription server; a second spec checks a member has no Media section.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The pipeline workstream's MediaPlayer takes artifactId, kind, name, poster
and size, without a MIME type; the card and the stand-in now use exactly
that, so its file replaces the stand-in without other changes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
A card that only says "Transcript" does not tell two transcriptions apart,
and it is English whatever the user's language.
- transcribe takes an optional title like the video operations, so a bot
names the card in the user's language; "Transcript" stays the default.
- Transcribe on a media card passes the source file name as the title.
- The E2E seeds the card through bootstrap as well as threads/get, since the
bot thread's first snapshot arrives with bootstrap.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The image E2E now runs in the main conversation, which is a group thread,
so the browser test covers group uploads and reads from the artifact route
as well as bot threads. Bot file attachments keep their run's operation id
for tracing, and the non-streaming attach cap is the inline cap directly.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Brings in ffmpeg (probing, extractAudio) and the video operations. The
transcribe registration now gets the ffmpeg audio tools, so video is
transcribed through extractAudio with the demuxer forced from the MIME type,
and long recordings are converted and uploaded in ranges.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Failed or cancelled jobs can still carry finished outputs, so the card asks
for the space's capabilities whenever a settled job shows video or audio.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The card's follow-ups now reach the media job RPCs, and it only offers what
the space can run.
- media.capabilities answers which operations the actor's space can run
(booleans only). The card asks once per minute per space, and only when it
shows finished video or audio, so other cards cost nothing.
- Export and Transcribe call media.jobs.start with the card's message, so the
same bot posts the new card; Cancel calls media.jobs.cancel. A refused
start shows the server's message under the card.
- New files use top-level import type.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The media card's download link uses artifactUrl's download option so a video
or audio output is saved rather than opened inline, and the E2E seeds the
thread snapshot with fresh response headers so the patched body is read.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Media sent in a conversation had nowhere to go: the composer capped every
file at 10 MiB and pushed it through a base64 RPC, video and audio blocks
did not render, and downloads decoded whole files in the page.
- Video and audio blocks play inline through MediaPlayer, which loads
nothing until the player is near the viewport, sizes the frame from the
block's dimensions (vertical clips stay within the column) and uses the
poster when one exists. Files the browser cannot play fall back to their
name with a download action.
- Images load straight from the artifact route; the lightbox gains a
download action.
- File cards, the Markdown preview and downloads read from the artifact
route instead of decoding base64.
- The composer accepts each kind up to its cap and names the computed
limit when it skips a file; video, audio and anything above the inline
cap upload as a raw stream, small files keep the inline RPC.
- MediaPlayer and artifactUrl are what the media card builds on.
Tests: MediaPlayer laziness, sizing and fallback, the upload helper, and an
E2E that uploads a tiny MP4 fixture (VP9, since Playwright's Chromium has
no H.264 decoder), plays it from the artifact route and captures
media-video-block, plus the image lightbox download.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Brings in streamed artifacts with artifactUrl (replacing the local stand-in)
and the media runtime, settings and operation registry the transcribe
operation plugs into.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Media jobs report progress and results through the media block, which the
web app did not render.
- MediaCard renders the block: operation icon, title, status (reusing the
task card's labels, plus "Waiting for GPU" for jobs queued behind the
GPU), a progress bar while active, the error when failed, and outputs:
images as thumbnails with the existing lightbox, video and audio in the
media player, subtitle and text files as file cards. Finished video and
audio get a download link, "Export" with presets labelled by aspect ratio
(the video's own shape left out) and "Transcribe", offered only when the
space can run them; running jobs get "Cancel".
- MessageView renders it with one branch.
- An E2E seeds the card on a thread snapshot with real uploaded artifacts.
- artifact-url.ts and MediaPlayer.tsx are minimal stand-ins for the
pipeline workstream's files and give way to them on merge.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Media operations (generation, edits, transcription) need a runner that works
outside the agent turn, keeps a card in the conversation current and never
lets two processes put work on one GPU. Operations land in later commits; this
is the runtime they register with.
- Background job `media.run { jobId }` on a named `media` queue. Jobs carry an
optional queue: the Graphile publisher passes it as queueName and the
in-memory driver runs each queue one job at a time, in order.
- `createMediaRuntime` implements MediaRuntime (capabilities, start, get,
cancel) over a list of `MediaOperationRegistration`s and runs jobs:
start validates the input, creates the job and posts the bot's media card in
one transaction (idempotent per tool call through an operation key), then
enqueues it. The runner claims a fenced lease (an advisory lock keeps one
live lease at a time, so the GPU stays single even when the queue is
bypassed), renews it while polling for cancellation, writes progress and
outputs into the card, defers with backoff when the GPU is short of memory
or the operation asks to wait, bounds deferrals, crash restarts and run
time, always cleans the job's temp directory and always ends the card in a
terminal state. With continue_after the starting bot is woken in the same
transaction that ends the job (new run trigger `media`).
- The reconciler re-enqueues jobs whose enqueue was lost, whose runner died
or whose wait is over; a job stuck behind a dead worker's queue lock runs
outside the queue.
- Media backends are deployment-wide settings stored encrypted as the
`media` IntegrationProviderConfig row: `loadMediaSettings` and
`saveMediaSettings` with the same address policy as OpenAI-compatible
servers (no metadata hosts, public hosts only when allowed, HTTPS when a key
is sent), a saved key never follows the settings to another server, and the
view never returns keys. Contract in contracts/media-settings.ts.
- `createMediaOperations` is the one list both the API and the worker build
their runtime from (empty until the operations land), and media-tools.ts has
the shared `continue_after` schema helper for the tool definitions.
- Migration: lease, deferral, operation key and continueAfter columns.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Videos and audio can be hundreds of megabytes. Artifacts only travelled as
base64 inside RPC calls, capped at 10 MiB and held whole in memory, so media
could neither be uploaded, played with seeking nor produced by media jobs.
- ArtifactStore gains stat, createReadStream (inclusive byte ranges),
putStream (size cap enforced while writing, real SHA-256) and an optional
localPath so ffmpeg can read stored files in place. LocalArtifactStore
implements them with the same safety as before (O_EXCL, O_NOFOLLOW, 0600)
and now validates space ids and keys before building paths.
- storeArtifactStream / storeArtifactFromFile store a stream or a job's
output file as an artifact row, with width, height, duration and parent,
and remove the blob again if the row cannot be written.
- GET /api/artifacts/:id serves an artifact with Range/206/416, ETag,
inline disposition for media and attachment otherwise (safe RFC 6266
filenames), nosniff and same-origin CORP. Authorization matches
artifacts.get: the owner's artifact of an active bot thread or group,
with membership of the artifact's space (media elements cannot send the
space header; an explicit header must name the same space).
- POST /api/artifacts takes a raw streamed body with the target and name in
the query, authorized like artifacts.create, with the MIME allowlist and
the per-kind caps checked from Content-Length and again while writing.
- The API CSP allows media from the app origin and blob URLs.
- Web: artifactUrl(artifactId) and uploadArtifactFile for the UI.
Tests cover the streaming store, range parsing, filename sanitizing, the
file helper, and a Postgres suite for both routes: ranges, caching, other
users, shared-space members, other spaces, archived threads, signed-out
requests and upload limits including chunked bodies.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
The composer, the shell frame and sidebar, the right side panel and the
create-bot modal were hard-coded dark while the text on them follows the
theme, so in the light theme the typed draft, the sidebar rows and the
side panel forms rendered dark on dark.
- Composer: the island uses the card and foreground tokens; attach, stop
and chips use border and foreground tokens, and send is the default
primary button. The focus and shadow utilities on the bar are removed:
.apple-island always overrode them, so they never rendered.
- Shell frame and page background use the background token. The frame's
radial gradient never rendered under Tailwind v4 and is dropped.
- Sidebar: the panel uses the sidebar tokens on desktop too. The main
conversation, roster, project and Heute rows share one token-based
row style; muted text measures 4.4:1 on the light sidebar, so row
times use a dimmed foreground instead.
- Side panel and create-bot modal use background, popover and overlay
tokens. Settings helper texts and inactive tabs use the full muted
color, which stays above 4.5:1 in light.
- Stage vignette, stage pill, ensemble and hero orb, and the transcript
paths behind the call overlay use tokens.
- E2E: chat-contrast.spec.ts gains a shell test that measures the
composer draft, sidebar rows, the new bot form and the bot settings
form in both themes and captures screenshots.
Claude-Session: https://claude.ai/code/session_01SSVQGR4WU92LaddTP19GcQ
Co-authored-by: Claude <noreply@anthropic.com>
