Media operations (generation, edits, transcription) need a runner that works
outside the agent turn, keeps a card in the conversation current and never
lets two processes put work on one GPU. Operations land in later commits; this
is the runtime they register with.
- Background job `media.run { jobId }` on a named `media` queue. Jobs carry an
optional queue: the Graphile publisher passes it as queueName and the
in-memory driver runs each queue one job at a time, in order.
- `createMediaRuntime` implements MediaRuntime (capabilities, start, get,
cancel) over a list of `MediaOperationRegistration`s and runs jobs:
start validates the input, creates the job and posts the bot's media card in
one transaction (idempotent per tool call through an operation key), then
enqueues it. The runner claims a fenced lease (an advisory lock keeps one
live lease at a time, so the GPU stays single even when the queue is
bypassed), renews it while polling for cancellation, writes progress and
outputs into the card, defers with backoff when the GPU is short of memory
or the operation asks to wait, bounds deferrals, crash restarts and run
time, always cleans the job's temp directory and always ends the card in a
terminal state. With continue_after the starting bot is woken in the same
transaction that ends the job (new run trigger `media`).
- The reconciler re-enqueues jobs whose enqueue was lost, whose runner died
or whose wait is over; a job stuck behind a dead worker's queue lock runs
outside the queue.
- Media backends are deployment-wide settings stored encrypted as the
`media` IntegrationProviderConfig row: `loadMediaSettings` and
`saveMediaSettings` with the same address policy as OpenAI-compatible
servers (no metadata hosts, public hosts only when allowed, HTTPS when a key
is sent), a saved key never follows the settings to another server, and the
view never returns keys. Contract in contracts/media-settings.ts.
- `createMediaOperations` is the one list both the API and the worker build
their runtime from (empty until the operations land), and media-tools.ts has
the shared `continue_after` schema helper for the tool definitions.
- Migration: lease, deferral, operation key and continueAfter columns.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
Groundwork for image and video generation, video editing and transcription.
Nothing calls it yet; the media runner, providers, tools and UI build on it.
- Contracts: video, audio and subtitle MIME types, per-kind size caps for
streamed and generated artifacts, video/audio message blocks and the live
media card block, media operations, job statuses and output presets
(9:16, 16:9, 1:1, 4:5, 4:3, 21:9).
- Core: attachment kinds include video and audio, artifacts become video or
audio blocks, and agent history keeps media artifact ids so later turns
can refer to them.
- Schema: artifacts record width, height, duration and the artifact they
were made from; media_jobs tracks generations, edits and transcriptions.
- Adapter kit: provider-neutral image generation, video generation and
transcription interfaces, the media job context operations run in, and
the MediaRuntime that tools use to start jobs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u
