5e2094f2ad
FastAPI service in infra/media-worker implementing every MediaInferenceProvider operation behind HTTP API v1, plus capabilities. Bearer auth, MEDIA_ROOT realpath enforcement, strict request schemas, cancellation on disconnect, GPU/CPU gates with 503 overflow, input size/duration limits, lazy SHA256-pinned model downloads, fail-closed redaction with atomic output, offline pytest suite with fake backends, CPU/CUDA/ROCm Dockerfile and a license review of every model. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0138YqDj3ttfayRwuMKKrqVj
75 lines
3.6 KiB
Markdown
75 lines
3.6 KiB
Markdown
# media-worker
|
|
|
|
Local inference sidecar for Media Studio (Python 3.11, FastAPI). It implements
|
|
every `MediaInferenceProvider` operation (`packages/adapter-kit/src/media.ts`)
|
|
behind HTTP API v1; the TypeScript client is `packages/adapters/src/media-sidecar.ts`.
|
|
|
|
## API
|
|
|
|
- `GET /healthz` → `{"ok": true, "version": "..."}` (no token)
|
|
- `GET /v1/capabilities` → `MediaCapabilities`
|
|
- `POST /v1/<operation>` with the TypeScript request object (minus `signal`)
|
|
- Errors: `{"error": {"code", "message", "operation"}}` with
|
|
`bad_request`/`invalid_path` 400, `unauthorized` 401, `too_large` 413,
|
|
`unsupported` 501, `busy` 503, `internal` 500.
|
|
|
|
Every route but `/healthz` needs `Authorization: Bearer $MEDIA_WORKER_TOKEN`.
|
|
Paths must be absolute and resolve (symlinks included) inside `MEDIA_ROOT`.
|
|
A request stops when its client disconnects (between frames or chunks) or
|
|
after `MEDIA_MAX_SECONDS`.
|
|
|
|
## Configuration
|
|
|
|
| Variable | Default | |
|
|
|---|---|---|
|
|
| `MEDIA_WORKER_TOKEN` | — | required, ≥ 32 chars (`MEDIA_WORKER_INSECURE_DEV=1` lifts this for tests) |
|
|
| `MEDIA_ROOT` | — | required; mount it at the same absolute path as the API's `DATA_DIR/media` |
|
|
| `MEDIA_MODELS_DIR` | `/models` | weight cache (volume) |
|
|
| `MEDIA_WORKER_HOST` / `MEDIA_WORKER_PORT` | `127.0.0.1` / `8790` | |
|
|
| `MEDIA_GPU_CONCURRENCY` | `1` | model operations running at once |
|
|
| `MEDIA_CPU_CONCURRENCY` | `2` | scenes, redact, audio, saliency running at once |
|
|
| `MEDIA_QUEUE_SIZE` | `8` | waiting requests per gate before `503 busy` |
|
|
| `MEDIA_MAX_INPUT_BYTES` | 20 GiB | larger inputs → 413 |
|
|
| `MEDIA_MAX_SECONDS` | 14400 | longer inputs → 413; longer processing is cancelled |
|
|
| `MEDIA_MODEL_DOWNLOAD` | `1` | `0` = offline; missing weights → 501 |
|
|
| `MEDIA_OPTIONAL_BACKENDS` | empty | opt-in for backends that are not clean for commercial use, e.g. `faceIdentity` |
|
|
| `MEDIA_ACCELERATOR` | build arg | `cpu`, `cuda` or `rocm`; falls back to cpu when no GPU is visible |
|
|
|
|
## Models
|
|
|
|
Backends, licenses, URLs, SHA256 and sizes are pinned in `app/models.py`; the
|
|
license review is in `LICENSES.md`. Weights download on first use into
|
|
`MEDIA_MODELS_DIR/<backend>/`, are checked against the pinned SHA256 and size,
|
|
and are deleted on mismatch (the operation then reports 501). Warm the volume
|
|
ahead of time with `python -m app prefetch`.
|
|
|
|
`synthesizeIdentity` always answers 501: there is no commercially licensed
|
|
generator and swap pipeline, and the request schema rejects any extra field,
|
|
so a target-face image cannot be passed.
|
|
|
|
## Redaction geometry
|
|
|
|
Boxes interpolate linearly between keyframes; a single keyframe is a static
|
|
region for the whole clip. `paddingPct` grows the box by that share of its
|
|
size, `featherPct` fades the effect outward over that share of the shorter
|
|
side (the region itself stays fully covered). Partial modes use the first two
|
|
landmarks as eyes and the third as the nose. Output is FFV1 for `.mkv`,
|
|
otherwise x264 CRF 12; audio is copied and metadata dropped. Any frame error is
|
|
a 500 and nothing is left at `outPath`.
|
|
|
|
## Build and test
|
|
|
|
```sh
|
|
docker build --build-arg ACCELERATOR=cpu -t rakazo/media-worker infra/media-worker
|
|
docker build --target test -t rakazo/media-worker:test infra/media-worker
|
|
docker run --rm rakazo/media-worker:test # offline pytest, fake model backends
|
|
```
|
|
|
|
Without Docker: `pip install -r requirements.txt -r requirements-dev.txt` and
|
|
`python -m pytest` in this directory. Operations whose ML packages are missing
|
|
report `available: false`.
|
|
|
|
CUDA uses the torch cu126 wheels and `onnxruntime-gpu`. ROCm uses the torch
|
|
rocm7.1 wheels for SAM 2; ONNX Runtime and CTranslate2 have no ROCm wheels on
|
|
PyPI, so SigLIP, LaMa and Whisper run on the CPU there.
|