# media-worker

Local inference sidecar for Media Studio (Python 3.11, FastAPI). It implements
every `MediaInferenceProvider` operation (`packages/adapter-kit/src/media.ts`)
behind HTTP API v1; the TypeScript client is `packages/adapters/src/media-sidecar.ts`.

## API

- `GET /healthz` → `{"ok": true, "version": "..."}` (no token)
- `GET /v1/capabilities` → `MediaCapabilities`
- `POST /v1/<operation>` with the TypeScript request object (minus `signal`)
- Errors: `{"error": {"code", "message", "operation"}}` with
  `bad_request`/`invalid_path` 400, `unauthorized` 401, `too_large` 413,
  `unsupported` 501, `busy` 503, `internal` 500.

Every route but `/healthz` needs `Authorization: Bearer $MEDIA_WORKER_TOKEN`.
Paths must be absolute and resolve (symlinks included) inside `MEDIA_ROOT`.
A request stops when its client disconnects (between frames or chunks) or
after `MEDIA_MAX_SECONDS`.

## Configuration

| Variable | Default | |
|---|---|---|
| `MEDIA_WORKER_TOKEN` | — | required, ≥ 32 chars (`MEDIA_WORKER_INSECURE_DEV=1` lifts this for tests) |
| `MEDIA_ROOT` | — | required; mount it at the same absolute path as the API's `DATA_DIR/media` |
| `MEDIA_MODELS_DIR` | `/models` | weight cache (volume) |
| `MEDIA_WORKER_HOST` / `MEDIA_WORKER_PORT` | `127.0.0.1` / `8790` | |
| `MEDIA_GPU_CONCURRENCY` | `1` | model operations running at once |
| `MEDIA_CPU_CONCURRENCY` | `2` | scenes, redact, audio, saliency running at once |
| `MEDIA_QUEUE_SIZE` | `8` | waiting requests per gate before `503 busy` |
| `MEDIA_MAX_INPUT_BYTES` | 20 GiB | larger inputs → 413 |
| `MEDIA_MAX_SECONDS` | 14400 | longer inputs → 413; longer processing is cancelled |
| `MEDIA_MODEL_DOWNLOAD` | `1` | `0` = offline; missing weights → 501 |
| `MEDIA_OPTIONAL_BACKENDS` | empty | opt-in for backends that are not clean for commercial use, e.g. `faceIdentity` |
| `MEDIA_ACCELERATOR` | build arg | `cpu`, `cuda` or `rocm`; falls back to cpu when no GPU is visible |

## Models

Backends, licenses, URLs, SHA256 and sizes are pinned in `app/models.py`; the
license review is in `LICENSES.md`. Weights download on first use into
`MEDIA_MODELS_DIR/<backend>/`, are checked against the pinned SHA256 and size,
and are deleted on mismatch (the operation then reports 501). Warm the volume
ahead of time with `python -m app prefetch`.

`synthesizeIdentity` always answers 501: there is no commercially licensed
generator and swap pipeline, and the request schema rejects any extra field,
so a target-face image cannot be passed.

## Redaction geometry

Boxes interpolate linearly between keyframes; a single keyframe is a static
region for the whole clip. `paddingPct` grows the box by that share of its
size, `featherPct` fades the effect outward over that share of the shorter
side (the region itself stays fully covered). Partial modes use the first two
landmarks as eyes and the third as the nose. Output is FFV1 for `.mkv`,
otherwise x264 CRF 12; audio is copied and metadata dropped. Any frame error is
a 500 and nothing is left at `outPath`.

## Build and test

```sh
docker build --build-arg ACCELERATOR=cpu -t rakazo/media-worker infra/media-worker
docker build --target test -t rakazo/media-worker:test infra/media-worker
docker run --rm rakazo/media-worker:test          # offline pytest, fake model backends
```

Without Docker: `pip install -r requirements.txt -r requirements-dev.txt` and
`python -m pytest` in this directory. Operations whose ML packages are missing
report `available: false`.

CUDA uses the torch cu126 wheels and `onnxruntime-gpu`. ROCm uses the torch
rocm7.1 wheels for SAM 2; ONNX Runtime and CTranslate2 have no ROCm wheels on
PyPI, so SigLIP, LaMa and Whisper run on the CPU there.