evOo-local/infra/media-worker/README.md

media-worker

Local inference sidecar for Media Studio (Python 3.11, FastAPI). It implements every MediaInferenceProvider operation (packages/adapter-kit/src/media.ts) behind HTTP API v1; the TypeScript client is packages/adapters/src/media-sidecar.ts.

API

• GET /healthz → {"ok": true, "version": "..."} (no token)
• GET /v1/capabilities → MediaCapabilities
• POST /v1/<operation> with the TypeScript request object (minus signal)
• Errors: {"error": {"code", "message", "operation"}} with bad_request/invalid_path 400, unauthorized 401, too_large 413, unsupported 501, busy 503, internal 500.

Every route but /healthz needs Authorization: Bearer $MEDIA_WORKER_TOKEN. Paths must be absolute and resolve (symlinks included) inside MEDIA_ROOT. A request stops when its client disconnects (between frames or chunks) or after MEDIA_MAX_SECONDS.

Configuration

Variable	Default
MEDIA_WORKER_TOKEN	—	required, ≥ 32 chars (MEDIA_WORKER_INSECURE_DEV=1 lifts this for tests)
MEDIA_ROOT	—	required; mount it at the same absolute path as the API's DATA_DIR/media
MEDIA_MODELS_DIR	/models	weight cache (volume)
MEDIA_WORKER_HOST / MEDIA_WORKER_PORT	127.0.0.1 / 8790
MEDIA_GPU_CONCURRENCY	1	model operations running at once
MEDIA_CPU_CONCURRENCY	2	scenes, redact, audio, saliency running at once
MEDIA_QUEUE_SIZE	8	waiting requests per gate before 503 busy
MEDIA_MAX_INPUT_BYTES	20 GiB	larger inputs → 413
MEDIA_MAX_SECONDS	14400	longer inputs → 413; longer processing is cancelled
MEDIA_MODEL_DOWNLOAD	1	0 = offline; missing weights → 501
MEDIA_OPTIONAL_BACKENDS	empty	opt-in for backends that are not clean for commercial use, e.g. faceIdentity
MEDIA_ACCELERATOR	build arg	cpu, cuda or rocm; falls back to cpu when no GPU is visible

Models

Backends, licenses, URLs, SHA256 and sizes are pinned in app/models.py; the license review is in LICENSES.md. Weights download on first use into MEDIA_MODELS_DIR/<backend>/, are checked against the pinned SHA256 and size, and are deleted on mismatch (the operation then reports 501). Warm the volume ahead of time with python -m app prefetch.

synthesizeIdentity always answers 501: there is no commercially licensed generator and swap pipeline, and the request schema rejects any extra field, so a target-face image cannot be passed.

Redaction geometry

Boxes interpolate linearly between keyframes; a single keyframe is a static region for the whole clip. paddingPct grows the box by that share of its size, featherPct fades the effect outward over that share of the shorter side (the region itself stays fully covered). Partial modes use the first two landmarks as eyes and the third as the nose. Output is FFV1 for .mkv, otherwise x264 CRF 12; audio is copied and metadata dropped. Any frame error is a 500 and nothing is left at outPath.

Build and test

docker build --build-arg ACCELERATOR=cpu -t rakazo/media-worker infra/media-worker
docker build --target test -t rakazo/media-worker:test infra/media-worker
docker run --rm rakazo/media-worker:test          # offline pytest, fake model backends

Without Docker: pip install -r requirements.txt -r requirements-dev.txt and python -m pytest in this directory. Operations whose ML packages are missing report available: false.

CUDA uses the torch cu126 wheels and onnxruntime-gpu. ROCm uses the torch rocm7.1 wheels for SAM 2; ONNX Runtime and CTranslate2 have no ROCm wheels on PyPI, so SigLIP, LaMa and Whisper run on the CPU there.