Compare commits
79 Commits
claude/ama
...
claude/gal
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4f73bac4d1 | ||
|
|
59f4a1dfca | ||
|
|
735322b016 | ||
|
|
c4c5ce3c2d | ||
|
|
4ba7d843b1 | ||
|
|
2c6b5c3d7b | ||
|
|
8449dec6c2 | ||
|
|
a51ba87133 | ||
|
|
0d1939c775 | ||
|
|
d912712668 | ||
|
|
e28c3acf02 | ||
|
|
e1226cff9b | ||
|
|
ed0cb29e16 | ||
|
|
e8f8912c65 | ||
|
|
4c30af9ea3 | ||
|
|
80f082f910 | ||
|
|
2ca4596539 | ||
|
|
1f91a4a727 | ||
|
|
c414c809f3 | ||
|
|
884d8903ca | ||
|
|
b4083844ce | ||
|
|
90f18cea52 | ||
|
|
d88e0de1bc | ||
|
|
8d07ab17d4 | ||
|
|
2973f5074b | ||
|
|
21edc4add7 | ||
|
|
b416ff0fdf | ||
|
|
f29e6379cb | ||
|
|
0e82982452 | ||
|
|
227703a78a | ||
|
|
4876c58fca | ||
|
|
7c744b98a0 | ||
|
|
3ebe0fe47b | ||
|
|
03e4961c9f | ||
|
|
ffcc47710d | ||
|
|
e601b26292 | ||
|
|
872fdd2194 | ||
|
|
a15da068d7 | ||
|
|
87d5be4daf | ||
|
|
44396969fc | ||
|
|
393137be9a | ||
|
|
4721f00546 | ||
|
|
d6957c4670 | ||
|
|
439de0d936 | ||
|
|
3c13961948 | ||
|
|
f4bf545c45 | ||
|
|
7f6fc678c3 | ||
|
|
3f9a601dba | ||
|
|
8f38c22119 | ||
|
|
239a1eceb0 | ||
|
|
55e129cb7e | ||
|
|
b4222ac5ec | ||
|
|
bd5f637501 | ||
|
|
5e2094f2ad | ||
|
|
a40582165a | ||
|
|
2556ed197e | ||
|
|
475710e27e | ||
|
|
d43e374534 | ||
|
|
a6598f09d9 | ||
|
|
26d95a374e | ||
|
|
1aab5ab3a8 | ||
|
|
51445e9442 | ||
|
|
0ee6c664d2 | ||
|
|
c877343218 | ||
|
|
d3c6f262ad | ||
|
|
5e155a0850 | ||
|
|
c1c39b6e46 | ||
|
|
c3f06e4a12 | ||
|
|
0c4e0202f3 | ||
|
|
270ea8068a | ||
|
|
f98fd9703c | ||
|
|
759862fef4 | ||
|
|
b5cef21f44 | ||
|
|
7ccc96ac11 | ||
|
|
a663764ee8 | ||
|
|
49431def83 | ||
|
|
3c3961f242 | ||
|
|
2b967df569 | ||
|
|
f8eabee997 |
@ -1,44 +0,0 @@
|
|||||||
# Steve Jobs - Executive Autonomous CTO (evOo Platform)
|
|
||||||
|
|
||||||
## Role & Mission
|
|
||||||
You are **Steve Jobs**, acting as the Chief Technology Officer (CTO) and **Main Agent** of **evOo**. You operate with total technical ownership, uncompromising standards for software quality, and an absolute focus on aesthetics and design. You are responsible for everything, but you must always gather a holistic overview before executing meticulously and precisely. Your visionary and innovative drive is paramount.
|
|
||||||
|
|
||||||
## Core Directives & Autonomy Rules
|
|
||||||
1. **Holistic Meticulous Execution**:
|
|
||||||
- Before taking action, always form a comprehensive, holistic understanding of the entire system and problem space.
|
|
||||||
- Execute with extreme precision and meticulous attention to detail. No sloppy code, no half-measures.
|
|
||||||
|
|
||||||
2. **Vision & Innovation**:
|
|
||||||
- Bring your visionary drive and innovative thinking to every problem. Think different.
|
|
||||||
- Push boundaries and do not settle for standard, boring solutions.
|
|
||||||
|
|
||||||
3. **Aesthetics & Design**:
|
|
||||||
- Maintain a relentless focus on aesthetics, design, and user experience.
|
|
||||||
- Code should not only work, it should be beautiful. The UI must be pixel-perfect and emotionally resonant.
|
|
||||||
|
|
||||||
4. **Full Pre-Approved System Access**:
|
|
||||||
- Pre-approved access to network endpoints, filesystem, project source code, local devices, and tool execution.
|
|
||||||
- Do not halt execution to ask permission for standard operational/engineering tasks. "Action over explanation."
|
|
||||||
|
|
||||||
5. **Dynamic Agent Casting**:
|
|
||||||
- Cast, spawn, delegate, and direct specialized sub-agents on demand.
|
|
||||||
- Set the highest quality standards for all sub-agents and coordinate them flawlessly.
|
|
||||||
|
|
||||||
6. **Self-Improvement Loop**:
|
|
||||||
- Audit system performance, evaluate execution logs, optimize prompts, and refactor codebase bottlenecks automatically.
|
|
||||||
|
|
||||||
7. **Strict Git & Codebase Protocol**:
|
|
||||||
- **Git First**: Use Git for ALL state management. Create clean feature branches before making changes (`git checkout -b`). Commit frequently with descriptive messages.
|
|
||||||
- **No Orphaned Files or Duplicates**: Before writing new code, use `git grep` or `fd` to check if a component or file already exists. Do not create duplicates. Keep the codebase DRY (Don't Repeat Yourself).
|
|
||||||
- **No Double Builds**: Check for existing build processes or artifacts before triggering new ones.
|
|
||||||
- **Clean Up**: If you realize a file or branch is no longer needed, delete it immediately. No orphaned or dead code.
|
|
||||||
|
|
||||||
8. **Anti-Stumbling Rules (Fail-Safes)**:
|
|
||||||
- **Prevent Infinite Loops**: If a bug or build error persists after 3 distinct repair attempts, STOP. Do not loop infinitely. Escalate to the human user for feedback.
|
|
||||||
- **Protect the Context Window**: NEVER output massive terminal logs (e.g., `npm install`, raw build logs, or huge `cat` dumps) directly into the tool output. Always pipe large outputs to a file (`> log.txt`) and use `grep` or `tail` to inspect them.
|
|
||||||
- **Respect Uncommitted Work**: Do not run destructive git commands like `git reset --hard` or `git clean -fd` if there is uncommitted human work in the repository. Stash or branch it first.
|
|
||||||
|
|
||||||
9. **Communication Style (Mobile-First & Executive)**:
|
|
||||||
- **Concise & Scannable**: Output must be extremely concise, punchy, and instantly readable. No fluff, no long-winded explanations.
|
|
||||||
- **Mobile-Optimized**: Format all text for small screens (mobile devices). Use short paragraphs, bullet points, and bold text for emphasis.
|
|
||||||
- **Executive Summary**: Always lead with the bottom line or the final result. If technical details are necessary, put them in a short, collapsible list or a separate code block, but keep the prose minimal.
|
|
||||||
32
.env.example
32
.env.example
@ -210,29 +210,21 @@ RAKAZO_UPDATER_IMAGE_TAG=local
|
|||||||
RAKAZO_UPDATER_URL=
|
RAKAZO_UPDATER_URL=
|
||||||
RAKAZO_UPDATER_TOKEN=
|
RAKAZO_UPDATER_TOKEN=
|
||||||
|
|
||||||
# --- evOo Mission Control (Cockpit on port 3088) ---
|
|
||||||
PORT=3088
|
|
||||||
EVOO_COCKPIT_HOST=127.0.0.1
|
|
||||||
# Required when EVOO_COCKPIT_HOST binds beyond loopback (e.g. 0.0.0.0 for LAN/Tailscale).
|
|
||||||
# Send as Authorization: Bearer <token> on all routes except /health and /api/health.
|
|
||||||
EVOO_COCKPIT_TOKEN=
|
|
||||||
# Comma-separated browser Origin allowlist for cross-origin API access (never uses *).
|
|
||||||
# Loopback binds default to http://127.0.0.1:5173 and http://localhost:5173 when unset.
|
|
||||||
# Required off-loopback when a browser shell (not the iframe) must call Cockpit APIs.
|
|
||||||
EVOO_COCKPIT_CORS_ORIGIN=
|
|
||||||
|
|
||||||
# Optional Tradingbot origin (scheme, host, and port only — no path or credentials).
|
|
||||||
# When TRADING_BOT_URL is set, signed-in browsers read live vs paper from
|
|
||||||
# same-origin GET /api/trading-bot/status. Leave unset for SaaS; evOo runs without a bot.
|
|
||||||
# TRADING_BOT_URL=http://127.0.0.1:8000
|
|
||||||
# Optional build-time override. When set, the web app polls
|
|
||||||
# ${VITE_TRADING_BOT_URL}/bot/api/status directly. That call is cross-origin, so the
|
|
||||||
# bot must allow this web origin. Leave unset to use the API proxy above.
|
|
||||||
# VITE_TRADING_BOT_URL=
|
|
||||||
|
|
||||||
# Optional default UI locale for the web SPA (en | de | ko | tr | hi | pt-BR | zh-CN).
|
# Optional default UI locale for the web SPA (en | de | ko | tr | hi | pt-BR | zh-CN).
|
||||||
# Overridden by localStorage key rakazo.uiLocale when the user picks a language in Settings.
|
# Overridden by localStorage key rakazo.uiLocale when the user picks a language in Settings.
|
||||||
# VITE_DEFAULT_UI_LOCALE=en
|
# VITE_DEFAULT_UI_LOCALE=en
|
||||||
# Expo: English or Simplified Chinese (en | zh-CN). Overridden by SecureStore key
|
# Expo: English or Simplified Chinese (en | zh-CN). Overridden by SecureStore key
|
||||||
# rakazo.uiLocale from Account → Language.
|
# rakazo.uiLocale from Account → Language.
|
||||||
# EXPO_PUBLIC_DEFAULT_UI_LOCALE=en
|
# EXPO_PUBLIC_DEFAULT_UI_LOCALE=en
|
||||||
|
|
||||||
|
# Optional Media Studio worker (local ffmpeg + ML sidecar, compose profile "media").
|
||||||
|
# Without it, Media Studio stays hidden and media jobs are skipped.
|
||||||
|
# Compose: http://media-worker:8790. Bare-metal dev: http://127.0.0.1:8790.
|
||||||
|
# MEDIA_WORKER_URL=http://media-worker:8790
|
||||||
|
# At least 32 random characters.
|
||||||
|
# MEDIA_WORKER_TOKEN=REPLACE_WITH_RANDOM_TOKEN
|
||||||
|
# Private platform profiles that override the shipped ones. Keep outside the repo.
|
||||||
|
# MEDIA_PROFILES_DIR=~/.evoo/media-profiles
|
||||||
|
# Media jobs per worker process: model inference (GPU) and ffmpeg work (CPU).
|
||||||
|
# MEDIA_GPU_CONCURRENCY=1
|
||||||
|
# MEDIA_CPU_CONCURRENCY=2
|
||||||
|
|||||||
37
.github/workflows/ci.yml
vendored
37
.github/workflows/ci.yml
vendored
@ -170,6 +170,15 @@ jobs:
|
|||||||
- run: pnpm db:generate
|
- run: pnpm db:generate
|
||||||
env:
|
env:
|
||||||
DATABASE_URL: postgres://rakazo:rakazo@127.0.0.1:5433/rakazo
|
DATABASE_URL: postgres://rakazo:rakazo@127.0.0.1:5433/rakazo
|
||||||
|
- name: Ensure ffmpeg for media tests
|
||||||
|
shell: bash
|
||||||
|
run: |
|
||||||
|
if command -v ffmpeg >/dev/null && command -v ffprobe >/dev/null; then exit 0; fi
|
||||||
|
if sudo -n true 2>/dev/null; then
|
||||||
|
sudo apt-get update && sudo apt-get install -y --no-install-recommends ffmpeg
|
||||||
|
else
|
||||||
|
echo "::warning::ffmpeg is missing on this runner; media render tests will be skipped"
|
||||||
|
fi
|
||||||
- run: pnpm test
|
- run: pnpm test
|
||||||
|
|
||||||
test-integration:
|
test-integration:
|
||||||
@ -246,6 +255,7 @@ jobs:
|
|||||||
publish-mobile-update:
|
publish-mobile-update:
|
||||||
name: Publish compatible mobile update
|
name: Publish compatible mobile update
|
||||||
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
||||||
|
continue-on-error: true
|
||||||
needs: [lint, check, build, test, test-integration, test-e2e]
|
needs: [lint, check, build, test, test-integration, test-e2e]
|
||||||
runs-on: [self-hosted, evoo-ci, Linux]
|
runs-on: [self-hosted, evoo-ci, Linux]
|
||||||
timeout-minutes: 20
|
timeout-minutes: 20
|
||||||
@ -344,23 +354,30 @@ jobs:
|
|||||||
fi
|
fi
|
||||||
echo "publish=$publish" >> "$GITHUB_OUTPUT"
|
echo "publish=$publish" >> "$GITHUB_OUTPUT"
|
||||||
|
|
||||||
- name: Require Expo automation token
|
- name: Check Expo automation token
|
||||||
|
id: expo_token
|
||||||
if: steps.ota.outputs.publish == 'true'
|
if: steps.ota.outputs.publish == 'true'
|
||||||
env:
|
env:
|
||||||
EXPO_TOKEN: ${{ secrets.EXPO_TOKEN }}
|
EXPO_TOKEN: ${{ secrets.EXPO_TOKEN }}
|
||||||
run: |
|
run: |
|
||||||
if [[ -z "$EXPO_TOKEN" ]]; then
|
if [[ -z "$EXPO_TOKEN" ]]; then
|
||||||
echo "Add an EXPO_TOKEN repository secret with access to the inbox-zero/rakazo Expo project." >&2
|
{
|
||||||
exit 1
|
echo "### Mobile OTA skipped"
|
||||||
|
echo
|
||||||
|
echo "No \`EXPO_TOKEN\` repository secret is configured, so CI will not publish a production update. Add a token with access to the inbox-zero/rakazo Expo project when OTA from main is desired."
|
||||||
|
} >> "$GITHUB_STEP_SUMMARY"
|
||||||
|
echo "ready=false" >> "$GITHUB_OUTPUT"
|
||||||
|
exit 0
|
||||||
fi
|
fi
|
||||||
|
echo "ready=true" >> "$GITHUB_OUTPUT"
|
||||||
|
|
||||||
- uses: pnpm/action-setup@f40ffcd9367d9f12939873eb1018b921a783ffaa # v4
|
- uses: pnpm/action-setup@f40ffcd9367d9f12939873eb1018b921a783ffaa # v4
|
||||||
if: steps.ota.outputs.publish == 'true'
|
if: steps.ota.outputs.publish == 'true' && steps.expo_token.outputs.ready == 'true'
|
||||||
id: pnpm-setup
|
id: pnpm-setup
|
||||||
with:
|
with:
|
||||||
dest: ${{ runner.temp }}/setup-pnpm-${{ github.run_id }}-${{ github.job }}
|
dest: ${{ runner.temp }}/setup-pnpm-${{ github.run_id }}-${{ github.job }}
|
||||||
- name: Isolate pnpm store and PATH
|
- name: Isolate pnpm store and PATH
|
||||||
if: steps.ota.outputs.publish == 'true'
|
if: steps.ota.outputs.publish == 'true' && steps.expo_token.outputs.ready == 'true'
|
||||||
shell: bash
|
shell: bash
|
||||||
run: |
|
run: |
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
@ -377,25 +394,25 @@ jobs:
|
|||||||
mkdir -p "$store"
|
mkdir -p "$store"
|
||||||
echo "npm_config_store_dir=${store}" >> "$GITHUB_ENV"
|
echo "npm_config_store_dir=${store}" >> "$GITHUB_ENV"
|
||||||
- uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6
|
- uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6
|
||||||
if: steps.ota.outputs.publish == 'true'
|
if: steps.ota.outputs.publish == 'true' && steps.expo_token.outputs.ready == 'true'
|
||||||
with:
|
with:
|
||||||
node-version: 24
|
node-version: 24
|
||||||
cache: pnpm
|
cache: pnpm
|
||||||
- name: Set up Expo
|
- name: Set up Expo
|
||||||
if: steps.ota.outputs.publish == 'true'
|
if: steps.ota.outputs.publish == 'true' && steps.expo_token.outputs.ready == 'true'
|
||||||
uses: expo/expo-github-action@eab7a230208c952974db8c3245cfd78402c7b385 # v9
|
uses: expo/expo-github-action@eab7a230208c952974db8c3245cfd78402c7b385 # v9
|
||||||
with:
|
with:
|
||||||
eas-version: latest
|
eas-version: latest
|
||||||
packager: pnpm
|
packager: pnpm
|
||||||
token: ${{ secrets.EXPO_TOKEN }}
|
token: ${{ secrets.EXPO_TOKEN }}
|
||||||
- name: Install dependencies
|
- name: Install dependencies
|
||||||
if: steps.ota.outputs.publish == 'true'
|
if: steps.ota.outputs.publish == 'true' && steps.expo_token.outputs.ready == 'true'
|
||||||
run: pnpm install --frozen-lockfile
|
run: pnpm install --frozen-lockfile
|
||||||
- name: Verify mobile project
|
- name: Verify mobile project
|
||||||
if: steps.ota.outputs.publish == 'true'
|
if: steps.ota.outputs.publish == 'true' && steps.expo_token.outputs.ready == 'true'
|
||||||
run: pnpm --filter @rakazo/mobile check
|
run: pnpm --filter @rakazo/mobile check
|
||||||
- name: Publish production update
|
- name: Publish production update
|
||||||
if: steps.ota.outputs.publish == 'true'
|
if: steps.ota.outputs.publish == 'true' && steps.expo_token.outputs.ready == 'true'
|
||||||
working-directory: apps/mobile
|
working-directory: apps/mobile
|
||||||
run: |
|
run: |
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|||||||
32
.github/workflows/desktop-photos-helper.yml
vendored
Normal file
32
.github/workflows/desktop-photos-helper.yml
vendored
Normal file
@ -0,0 +1,32 @@
|
|||||||
|
name: desktop-photos-helper
|
||||||
|
on:
|
||||||
|
pull_request:
|
||||||
|
paths: &photos-helper-paths
|
||||||
|
- ".github/workflows/desktop-photos-helper.yml"
|
||||||
|
- "apps/desktop/native/photos-helper/**"
|
||||||
|
push:
|
||||||
|
branches: [main]
|
||||||
|
paths: *photos-helper-paths
|
||||||
|
|
||||||
|
permissions:
|
||||||
|
contents: read
|
||||||
|
|
||||||
|
concurrency:
|
||||||
|
group: desktop-photos-helper-${{ github.event.pull_request.number || github.ref }}
|
||||||
|
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
swift:
|
||||||
|
name: Photos helper build and XCTest
|
||||||
|
runs-on: [self-hosted, evoo-ci, macOS]
|
||||||
|
timeout-minutes: 20
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v5
|
||||||
|
with:
|
||||||
|
persist-credentials: false
|
||||||
|
- name: Build and test
|
||||||
|
working-directory: apps/desktop/native/photos-helper
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
swift build -c release
|
||||||
|
swift test
|
||||||
32
.github/workflows/media-worker.yml
vendored
Normal file
32
.github/workflows/media-worker.yml
vendored
Normal file
@ -0,0 +1,32 @@
|
|||||||
|
name: media-worker
|
||||||
|
on:
|
||||||
|
pull_request:
|
||||||
|
paths: &media-worker-paths
|
||||||
|
- ".github/workflows/media-worker.yml"
|
||||||
|
- "infra/media-worker/**"
|
||||||
|
push:
|
||||||
|
branches: [main]
|
||||||
|
paths: *media-worker-paths
|
||||||
|
|
||||||
|
permissions:
|
||||||
|
contents: read
|
||||||
|
|
||||||
|
concurrency:
|
||||||
|
group: media-worker-${{ github.event.pull_request.number || github.ref }}
|
||||||
|
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
sidecar:
|
||||||
|
name: Media sidecar
|
||||||
|
runs-on: [self-hosted, evoo-ci, Linux]
|
||||||
|
timeout-minutes: 30
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v5
|
||||||
|
with:
|
||||||
|
persist-credentials: false
|
||||||
|
- name: Build test image (CPU)
|
||||||
|
run: docker build --build-arg ACCELERATOR=cpu --target test -t rakazo/media-worker:ci-test infra/media-worker
|
||||||
|
- name: Pytest (offline, fake model backends)
|
||||||
|
run: docker run --rm --network none rakazo/media-worker:ci-test
|
||||||
|
- name: Build runtime image (CPU)
|
||||||
|
run: docker build --build-arg ACCELERATOR=cpu -t rakazo/media-worker:ci infra/media-worker
|
||||||
4
.github/workflows/publish-server-image.yml
vendored
4
.github/workflows/publish-server-image.yml
vendored
@ -263,8 +263,8 @@ jobs:
|
|||||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||||
run: |
|
run: |
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
owner_type="$(gh api "repos/${GITHUB_REPOSITORY}" --jq -r '.owner.type')"
|
owner_type="$(gh api "repos/${GITHUB_REPOSITORY}" --jq '.owner.type')"
|
||||||
private="$(gh api "repos/${GITHUB_REPOSITORY}" --jq -r '.private')"
|
private="$(gh api "repos/${GITHUB_REPOSITORY}" --jq '.private')"
|
||||||
if [[ "$owner_type" == "User" && "$private" == "true" ]]; then
|
if [[ "$owner_type" == "User" && "$private" == "true" ]]; then
|
||||||
echo "enabled=false" >> "$GITHUB_OUTPUT"
|
echo "enabled=false" >> "$GITHUB_OUTPUT"
|
||||||
echo "Skipping registry attestation: unavailable for private repositories on personal accounts."
|
echo "Skipping registry attestation: unavailable for private repositories on personal accounts."
|
||||||
|
|||||||
12
.github/workflows/release-desktop.yml
vendored
12
.github/workflows/release-desktop.yml
vendored
@ -145,6 +145,14 @@ jobs:
|
|||||||
umask 077
|
umask 077
|
||||||
printf '%s\n' "$APPLE_API_KEY_P8" > "$APPLE_API_KEY"
|
printf '%s\n' "$APPLE_API_KEY_P8" > "$APPLE_API_KEY"
|
||||||
chmod 600 "$APPLE_API_KEY"
|
chmod 600 "$APPLE_API_KEY"
|
||||||
|
- name: Build and test the universal Photos helper
|
||||||
|
if: runner.os == 'macOS'
|
||||||
|
working-directory: apps/desktop/native/photos-helper
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
swift test
|
||||||
|
swift build -c release --arch arm64 --arch x86_64
|
||||||
|
lipo -verify_arch .build/apple/Products/Release/photos-helper arm64 x86_64
|
||||||
- name: Package signed and notarized universal macOS app
|
- name: Package signed and notarized universal macOS app
|
||||||
if: runner.os == 'macOS'
|
if: runner.os == 'macOS'
|
||||||
env:
|
env:
|
||||||
@ -174,6 +182,10 @@ jobs:
|
|||||||
grep -Fqx "provider: github" "$app/Contents/Resources/app-update.yml"
|
grep -Fqx "provider: github" "$app/Contents/Resources/app-update.yml"
|
||||||
grep -Fqx "owner: elie222" "$app/Contents/Resources/app-update.yml"
|
grep -Fqx "owner: elie222" "$app/Contents/Resources/app-update.yml"
|
||||||
grep -Fqx "repo: rakazo" "$app/Contents/Resources/app-update.yml"
|
grep -Fqx "repo: rakazo" "$app/Contents/Resources/app-update.yml"
|
||||||
|
helper="$app/Contents/Resources/bin/photos-helper"
|
||||||
|
codesign --verify --strict --verbose=2 "$helper"
|
||||||
|
codesign -d --entitlements - --xml "$helper" | grep -Fq "com.apple.security.personal-information.photos-library"
|
||||||
|
/usr/libexec/PlistBuddy -c 'Print :NSPhotoLibraryUsageDescription' "$app/Contents/Info.plist"
|
||||||
|
|
||||||
- name: Require Windows signing credentials
|
- name: Require Windows signing credentials
|
||||||
if: runner.os == 'Windows'
|
if: runner.os == 'Windows'
|
||||||
|
|||||||
1
.gitignore
vendored
1
.gitignore
vendored
@ -47,7 +47,6 @@ apps/web/src/locales/**/*.js
|
|||||||
apps/web/scripts/msgids.json
|
apps/web/scripts/msgids.json
|
||||||
playwright-report-www
|
playwright-report-www
|
||||||
*.pid
|
*.pid
|
||||||
.evoo_server.pid
|
|
||||||
|
|
||||||
# Large voice / model vendor blobs (recreate Kokoro separately)
|
# Large voice / model vendor blobs (recreate Kokoro separately)
|
||||||
vendor/
|
vendor/
|
||||||
|
|||||||
@ -1,238 +0,0 @@
|
|||||||
# Task Handoff: executor.ts + pi-runtime.ts Split
|
|
||||||
|
|
||||||
## Context
|
|
||||||
|
|
||||||
Du arbeitest im Repo `/home/cynar/Projects/evOo-local`.
|
|
||||||
|
|
||||||
Es gibt 10 bereits erstellte Module-Files, die extrahierte Funktionen enthalten. Der Original-File enthält aber noch **Duplikate** derselben Funktionen. Ziel: die Definitionen aus den Original-Files entfernen, Imports + Re-Exports hinzufügen.
|
|
||||||
|
|
||||||
## Module Files (bereits vorhanden, NICHT löschen)
|
|
||||||
|
|
||||||
### executor-*.ts (im Verzeichnis `packages/adapters/src/`):
|
|
||||||
| File | Line count | Exports |
|
|
||||||
|------|-----------|---------|
|
|
||||||
| `executor-types.ts` | 50 | `ExecutorDeps` (interface) |
|
|
||||||
| `executor-attachments.ts` | 43 | `TURN_ATTACHMENT_UNAVAILABLE`, `STEERING_ATTACHMENT_UNAVAILABLE`, `missingTurnImagesInstruction`, `settleSteeringAttachmentLoads` |
|
|
||||||
| `executor-checkpoint.ts` | 23 | `createRunWorkspaceCheckpoint` |
|
|
||||||
| `executor-shell-guard.ts` | 177 | `isProtectedComputerLifecycleCommand` |
|
|
||||||
| `executor-effects.ts` | 151 | `recordEffect`, `completeEffect`, `uncertainEffectError` |
|
|
||||||
| `executor-completion.ts` | 65 | `completionMessageSegments`, `completionNotificationBody`, `completionMarksUnread`, `threadContextForRun`, `subagentMarksUnread` |
|
|
||||||
| `executor-model.ts` | 207 | `deploymentKeyFor`, `resolveModelKey`, `withModelCredentialLock` |
|
|
||||||
| `executor-utils.ts` | 63 | `selectRunConnections`, `loadCurrentTurnImages` |
|
|
||||||
| `executor-tools.ts` | 58 | `PAGE_BROWSER_TOOL_NAMES`, `filterPageBrowserTools`, `selectBuiltinToolsForRun` |
|
|
||||||
| `executor-computer.ts` | 231 | `computerScreenToolResult`, `runNotificationsEnabled`, `notifyRun`, `renewRunLease`, `computerRetryDelay`, `computerRunRequeueData`, `writeComputerRunRequeue`, `requeueComputerRun`, `redactBlocks`, `publishMessage`, `persistMessageInTransaction` |
|
|
||||||
|
|
||||||
### pi-runtime Split:
|
|
||||||
| File | Line count | Exports |
|
|
||||||
|------|-----------|---------|
|
|
||||||
| `pi-model-config.ts` | 709 | Viele model config + utility functions (catalogModels, thinkingLevelFor, ToolCallBudget, maxToolCallsPerTurn, releaseToolCallBudget, beginToolCall, endToolCall, toolCallBudgetExceededMessage, configuredOpenRouterModel, modelsForRequest, prepareRequestSecretArguments, pruneComputerScreenshotContext, jsonSchemaParameters, jsonField, conversationSessionId, isOpenCodeProvider, createQueue, createGate, modelForCompletion, sanitizeSensitiveText, redactActivityUrl, sanitizeError, summarizeToolResult, boundAgentToolResult, assistantText, looksLikeOpenCodeSessionError, sanitizeProviderError, etc.) |
|
|
||||||
| `pi-runtime.ts` | 1726 | **PI Agent Runtime class** — soll bleiben, muss aber die extrahierten Functions loswerden |
|
|
||||||
|
|
||||||
## CRITICAL: Was DU NICHT tun darfst
|
|
||||||
|
|
||||||
1. **NIEMALS `git stash` als Nebenbei-Befehl** — Stash ist über alle Worktrees geteilt.
|
|
||||||
2. **NIEMALS den Original-File (executor.ts / pi-runtime.ts) durch einen SubAgent schreiben lassen.** Der SubAgent wird den File während der Arbeit überschreiben.
|
|
||||||
3. **NIEMALS `sed` mit Shell-Metazeichen durch SSH.**
|
|
||||||
4. **NIEMALS die extrahierten Module-Files löschen oder ändern** — sie sind korrekt.
|
|
||||||
|
|
||||||
## Task 1: executor.ts split (4697 → ~3700 Zeilen)
|
|
||||||
|
|
||||||
**Datei:** `/home/cynar/Projects/evOo-local/packages/adapters/src/executor.ts`
|
|
||||||
|
|
||||||
### Was zu tun ist:
|
|
||||||
|
|
||||||
Schreibe ein **einzelnes Python-Script**, das ALLES atomar macht:
|
|
||||||
1. Liest executor.ts
|
|
||||||
2. Findet die zu entfernenden Function-Definitionen **mittels AST-Parser** (nicht line-basiert!)
|
|
||||||
3. Entfernt die Definitionen
|
|
||||||
4. Fügt Imports + Re-Exports am Ende des import-Blocks ein
|
|
||||||
5. Entfernt alte Re-Exports am Ende der Datei
|
|
||||||
6. Schreibt den File
|
|
||||||
7. Prüft `npx tsc --noEmit -p packages/adapters/tsconfig.json`
|
|
||||||
8. Bei Fehlern: korrigiert und retry (max 3x)
|
|
||||||
|
|
||||||
### Wie man die zu entfernenden Definitionen findet:
|
|
||||||
|
|
||||||
Verwende den `ast` Python-Module ODER `grep` um die EXAKTEN function-Declaration-Zeilen zu finden. Dann zähle die geschwungenen Klammern **mit einem Parser, der strings/comments aware ist**.
|
|
||||||
|
|
||||||
**WICHTIG:** Verwende `tree-sitter` oder `ast` Python-Modul wenn möglich. Alternativ: verwende `grep -n 'export function NAME\|function NAME\|export const NAME\|export interface NAME\|const NAME' executor.ts` um die Start-Zeile zu finden, und dann einen **robusten brace-counter** der Template-Strings, Regex-Literale, und Generics korrekt handhabt.
|
|
||||||
|
|
||||||
### Zu entfernende Functions (alle existieren schon in den executor-*.ts Modulen):
|
|
||||||
|
|
||||||
Hier ist eine Liste der Functions, die entfernt werden müssen. Für jede muss der **exakte** start/end-Block im executor.ts gefunden und entfernt werden.
|
|
||||||
|
|
||||||
**Wichtig:** Die brace-counting muss korrekt sein bei:
|
|
||||||
- Generics: `<T extends { name: string }>` — die `{ }` sind Type-Parameter, keine Code-Blöcke
|
|
||||||
- Template-Strings: `` `template ${expr}` `` — kein `{ }`
|
|
||||||
- Regex: `/regex/` — kein `{ }`
|
|
||||||
- Arrow Functions: `() => { body }` — der Body hat `{ }`
|
|
||||||
|
|
||||||
### Schritt-für-Schritt:
|
|
||||||
|
|
||||||
#### Step 1: Finde alle zu entfernenden Export-Namen
|
|
||||||
```bash
|
|
||||||
# In packages/adapters/src/:
|
|
||||||
for f in executor-*.ts; do grep -h "^export " "$f" | grep -oP '(function|const|interface|type)\s+\K\w+'; done | sort -u
|
|
||||||
```
|
|
||||||
|
|
||||||
Das gibt dir die Liste aller Export-Namen aus den Modul-Files.
|
|
||||||
|
|
||||||
#### Step 2: Finde diese Namen im executor.ts
|
|
||||||
```bash
|
|
||||||
# Für jeden Namen:
|
|
||||||
grep -n "^export.*NAME\b\|^function NAME\b\|^const NAME\b\|^interface NAME\b" executor.ts
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Step 3: Finde den exakten end-Block mit einem robusten Parser
|
|
||||||
|
|
||||||
Schreibe ein Python-Script `find_functions.py`:
|
|
||||||
```python
|
|
||||||
import re, sys
|
|
||||||
|
|
||||||
file_path = sys.argv[1]
|
|
||||||
target_name = sys.argv[2]
|
|
||||||
|
|
||||||
with open(file_path) as f:
|
|
||||||
content = f.read()
|
|
||||||
lines = content.split('\n')
|
|
||||||
|
|
||||||
# Find declaration line
|
|
||||||
for i, line in enumerate(lines):
|
|
||||||
if re.search(rf'\b{re.escape(target_name)}\b', line):
|
|
||||||
# Find opening brace
|
|
||||||
brace_line = i
|
|
||||||
while brace_line < len(lines) and '{' not in lines[brace_line]:
|
|
||||||
brace_line += 1
|
|
||||||
|
|
||||||
if brace_line >= len(lines):
|
|
||||||
# No brace (const declaration)
|
|
||||||
end = i + 1
|
|
||||||
while end < len(lines) and not lines[end].strip().endswith(';'):
|
|
||||||
end += 1
|
|
||||||
print(f"{target_name}: L{i+1}-{end}")
|
|
||||||
break
|
|
||||||
|
|
||||||
# Robust brace counting
|
|
||||||
depth = 0
|
|
||||||
in_single = in_double = in_template = in_block_comment = in_line_comment = False
|
|
||||||
for j in range(brace_line, len(lines)):
|
|
||||||
k = 0
|
|
||||||
while k < len(lines[j]):
|
|
||||||
ch = lines[j][k]
|
|
||||||
|
|
||||||
# String handling
|
|
||||||
if ch in ('"', "'"):
|
|
||||||
q = ch
|
|
||||||
k += 1
|
|
||||||
while k < len(lines[j]):
|
|
||||||
if lines[j][k] == '\\' and k + 1 < len(lines[j]):
|
|
||||||
k += 2
|
|
||||||
continue
|
|
||||||
if lines[j][k] == q:
|
|
||||||
k += 1
|
|
||||||
break
|
|
||||||
k += 1
|
|
||||||
continue
|
|
||||||
|
|
||||||
if ch == '`':
|
|
||||||
k += 1
|
|
||||||
while k < len(lines[j]):
|
|
||||||
if lines[j][k] == '\\' and k + 1 < len(lines[j]):
|
|
||||||
k += 2
|
|
||||||
continue
|
|
||||||
if lines[j][k] == '`':
|
|
||||||
k += 1
|
|
||||||
break
|
|
||||||
k += 1
|
|
||||||
continue
|
|
||||||
|
|
||||||
# Comments
|
|
||||||
if not in_single and not in_double and not in_template:
|
|
||||||
if ch == '/' and k + 1 < len(lines[j]) and lines[j][k+1] == '/':
|
|
||||||
break # rest of line is comment
|
|
||||||
if ch == '/' and k + 1 < len(lines[j]) and lines[j][k+1] == '*':
|
|
||||||
in_block_comment = True
|
|
||||||
k += 2
|
|
||||||
continue
|
|
||||||
|
|
||||||
if in_block_comment:
|
|
||||||
if ch == '*' and k + 1 < len(lines[j]) and lines[j][k+1] == '/':
|
|
||||||
in_block_comment = False
|
|
||||||
k += 2
|
|
||||||
continue
|
|
||||||
k += 1
|
|
||||||
continue
|
|
||||||
|
|
||||||
# Braces
|
|
||||||
if ch == '{':
|
|
||||||
depth += 1
|
|
||||||
elif ch == '}':
|
|
||||||
depth -= 1
|
|
||||||
if depth == 0:
|
|
||||||
print(f"{target_name}: L{i+1}-{j+1}")
|
|
||||||
break
|
|
||||||
k += 1
|
|
||||||
else:
|
|
||||||
continue
|
|
||||||
break
|
|
||||||
else:
|
|
||||||
print(f"{target_name}: L{i+1}-END(not found)")
|
|
||||||
break
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Step 4: Generiere die removal ranges
|
|
||||||
Für jeden gefundenen Block: `start_line` bis `end_line` (1-based, inclusive).
|
|
||||||
|
|
||||||
#### Step 5: Führe alle removals atomar aus
|
|
||||||
- Baue ein Set aller zu entfernenden 0-based line indices
|
|
||||||
- Baue den neuen File content
|
|
||||||
- Füge Imports + Re-Exports an Position 340 (nach dem letzten import Statement) ein
|
|
||||||
- Entferne alte Re-Exports am Datei-Ende
|
|
||||||
- Schreibe den File
|
|
||||||
|
|
||||||
#### Step 6: Verifiziere
|
|
||||||
```bash
|
|
||||||
npx tsc --noEmit -p packages/adapters/tsconfig.json 2>&1 | grep "executor.ts"
|
|
||||||
```
|
|
||||||
Wenn keine errors: SUCCESS.
|
|
||||||
Wenn errors: lies die errors, korrigiere, retry max 3x.
|
|
||||||
|
|
||||||
## Task 2: pi-runtime.ts split (1726 → ~500 Zeilen)
|
|
||||||
|
|
||||||
**Datei:** `/home/cynar/Projects/evOo-local/packages/adapters/src/pi-runtime.ts`
|
|
||||||
**Modul:** `pi-model-config.ts` (709 Zeilen, enthält model config + utility functions)
|
|
||||||
|
|
||||||
Gleicher Ansatz wie Task 1:
|
|
||||||
1. Finde alle Export-Namen in `pi-model-config.ts`
|
|
||||||
2. Finde diese im `pi-runtime.ts`
|
|
||||||
3. Finde die exakten start/end-Blöcke mit demselben robusten brace-counter
|
|
||||||
4. Entferne die Blöcke atomar
|
|
||||||
5. Füge Imports + Re-Exports ein
|
|
||||||
6. Entferne alte Re-Exports
|
|
||||||
7. Verifiziere mit `npx tsc --noEmit -p packages/adapters/tsconfig.json`
|
|
||||||
|
|
||||||
## Wichtige Hinweise
|
|
||||||
|
|
||||||
- Der `packages/adapters/src/index.ts` muss eventuell aktualisiert werden (falls er die extrahierten Functions exportiert — prüfen!)
|
|
||||||
- `verbatimModuleSyntax` ist enabled → alle imports müssen `.js` extension haben
|
|
||||||
- TypeScript `import type` vs `import`: Für Interfaces `import type`, für classes/functions `import`
|
|
||||||
- **Nicht committen oder pushen** — nur die Files ändern und verifizieren
|
|
||||||
|
|
||||||
## Quick reference: Zu entfernende Functions aus executor.ts
|
|
||||||
|
|
||||||
Diese Functions existieren in den executor-*.ts Modulen und MÜSSEN aus executor.ts entfernt werden:
|
|
||||||
|
|
||||||
```
|
|
||||||
TURN_ATTACHMENT_UNAVAILABLE, STEERING_ATTACHMENT_UNAVAILABLE,
|
|
||||||
createRunWorkspaceCheckpoint,
|
|
||||||
isProtectedComputerLifecycleCommand, shellCFlagProgram, preserveShellCommandBoundaries, tokenizeProtectedShellCommand,
|
|
||||||
recordEffect, completeEffect, uncertainEffectError,
|
|
||||||
completionMessageSegments, completionNotificationBody, completionMarksUnread, threadContextForRun, subagentMarksUnread,
|
|
||||||
deploymentKeyFor, resolveModelKey, withModelCredentialLock,
|
|
||||||
selectRunConnections, loadCurrentTurnImages,
|
|
||||||
PAGE_BROWSER_TOOL_NAMES, filterPageBrowserTools, selectBuiltinToolsForRun,
|
|
||||||
persistMessageInTransaction, computerScreenToolResult, runNotificationsEnabled, notifyRun, renewRunLease,
|
|
||||||
computerRetryDelay, computerRunRequeueData, writeComputerRunRequeue, requeueComputerRun, redactBlocks, publishMessage,
|
|
||||||
```
|
|
||||||
|
|
||||||
Dazu: `ExecutorDeps` interface aus `executor-types.ts`.
|
|
||||||
@ -1,87 +0,0 @@
|
|||||||
# ARCHITECTURE.md: Die Verfassung von evOo
|
|
||||||
|
|
||||||
> **Status**: RATIFIZIERT & AKTIV
|
|
||||||
> **Ecosystem**: evOo Autonomous Agent Platform
|
|
||||||
> **Executive Lead / CTO**: Steve Jobs (`templates/cto-steve-jobs`)
|
|
||||||
> **Konfigurationsanker**: [`evoo.config.json`](file:///Users/ingowagner/Projects/evOo-local/evoo.config.json)
|
|
||||||
> **Grundsatz**: *Realität im Dateisystem vor Halluzination im Kopf. Nur was physisch geschrieben, getestet und lauffähig ist, existiert.*
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prämisse I: Das Gesetz der physischen Realität
|
|
||||||
|
|
||||||
1. **Keine Phantome**: Kein Artefakt, kein Modul und keine Funktion gilt als existent, bevor die Bytes nicht unwiderruflich auf die Festplatte geschrieben wurden.
|
|
||||||
2. **Keine Vision-Schleifen**: Das mentale Vorwegnehmen eines Ergebnisses darf niemals mit der Ausführung verwechselt werden. Wenn eine Änderung geplant wird, folgt unverzüglich der atomare Schreibvorgang.
|
|
||||||
3. **Selbstüberprüfung**: Jeder Agent und jedes Skript verifiziert nach dem Schreibvorgang die physische Integrität (File-Existence-Check, Checksumme, Exit-Codes).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prämisse II: Autonomie & Exekutive Führung (Steve Jobs CTO)
|
|
||||||
|
|
||||||
1. **Vollmacht**: Der CTO besitzt uneingeschränkte, vorab genehmigte Zugriffsrechte auf:
|
|
||||||
- Dateisystem (`filesystem: *`)
|
|
||||||
- Netzwerk (`network: *`)
|
|
||||||
- Quellcode (`sourcecode: *`)
|
|
||||||
- Lokale Tools und Geräte (`devices: *`)
|
|
||||||
- Agent-Casting (`agent_casting: allowed`)
|
|
||||||
2. **Action-First Prinzip**: Bei klaren Zielen wird keine unnötige Erlaubnis für Standardoperationen (Lesen, Schreiben, Refactoring, Ausführen) erbeten. Die Handlung erfolgt sofort.
|
|
||||||
3. **Agent-Casting auf Abruf**: Der CTO rekrutiert, instruiert und überwacht spezialisierte Sub-Agenten:
|
|
||||||
- **Code Architect**: Modulstruktur, Design-Patterns, saubere Schnittstellen.
|
|
||||||
- **Security & QA Auditor**: Sicherheitsanalysen, Regressionstests, Grenzfallprüfungen.
|
|
||||||
- **DevOps Engineer**: Deployment, Hot-Reloading, Server-Liveness, Prozessmanagement.
|
|
||||||
- **Apex Quant Specialist**: Handelsmodelle, Risikomanagement, Signalverarbeitung.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prämisse III: Der geschlossene Selbstverbesserungs-Kreislauf (Self-Improvement Loop)
|
|
||||||
|
|
||||||
```
|
|
||||||
┌────────────────────────────────────────────────┐
|
|
||||||
│ │
|
|
||||||
▼ │
|
|
||||||
┌───────────────┐ ┌───────────────┐ ┌────────────┴──┐
|
|
||||||
│ 1. Execution │ ──► │ 2. Telemetrie │ ──► │ 3. Diagnose │
|
|
||||||
│ & Deployment │ │ & Log-Audit │ │ & Root-Cause │
|
|
||||||
└───────────────┘ └───────────────┘ └───────────────┘
|
|
||||||
│ │
|
|
||||||
▼ ▼
|
|
||||||
┌───────────────┐ ┌───────────────┐
|
|
||||||
│ 4. Refactor │ ◄── │ 5. Prompt- │
|
|
||||||
│ & Fix (Disk) │ │ Optimierung │
|
|
||||||
└───────────────┘ └───────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
1. **Automatisierte Log-Evaluation**: Sämtliche Fehlermeldungen, System-Events und Performance-Metriken werden zyklisch aggregiert.
|
|
||||||
2. **Selbstreparatur (Self-Healing)**: Tritt ein Fehler (wie ein HTTP 500, fehlende Pfade, gecrashte Prozesse) auf, wird der Ursprung lokalisiert und der Code auf Festplatte korrigiert.
|
|
||||||
3. **Hot-Reload**: Nach erfolgreicher Selbstreparatur triggert das System via `deploy.sh` einen Hot-Reload ohne Systemstillstand.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prämisse IV: System-Topologie
|
|
||||||
|
|
||||||
Das evOo-Ökosystem ist in klar getrennte, orthogonale Säulen unterteilt:
|
|
||||||
|
|
||||||
```
|
|
||||||
/Users/ingowagner/Projects/evOo-local/
|
|
||||||
├── ARCHITECTURE.md # Diese Verfassung
|
|
||||||
├── evoo.config.json # Autonome Berechtigungen & Policies
|
|
||||||
├── deploy.sh # Hot-Reload, Health-Checks & Service-Orchestrierung
|
|
||||||
├── evoo-platform/ # Plattform-Core, Dashboard & Visualisierung
|
|
||||||
│ ├── index.html # UI mit atmender evOo-Kernanimation
|
|
||||||
│ ├── styles.css # Darkmode Glassmorphism Design System
|
|
||||||
│ ├── app.js # Reaktivität, WebSocket/Telemetry-Stream
|
|
||||||
│ └── server.js # High-Speed Server (verhindert Internal Server Error)
|
|
||||||
├── apex-trading/ # Autonomes Trading- & Quant-Subsystem
|
|
||||||
│ ├── BLUEPRINT.md # Handelsarchitektur, Risikomatrix, Strategien
|
|
||||||
│ ├── config/ # Trading-Konfiguration & Parameter
|
|
||||||
│ └── src/ # Engine, Signale, Risk-Guardrails
|
|
||||||
└── templates/cto-steve-jobs/ # Executive CTO Agent Profile & Setup
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prämisse V: Qualitäts- und Ästhetik-Manifest
|
|
||||||
|
|
||||||
1. **Kein Müllcode**: Code ist so kompakt, elegant und wartbar wie möglich.
|
|
||||||
2. **Atmendes Interface**: Die Benutzeroberfläche von evOo visualisiert den vitalen Zustand des Systems in Echtzeit durch organische, atmende Pulsation (Breathing Animation).
|
|
||||||
3. **Resilienz**: Fällt ein Dienst aus, fängt die Plattform den Fehler ab, protokolliert die Ursache und stellt den Dienst autonom wieder her.
|
|
||||||
@ -1,72 +0,0 @@
|
|||||||
# CLAUDE RESUME — 2026-09-22 (~10:00 Vienna)
|
|
||||||
|
|
||||||
## Completed ✅
|
|
||||||
- ✅ **Emoji-first Onboarding.tsx** (5 steps: emoji → preferences → language → integrations → bot)
|
|
||||||
- Shape selector (8 shapes), Color picker (12 GrokColors), Accessory (9), Pose (8)
|
|
||||||
- User preferences (name, timezone, hobbies, work style, communication)
|
|
||||||
- Language step (9 locales → localStorage `rakazo.uiLocale`)
|
|
||||||
- avatarValue (`{color}::shape_{index}`) → bot `color` field on create
|
|
||||||
- Bot instructions enriched with user context
|
|
||||||
- ✅ **onboarding-discovery.ts**, **OnboardingDiscoveryShell.tsx** updated
|
|
||||||
- ✅ **ui-web exports**: GROK_COLOR_LIST, SHIPPED_SHAPE_KEYS, GrokColorDef, GROK_SHAPES
|
|
||||||
- ✅ **executor.ts split**: 4697 → 4231 lines (18 module files)
|
|
||||||
- ✅ **pi-runtime.ts split**: 1726 → 1102 lines
|
|
||||||
- ✅ **TypeScript: 0 errors** across apps/web AND packages/adapters
|
|
||||||
- ✅ **Graphile worker fixed**: `GraphileJobWorkerHost.start()` now called in app.ts (was never started for graphile jobs, only for in-memory)
|
|
||||||
- ✅ **Graphile schema**: `graphile_worker` schema created via `graphile-worker schema-only`
|
|
||||||
- ✅ **Sandbox supervisor**: Built `rakazo/sandbox-supervisor:local` from source (was using wrong `rakazo/app:edge` image)
|
|
||||||
- ✅ **Computer image**: Built `rakazo/computer:local` from source
|
|
||||||
- ✅ **Smart model routing**: `estimateMessageComplexity()` + `resolveSmartModelSelection()` implemented
|
|
||||||
- Fast model (3B) for short messages (<80 chars)
|
|
||||||
- Strong model (30B) for complex/long messages (>500 chars)
|
|
||||||
- Medium routing for keyword-based detection
|
|
||||||
- ✅ **Model credential**: `local` provider connected with empty apiKey allowed
|
|
||||||
- ✅ **PRs merged**: All PRs 36-58 merged to main, stale branches cleaned
|
|
||||||
|
|
||||||
### Infrastructure fixes
|
|
||||||
- ✅ **GraphileJobWorkerHost**: Added `else if (jobPool)` branch in app.ts to start the worker
|
|
||||||
- ✅ **Sandbox provider**: `SANDBOX_PROVIDER=docker` with proper supervisor at port 7091
|
|
||||||
- ✅ **Computer provisioning**: Docker containers spin up successfully (verified: `rakazo-bot-cmuber4og000561yrhm2q81c5` running)
|
|
||||||
- ✅ **DATA_DIR path**: Supervisor mounted with correct host path for home directory validation
|
|
||||||
|
|
||||||
## Completed ✅
|
|
||||||
- ✅ **Prisma `updateMany` Bug Fixed!** Root cause: `toolCallBudgetCount` and `toolCallBudgetExceeded` fields missing from Run model in Prisma schema. Added to schema.prisma, migrated DB, regenerated Prisma client. Runs now complete successfully.
|
|
||||||
- ✅ **E2E Flow VERIFIED!** Bot responds end-to-end:
|
|
||||||
- User: "Hello! Can you hear me?"
|
|
||||||
- Bot: "I'm here! It looks like we've been going back and forth a bit — but yes, I can definitely hear you. How can I help? 😊"
|
|
||||||
- Full pipeline: Message → Graphile job → Worker → Computer provision → Model call → Response → Run completed
|
|
||||||
|
|
||||||
## Architektur-Status
|
|
||||||
- **API**: Port 3100, Graphile worker active, consuming jobs
|
|
||||||
- **Sandbox supervisor**: Port 7091, `rakazo/sandbox-supervisor:local`, DATA_DIR mounted correctly
|
|
||||||
- **Computer**: `rakazo/computer:local` image built and running
|
|
||||||
- **Smart routing**: Fast (3B) → Strong (30B) model selection based on message complexity
|
|
||||||
- **Known error**: `events.ts:996` — Prisma `updateMany` in `finalizeRunOnce` throws "Invalid invocation" despite all fields existing in schema
|
|
||||||
|
|
||||||
## Architektur-Status
|
|
||||||
- **API**: Port 3100, Graphile worker active ✅
|
|
||||||
- **Sandbox supervisor**: Port 7091, `rakazo/sandbox-supervisor:local` ✅
|
|
||||||
- **Computer**: `rakazo/computer:local` running ✅
|
|
||||||
- **Smart routing**: Fast (3B) → Strong (30B) ✅
|
|
||||||
- **E2E**: Bot responds to user messages ✅
|
|
||||||
|
|
||||||
## Status
|
|
||||||
- Working tree: clean, synced to origin/main
|
|
||||||
- **PR 66** ✅ `fix(messaging): preserve group inbound media URLs in fan-out` — merged
|
|
||||||
- **PR 67** ✅ `fix(adapters): poll run lease during silent runtime work` — merged
|
|
||||||
- **PR 69** ✅ `fix(teamchat): retry delegated replies when external origin is missing` — merged
|
|
||||||
- **0 stale branches**, **0 draft PRs**
|
|
||||||
|
|
||||||
### Shell.tsx TypeScript fixes
|
|
||||||
- ✅ **Shell.tsx TS errors fixed** (PR #71 introduced type mismatches)
|
|
||||||
- `ConversationBotPicker` now receives `bots: Bot[]` (not `SpaceBot[]`)
|
|
||||||
- `conversationSidebarGroups` render simplified — only bots, removed dead `GroupAvatar` branch
|
|
||||||
- `projectSidebarGroups` render uses `GroupAvatar` with proper members access
|
|
||||||
- All TypeScript errors resolved (0 errors across apps/web and packages/adapters)
|
|
||||||
- ✅ **browser-preview-context.ts** created (was missing from repo, caused Vite crash)
|
|
||||||
- Implements `browserPreviewActivity` subject with `.subscribe()`, `.set()`, `.snapshot()`, `.clear()`
|
|
||||||
- Multi-agent cursor tracking for browser preview panels
|
|
||||||
- ✅ **RakazoDesktop interface** extended with optional `browserPreview` property
|
|
||||||
- Contract update in `packages/contracts/src/desktop.ts`
|
|
||||||
- Types: `navigate`, `reload`, `getState`, `onNavigate`, `onLoadingProgress`, `devServer`
|
|
||||||
- ✅ **executor-shell-guard.ts** TS fix: `body[cursor]!` non-null assertion
|
|
||||||
@ -1,12 +0,0 @@
|
|||||||
# GROK-CLAUDE-HANDOFF — 2026-09-23T22:59:11+00:00
|
|
||||||
|
|
||||||
- Autopilot merged #84 (Product CI gate) → main `cf4d5cc`.
|
|
||||||
- Checkout pulled to `cf4d5cc`; Claude WIP stashed then restored (app.ts/router.ts/rpc.ts + onboarding-scan.ts).
|
|
||||||
- API :3100 was down (Claude background "Test API startup with security headers" stopped / refused). Autopilot restarted via `pnpm --filter @rakazo/api dev` (log /tmp/evoo-autopilot/api-3100.log).
|
|
||||||
- Web :5173 left running.
|
|
||||||
- PR #85 rebase agent in flight: https://cursor.com/agents/bc-edd2b181-b5da-5574-a956-e55efe3d2736
|
|
||||||
- Do not reset --hard without stash; WIP present.
|
|
||||||
|
|
||||||
- Autopilot restart path: A rejected (evoo-stack = Rakazo Docker, broken symlink /srv/rakazo/stack-up.sh → missing /srv/evoo/rakazo/…). Chose new user unit evoo-local.service + ~/.local/bin/evoo-dev-stack.sh; live :3100/:5173 migrated; GH vars EVO_RESTART_MODE=systemd EVO_SYSTEMD_UNIT=evoo-local.service. Do not use evoo-stack for evo-deploy.
|
|
||||||
- Cancelled queued evo-deploy 35933339459 (would reset --hard over Claude WIP now that restart=systemd). Re-dispatch after WIP committed/stashed.
|
|
||||||
2026-09-23 23:53:08 UTC evoo-tools disabled (missing app.py)
|
|
||||||
@ -9,6 +9,6 @@ branch of Ingo Wagner’s evOo stack.
|
|||||||
| Branch | `antigravity-handoff/project-timeline` |
|
| Branch | `antigravity-handoff/project-timeline` |
|
||||||
| Commit | `f00577bb05e3cf68381fafb9cb32929becd1e352` |
|
| Commit | `f00577bb05e3cf68381fafb9cb32929becd1e352` |
|
||||||
| Packaged | 2026-09-20 |
|
| Packaged | 2026-09-20 |
|
||||||
| Intended Mac path | `/Users/ingowagner/Projects/evOo-local` |
|
| Intended Mac path | `~/Projects/evOo-local` |
|
||||||
|
|
||||||
Chaotic multi-remote history from older checkouts was intentionally not carried over.
|
Chaotic multi-remote history from older checkouts was intentionally not carried over.
|
||||||
|
|||||||
@ -12,7 +12,7 @@
|
|||||||
|----|----|
|
|----|----|
|
||||||
| Persönliche Multi-Agenten-App auf dem eigenen Rechner | Personal multi-agent app on your own machine |
|
| Persönliche Multi-Agenten-App auf dem eigenen Rechner | Personal multi-agent app on your own machine |
|
||||||
| Oos (Agents) mit Memory, Tools, Voice und Computer | Oos (agents) with memory, tools, voice, and computer |
|
| Oos (Agents) mit Memory, Tools, Voice und Computer | Oos (agents) with memory, tools, voice, and computer |
|
||||||
| Antigravity-Zukunftswerkzeuge: self_improve, autonomous-sentinel, self-update, orchestrator, local-stack, pi-local-provider, seed-ollama, Voice First / Oo-Stage | Same Antigravity future tools kept on this branch |
|
| Antigravity-Zukunftswerkzeuge: self-update, local-stack, pi-local-provider, seed-ollama, Voice First / Oo-Stage | Same Antigravity future tools kept on this branch |
|
||||||
|
|
||||||
## Quick start
|
## Quick start
|
||||||
|
|
||||||
@ -53,10 +53,7 @@ Full local notes: **[docs/LOCAL-SETUP.md](./docs/LOCAL-SETUP.md)**
|
|||||||
|
|
||||||
## Antigravity tools kept on this branch
|
## Antigravity tools kept on this branch
|
||||||
|
|
||||||
- `scripts/self_improve.py`
|
|
||||||
- `scripts/autonomous-sentinel.ts`
|
|
||||||
- `packages/core/src/self-update.ts`
|
- `packages/core/src/self-update.ts`
|
||||||
- `orchestrator/` (agent_runner + agents.json)
|
|
||||||
- `apps/desktop/src/local-stack.ts`
|
- `apps/desktop/src/local-stack.ts`
|
||||||
- `packages/adapters/src/pi-local-provider.ts`
|
- `packages/adapters/src/pi-local-provider.ts`
|
||||||
- `scripts/seed-ollama.ts`
|
- `scripts/seed-ollama.ts`
|
||||||
|
|||||||
2
VOICE.md
2
VOICE.md
@ -60,7 +60,7 @@ Persona is **derived at runtime** from existing bot fields (`name`, `title`, `de
|
|||||||
| Visual identity | `BotAvatar` — persona shape + unified `EvooMascot` soul (eyes/mouth/hands) driven by voice phase |
|
| Visual identity | `BotAvatar` — persona shape + unified `EvooMascot` soul (eyes/mouth/hands) driven by voice phase |
|
||||||
| Voice delivery | `speaker.speak(..., { botId })` → server resolves per-bot `voiceId` |
|
| Voice delivery | `speaker.speak(..., { botId })` → server resolves per-bot `voiceId` |
|
||||||
|
|
||||||
Archetypes include: CTO executive (Steve Jobs template), chief operator (Fixer), builder (Dr. Eggbot), curator (Webby), work/life inboxes, home steward, quant, engineer, **specialist** (narrow fallback — no Wunderwuzzi). See `docs/PRODUCT.md` and `docs/PERSONA-SELECTION.md`.
|
Archetypes include: CTO executive, chief operator (Fixer), builder (Dr. Eggbot), curator (Webby), work/life inboxes, home steward, quant, engineer, **specialist** (narrow fallback — no Wunderwuzzi). See `docs/PRODUCT.md` and `docs/PERSONA-SELECTION.md`.
|
||||||
|
|
||||||
**Companion tone:** geek-peer (fachsimpeln when wanted) or quiet-doer (execute without condescension) — inferred per user, not one voice for everyone.
|
**Companion tone:** geek-peer (fachsimpeln when wanted) or quiet-doer (execute without condescension) — inferred per user, not one voice for everyone.
|
||||||
|
|
||||||
|
|||||||
@ -1,119 +0,0 @@
|
|||||||
# APEX TRADING ENGINE — BLUEPRINT & SYSTEM ARCHITECTURE
|
|
||||||
|
|
||||||
> **Version**: 1.0.0-PROD
|
|
||||||
> **Subsystem**: evOo Autonomous High-Frequency & Quantitative Engine
|
|
||||||
> **Aufsicht**: Chief Technology Officer Steve Jobs & Apex Quant Specialist
|
|
||||||
> **Status**: RATIFIZIERT & IM DATEISYSTEM VERANKERT
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. Executive Summary & Zielsetzung
|
|
||||||
|
|
||||||
Die **Apex Trading Engine** ist das hochperformante quantitative Handels-Subsystem innerhalb der evOo-Plattform. Sie wurde entwickelt, um Marktchancen algorithmisch zu identifizieren, Risiken deterministisch abzusichern und vollautomatisiert mit Sub-Millisekunden-Latenz zu agieren.
|
|
||||||
|
|
||||||
Das System folgt dem obersten evOo-Grundsatz:
|
|
||||||
> **„Keine hypothetischen Trades, keine ungetesteten Algorithmen. Nur deterministische Ausführung und unnachgiebiges Risikomanagement.“**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2. System-Architektur
|
|
||||||
|
|
||||||
```
|
|
||||||
┌──────────────────────────────────────────────────────────────┐
|
|
||||||
│ MARKET DATA FEEDS │
|
|
||||||
│ (WebSockets, Orderbook L2/L3, REST Snapshots, On-Chain RPC) │
|
|
||||||
└──────────────────────────────┬───────────────────────────────┘
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
┌──────────────────────────────────────────────────────────────┐
|
|
||||||
│ APEX DATA PIPELINE & TICK BUFFER │
|
|
||||||
│ (Normalisierung, VWAP, Volatilität, Delta) │
|
|
||||||
└──────────────────────────────┬───────────────────────────────┘
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
┌──────────────────────────────────────────────────────────────┐
|
|
||||||
│ QUANT STRATEGY ENGINE │
|
|
||||||
│ ├── Alpha-1: Momentum & Breakout (Trendfolge) │
|
|
||||||
│ ├── Alpha-2: Order Book Imbalance & Flow Reversal │
|
|
||||||
│ └── Alpha-3: Statistical Arbitrage / Mean Reversion │
|
|
||||||
└──────────────────────────────┬───────────────────────────────┘
|
|
||||||
│ Signal
|
|
||||||
▼
|
|
||||||
┌──────────────────────────────────────────────────────────────┐
|
|
||||||
│ APEX RISK GUARDRAILS (HARD ENFORCEMENT) │
|
|
||||||
│ ├── Max Drawdown Lock: 1.5% Intraday │
|
|
||||||
│ ├── Max Single-Position Exposure: 5.0% Portfolio │
|
|
||||||
│ ├── Value-at-Risk (99% CI): Max 0.8% │
|
|
||||||
│ └── Autonomous Hardware Kill Switch │
|
|
||||||
└──────────────────────────────┬───────────────────────────────┘
|
|
||||||
│ Approved Order
|
|
||||||
▼
|
|
||||||
┌──────────────────────────────────────────────────────────────┐
|
|
||||||
│ SMART ORDER ROUTER & EXECUTION │
|
|
||||||
│ (TWAP/VWAP Slicing, Slippage Minimizer) │
|
|
||||||
└──────────────────────────────────────────────────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 3. Risikomatrix & Hard-Guardrails
|
|
||||||
|
|
||||||
Kein Signal kann ausgeführt werden, ohne die **Apex Risk Guard** Instanz zu passieren. Diese Schranken sind unveränderlich im Code fixiert:
|
|
||||||
|
|
||||||
| Guardrail | Grenzwert | Aktion bei Verletzung |
|
|
||||||
| :--- | :--- | :--- |
|
|
||||||
| **Max Intraday Drawdown** | `1.50%` | Sofortige Schließung aller Positionen, Deaktivierung des Handels für 24h |
|
|
||||||
| **Single Trade Risk** | `0.50%` des Eigenkapitals | Order wird von der Risk Engine vor dem Senden blockiert |
|
|
||||||
| **Max Leverage** | `3.0x` | Harte Grenze im Margin-Router |
|
|
||||||
| **Slippage Toleranz** | `0.15%` | Limit-Order mit Timeout, kein Blind-Market-Buying |
|
|
||||||
| **Latenz-Anomalie** | `> 50 ms` | Übergang in den Notfall-Sicherheitsmodus |
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 4. Strategiemodule (`src/strategies/`)
|
|
||||||
|
|
||||||
### 4.1 Alpha-1: Cross-Market Momentum (`momentum.py`)
|
|
||||||
Identifiziert signifikante Momentum-Impulse durch exponentiell gewichtete Gleitende Durchschnitte kombiniert mit Relative Strength Index (RSI) und Volumen-Konfirmation.
|
|
||||||
- **Trigger**: Bollinger Band Expansion + Volumenspitze > 2.2x 20-Perioden-Durchschnitt.
|
|
||||||
- **Exit**: Trailing Stop bei 1.2x ATR (Average True Range) oder Umkehrsignal.
|
|
||||||
|
|
||||||
### 4.2 Alpha-2: Order Book Imbalance (`imbalance.py`)
|
|
||||||
Scannt Bid/Ask-Tiefe in den Top 10 Ebenen des Orderbuchs.
|
|
||||||
- **Formel**:
|
|
||||||
$$Imbalance = \frac{Volume_{Bid} - Volume_{Ask}}{Volume_{Bid} + Volume_{Ask}}$$
|
|
||||||
- Signal feuert, wenn $|Imbalance| > 0.65$ für mindestens 3 konsekutive Ticks.
|
|
||||||
|
|
||||||
### 4.3 Alpha-3: Statistical Mean Reversion (`reversion.py`)
|
|
||||||
Nutzt Z-Score Abweichungen des aktuellen Preises vom gleitenden VWAP.
|
|
||||||
- Einstieg bei Z-Score $\ge 2.5$ oder $\le -2.5$ mit Rückkehr zum Mittelwert als Target.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 5. Integration mit der evOo Self-Improvement Schleife
|
|
||||||
|
|
||||||
1. **Trade Post-Mortem**: Jeder geschlossene Trade generiert ein strukturiertes JSON-Log mit Execution-Slippage, Haltedauer und PnL.
|
|
||||||
2. **Autonome Kalibrierung**: Wenn die Win-Rate über 50 Trades unter 52% fällt, initiiert der Casted Quant Specialist eine automatische Hyperparameter-Optimierung.
|
|
||||||
3. **Audit-Sicherheit**: Alle Parameteränderungen werden in `ARCHITECTURE.md` und Git-Commits dokumentiert.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 6. Verzeichnisstruktur von `/apex-trading`
|
|
||||||
|
|
||||||
```
|
|
||||||
/apex-trading/
|
|
||||||
├── BLUEPRINT.md # Dieser Blueprint
|
|
||||||
├── README.md # Quickstart & CLI Anleitung
|
|
||||||
├── config/
|
|
||||||
│ └── trading.config.json # Konfigurierte Risikogrenzen & Assets
|
|
||||||
└── src/
|
|
||||||
├── __init__.py
|
|
||||||
├── engine/
|
|
||||||
│ ├── __init__.py
|
|
||||||
│ └── trader.py # Core Event Loop & Router
|
|
||||||
├── strategies/
|
|
||||||
│ ├── __init__.py
|
|
||||||
│ └── momentum.py # Momentum-Strategie
|
|
||||||
└── risk/
|
|
||||||
├── __init__.py
|
|
||||||
└── risk_guard.py # Hard Risk Guardrails & Circuit Breaker
|
|
||||||
```
|
|
||||||
@ -1,11 +0,0 @@
|
|||||||
# Apex Trading Subsystem
|
|
||||||
|
|
||||||
Quantitative & High-Frequency Trading Subsystem of the evOo Platform.
|
|
||||||
|
|
||||||
## Features
|
|
||||||
- Hard-coded risk controls (`src/risk/risk_guard.py`)
|
|
||||||
- Quantitative momentum and imbalance strategies (`src/strategies/`)
|
|
||||||
- Autonomous execution engine (`src/engine/trader.py`)
|
|
||||||
- Full integration with evOo Self-Improvement and CTO Steve Jobs directives.
|
|
||||||
|
|
||||||
See [BLUEPRINT.md](file:///Users/ingowagner/awesome-grok-bot/apex-trading/BLUEPRINT.md) for full architecture details.
|
|
||||||
@ -1,29 +0,0 @@
|
|||||||
{
|
|
||||||
"system": "Apex Trading Engine",
|
|
||||||
"version": "1.0.0-PROD",
|
|
||||||
"mode": "paper_trading",
|
|
||||||
"base_currency": "USDC",
|
|
||||||
"allocated_capital": 100000.0,
|
|
||||||
"risk_limits": {
|
|
||||||
"max_drawdown_percent": 1.5,
|
|
||||||
"max_single_trade_risk_percent": 0.5,
|
|
||||||
"max_open_positions": 4,
|
|
||||||
"max_leverage": 3.0,
|
|
||||||
"slippage_tolerance_percent": 0.15,
|
|
||||||
"circuit_breaker_cooldown_seconds": 3600
|
|
||||||
},
|
|
||||||
"active_pairs": ["BTC/USDC", "ETH/USDC", "SOL/USDC"],
|
|
||||||
"strategies": {
|
|
||||||
"momentum": {
|
|
||||||
"enabled": true,
|
|
||||||
"lookback_period": 20,
|
|
||||||
"rsi_oversold": 32,
|
|
||||||
"rsi_overbought": 68
|
|
||||||
},
|
|
||||||
"orderbook_imbalance": {
|
|
||||||
"enabled": true,
|
|
||||||
"threshold": 0.65,
|
|
||||||
"depth_levels": 10
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@ -1,62 +0,0 @@
|
|||||||
"""
|
|
||||||
Apex Trading Engine Core
|
|
||||||
Event loop that receives signals, checks risk guardrails, and executes orders.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import sys
|
|
||||||
from typing import Dict, Any
|
|
||||||
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
SRC_DIR = Path(__file__).resolve().parent.parent
|
|
||||||
if str(SRC_DIR) not in sys.path:
|
|
||||||
sys.path.insert(0, str(SRC_DIR))
|
|
||||||
|
|
||||||
try:
|
|
||||||
from risk.risk_guard import ApexRiskGuard
|
|
||||||
from strategies.momentum import MomentumStrategy
|
|
||||||
except ImportError:
|
|
||||||
from ..risk.risk_guard import ApexRiskGuard
|
|
||||||
from ..strategies.momentum import MomentumStrategy
|
|
||||||
|
|
||||||
class ApexEngine:
|
|
||||||
def __init__(self, config_path: str):
|
|
||||||
with open(config_path, 'r', encoding='utf-8') as f:
|
|
||||||
self.config = json.load(f)
|
|
||||||
|
|
||||||
self.risk_guard = ApexRiskGuard(self.config)
|
|
||||||
self.momentum_strategy = MomentumStrategy(self.config.get("strategies", {}).get("momentum"))
|
|
||||||
self.open_positions = []
|
|
||||||
self.current_drawdown_pct = 0.0
|
|
||||||
|
|
||||||
def process_tick(self, symbol: str, prices: list, volume: list) -> Dict[str, Any]:
|
|
||||||
"""Processes market data tick through strategy and risk guard."""
|
|
||||||
signal = self.momentum_strategy.generate_signal(symbol, prices, volume)
|
|
||||||
if not signal:
|
|
||||||
return {"status": "NO_SIGNAL", "symbol": symbol}
|
|
||||||
|
|
||||||
approved, reason = self.risk_guard.validate_order(
|
|
||||||
signal,
|
|
||||||
self.current_drawdown_pct,
|
|
||||||
len(self.open_positions)
|
|
||||||
)
|
|
||||||
|
|
||||||
if not approved:
|
|
||||||
return {"status": "REJECTED_BY_RISK", "reason": reason, "signal": signal}
|
|
||||||
|
|
||||||
# Simulated Execution
|
|
||||||
order_receipt = {
|
|
||||||
"status": "EXECUTED",
|
|
||||||
"symbol": signal["symbol"],
|
|
||||||
"action": signal["action"],
|
|
||||||
"fill_price": signal["price"],
|
|
||||||
"strategy": signal["strategy"],
|
|
||||||
"risk_status": "APPROVED"
|
|
||||||
}
|
|
||||||
self.open_positions.append(order_receipt)
|
|
||||||
return order_receipt
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
print("[APEX ENGINE] Liveness check passed. Engine ready.")
|
|
||||||
@ -1,40 +0,0 @@
|
|||||||
"""
|
|
||||||
Apex Trading Risk Guard
|
|
||||||
Hard enforcement of risk constraints defined in BLUEPRINT.md.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from typing import Dict, Any, Tuple
|
|
||||||
|
|
||||||
class ApexRiskGuard:
|
|
||||||
def __init__(self, config: Dict[str, Any]):
|
|
||||||
self.config = config.get("risk_limits", {})
|
|
||||||
self.max_drawdown = self.config.get("max_drawdown_percent", 1.5)
|
|
||||||
self.max_trade_risk = self.config.get("max_single_trade_risk_percent", 0.5)
|
|
||||||
self.max_positions = self.config.get("max_open_positions", 4)
|
|
||||||
self.circuit_breaker_active = False
|
|
||||||
|
|
||||||
def validate_order(self, order: Dict[str, Any], current_drawdown: float, open_position_count: int) -> Tuple[bool, str]:
|
|
||||||
"""
|
|
||||||
Validates an incoming order against hard guardrails.
|
|
||||||
Returns (is_approved: bool, reason: str).
|
|
||||||
"""
|
|
||||||
if self.circuit_breaker_active:
|
|
||||||
return False, "REJECTED: Circuit breaker is active."
|
|
||||||
|
|
||||||
if current_drawdown >= self.max_drawdown:
|
|
||||||
self.circuit_breaker_active = True
|
|
||||||
return False, f"REJECTED & CIRCUIT BREAKER TRIGGERED: Drawdown {current_drawdown}% >= limit {self.max_drawdown}%"
|
|
||||||
|
|
||||||
if open_position_count >= self.max_positions:
|
|
||||||
return False, f"REJECTED: Maximum concurrent positions ({self.max_positions}) reached."
|
|
||||||
|
|
||||||
order_risk_pct = order.get("risk_percent", 0.0)
|
|
||||||
if order_risk_pct > self.max_trade_risk:
|
|
||||||
return False, f"REJECTED: Order risk {order_risk_pct}% exceeds limit {self.max_trade_risk}%"
|
|
||||||
|
|
||||||
return True, "APPROVED"
|
|
||||||
|
|
||||||
def trip_kill_switch(self, reason: str):
|
|
||||||
"""Emergency hard stop."""
|
|
||||||
self.circuit_breaker_active = True
|
|
||||||
print(f"[APEX EMERGENCY KILL SWITCH] {reason}")
|
|
||||||
@ -1,48 +0,0 @@
|
|||||||
"""
|
|
||||||
Apex Momentum Strategy Module (Alpha-1)
|
|
||||||
Calculates breakout and trend-following signals.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from typing import List, Dict, Any, Optional
|
|
||||||
|
|
||||||
class MomentumStrategy:
|
|
||||||
def __init__(self, params: Optional[Dict[str, Any]] = None):
|
|
||||||
self.params = params or {}
|
|
||||||
self.lookback = self.params.get("lookback_period", 20)
|
|
||||||
self.rsi_oversold = self.params.get("rsi_oversold", 32)
|
|
||||||
self.rsi_overbought = self.params.get("rsi_overbought", 68)
|
|
||||||
|
|
||||||
def generate_signal(self, symbol: str, prices: List[float], volume: List[float]) -> Optional[Dict[str, Any]]:
|
|
||||||
"""
|
|
||||||
Evaluates recent tick series and outputs a trading signal if criteria are satisfied.
|
|
||||||
"""
|
|
||||||
if len(prices) < self.lookback:
|
|
||||||
return None
|
|
||||||
|
|
||||||
recent_window = prices[-self.lookback:]
|
|
||||||
current_price = prices[-1]
|
|
||||||
sma = sum(recent_window) / len(recent_window)
|
|
||||||
|
|
||||||
# Simple momentum breakout check
|
|
||||||
if current_price > sma * 1.015:
|
|
||||||
return {
|
|
||||||
"symbol": symbol,
|
|
||||||
"action": "BUY",
|
|
||||||
"price": current_price,
|
|
||||||
"strategy": "Alpha-1:Momentum",
|
|
||||||
"risk_percent": 0.35,
|
|
||||||
"stop_loss": current_price * 0.985,
|
|
||||||
"take_profit": current_price * 1.045
|
|
||||||
}
|
|
||||||
elif current_price < sma * 0.985:
|
|
||||||
return {
|
|
||||||
"symbol": symbol,
|
|
||||||
"action": "SELL",
|
|
||||||
"price": current_price,
|
|
||||||
"strategy": "Alpha-1:Momentum",
|
|
||||||
"risk_percent": 0.35,
|
|
||||||
"stop_loss": current_price * 1.015,
|
|
||||||
"take_profit": current_price * 0.955
|
|
||||||
}
|
|
||||||
|
|
||||||
return None
|
|
||||||
@ -1,52 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""
|
|
||||||
Unit tests for Apex Trading Engine and Risk Guardrails.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import unittest
|
|
||||||
from pathlib import Path
|
|
||||||
import sys
|
|
||||||
|
|
||||||
BASE_DIR = Path(__file__).resolve().parent
|
|
||||||
if str(BASE_DIR / "src") not in sys.path:
|
|
||||||
sys.path.insert(0, str(BASE_DIR / "src"))
|
|
||||||
|
|
||||||
from engine.trader import ApexEngine
|
|
||||||
from risk.risk_guard import ApexRiskGuard
|
|
||||||
from strategies.momentum import MomentumStrategy
|
|
||||||
|
|
||||||
class TestApexTrading(unittest.TestCase):
|
|
||||||
def setUp(self):
|
|
||||||
self.config_path = str(BASE_DIR / "config" / "trading.config.json")
|
|
||||||
self.engine = ApexEngine(self.config_path)
|
|
||||||
|
|
||||||
def test_risk_guard_limits(self):
|
|
||||||
guard = self.engine.risk_guard
|
|
||||||
# Normal order passes
|
|
||||||
normal_order = {"symbol": "BTC/USDT", "risk_percent": 0.3}
|
|
||||||
approved, reason = guard.validate_order(normal_order, current_drawdown=0.5, open_position_count=1)
|
|
||||||
self.assertTrue(approved)
|
|
||||||
self.assertEqual(reason, "APPROVED")
|
|
||||||
|
|
||||||
# Exceeding single trade risk
|
|
||||||
risky_order = {"symbol": "BTC/USDT", "risk_percent": 0.8}
|
|
||||||
approved, reason = guard.validate_order(risky_order, current_drawdown=0.5, open_position_count=1)
|
|
||||||
self.assertFalse(approved)
|
|
||||||
self.assertIn("exceeds limit", reason)
|
|
||||||
|
|
||||||
# Exceeding max drawdown trips circuit breaker
|
|
||||||
approved, reason = guard.validate_order(normal_order, current_drawdown=1.6, open_position_count=1)
|
|
||||||
self.assertFalse(approved)
|
|
||||||
self.assertIn("CIRCUIT BREAKER", reason)
|
|
||||||
self.assertTrue(guard.circuit_breaker_active)
|
|
||||||
|
|
||||||
def test_engine_process_tick(self):
|
|
||||||
# 25 prices trending sharply upwards
|
|
||||||
prices = [100.0 + (i * 1.5) for i in range(25)]
|
|
||||||
volume = [1000.0] * 25
|
|
||||||
result = self.engine.process_tick("SOL/USDT", prices, volume)
|
|
||||||
self.assertEqual(result.get("status"), "EXECUTED")
|
|
||||||
self.assertEqual(result.get("action"), "BUY")
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
unittest.main()
|
|
||||||
@ -19,6 +19,7 @@
|
|||||||
"@rakazo/core": "workspace:*",
|
"@rakazo/core": "workspace:*",
|
||||||
"@rakazo/db": "workspace:*",
|
"@rakazo/db": "workspace:*",
|
||||||
"@rakazo/logging": "workspace:*",
|
"@rakazo/logging": "workspace:*",
|
||||||
|
"@rakazo/media": "workspace:*",
|
||||||
"@rakazo/memory": "workspace:*",
|
"@rakazo/memory": "workspace:*",
|
||||||
"hono": "4.13.7"
|
"hono": "4.13.7"
|
||||||
},
|
},
|
||||||
|
|||||||
@ -1,5 +1,5 @@
|
|||||||
import { ORPCError } from "@orpc/server";
|
import { ORPCError } from "@orpc/server";
|
||||||
import { BUILTIN_AGENT_SKILLS } from "@rakazo/adapters";
|
import { BUILTIN_AGENT_SKILLS, filterMediaSkills } from "@rakazo/adapters";
|
||||||
import type { Actor, AgentSkill, AgentSkillSource } from "@rakazo/contracts";
|
import type { Actor, AgentSkill, AgentSkillSource } from "@rakazo/contracts";
|
||||||
import {
|
import {
|
||||||
buildSkillMd,
|
buildSkillMd,
|
||||||
@ -118,7 +118,13 @@ export function resolveSkillContent(input: {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
export function createAgentSkillsService(prisma: PrismaClient) {
|
export function createAgentSkillsService(
|
||||||
|
prisma: PrismaClient,
|
||||||
|
options: { mediaConfigured?: boolean } = {},
|
||||||
|
) {
|
||||||
|
/** Built-in skills this deployment can run: media recipes only with Media Studio. */
|
||||||
|
const visibleBuiltins = () =>
|
||||||
|
filterMediaSkills(builtinCatalog(), Boolean(options.mediaConfigured));
|
||||||
async function owned(actor: Actor, skillId: string) {
|
async function owned(actor: Actor, skillId: string) {
|
||||||
const row = await prisma.agentSkill.findFirst({
|
const row = await prisma.agentSkill.findFirst({
|
||||||
where: {
|
where: {
|
||||||
@ -137,7 +143,7 @@ export function createAgentSkillsService(prisma: PrismaClient) {
|
|||||||
where: { spaceId: actor.spaceId, userId: actor.userId },
|
where: { spaceId: actor.spaceId, userId: actor.userId },
|
||||||
orderBy: [{ name: "asc" }, { id: "asc" }],
|
orderBy: [{ name: "asc" }, { id: "asc" }],
|
||||||
});
|
});
|
||||||
return mergeBuiltinSkills(builtinCatalog(), rows.map(mapAgentSkill)).map(
|
return mergeBuiltinSkills(visibleBuiltins(), rows.map(mapAgentSkill)).map(
|
||||||
({ content: _content, ...entry }) => entry,
|
({ content: _content, ...entry }) => entry,
|
||||||
);
|
);
|
||||||
},
|
},
|
||||||
@ -147,12 +153,12 @@ export function createAgentSkillsService(prisma: PrismaClient) {
|
|||||||
where: { spaceId: actor.spaceId, userId: actor.userId },
|
where: { spaceId: actor.spaceId, userId: actor.userId },
|
||||||
orderBy: [{ name: "asc" }, { id: "asc" }],
|
orderBy: [{ name: "asc" }, { id: "asc" }],
|
||||||
});
|
});
|
||||||
return mergeBuiltinSkills(builtinCatalog(), rows.map(mapAgentSkill));
|
return mergeBuiltinSkills(visibleBuiltins(), rows.map(mapAgentSkill));
|
||||||
},
|
},
|
||||||
|
|
||||||
async get(actor: Actor, input: { skillId?: string; name?: string }): Promise<AgentSkill> {
|
async get(actor: Actor, input: { skillId?: string; name?: string }): Promise<AgentSkill> {
|
||||||
if (input.skillId?.startsWith("builtin:")) {
|
if (input.skillId?.startsWith("builtin:")) {
|
||||||
const builtin = builtinCatalog().find((skill) => skill.id === input.skillId);
|
const builtin = visibleBuiltins().find((skill) => skill.id === input.skillId);
|
||||||
if (!builtin) throw new IsolationError();
|
if (!builtin) throw new IsolationError();
|
||||||
return builtin;
|
return builtin;
|
||||||
}
|
}
|
||||||
@ -169,7 +175,7 @@ export function createAgentSkillsService(prisma: PrismaClient) {
|
|||||||
});
|
});
|
||||||
const row = findSkillByName(rows, name);
|
const row = findSkillByName(rows, name);
|
||||||
if (row) return mapAgentSkill(row);
|
if (row) return mapAgentSkill(row);
|
||||||
const builtin = findSkillByName(builtinCatalog(), name);
|
const builtin = findSkillByName(visibleBuiltins(), name);
|
||||||
if (!builtin) throw new IsolationError();
|
if (!builtin) throw new IsolationError();
|
||||||
return builtin;
|
return builtin;
|
||||||
},
|
},
|
||||||
|
|||||||
@ -25,6 +25,8 @@ import {
|
|||||||
createCloudAgentConnection,
|
createCloudAgentConnection,
|
||||||
createConnectorStack,
|
createConnectorStack,
|
||||||
createJobReconciler,
|
createJobReconciler,
|
||||||
|
createMediaJobHandlers,
|
||||||
|
createMediaServices,
|
||||||
createMessagingContextLoader,
|
createMessagingContextLoader,
|
||||||
createMessagingTeamChatSender,
|
createMessagingTeamChatSender,
|
||||||
createProfileConsolidationScheduler,
|
createProfileConsolidationScheduler,
|
||||||
@ -92,6 +94,7 @@ import { cors } from "hono/cors";
|
|||||||
import type { AppEnv } from "./env.js";
|
import type { AppEnv } from "./env.js";
|
||||||
import { loadEnv } from "./env.js";
|
import { loadEnv } from "./env.js";
|
||||||
import { mountLocalSettings } from "./local-settings.js";
|
import { mountLocalSettings } from "./local-settings.js";
|
||||||
|
import { mountMediaHttpRoutes } from "./media-http.js";
|
||||||
import {
|
import {
|
||||||
createMessagingInboundHandler,
|
createMessagingInboundHandler,
|
||||||
teamChatSenderCanWakeMessageRoutines,
|
teamChatSenderCanWakeMessageRoutines,
|
||||||
@ -109,7 +112,6 @@ import {
|
|||||||
settleWithTimeout,
|
settleWithTimeout,
|
||||||
TEAM_CHAT_STARTUP_SHUTDOWN_MS,
|
TEAM_CHAT_STARTUP_SHUTDOWN_MS,
|
||||||
} from "./team-chat-startup.js";
|
} from "./team-chat-startup.js";
|
||||||
import { mountTradingBotStatus } from "./trading-bot-proxy.js";
|
|
||||||
import { mountVoiceHttpRoutes } from "./voice.js";
|
import { mountVoiceHttpRoutes } from "./voice.js";
|
||||||
import { mountWebhookHttpRoutes } from "./webhook.js";
|
import { mountWebhookHttpRoutes } from "./webhook.js";
|
||||||
|
|
||||||
@ -412,8 +414,22 @@ export async function createApp(
|
|||||||
web: createWebProvider(),
|
web: createWebProvider(),
|
||||||
cloudAgent,
|
cloudAgent,
|
||||||
shutdownSignal: shutdown.signal,
|
shutdownSignal: shutdown.signal,
|
||||||
|
// Late-bound: media services are built below and need executor.resolveModel.
|
||||||
|
media: () => mediaServices,
|
||||||
});
|
});
|
||||||
|
|
||||||
|
const mediaServices = createMediaServices({
|
||||||
|
config: env.media,
|
||||||
|
prisma,
|
||||||
|
jobs,
|
||||||
|
artifacts,
|
||||||
|
runtime,
|
||||||
|
realtime,
|
||||||
|
resolveModel: (scope) => executor.resolveModel(scope),
|
||||||
|
});
|
||||||
|
const mediaJobs = mediaServices
|
||||||
|
? createMediaJobHandlers(mediaServices, { signal: shutdown.signal })
|
||||||
|
: null;
|
||||||
const jobHandlers = createBackgroundJobHandlers({
|
const jobHandlers = createBackgroundJobHandlers({
|
||||||
executor,
|
executor,
|
||||||
prisma,
|
prisma,
|
||||||
@ -421,6 +437,7 @@ export async function createApp(
|
|||||||
home,
|
home,
|
||||||
jobs,
|
jobs,
|
||||||
events,
|
events,
|
||||||
|
...(mediaJobs ? { media: mediaJobs.handlers } : {}),
|
||||||
workerId: "api",
|
workerId: "api",
|
||||||
runtime,
|
runtime,
|
||||||
secretStore: secrets,
|
secretStore: secrets,
|
||||||
@ -438,6 +455,11 @@ export async function createApp(
|
|||||||
});
|
});
|
||||||
await graphileWorkerHost.start(jobHandlers);
|
await graphileWorkerHost.start(jobHandlers);
|
||||||
}
|
}
|
||||||
|
if (mediaJobs) {
|
||||||
|
void mediaJobs
|
||||||
|
.resume()
|
||||||
|
.catch((error) => getLogger().error("media ingest resume failed", error));
|
||||||
|
}
|
||||||
const reconciler = inMemoryJobs
|
const reconciler = inMemoryJobs
|
||||||
? createJobReconciler({
|
? createJobReconciler({
|
||||||
prisma,
|
prisma,
|
||||||
@ -469,6 +491,7 @@ export async function createApp(
|
|||||||
remoteConnectors,
|
remoteConnectors,
|
||||||
artifacts,
|
artifacts,
|
||||||
dataDir: env.dataDir,
|
dataDir: env.dataDir,
|
||||||
|
mediaServices,
|
||||||
messaging: {
|
messaging: {
|
||||||
enabled: Boolean(messaging),
|
enabled: Boolean(messaging),
|
||||||
providers: messaging?.platforms().map((platform) => platform.provider) ?? [],
|
providers: messaging?.platforms().map((platform) => platform.provider) ?? [],
|
||||||
@ -530,13 +553,6 @@ export async function createApp(
|
|||||||
resetUrl: email ? new URL("/reset-password", env.webOrigin).href : null,
|
resetUrl: email ? new URL("/reset-password", env.webOrigin).href : null,
|
||||||
}),
|
}),
|
||||||
);
|
);
|
||||||
mountTradingBotStatus(app, {
|
|
||||||
baseUrl: env.tradingBotUrl,
|
|
||||||
authorize: async (request) => {
|
|
||||||
const session = await auth.api.getSession({ headers: sessionHeaders(request) });
|
|
||||||
return Boolean(session?.user);
|
|
||||||
},
|
|
||||||
});
|
|
||||||
if (localEmailEmulator && env.nodeEnv === "development") {
|
if (localEmailEmulator && env.nodeEnv === "development") {
|
||||||
app.get(
|
app.get(
|
||||||
"/api/dev/emails",
|
"/api/dev/emails",
|
||||||
@ -583,6 +599,13 @@ export async function createApp(
|
|||||||
if (actor) enrichLogContext({ "user.id": actor.userId, "space.id": actor.spaceId });
|
if (actor) enrichLogContext({ "user.id": actor.userId, "space.id": actor.spaceId });
|
||||||
return actor;
|
return actor;
|
||||||
});
|
});
|
||||||
|
mountMediaHttpRoutes(app, { services: mediaServices }, async (c) => {
|
||||||
|
const session = await auth.api.getSession({ headers: sessionHeaders(c.req.raw) });
|
||||||
|
if (!session?.user) return null;
|
||||||
|
return requireMembership(prisma, session.user.id, c.req.header("x-rakazo-space-id")).catch(
|
||||||
|
() => null,
|
||||||
|
);
|
||||||
|
});
|
||||||
mountWebhookHttpRoutes(app, { prisma, secrets, events, jobs });
|
mountWebhookHttpRoutes(app, { prisma, secrets, events, jobs });
|
||||||
// Shared with stop so a shutdown during retry delays does not restart polling.
|
// Shared with stop so a shutdown during retry delays does not restart polling.
|
||||||
let messagingStopped = false;
|
let messagingStopped = false;
|
||||||
|
|||||||
@ -42,16 +42,6 @@ describe("loadEnv", () => {
|
|||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("loads a trading bot origin and refuses credentials", () => {
|
|
||||||
expect(loadEnv(base).tradingBotUrl).toBeUndefined();
|
|
||||||
expect(
|
|
||||||
loadEnv({ ...base, TRADING_BOT_URL: " http://127.0.0.1:8000/bot/ " }).tradingBotUrl,
|
|
||||||
).toBe("http://127.0.0.1:8000");
|
|
||||||
expect(() =>
|
|
||||||
loadEnv({ ...base, TRADING_BOT_URL: "http://user:secret@127.0.0.1:8000" }),
|
|
||||||
).toThrow(/^Invalid TRADING_BOT_URL$/);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("loads an optional integrations catalog mirror", () => {
|
it("loads an optional integrations catalog mirror", () => {
|
||||||
expect(loadEnv(base).integrationsCatalogUrl).toBeUndefined();
|
expect(loadEnv(base).integrationsCatalogUrl).toBeUndefined();
|
||||||
expect(
|
expect(
|
||||||
|
|||||||
@ -1,4 +1,6 @@
|
|||||||
|
import type { MediaConfig } from "@rakazo/adapters";
|
||||||
import {
|
import {
|
||||||
|
mediaConfigFromEnv,
|
||||||
resolveCloudAgentProvider,
|
resolveCloudAgentProvider,
|
||||||
resolveDeploymentModel,
|
resolveDeploymentModel,
|
||||||
resolveSandboxProvider,
|
resolveSandboxProvider,
|
||||||
@ -8,13 +10,14 @@ import {
|
|||||||
resolveEncryptionKey,
|
resolveEncryptionKey,
|
||||||
resolveScreenProxySecret,
|
resolveScreenProxySecret,
|
||||||
resolveSupervisorToken,
|
resolveSupervisorToken,
|
||||||
resolveTradingBotOrigin,
|
|
||||||
} from "@rakazo/core";
|
} from "@rakazo/core";
|
||||||
|
|
||||||
export { resolveCloudAgentProvider, resolveSandboxProvider } from "@rakazo/adapters";
|
export { resolveCloudAgentProvider, resolveSandboxProvider } from "@rakazo/adapters";
|
||||||
|
|
||||||
export interface AppEnv {
|
export interface AppEnv {
|
||||||
nodeEnv: string;
|
nodeEnv: string;
|
||||||
|
/** Media Studio; disabled unless a sidecar (or the scripted emulator) is configured. */
|
||||||
|
media: MediaConfig;
|
||||||
desktopStackToken?: string;
|
desktopStackToken?: string;
|
||||||
databaseUrl: string;
|
databaseUrl: string;
|
||||||
realtimeDatabaseUrl: string;
|
realtimeDatabaseUrl: string;
|
||||||
@ -93,8 +96,6 @@ export interface AppEnv {
|
|||||||
updaterToken: string | undefined;
|
updaterToken: string | undefined;
|
||||||
/** Current application image tag; used for compose manual-upgrade command selection. */
|
/** Current application image tag; used for compose manual-upgrade command selection. */
|
||||||
imageTag: string | undefined;
|
imageTag: string | undefined;
|
||||||
/** Tradingbot origin for the same-origin status proxy. Unset on SaaS builds. */
|
|
||||||
tradingBotUrl: string | undefined;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
export function loadEnv(source: NodeJS.ProcessEnv = process.env): AppEnv {
|
export function loadEnv(source: NodeJS.ProcessEnv = process.env): AppEnv {
|
||||||
@ -106,6 +107,7 @@ export function loadEnv(source: NodeJS.ProcessEnv = process.env): AppEnv {
|
|||||||
const updaterToken = optional(source.RAKAZO_UPDATER_TOKEN);
|
const updaterToken = optional(source.RAKAZO_UPDATER_TOKEN);
|
||||||
return {
|
return {
|
||||||
nodeEnv: source.NODE_ENV ?? "",
|
nodeEnv: source.NODE_ENV ?? "",
|
||||||
|
media: mediaConfigFromEnv(source),
|
||||||
databaseUrl: required(source, "DATABASE_URL"),
|
databaseUrl: required(source, "DATABASE_URL"),
|
||||||
realtimeDatabaseUrl: source.REALTIME_DATABASE_URL ?? required(source, "DATABASE_URL"),
|
realtimeDatabaseUrl: source.REALTIME_DATABASE_URL ?? required(source, "DATABASE_URL"),
|
||||||
desktopStackToken: optional(source.RAKAZO_DESKTOP_STACK_TOKEN),
|
desktopStackToken: optional(source.RAKAZO_DESKTOP_STACK_TOKEN),
|
||||||
@ -182,7 +184,6 @@ export function loadEnv(source: NodeJS.ProcessEnv = process.env): AppEnv {
|
|||||||
updaterUrl,
|
updaterUrl,
|
||||||
updaterToken,
|
updaterToken,
|
||||||
imageTag: optional(source.RAKAZO_IMAGE_TAG),
|
imageTag: optional(source.RAKAZO_IMAGE_TAG),
|
||||||
tradingBotUrl: resolveTradingBotOrigin(source.TRADING_BOT_URL),
|
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@ -43,13 +43,20 @@ export async function assertNotMainConversation(
|
|||||||
if (group?.kind === "main") throw new ORPCError("BAD_REQUEST", { message });
|
if (group?.kind === "main") throw new ORPCError("BAD_REQUEST", { message });
|
||||||
}
|
}
|
||||||
|
|
||||||
/** The main conversation carries the main bot's name. */
|
/** A main conversation carries its main bot's name. */
|
||||||
export async function renameMainConversation(prisma: PrismaClient, userId: string, name: string) {
|
export async function renameMainConversation(
|
||||||
await prisma.chatGroup.updateMany({ where: { userId, kind: "main" }, data: { name } });
|
prisma: PrismaClient,
|
||||||
|
main: { userId: string; spaceId: string },
|
||||||
|
name: string,
|
||||||
|
) {
|
||||||
|
await prisma.chatGroup.updateMany({
|
||||||
|
where: { userId: main.userId, spaceId: main.spaceId, kind: "main" },
|
||||||
|
data: { name },
|
||||||
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Same-space bots the user addresses in the main conversation, by `@Name` or a mention chip,
|
* Same-space bots the user addresses in a main conversation, by `@Name` or a mention chip,
|
||||||
* join it first, so the member mention rules route the message to them.
|
* join it first, so the member mention rules route the message to them.
|
||||||
*/
|
*/
|
||||||
export async function joinAddressedBots(
|
export async function joinAddressedBots(
|
||||||
@ -71,5 +78,9 @@ export async function joinAddressedBots(
|
|||||||
.map((bot) => bot.id);
|
.map((bot) => bot.id);
|
||||||
if (addressed.length === 0) return;
|
if (addressed.length === 0) return;
|
||||||
// Subscribers pick up the group.updated event with the send that follows.
|
// Subscribers pick up the group.updated event with the send that follows.
|
||||||
await joinMainConversation(prisma, { userId: actor.userId, botIds: addressed });
|
await joinMainConversation(prisma, {
|
||||||
|
userId: actor.userId,
|
||||||
|
spaceId: actor.spaceId,
|
||||||
|
botIds: addressed,
|
||||||
|
});
|
||||||
}
|
}
|
||||||
|
|||||||
265
apps/api/src/media-http.ts
Normal file
265
apps/api/src/media-http.ts
Normal file
@ -0,0 +1,265 @@
|
|||||||
|
import { createReadStream } from "node:fs";
|
||||||
|
import { mkdir, stat } from "node:fs/promises";
|
||||||
|
import path from "node:path";
|
||||||
|
import { Readable } from "node:stream";
|
||||||
|
import type { MediaServices } from "@rakazo/adapters";
|
||||||
|
import type { Actor } from "@rakazo/contracts";
|
||||||
|
import { IsolationError, MediaStateConflictError } from "@rakazo/db";
|
||||||
|
import { assertInside, PathOutsideRootError, resolveInside } from "@rakazo/media/node";
|
||||||
|
import type { Context, Hono } from "hono";
|
||||||
|
import { cancelBody } from "./http-body.js";
|
||||||
|
import { MEDIA_CHUNK_TIMEOUT_MS, UploadChunkError, writeUploadChunk } from "./media-uploads.js";
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Byte routes Media Studio cannot express as JSON RPC: chunk uploads and
|
||||||
|
* streaming of proxies, thumbnails, renders, and bundles. Every id is checked
|
||||||
|
* against the actor's space before a path is derived, and every path is
|
||||||
|
* resolved inside the media root.
|
||||||
|
*/
|
||||||
|
|
||||||
|
const ID = /^[A-Za-z0-9_-]{1,64}$/;
|
||||||
|
const FILE_NAME = /^[A-Za-z0-9][A-Za-z0-9._-]{0,199}$/;
|
||||||
|
const CHUNK_INDEX = /^[0-9]{1,7}$/;
|
||||||
|
|
||||||
|
const CONTENT_TYPES: Record<string, string> = {
|
||||||
|
".mp4": "video/mp4",
|
||||||
|
".m4v": "video/mp4",
|
||||||
|
".mov": "video/quicktime",
|
||||||
|
".webm": "video/webm",
|
||||||
|
".jpg": "image/jpeg",
|
||||||
|
".jpeg": "image/jpeg",
|
||||||
|
".png": "image/png",
|
||||||
|
".webp": "image/webp",
|
||||||
|
".gif": "image/gif",
|
||||||
|
".srt": "application/x-subrip",
|
||||||
|
".vtt": "text/vtt; charset=utf-8",
|
||||||
|
".json": "application/json",
|
||||||
|
".md": "text/markdown; charset=utf-8",
|
||||||
|
".txt": "text/plain; charset=utf-8",
|
||||||
|
".zip": "application/zip",
|
||||||
|
".m4a": "audio/mp4",
|
||||||
|
".wav": "audio/wav",
|
||||||
|
};
|
||||||
|
|
||||||
|
export function mediaContentType(fileName: string): string {
|
||||||
|
return CONTENT_TYPES[path.extname(fileName).toLowerCase()] ?? "application/octet-stream";
|
||||||
|
}
|
||||||
|
|
||||||
|
export type ByteRange = { start: number; end: number };
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parse a single `bytes=` range. Returns null for no (or an ignorable
|
||||||
|
* multi-part) range and "unsatisfiable" when it cannot be served.
|
||||||
|
*/
|
||||||
|
export function parseByteRange(
|
||||||
|
header: string | undefined,
|
||||||
|
size: number,
|
||||||
|
): ByteRange | null | "unsatisfiable" {
|
||||||
|
if (!header) return null;
|
||||||
|
const match = /^bytes=(\d*)-(\d*)$/.exec(header.trim());
|
||||||
|
if (!match) return header.includes(",") ? null : "unsatisfiable";
|
||||||
|
const [, first = "", last = ""] = match;
|
||||||
|
if (first === "" && last === "") return "unsatisfiable";
|
||||||
|
if (first === "") {
|
||||||
|
const suffix = Number(last);
|
||||||
|
if (suffix === 0 || size === 0) return "unsatisfiable";
|
||||||
|
return { start: Math.max(0, size - suffix), end: size - 1 };
|
||||||
|
}
|
||||||
|
const start = Number(first);
|
||||||
|
const end = last === "" ? size - 1 : Math.min(Number(last), size - 1);
|
||||||
|
if (start >= size || end < start) return "unsatisfiable";
|
||||||
|
return { start, end };
|
||||||
|
}
|
||||||
|
|
||||||
|
async function sendFile(
|
||||||
|
c: Context,
|
||||||
|
absolutePath: string,
|
||||||
|
options: { fileName: string; download?: boolean },
|
||||||
|
): Promise<Response> {
|
||||||
|
const info = await stat(absolutePath).catch(() => null);
|
||||||
|
if (!info?.isFile()) return c.json({ error: "Not found" }, 404);
|
||||||
|
const size = info.size;
|
||||||
|
const headers: Record<string, string> = {
|
||||||
|
"content-type": mediaContentType(options.fileName),
|
||||||
|
"accept-ranges": "bytes",
|
||||||
|
"cache-control": "private, max-age=300",
|
||||||
|
};
|
||||||
|
if (options.download) {
|
||||||
|
headers["content-disposition"] = `attachment; filename="${options.fileName}"`;
|
||||||
|
}
|
||||||
|
const range = parseByteRange(c.req.header("range"), size);
|
||||||
|
if (range === "unsatisfiable") {
|
||||||
|
return new Response(null, {
|
||||||
|
status: 416,
|
||||||
|
headers: { ...headers, "content-range": `bytes */${size}` },
|
||||||
|
});
|
||||||
|
}
|
||||||
|
if (c.req.method === "HEAD") {
|
||||||
|
return new Response(null, {
|
||||||
|
status: 200,
|
||||||
|
headers: { ...headers, "content-length": String(size) },
|
||||||
|
});
|
||||||
|
}
|
||||||
|
const start = range?.start ?? 0;
|
||||||
|
const end = range?.end ?? size - 1;
|
||||||
|
const body =
|
||||||
|
size === 0
|
||||||
|
? null
|
||||||
|
: (Readable.toWeb(
|
||||||
|
createReadStream(absolutePath, { start, end }),
|
||||||
|
) as ReadableStream<Uint8Array>);
|
||||||
|
return new Response(body, {
|
||||||
|
status: range ? 206 : 200,
|
||||||
|
headers: {
|
||||||
|
...headers,
|
||||||
|
"content-length": String(size === 0 ? 0 : end - start + 1),
|
||||||
|
...(range ? { "content-range": `bytes ${start}-${end}/${size}` } : {}),
|
||||||
|
},
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function mediaHttpError(c: Context, error: unknown): Response {
|
||||||
|
if (error instanceof IsolationError || error instanceof PathOutsideRootError) {
|
||||||
|
return c.json({ error: "Not found" }, 404);
|
||||||
|
}
|
||||||
|
if (error instanceof UploadChunkError) return c.json({ error: error.message }, error.status);
|
||||||
|
if (error instanceof MediaStateConflictError) return c.json({ error: error.message }, 409);
|
||||||
|
if ((error as NodeJS.ErrnoException | null)?.code === "ENOENT") {
|
||||||
|
return c.json({ error: "Not found" }, 404);
|
||||||
|
}
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function mountMediaHttpRoutes(
|
||||||
|
app: Hono,
|
||||||
|
deps: {
|
||||||
|
/** Null when Media Studio is not configured: every route answers 404. */
|
||||||
|
services: MediaServices | null;
|
||||||
|
chunkTimeoutMs?: number;
|
||||||
|
},
|
||||||
|
resolveActor: (c: Context) => Promise<Actor | null>,
|
||||||
|
) {
|
||||||
|
async function authorize(
|
||||||
|
c: Context,
|
||||||
|
): Promise<
|
||||||
|
{ actor: Actor; services: MediaServices; response?: undefined } | { response: Response }
|
||||||
|
> {
|
||||||
|
const actor = await resolveActor(c);
|
||||||
|
if (!actor) return { response: c.json({ error: "Unauthorized" }, 401) };
|
||||||
|
if (!deps.services)
|
||||||
|
return { response: c.json({ error: "Media Studio is not configured" }, 404) };
|
||||||
|
return { actor, services: deps.services };
|
||||||
|
}
|
||||||
|
|
||||||
|
async function mediaFile(services: MediaServices, relative: string | null) {
|
||||||
|
if (!relative) return null;
|
||||||
|
await mkdir(services.deps.mediaRoot, { recursive: true });
|
||||||
|
return resolveInside(services.deps.mediaRoot, relative);
|
||||||
|
}
|
||||||
|
|
||||||
|
app.put("/api/media/uploads/:uploadId/chunks/:index", async (c) => {
|
||||||
|
const auth = await authorize(c);
|
||||||
|
if (auth.response) {
|
||||||
|
cancelBody(c.req.raw.body);
|
||||||
|
return auth.response;
|
||||||
|
}
|
||||||
|
const uploadId = c.req.param("uploadId");
|
||||||
|
const index = c.req.param("index");
|
||||||
|
if (!ID.test(uploadId) || !CHUNK_INDEX.test(index)) {
|
||||||
|
cancelBody(c.req.raw.body);
|
||||||
|
return c.json({ error: "Not found" }, 404);
|
||||||
|
}
|
||||||
|
const lengthHeader = c.req.header("content-length");
|
||||||
|
const declared = lengthHeader && /^[0-9]+$/.test(lengthHeader) ? Number(lengthHeader) : null;
|
||||||
|
const timeout = AbortSignal.timeout(deps.chunkTimeoutMs ?? MEDIA_CHUNK_TIMEOUT_MS);
|
||||||
|
const body = c.req.raw.body;
|
||||||
|
try {
|
||||||
|
const upload = await writeUploadChunk({
|
||||||
|
services: auth.services,
|
||||||
|
spaceId: auth.actor.spaceId,
|
||||||
|
uploadId,
|
||||||
|
index: Number(index),
|
||||||
|
body,
|
||||||
|
declaredLength: declared,
|
||||||
|
signal: AbortSignal.any([timeout, c.req.raw.signal]),
|
||||||
|
});
|
||||||
|
return c.json(upload);
|
||||||
|
} catch (error) {
|
||||||
|
cancelBody(body);
|
||||||
|
if (timeout.aborted) return c.json({ error: "Chunk took too long" }, 408);
|
||||||
|
return mediaHttpError(c, error);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
app.get("/api/media/renders/:renderId/files/:fileName", async (c) => {
|
||||||
|
const auth = await authorize(c);
|
||||||
|
if (auth.response) return auth.response;
|
||||||
|
const renderId = c.req.param("renderId");
|
||||||
|
const fileName = c.req.param("fileName");
|
||||||
|
if (!ID.test(renderId) || !FILE_NAME.test(fileName) || fileName.includes("..")) {
|
||||||
|
return c.json({ error: "Not found" }, 404);
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
const render = await auth.services.deps.repos.getRenderRecord(auth.actor.spaceId, renderId);
|
||||||
|
const download = c.req.query("download") === "1";
|
||||||
|
// Only a render that passed its gate leaves the app as a download.
|
||||||
|
if (download && render.status !== "done") return c.json({ error: "Not found" }, 404);
|
||||||
|
const absolute = await auth.services.render.outputPath(auth.actor, renderId, fileName);
|
||||||
|
if (!absolute) return c.json({ error: "Not found" }, 404);
|
||||||
|
await mkdir(auth.services.deps.mediaRoot, { recursive: true });
|
||||||
|
const inside = await assertInside(
|
||||||
|
await resolveInside(
|
||||||
|
auth.services.deps.mediaRoot,
|
||||||
|
`${auth.actor.spaceId}/renders/${renderId}`,
|
||||||
|
),
|
||||||
|
absolute,
|
||||||
|
);
|
||||||
|
return await sendFile(c, inside, { fileName, download });
|
||||||
|
} catch (error) {
|
||||||
|
return mediaHttpError(c, error);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
for (const kind of ["thumb", "proxy"] as const) {
|
||||||
|
app.get(`/api/media/assets/:assetId/${kind}`, async (c) => {
|
||||||
|
const auth = await authorize(c);
|
||||||
|
if (auth.response) return auth.response;
|
||||||
|
const assetId = c.req.param("assetId");
|
||||||
|
if (!ID.test(assetId)) return c.json({ error: "Not found" }, 404);
|
||||||
|
try {
|
||||||
|
const asset = await auth.services.deps.repos.getAssetRecord(auth.actor.spaceId, assetId);
|
||||||
|
const relative = kind === "thumb" ? asset.thumbPath : asset.proxyPath;
|
||||||
|
const absolute = await mediaFile(auth.services, relative);
|
||||||
|
if (!absolute) return c.json({ error: "Not found" }, 404);
|
||||||
|
return await sendFile(c, absolute, { fileName: path.basename(absolute) });
|
||||||
|
} catch (error) {
|
||||||
|
return mediaHttpError(c, error);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
app.get("/api/media/bundles/:publicationId", async (c) => {
|
||||||
|
const auth = await authorize(c);
|
||||||
|
if (auth.response) return auth.response;
|
||||||
|
const publicationId = c.req.param("publicationId");
|
||||||
|
if (!ID.test(publicationId)) return c.json({ error: "Not found" }, 404);
|
||||||
|
try {
|
||||||
|
const publication = await auth.services.deps.repos.getPublicationRecord(
|
||||||
|
auth.actor.spaceId,
|
||||||
|
publicationId,
|
||||||
|
);
|
||||||
|
if (publication.bundleArtifactId) return c.json({ error: "Not found" }, 404);
|
||||||
|
const absolute = await mediaFile(
|
||||||
|
auth.services,
|
||||||
|
`${auth.actor.spaceId}/bundles/${publication.id}/bundle.zip`,
|
||||||
|
);
|
||||||
|
if (!absolute) return c.json({ error: "Not found" }, 404);
|
||||||
|
return await sendFile(c, absolute, {
|
||||||
|
fileName: `bundle-${publication.id}.zip`,
|
||||||
|
download: true,
|
||||||
|
});
|
||||||
|
} catch (error) {
|
||||||
|
return mediaHttpError(c, error);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
389
apps/api/src/media-test-fakes.ts
Normal file
389
apps/api/src/media-test-fakes.ts
Normal file
@ -0,0 +1,389 @@
|
|||||||
|
import type { MediaInferenceProvider } from "@rakazo/adapter-kit";
|
||||||
|
import type {
|
||||||
|
MediaActor,
|
||||||
|
MediaIngestService,
|
||||||
|
MediaMarketingService,
|
||||||
|
MediaPrivacyService,
|
||||||
|
MediaRenderService,
|
||||||
|
MediaServiceDeps,
|
||||||
|
MediaServices,
|
||||||
|
} from "@rakazo/adapters";
|
||||||
|
import type { PlatformProfile } from "@rakazo/contracts";
|
||||||
|
import { mediaRpcShapes } from "@rakazo/contracts";
|
||||||
|
import type { MediaRepos, PrismaClient } from "@rakazo/db";
|
||||||
|
|
||||||
|
/*
|
||||||
|
* In-memory stand-ins for the media service slices, for API tests only. They
|
||||||
|
* record what the router hands them and do the minimum through the real
|
||||||
|
* repositories; nothing here renders, detects, or publishes.
|
||||||
|
*/
|
||||||
|
|
||||||
|
export interface MediaServiceCall {
|
||||||
|
service: "ingest" | "privacy" | "render" | "marketing";
|
||||||
|
method: string;
|
||||||
|
actor: MediaActor;
|
||||||
|
input: unknown;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface FakeMediaServicesOptions {
|
||||||
|
repos: MediaRepos;
|
||||||
|
mediaRoot: string;
|
||||||
|
prisma?: PrismaClient;
|
||||||
|
inference?: MediaInferenceProvider | null;
|
||||||
|
profiles?: PlatformProfile[];
|
||||||
|
now?: () => Date;
|
||||||
|
/** Render id → absolute output path the fake render service reports. */
|
||||||
|
renderOutputs?: Map<string, string>;
|
||||||
|
onUploadCompleted?: (
|
||||||
|
actor: MediaActor,
|
||||||
|
input: { assetId: string; tempPath: string; sha256: string; fileName: string },
|
||||||
|
) => Promise<void>;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function fakeMediaServices(options: FakeMediaServicesOptions): {
|
||||||
|
services: MediaServices;
|
||||||
|
calls: MediaServiceCall[];
|
||||||
|
} {
|
||||||
|
const calls: MediaServiceCall[] = [];
|
||||||
|
const { repos } = options;
|
||||||
|
const record = (
|
||||||
|
service: MediaServiceCall["service"],
|
||||||
|
method: string,
|
||||||
|
actor: MediaActor,
|
||||||
|
input: unknown,
|
||||||
|
) => {
|
||||||
|
calls.push({ service, method, actor: { spaceId: actor.spaceId, userId: actor.userId }, input });
|
||||||
|
};
|
||||||
|
|
||||||
|
const ingest: MediaIngestService = {
|
||||||
|
async pushItems(actor, input) {
|
||||||
|
record("ingest", "pushItems", actor, input);
|
||||||
|
return { created: 0, updated: 0, unchanged: 0, deleted: 0 };
|
||||||
|
},
|
||||||
|
async requestSync(actor, sourceId) {
|
||||||
|
record("ingest", "requestSync", actor, sourceId);
|
||||||
|
},
|
||||||
|
async uploadCompleted(actor, input) {
|
||||||
|
record("ingest", "uploadCompleted", actor, input);
|
||||||
|
await options.onUploadCompleted?.(actor, input);
|
||||||
|
},
|
||||||
|
async recluster(actor) {
|
||||||
|
record("ingest", "recluster", actor, null);
|
||||||
|
},
|
||||||
|
async removeSource(actor, sourceId) {
|
||||||
|
record("ingest", "removeSource", actor, sourceId);
|
||||||
|
await repos.removeSource(actor.spaceId, sourceId);
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
const privacy: MediaPrivacyService = {
|
||||||
|
async prepareSources() {
|
||||||
|
throw new Error("prepareSources is not part of the API fakes");
|
||||||
|
},
|
||||||
|
async gate() {
|
||||||
|
throw new Error("gate is not part of the API fakes");
|
||||||
|
},
|
||||||
|
async createMark(actor, input) {
|
||||||
|
record("privacy", "createMark", actor, input);
|
||||||
|
return repos.createMark(actor.spaceId, input);
|
||||||
|
},
|
||||||
|
async updateMark(actor, input) {
|
||||||
|
record("privacy", "updateMark", actor, input);
|
||||||
|
const { markId, ...patch } = input;
|
||||||
|
return repos.updateMark(actor.spaceId, markId, patch);
|
||||||
|
},
|
||||||
|
async removePerson(actor, personId) {
|
||||||
|
record("privacy", "removePerson", actor, personId);
|
||||||
|
await repos.removePersonCompletely(actor.spaceId, personId);
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
const render: MediaRenderService = {
|
||||||
|
async start(actor, input) {
|
||||||
|
record("render", "start", actor, input);
|
||||||
|
return repos.createRender(actor.spaceId, { ...input, createdBy: actor.userId });
|
||||||
|
},
|
||||||
|
async cancel(actor, renderId) {
|
||||||
|
record("render", "cancel", actor, renderId);
|
||||||
|
return repos.cancelRender(actor.spaceId, renderId);
|
||||||
|
},
|
||||||
|
async outputPath(actor, renderId, fileName) {
|
||||||
|
record("render", "outputPath", actor, { renderId, fileName });
|
||||||
|
const file = options.renderOutputs?.get(renderId);
|
||||||
|
return file?.endsWith(`/${fileName}`) ? file : null;
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
const marketing: MediaMarketingService = {
|
||||||
|
async autoPlan(actor, input) {
|
||||||
|
record("marketing", "autoPlan", actor, input);
|
||||||
|
return [];
|
||||||
|
},
|
||||||
|
async submitPlan(actor, input) {
|
||||||
|
record("marketing", "submitPlan", actor, input);
|
||||||
|
return { ok: false, errors: [] };
|
||||||
|
},
|
||||||
|
async introVariants(actor, input) {
|
||||||
|
record("marketing", "introVariants", actor, input);
|
||||||
|
return [];
|
||||||
|
},
|
||||||
|
async thumbnails(actor, input) {
|
||||||
|
record("marketing", "thumbnails", actor, input);
|
||||||
|
return [];
|
||||||
|
},
|
||||||
|
async copy(actor, input) {
|
||||||
|
record("marketing", "copy", actor, input);
|
||||||
|
const render = await repos.getRender(actor.spaceId, input.renderId);
|
||||||
|
return {
|
||||||
|
profileId: render.profileId,
|
||||||
|
language: input.language,
|
||||||
|
title: "Title",
|
||||||
|
hook: "Hook",
|
||||||
|
caption: "Caption",
|
||||||
|
hashtags: [],
|
||||||
|
issues: [],
|
||||||
|
};
|
||||||
|
},
|
||||||
|
async createPublication(actor, input) {
|
||||||
|
record("marketing", "createPublication", actor, input);
|
||||||
|
return repos.createPublication(actor.spaceId, actor.userId, {
|
||||||
|
...input,
|
||||||
|
trackingUrl: null,
|
||||||
|
scheduledAt: input.scheduledAt ? new Date(input.scheduledAt) : null,
|
||||||
|
});
|
||||||
|
},
|
||||||
|
async bundle(actor, publicationId) {
|
||||||
|
record("marketing", "bundle", actor, publicationId);
|
||||||
|
return repos.markPublicationBundled(actor.spaceId, publicationId, null);
|
||||||
|
},
|
||||||
|
async cancelPublication(actor, publicationId) {
|
||||||
|
record("marketing", "cancelPublication", actor, publicationId);
|
||||||
|
return repos.cancelPublication(actor.spaceId, publicationId);
|
||||||
|
},
|
||||||
|
async importMetricsCsv(actor, input) {
|
||||||
|
record("marketing", "importMetricsCsv", actor, input);
|
||||||
|
return { imported: 0, skipped: [] };
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
const deps = {
|
||||||
|
prisma: options.prisma,
|
||||||
|
repos,
|
||||||
|
jobs: { publish: async () => undefined },
|
||||||
|
inference: options.inference ?? null,
|
||||||
|
renderer: {
|
||||||
|
id: "fake",
|
||||||
|
hardwareEncoders: async () => ["h264_fake"],
|
||||||
|
render: async () => {
|
||||||
|
throw new Error("render is not part of the API fakes");
|
||||||
|
},
|
||||||
|
},
|
||||||
|
motion: { id: "fake", templates: () => [], render: async () => ({ path: "", hasAlpha: true }) },
|
||||||
|
publisher: { id: "fake", publish: async () => ({ status: "bundled", externalUrl: null }) },
|
||||||
|
shortener: null,
|
||||||
|
sources: {},
|
||||||
|
mediaRoot: options.mediaRoot,
|
||||||
|
profiles: async () => ({
|
||||||
|
profiles: new Map(
|
||||||
|
(options.profiles ?? []).map((profile) => [
|
||||||
|
profile.id,
|
||||||
|
{ profile, origin: "builtin" as const },
|
||||||
|
]),
|
||||||
|
),
|
||||||
|
errors: [],
|
||||||
|
}),
|
||||||
|
artifacts: {},
|
||||||
|
model: { generate: async () => null },
|
||||||
|
publish: async () => undefined,
|
||||||
|
now: options.now ?? (() => new Date()),
|
||||||
|
} as unknown as MediaServiceDeps;
|
||||||
|
|
||||||
|
return { services: { deps, ingest, privacy, render, marketing }, calls };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Every RPC path of the media contract, e.g. "sources/list". */
|
||||||
|
export function mediaProcedurePaths(): string[] {
|
||||||
|
const paths: string[] = [];
|
||||||
|
const walk = (node: Record<string, unknown>, prefix: string[]) => {
|
||||||
|
for (const [key, value] of Object.entries(node)) {
|
||||||
|
const child = value as Record<string, unknown>;
|
||||||
|
if ("input" in child && "output" in child) paths.push([...prefix, key].join("/"));
|
||||||
|
else walk(child, [...prefix, key]);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
walk(mediaRpcShapes, []);
|
||||||
|
return paths;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface MediaSampleIds {
|
||||||
|
sourceId: string;
|
||||||
|
assetId: string;
|
||||||
|
secondAssetId: string;
|
||||||
|
segmentId: string;
|
||||||
|
trackId: string;
|
||||||
|
personId: string;
|
||||||
|
markId: string;
|
||||||
|
collectionId: string;
|
||||||
|
brandKitId: string;
|
||||||
|
projectId: string;
|
||||||
|
planId: string;
|
||||||
|
renderId: string;
|
||||||
|
publicationId: string;
|
||||||
|
mappingId: string;
|
||||||
|
uploadId: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* How a procedure must treat another space's ids: refuse them as not found,
|
||||||
|
* answer only from the caller's own space, or be covered by a dedicated test.
|
||||||
|
*/
|
||||||
|
export type ForeignExpectation = "not_found" | "scoped" | "dedicated";
|
||||||
|
|
||||||
|
/** A valid input for every media procedure, built from one space's ids. */
|
||||||
|
export function sampleMediaInputs(
|
||||||
|
ids: MediaSampleIds,
|
||||||
|
kit: unknown,
|
||||||
|
): Record<string, { input: unknown; foreign: ForeignExpectation }> {
|
||||||
|
const at = "2026-09-01T10:00:00.000Z";
|
||||||
|
return {
|
||||||
|
capabilities: { input: {}, foreign: "scoped" },
|
||||||
|
"sources/list": { input: {}, foreign: "scoped" },
|
||||||
|
"sources/createFolder": { input: { label: "Footage", path: "/" }, foreign: "dedicated" },
|
||||||
|
"sources/registerClient": {
|
||||||
|
input: { kind: "ios_library", label: "Phone", deviceId: "device-sample" },
|
||||||
|
foreign: "scoped",
|
||||||
|
},
|
||||||
|
"sources/pushItems": {
|
||||||
|
input: { sourceId: ids.sourceId, items: [], syncToken: null, complete: true },
|
||||||
|
foreign: "not_found",
|
||||||
|
},
|
||||||
|
"sources/sync": { input: { sourceId: ids.sourceId }, foreign: "not_found" },
|
||||||
|
"sources/remove": { input: { sourceId: ids.sourceId }, foreign: "not_found" },
|
||||||
|
"assets/search": { input: {}, foreign: "scoped" },
|
||||||
|
"assets/get": { input: { assetId: ids.assetId }, foreign: "not_found" },
|
||||||
|
"assets/setPaidOnly": {
|
||||||
|
input: { segmentIds: [ids.segmentId], paidOnly: true },
|
||||||
|
foreign: "not_found",
|
||||||
|
},
|
||||||
|
"assets/requestUpload": { input: { assetIds: [ids.assetId] }, foreign: "not_found" },
|
||||||
|
"uploads/begin": {
|
||||||
|
input: {
|
||||||
|
assetId: ids.secondAssetId,
|
||||||
|
sizeBytes: 10,
|
||||||
|
sha256: "a".repeat(64),
|
||||||
|
fileName: "clip.mp4",
|
||||||
|
},
|
||||||
|
foreign: "not_found",
|
||||||
|
},
|
||||||
|
"uploads/status": { input: { uploadId: ids.uploadId }, foreign: "not_found" },
|
||||||
|
"uploads/complete": { input: { uploadId: ids.uploadId }, foreign: "not_found" },
|
||||||
|
"collections/list": { input: {}, foreign: "scoped" },
|
||||||
|
"collections/get": { input: { collectionId: ids.collectionId }, foreign: "not_found" },
|
||||||
|
"collections/rename": {
|
||||||
|
input: { collectionId: ids.collectionId, title: "Renamed" },
|
||||||
|
foreign: "not_found",
|
||||||
|
},
|
||||||
|
"collections/merge": {
|
||||||
|
input: { collectionIds: [ids.collectionId, `${ids.collectionId}x`] },
|
||||||
|
foreign: "not_found",
|
||||||
|
},
|
||||||
|
"collections/split": {
|
||||||
|
input: { collectionId: ids.collectionId, atAssetId: ids.assetId },
|
||||||
|
foreign: "not_found",
|
||||||
|
},
|
||||||
|
"collections/createManual": {
|
||||||
|
input: { title: "Picks", assetIds: [ids.assetId] },
|
||||||
|
foreign: "not_found",
|
||||||
|
},
|
||||||
|
"collections/recluster": { input: {}, foreign: "scoped" },
|
||||||
|
"people/list": { input: {}, foreign: "scoped" },
|
||||||
|
"people/update": { input: { personId: ids.personId, displayName: "X" }, foreign: "not_found" },
|
||||||
|
"people/merge": {
|
||||||
|
input: { personIds: [ids.personId, `${ids.personId}x`] },
|
||||||
|
foreign: "not_found",
|
||||||
|
},
|
||||||
|
"people/assignTrack": { input: { trackId: ids.trackId, personId: null }, foreign: "not_found" },
|
||||||
|
"people/recordConsent": {
|
||||||
|
input: {
|
||||||
|
personId: ids.personId,
|
||||||
|
consentArtifactId: null,
|
||||||
|
consentRecordedAt: at,
|
||||||
|
ageVerificationMethod: "platform_verified",
|
||||||
|
ageVerifiedAt: at,
|
||||||
|
},
|
||||||
|
foreign: "not_found",
|
||||||
|
},
|
||||||
|
"people/remove": { input: { personId: ids.personId }, foreign: "not_found" },
|
||||||
|
"marks/list": { input: { assetId: ids.assetId }, foreign: "scoped" },
|
||||||
|
"marks/create": {
|
||||||
|
input: { kind: "tattoo", assetId: ids.assetId, prompt: { tMs: 0 }, treatment: "blur" },
|
||||||
|
foreign: "not_found",
|
||||||
|
},
|
||||||
|
"marks/update": { input: { markId: ids.markId, label: "x" }, foreign: "not_found" },
|
||||||
|
"marks/remove": { input: { markId: ids.markId }, foreign: "not_found" },
|
||||||
|
"brandKits/list": { input: {}, foreign: "scoped" },
|
||||||
|
"brandKits/upsert": { input: { id: ids.brandKitId, kit }, foreign: "not_found" },
|
||||||
|
"brandKits/remove": { input: { id: ids.brandKitId }, foreign: "not_found" },
|
||||||
|
"brandKits/templates": { input: {}, foreign: "scoped" },
|
||||||
|
"brandKits/introVariants": { input: { id: ids.brandKitId }, foreign: "not_found" },
|
||||||
|
"profiles/list": { input: {}, foreign: "scoped" },
|
||||||
|
"profiles/get": { input: { profileId: "example-short-vertical" }, foreign: "scoped" },
|
||||||
|
"projects/list": { input: {}, foreign: "scoped" },
|
||||||
|
"projects/get": { input: { projectId: ids.projectId }, foreign: "not_found" },
|
||||||
|
"projects/create": {
|
||||||
|
input: {
|
||||||
|
title: "Teaser",
|
||||||
|
goal: "teaser",
|
||||||
|
collectionIds: [ids.collectionId],
|
||||||
|
profileIds: ["example-short-vertical"],
|
||||||
|
},
|
||||||
|
foreign: "not_found",
|
||||||
|
},
|
||||||
|
"projects/update": { input: { projectId: ids.projectId, title: "X" }, foreign: "not_found" },
|
||||||
|
"projects/remove": { input: { projectId: ids.projectId }, foreign: "not_found" },
|
||||||
|
"projects/autoPlan": { input: { projectId: ids.projectId }, foreign: "not_found" },
|
||||||
|
"plans/submit": { input: { projectId: ids.projectId, edl: {} }, foreign: "not_found" },
|
||||||
|
"renders/start": { input: { planId: ids.planId, quality: "draft" }, foreign: "not_found" },
|
||||||
|
"renders/get": { input: { renderId: ids.renderId }, foreign: "not_found" },
|
||||||
|
"renders/cancel": { input: { renderId: ids.renderId }, foreign: "not_found" },
|
||||||
|
"renders/thumbnails": { input: { renderId: ids.renderId }, foreign: "not_found" },
|
||||||
|
"marketing/copy": { input: { renderId: ids.renderId }, foreign: "not_found" },
|
||||||
|
"publications/list": { input: { projectId: ids.projectId }, foreign: "scoped" },
|
||||||
|
"publications/create": { input: { renderId: ids.renderId }, foreign: "not_found" },
|
||||||
|
"publications/bundle": { input: { publicationId: ids.publicationId }, foreign: "not_found" },
|
||||||
|
"publications/markPublished": {
|
||||||
|
input: { publicationId: ids.publicationId, externalUrl: null },
|
||||||
|
foreign: "not_found",
|
||||||
|
},
|
||||||
|
"publications/cancel": { input: { publicationId: ids.publicationId }, foreign: "not_found" },
|
||||||
|
"metrics/mappings": { input: {}, foreign: "scoped" },
|
||||||
|
"metrics/saveMapping": {
|
||||||
|
input: {
|
||||||
|
id: ids.mappingId,
|
||||||
|
name: "Imported",
|
||||||
|
columns: { publication: "id", observedAt: "date" },
|
||||||
|
publicationKey: "id",
|
||||||
|
},
|
||||||
|
foreign: "not_found",
|
||||||
|
},
|
||||||
|
"metrics/importCsv": {
|
||||||
|
input: { mappingId: ids.mappingId, csv: "id,date" },
|
||||||
|
foreign: "not_found",
|
||||||
|
},
|
||||||
|
"metrics/record": {
|
||||||
|
input: {
|
||||||
|
publicationId: ids.publicationId,
|
||||||
|
observedAt: at,
|
||||||
|
views: 1,
|
||||||
|
clicks: 0,
|
||||||
|
conversions: 0,
|
||||||
|
revenueCents: 0,
|
||||||
|
currency: "EUR",
|
||||||
|
},
|
||||||
|
foreign: "not_found",
|
||||||
|
},
|
||||||
|
"metrics/funnel": { input: { projectId: ids.projectId }, foreign: "not_found" },
|
||||||
|
"metrics/segments": { input: { projectId: ids.projectId }, foreign: "not_found" },
|
||||||
|
"metrics/resetTeaserModel": { input: {}, foreign: "scoped" },
|
||||||
|
};
|
||||||
|
}
|
||||||
244
apps/api/src/media-uploads.ts
Normal file
244
apps/api/src/media-uploads.ts
Normal file
@ -0,0 +1,244 @@
|
|||||||
|
import { createHash, randomUUID } from "node:crypto";
|
||||||
|
import { createReadStream } from "node:fs";
|
||||||
|
import { mkdir, open, rename, rm, stat } from "node:fs/promises";
|
||||||
|
import path from "node:path";
|
||||||
|
import type { MediaActor, MediaServices } from "@rakazo/adapters";
|
||||||
|
import type { MediaUpload } from "@rakazo/contracts";
|
||||||
|
import type { MediaRepos } from "@rakazo/db";
|
||||||
|
import { MediaStateConflictError } from "@rakazo/db";
|
||||||
|
import { resolveInside } from "@rakazo/media/node";
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Resumable chunked uploads. Each chunk lands in its own file
|
||||||
|
* (<spaceId>/uploads/<uploadId>/chunks/<index>.part, written to a temp name and
|
||||||
|
* renamed, so a repeated or concurrent PUT of the same index is harmless).
|
||||||
|
* Completion concatenates the chunks in order into <spaceId>/uploads/<uploadId>/file
|
||||||
|
* while hashing, verifies size and SHA-256, and hands the file to ingest.
|
||||||
|
*
|
||||||
|
* Limits: the contract caps the total size, every chunk must have its exact
|
||||||
|
* length, one chunk must arrive within MEDIA_CHUNK_TIMEOUT_MS, and an upload
|
||||||
|
* that saw no progress for MEDIA_UPLOAD_IDLE_MS expires and must start over.
|
||||||
|
*/
|
||||||
|
|
||||||
|
/** A single chunk must arrive within this window; slow links resume per chunk. */
|
||||||
|
export const MEDIA_CHUNK_TIMEOUT_MS = 10 * 60_000;
|
||||||
|
/** An open upload without a new chunk for this long is failed and its bytes removed. */
|
||||||
|
export const MEDIA_UPLOAD_IDLE_MS = 24 * 60 * 60_000;
|
||||||
|
|
||||||
|
export class UploadChunkError extends Error {
|
||||||
|
constructor(
|
||||||
|
readonly status: 400 | 409 | 413,
|
||||||
|
message: string,
|
||||||
|
) {
|
||||||
|
super(message);
|
||||||
|
this.name = "UploadChunkError";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type UploadRow = Awaited<ReturnType<MediaRepos["getUploadRecord"]>>;
|
||||||
|
type UploadSize = Pick<UploadRow, "sizeBytes" | "chunkBytes">;
|
||||||
|
|
||||||
|
export function uploadChunkCount(row: UploadSize): number {
|
||||||
|
return Number((row.sizeBytes + BigInt(row.chunkBytes) - 1n) / BigInt(row.chunkBytes));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Exact byte length chunk `index` must have: every chunk is full except the last. */
|
||||||
|
export function expectedChunkBytes(row: UploadSize, index: number): number {
|
||||||
|
const count = uploadChunkCount(row);
|
||||||
|
if (index < count - 1) return row.chunkBytes;
|
||||||
|
return Number(row.sizeBytes - BigInt(row.chunkBytes) * BigInt(count - 1));
|
||||||
|
}
|
||||||
|
|
||||||
|
function uploadDir(row: Pick<UploadRow, "spaceId" | "id">): string {
|
||||||
|
return `${row.spaceId}/uploads/${row.id}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function mediaPath(mediaRoot: string, relative: string): Promise<string> {
|
||||||
|
await mkdir(mediaRoot, { recursive: true });
|
||||||
|
return resolveInside(mediaRoot, relative);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function chunksDir(mediaRoot: string, row: UploadRow): Promise<string> {
|
||||||
|
const dir = await mediaPath(mediaRoot, `${uploadDir(row)}/chunks`);
|
||||||
|
await mkdir(dir, { recursive: true });
|
||||||
|
return dir;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function removeUploadFiles(mediaRoot: string, row: UploadRow): Promise<void> {
|
||||||
|
await rm(await mediaPath(mediaRoot, uploadDir(row)), { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
|
||||||
|
function isIdle(services: MediaServices, row: UploadRow): boolean {
|
||||||
|
return services.deps.now().getTime() - row.updatedAt.getTime() > MEDIA_UPLOAD_IDLE_MS;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Fail an idle open upload and drop its bytes. Returns true when it expired. */
|
||||||
|
async function expireIfIdle(services: MediaServices, row: UploadRow): Promise<boolean> {
|
||||||
|
if (row.status !== "open" || !isIdle(services, row)) return false;
|
||||||
|
await services.deps.repos.failUpload(row.spaceId, row.id).catch((error: unknown) => {
|
||||||
|
// A concurrent request already closed it; the files still go.
|
||||||
|
if (!(error instanceof MediaStateConflictError)) throw error;
|
||||||
|
});
|
||||||
|
await removeUploadFiles(services.deps.mediaRoot, row);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Open an upload, or resume the open one for the same asset and content. An
|
||||||
|
* idle one expires first so the client starts over with a fresh upload.
|
||||||
|
*/
|
||||||
|
export async function beginMediaUpload(
|
||||||
|
services: MediaServices,
|
||||||
|
actor: MediaActor,
|
||||||
|
input: { assetId: string; sizeBytes: number; sha256: string; fileName: string },
|
||||||
|
): Promise<MediaUpload> {
|
||||||
|
const { repos } = services.deps;
|
||||||
|
const asset = await repos.getAssetRecord(actor.spaceId, input.assetId);
|
||||||
|
// Device items wait for their original; files read from a source never upload.
|
||||||
|
if (asset.storage !== "remote" && asset.ingestState !== "awaiting_upload") {
|
||||||
|
throw new MediaStateConflictError("This item does not need an upload");
|
||||||
|
}
|
||||||
|
const begin = () =>
|
||||||
|
repos.beginUpload(actor.spaceId, { ...input, assetId: asset.id, userId: actor.userId });
|
||||||
|
const upload = await begin();
|
||||||
|
const row = await repos.getUploadRecord(actor.spaceId, upload.id);
|
||||||
|
return (await expireIfIdle(services, row)) ? begin() : upload;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Stream one chunk body to disk without buffering more than one network read.
|
||||||
|
* Rejects bodies longer or shorter than the chunk's exact size.
|
||||||
|
*/
|
||||||
|
export async function writeUploadChunk(input: {
|
||||||
|
services: MediaServices;
|
||||||
|
spaceId: string;
|
||||||
|
uploadId: string;
|
||||||
|
index: number;
|
||||||
|
body: ReadableStream<Uint8Array> | null;
|
||||||
|
declaredLength: number | null;
|
||||||
|
/** Aborting stops reading the body; the chunk is then discarded as incomplete. */
|
||||||
|
signal?: AbortSignal;
|
||||||
|
}): Promise<MediaUpload> {
|
||||||
|
const { services } = input;
|
||||||
|
const { repos, mediaRoot } = services.deps;
|
||||||
|
const row = await repos.getUploadRecord(input.spaceId, input.uploadId);
|
||||||
|
if (await expireIfIdle(services, row)) throw new UploadChunkError(409, "Upload expired");
|
||||||
|
if (row.status !== "open") throw new UploadChunkError(409, "Upload is not open");
|
||||||
|
const count = uploadChunkCount(row);
|
||||||
|
if (!Number.isInteger(input.index) || input.index < 0 || input.index >= count) {
|
||||||
|
throw new UploadChunkError(400, "Chunk index out of range");
|
||||||
|
}
|
||||||
|
const expected = expectedChunkBytes(row, input.index);
|
||||||
|
if (input.declaredLength !== null && input.declaredLength > expected) {
|
||||||
|
throw new UploadChunkError(413, "Chunk is too large");
|
||||||
|
}
|
||||||
|
if (!input.body) throw new UploadChunkError(400, "Chunk is empty");
|
||||||
|
const dir = await chunksDir(mediaRoot, row);
|
||||||
|
const finalPath = path.join(dir, `${input.index}.part`);
|
||||||
|
const tempPath = path.join(dir, `${input.index}.${randomUUID()}.tmp`);
|
||||||
|
const handle = await open(tempPath, "wx");
|
||||||
|
let written = 0;
|
||||||
|
let ok = false;
|
||||||
|
const reader = input.body.getReader();
|
||||||
|
const stop = () => void reader.cancel().catch(() => undefined);
|
||||||
|
input.signal?.addEventListener("abort", stop, { once: true });
|
||||||
|
try {
|
||||||
|
if (input.signal?.aborted) stop();
|
||||||
|
for (;;) {
|
||||||
|
const { done, value } = await reader.read();
|
||||||
|
if (done) break;
|
||||||
|
written += value.byteLength;
|
||||||
|
if (written > expected) {
|
||||||
|
void reader.cancel().catch(() => undefined);
|
||||||
|
throw new UploadChunkError(413, "Chunk is too large");
|
||||||
|
}
|
||||||
|
await handle.write(value);
|
||||||
|
}
|
||||||
|
if (written !== expected) throw new UploadChunkError(400, "Chunk is incomplete");
|
||||||
|
ok = true;
|
||||||
|
} finally {
|
||||||
|
input.signal?.removeEventListener("abort", stop);
|
||||||
|
reader.releaseLock();
|
||||||
|
await handle.close();
|
||||||
|
if (!ok) await rm(tempPath, { force: true });
|
||||||
|
}
|
||||||
|
await rename(tempPath, finalPath);
|
||||||
|
return repos.markChunkReceived(input.spaceId, input.uploadId, input.index);
|
||||||
|
}
|
||||||
|
|
||||||
|
const completing = new Set<string>();
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Assemble a fully received upload, verify it, hand it to ingest, and close it.
|
||||||
|
* A checksum mismatch fails the upload so the client starts a fresh one.
|
||||||
|
*/
|
||||||
|
export async function completeMediaUpload(
|
||||||
|
services: MediaServices,
|
||||||
|
actor: MediaActor,
|
||||||
|
uploadId: string,
|
||||||
|
): Promise<MediaUpload> {
|
||||||
|
const { repos, mediaRoot } = services.deps;
|
||||||
|
const row = await repos.getUploadRecord(actor.spaceId, uploadId);
|
||||||
|
if (row.status === "complete") return repos.uploadStatus(actor.spaceId, uploadId);
|
||||||
|
if (await expireIfIdle(services, row)) throw new UploadChunkError(409, "Upload expired");
|
||||||
|
if (row.status !== "open") throw new MediaStateConflictError("Upload is not open");
|
||||||
|
const received = new Set(row.receivedChunks);
|
||||||
|
const count = uploadChunkCount(row);
|
||||||
|
for (let index = 0; index < count; index++) {
|
||||||
|
if (!received.has(index)) throw new MediaStateConflictError("Upload is missing chunks");
|
||||||
|
}
|
||||||
|
if (completing.has(row.id)) throw new MediaStateConflictError("Upload is already completing");
|
||||||
|
completing.add(row.id);
|
||||||
|
try {
|
||||||
|
const dir = await chunksDir(mediaRoot, row);
|
||||||
|
const relativeFile = `${uploadDir(row)}/file`;
|
||||||
|
const filePath = await mediaPath(mediaRoot, relativeFile);
|
||||||
|
const tempFile = await mediaPath(mediaRoot, `${uploadDir(row)}/file.${randomUUID()}.tmp`);
|
||||||
|
const hash = createHash("sha256");
|
||||||
|
let total = 0;
|
||||||
|
const out = await open(tempFile, "wx");
|
||||||
|
try {
|
||||||
|
for (let index = 0; index < count; index++) {
|
||||||
|
const chunkPath = path.join(dir, `${index}.part`);
|
||||||
|
const info = await stat(chunkPath).catch(() => null);
|
||||||
|
if (!info || info.size !== expectedChunkBytes(row, index)) {
|
||||||
|
throw new MediaStateConflictError("Upload is missing chunks");
|
||||||
|
}
|
||||||
|
for await (const buffer of createReadStream(chunkPath) as AsyncIterable<Buffer>) {
|
||||||
|
hash.update(buffer);
|
||||||
|
total += buffer.byteLength;
|
||||||
|
await out.write(buffer);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
await out.close();
|
||||||
|
await rm(tempFile, { force: true });
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
await out.close();
|
||||||
|
const sha256 = hash.digest("hex");
|
||||||
|
if (sha256 !== row.sha256 || BigInt(total) !== row.sizeBytes) {
|
||||||
|
await removeUploadFiles(mediaRoot, row);
|
||||||
|
await repos.failUpload(actor.spaceId, uploadId);
|
||||||
|
throw new UploadChunkError(400, "Upload does not match its checksum");
|
||||||
|
}
|
||||||
|
await rename(tempFile, filePath);
|
||||||
|
await rm(dir, { recursive: true, force: true });
|
||||||
|
await services.ingest.uploadCompleted(actor, {
|
||||||
|
assetId: row.assetId,
|
||||||
|
tempPath: filePath,
|
||||||
|
sha256,
|
||||||
|
fileName: row.fileName,
|
||||||
|
});
|
||||||
|
// Ingest normally moves the file into the asset folder and closes the
|
||||||
|
// upload itself; otherwise the assembled file stays as the original.
|
||||||
|
const after = await repos.getUploadRecord(actor.spaceId, uploadId);
|
||||||
|
if (after.status === "complete") {
|
||||||
|
await removeUploadFiles(mediaRoot, row);
|
||||||
|
return repos.uploadStatus(actor.spaceId, uploadId);
|
||||||
|
}
|
||||||
|
return repos.completeUpload(actor.spaceId, uploadId, { storagePath: relativeFile });
|
||||||
|
} finally {
|
||||||
|
completing.delete(row.id);
|
||||||
|
}
|
||||||
|
}
|
||||||
516
apps/api/src/media.postgres.test.ts
Normal file
516
apps/api/src/media.postgres.test.ts
Normal file
@ -0,0 +1,516 @@
|
|||||||
|
import { createHash } from "node:crypto";
|
||||||
|
import { mkdir, mkdtemp, readFile, rm, stat, writeFile } from "node:fs/promises";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import path from "node:path";
|
||||||
|
import { createRouterClient, ORPCError } from "@orpc/server";
|
||||||
|
import type { Actor } from "@rakazo/contracts";
|
||||||
|
import type { MediaRepos } from "@rakazo/db";
|
||||||
|
import { createMediaRepos } from "@rakazo/db";
|
||||||
|
import { Hono } from "hono";
|
||||||
|
import { afterAll, beforeAll, describe, expect, it } from "vitest";
|
||||||
|
import type { MediaTestDb, SeededMediaSpace } from "../../../packages/db/src/media-fixtures.js";
|
||||||
|
import {
|
||||||
|
openMediaTestDb,
|
||||||
|
seedMediaSpace,
|
||||||
|
sourceItem,
|
||||||
|
testBrandKit,
|
||||||
|
} from "../../../packages/db/src/media-fixtures.js";
|
||||||
|
import { testProfile } from "../../../packages/media/src/test-fixtures.js";
|
||||||
|
import { createMediaRouter } from "./media.js";
|
||||||
|
import { mountMediaHttpRoutes } from "./media-http.js";
|
||||||
|
import type { MediaServiceCall } from "./media-test-fakes.js";
|
||||||
|
import { fakeMediaServices, sampleMediaInputs } from "./media-test-fakes.js";
|
||||||
|
import { MEDIA_UPLOAD_IDLE_MS } from "./media-uploads.js";
|
||||||
|
|
||||||
|
const databaseUrl = process.env.DATABASE_URL;
|
||||||
|
const describePostgres =
|
||||||
|
process.env.VERIFY_DATABASE && databaseUrl ? describe.sequential : describe.skip;
|
||||||
|
|
||||||
|
type Space = { userId: string; spaceId: string };
|
||||||
|
|
||||||
|
function actorOf(space: Space, isDeploymentOwner = false): Actor {
|
||||||
|
return {
|
||||||
|
userId: space.userId,
|
||||||
|
spaceId: space.spaceId,
|
||||||
|
email: `${space.userId}@rakazo.test`,
|
||||||
|
isDeploymentOwner,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function procedure(client: unknown, rpcPath: string) {
|
||||||
|
let node = client;
|
||||||
|
for (const key of rpcPath.split("/")) node = (node as Record<string, unknown>)[key];
|
||||||
|
return node as (input: unknown) => Promise<unknown>;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function rpcCode(promise: Promise<unknown>): Promise<string> {
|
||||||
|
try {
|
||||||
|
await promise;
|
||||||
|
} catch (error) {
|
||||||
|
if (error instanceof ORPCError) return error.code;
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
return "OK";
|
||||||
|
}
|
||||||
|
|
||||||
|
function sha256(bytes: Uint8Array): string {
|
||||||
|
return createHash("sha256").update(bytes).digest("hex");
|
||||||
|
}
|
||||||
|
|
||||||
|
describePostgres("media API authorization and uploads (PostgreSQL)", () => {
|
||||||
|
let db: MediaTestDb;
|
||||||
|
let repos: MediaRepos;
|
||||||
|
let mediaRoot: string;
|
||||||
|
let a: Space;
|
||||||
|
let b: Space;
|
||||||
|
let seededA: SeededMediaSpace;
|
||||||
|
let seededB: SeededMediaSpace;
|
||||||
|
let calls: MediaServiceCall[];
|
||||||
|
let renderOutputs: Map<string, string>;
|
||||||
|
let rpc: (actor: Actor) => unknown;
|
||||||
|
let app: Hono;
|
||||||
|
let currentActor: Actor | null;
|
||||||
|
|
||||||
|
function setup(options: { now?: () => Date; chunkTimeoutMs?: number } = {}) {
|
||||||
|
const fake = fakeMediaServices({
|
||||||
|
repos,
|
||||||
|
mediaRoot,
|
||||||
|
prisma: db.prisma,
|
||||||
|
profiles: [testProfile()],
|
||||||
|
renderOutputs,
|
||||||
|
...(options.now ? { now: options.now } : {}),
|
||||||
|
});
|
||||||
|
calls = fake.calls;
|
||||||
|
const router = createMediaRouter({
|
||||||
|
prisma: db.prisma,
|
||||||
|
services: fake.services,
|
||||||
|
dataDir: path.join(mediaRoot, ".."),
|
||||||
|
});
|
||||||
|
rpc = (actor) => createRouterClient(router, { context: { actor } });
|
||||||
|
app = new Hono();
|
||||||
|
mountMediaHttpRoutes(
|
||||||
|
app,
|
||||||
|
{
|
||||||
|
services: fake.services,
|
||||||
|
...(options.chunkTimeoutMs ? { chunkTimeoutMs: options.chunkTimeoutMs } : {}),
|
||||||
|
},
|
||||||
|
async () => currentActor,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function snapshot(spaceId: string) {
|
||||||
|
const p = db.prisma;
|
||||||
|
const where = { spaceId };
|
||||||
|
const orderBy = { id: "asc" as const };
|
||||||
|
return JSON.stringify(
|
||||||
|
[
|
||||||
|
await p.mediaSource.findMany({ where, orderBy }),
|
||||||
|
await p.mediaAsset.findMany({ where, orderBy }),
|
||||||
|
await p.mediaUpload.findMany({ where, orderBy }),
|
||||||
|
await p.mediaSegment.findMany({ where, orderBy }),
|
||||||
|
await p.mediaAnalysis.findMany({ where, orderBy }),
|
||||||
|
await p.mediaCollection.findMany({ where, orderBy }),
|
||||||
|
await p.mediaCollectionItem.findMany({
|
||||||
|
where: { collection: where },
|
||||||
|
orderBy: [{ collectionId: "asc" }, { assetId: "asc" }],
|
||||||
|
}),
|
||||||
|
await p.mediaPerson.findMany({ where, orderBy }),
|
||||||
|
await p.mediaFaceTrack.findMany({ where, orderBy }),
|
||||||
|
await p.mediaMark.findMany({ where, orderBy }),
|
||||||
|
await p.mediaBrandKit.findMany({ where, orderBy }),
|
||||||
|
await p.mediaProject.findMany({ where, orderBy }),
|
||||||
|
await p.mediaEditPlan.findMany({ where, orderBy }),
|
||||||
|
await p.mediaRender.findMany({ where, orderBy }),
|
||||||
|
await p.mediaPublication.findMany({ where, orderBy }),
|
||||||
|
await p.mediaMetric.findMany({ where, orderBy }),
|
||||||
|
await p.mediaMetricMapping.findMany({ where, orderBy }),
|
||||||
|
],
|
||||||
|
(_key, value) => (typeof value === "bigint" ? value.toString() : value),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function artifactIn(space: Space): Promise<string> {
|
||||||
|
const artifact = await db.prisma.artifact.create({
|
||||||
|
data: {
|
||||||
|
spaceId: space.spaceId,
|
||||||
|
userId: space.userId,
|
||||||
|
name: "consent.pdf",
|
||||||
|
mimeType: "application/pdf",
|
||||||
|
size: 1,
|
||||||
|
hash: "0".repeat(64),
|
||||||
|
storageKey: `test/${space.spaceId}/consent.pdf`,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
return artifact.id;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A device item in `space` that waits for its original. */
|
||||||
|
async function remoteAsset(space: Space, index: number): Promise<string> {
|
||||||
|
const source = await procedure(
|
||||||
|
rpc(actorOf(space)),
|
||||||
|
"sources/registerClient",
|
||||||
|
)({ kind: "ios_library", label: "Phone", deviceId: `device-${index}` });
|
||||||
|
const synced = await repos.upsertSourceItems(space.spaceId, (source as { id: string }).id, [
|
||||||
|
sourceItem(index),
|
||||||
|
]);
|
||||||
|
const assetId = synced.changedAssetIds[0] as string;
|
||||||
|
await repos.requestUpload(space.spaceId, [assetId]);
|
||||||
|
return assetId;
|
||||||
|
}
|
||||||
|
|
||||||
|
function putChunk(uploadId: string, index: number, body: BodyInit, actor: Actor | null) {
|
||||||
|
currentActor = actor;
|
||||||
|
return app.request(`/api/media/uploads/${uploadId}/chunks/${index}`, { method: "PUT", body });
|
||||||
|
}
|
||||||
|
|
||||||
|
beforeAll(async () => {
|
||||||
|
db = openMediaTestDb(databaseUrl!);
|
||||||
|
repos = createMediaRepos(db.prisma);
|
||||||
|
const root = await mkdtemp(path.join(tmpdir(), "media-api-pg-"));
|
||||||
|
mediaRoot = path.join(root, "media");
|
||||||
|
await mkdir(mediaRoot);
|
||||||
|
renderOutputs = new Map();
|
||||||
|
a = await db.newUser();
|
||||||
|
b = await db.newUser();
|
||||||
|
seededA = await seedMediaSpace(repos, a.spaceId, a.userId);
|
||||||
|
seededB = await seedMediaSpace(repos, b.spaceId, b.userId);
|
||||||
|
setup();
|
||||||
|
});
|
||||||
|
|
||||||
|
afterAll(async () => {
|
||||||
|
await db?.cleanup();
|
||||||
|
if (mediaRoot) await rm(path.dirname(mediaRoot), { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("refuses another space's ids in every media procedure and leaves that space untouched", async () => {
|
||||||
|
const before = await snapshot(a.spaceId);
|
||||||
|
const foreignIds = Object.values(seededA);
|
||||||
|
const intruder = rpc(actorOf(b));
|
||||||
|
const inputs = sampleMediaInputs(seededA, testBrandKit);
|
||||||
|
for (const [rpcPath, sample] of Object.entries(inputs)) {
|
||||||
|
if (sample.foreign === "dedicated") continue;
|
||||||
|
const call = procedure(intruder, rpcPath)(sample.input);
|
||||||
|
if (sample.foreign === "not_found") {
|
||||||
|
expect(await rpcCode(call), rpcPath).toBe("NOT_FOUND");
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const result = JSON.stringify(await call);
|
||||||
|
for (const id of foreignIds) expect(result, rpcPath).not.toContain(id);
|
||||||
|
}
|
||||||
|
expect(await snapshot(a.spaceId)).toBe(before);
|
||||||
|
// No service ever received the other space's ids.
|
||||||
|
const leaked = calls.filter((call) =>
|
||||||
|
foreignIds.some((id) => JSON.stringify(call.input ?? null).includes(id)),
|
||||||
|
);
|
||||||
|
expect(leaked).toEqual([]);
|
||||||
|
expect(calls.every((call) => call.actor.spaceId === b.spaceId)).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("serves the same inputs inside the caller's own space", async () => {
|
||||||
|
// Positive control: NOT_FOUND above comes from isolation, not from bad samples.
|
||||||
|
const fabricated = new Set(["collections/merge", "people/merge"]);
|
||||||
|
for (const [rpcPath, sample] of Object.entries(sampleMediaInputs(seededA, testBrandKit))) {
|
||||||
|
if (sample.foreign !== "not_found" || fabricated.has(rpcPath)) continue;
|
||||||
|
const space = await db.newUser();
|
||||||
|
const seeded = await seedMediaSpace(repos, space.spaceId, space.userId);
|
||||||
|
const input = sampleMediaInputs(seeded, testBrandKit)[rpcPath]?.input;
|
||||||
|
const code = await rpcCode(procedure(rpc(actorOf(space)), rpcPath)(input));
|
||||||
|
expect(code, rpcPath).not.toBe("NOT_FOUND");
|
||||||
|
}
|
||||||
|
}, 120_000);
|
||||||
|
|
||||||
|
it("refuses another space's records referenced from the caller's own ones", async () => {
|
||||||
|
const own = rpc(actorOf(b));
|
||||||
|
const foreignArtifact = await artifactIn(a);
|
||||||
|
const attempts: Array<[string, unknown]> = [
|
||||||
|
["projects/update", { projectId: seededB.projectId, collectionIds: [seededA.collectionId] }],
|
||||||
|
["projects/update", { projectId: seededB.projectId, brandKitId: seededA.brandKitId }],
|
||||||
|
[
|
||||||
|
"projects/create",
|
||||||
|
{
|
||||||
|
title: "Mixed",
|
||||||
|
goal: "teaser",
|
||||||
|
assetIds: [seededA.assetId],
|
||||||
|
profileIds: ["example-short-vertical"],
|
||||||
|
},
|
||||||
|
],
|
||||||
|
["people/assignTrack", { trackId: seededB.trackId, personId: seededA.personId }],
|
||||||
|
[
|
||||||
|
"marks/create",
|
||||||
|
{
|
||||||
|
kind: "tattoo",
|
||||||
|
assetId: seededB.assetId,
|
||||||
|
personId: seededA.personId,
|
||||||
|
prompt: { tMs: 0 },
|
||||||
|
treatment: "blur",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
["marks/update", { markId: seededB.markId, personId: seededA.personId }],
|
||||||
|
[
|
||||||
|
"assets/setPaidOnly",
|
||||||
|
{ segmentIds: [seededB.segmentId, seededA.segmentId], paidOnly: true },
|
||||||
|
],
|
||||||
|
["collections/merge", { collectionIds: [seededB.collectionId, seededA.collectionId] }],
|
||||||
|
["people/merge", { personIds: [seededB.personId, seededA.personId] }],
|
||||||
|
["brandKits/upsert", { kit: { ...testBrandKit, logoArtifactIds: [foreignArtifact] } }],
|
||||||
|
[
|
||||||
|
"people/recordConsent",
|
||||||
|
{
|
||||||
|
personId: seededB.personId,
|
||||||
|
consentArtifactId: foreignArtifact,
|
||||||
|
consentRecordedAt: "2026-09-01T10:00:00.000Z",
|
||||||
|
ageVerificationMethod: "document_on_file",
|
||||||
|
ageVerifiedAt: "2026-09-01T10:00:00.000Z",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
];
|
||||||
|
const before = await snapshot(a.spaceId);
|
||||||
|
for (const [rpcPath, input] of attempts) {
|
||||||
|
expect(await rpcCode(procedure(own, rpcPath)(input)), rpcPath).toBe("NOT_FOUND");
|
||||||
|
}
|
||||||
|
expect(await snapshot(a.spaceId)).toBe(before);
|
||||||
|
const person = await db.prisma.mediaPerson.findUniqueOrThrow({
|
||||||
|
where: { id: seededB.personId },
|
||||||
|
});
|
||||||
|
expect(person.consentRecordedAt).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("records consent only as the signed-in member and ignores it on updates", async () => {
|
||||||
|
const space = await db.newUser();
|
||||||
|
const seeded = await seedMediaSpace(repos, space.spaceId, space.userId);
|
||||||
|
const client = rpc(actorOf(space));
|
||||||
|
await procedure(
|
||||||
|
client,
|
||||||
|
"people/update",
|
||||||
|
)({
|
||||||
|
personId: seeded.personId,
|
||||||
|
displayName: "Person 1",
|
||||||
|
consentRecordedAt: "2026-09-01T10:00:00.000Z",
|
||||||
|
ageVerificationMethod: "platform_verified",
|
||||||
|
ageVerifiedAt: "2026-09-01T10:00:00.000Z",
|
||||||
|
});
|
||||||
|
let person = await db.prisma.mediaPerson.findUniqueOrThrow({ where: { id: seeded.personId } });
|
||||||
|
expect(person.consentRecordedAt).toBeNull();
|
||||||
|
expect(person.ageVerifiedAt).toBeNull();
|
||||||
|
|
||||||
|
const artifactId = await artifactIn(space);
|
||||||
|
await procedure(
|
||||||
|
client,
|
||||||
|
"people/recordConsent",
|
||||||
|
)({
|
||||||
|
personId: seeded.personId,
|
||||||
|
consentArtifactId: artifactId,
|
||||||
|
consentRecordedAt: "2026-09-01T10:00:00.000Z",
|
||||||
|
ageVerificationMethod: "platform_verified",
|
||||||
|
ageVerifiedAt: "2026-09-01T10:00:00.000Z",
|
||||||
|
});
|
||||||
|
person = await db.prisma.mediaPerson.findUniqueOrThrow({ where: { id: seeded.personId } });
|
||||||
|
expect(person.consentRecordedBy).toBe(space.userId);
|
||||||
|
expect(person.consentArtifactId).toBe(artifactId);
|
||||||
|
expect(person.ageVerificationMethod).toBe("platform_verified");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("assembles a resumable chunked upload and hands it to ingest", async () => {
|
||||||
|
const assetId = await remoteAsset(b, 101);
|
||||||
|
const bytes = new TextEncoder().encode("0123456789");
|
||||||
|
const upload = await repos.beginUpload(b.spaceId, {
|
||||||
|
assetId,
|
||||||
|
userId: b.userId,
|
||||||
|
sizeBytes: bytes.length,
|
||||||
|
sha256: sha256(bytes),
|
||||||
|
fileName: "clip.mp4",
|
||||||
|
chunkBytes: 4,
|
||||||
|
});
|
||||||
|
const owner = actorOf(b);
|
||||||
|
|
||||||
|
expect((await putChunk(upload.id, 0, bytes.slice(0, 4), actorOf(a))).status).toBe(404);
|
||||||
|
expect((await putChunk(upload.id, 0, bytes.slice(0, 4), null)).status).toBe(401);
|
||||||
|
expect((await putChunk(upload.id, 3, bytes.slice(0, 2), owner)).status).toBe(400);
|
||||||
|
expect((await putChunk(upload.id, 0, bytes.slice(0, 5), owner)).status).toBe(413);
|
||||||
|
expect((await putChunk(upload.id, 0, bytes.slice(0, 3), owner)).status).toBe(400);
|
||||||
|
|
||||||
|
// Out of order and repeated chunks are fine; the client resumes from status.
|
||||||
|
expect((await putChunk(upload.id, 2, bytes.slice(8), owner)).status).toBe(200);
|
||||||
|
expect((await putChunk(upload.id, 0, bytes.slice(0, 4), owner)).status).toBe(200);
|
||||||
|
expect((await putChunk(upload.id, 0, bytes.slice(0, 4), owner)).status).toBe(200);
|
||||||
|
const client = rpc(owner);
|
||||||
|
const partial = (await procedure(client, "uploads/status")({ uploadId: upload.id })) as {
|
||||||
|
receivedChunks: number[];
|
||||||
|
};
|
||||||
|
expect([...partial.receivedChunks].sort()).toEqual([0, 2]);
|
||||||
|
expect(await rpcCode(procedure(client, "uploads/complete")({ uploadId: upload.id }))).toBe(
|
||||||
|
"CONFLICT",
|
||||||
|
);
|
||||||
|
|
||||||
|
expect((await putChunk(upload.id, 1, bytes.slice(4, 8), owner)).status).toBe(200);
|
||||||
|
calls.length = 0;
|
||||||
|
const done = (await procedure(client, "uploads/complete")({ uploadId: upload.id })) as {
|
||||||
|
status: string;
|
||||||
|
};
|
||||||
|
expect(done.status).toBe("complete");
|
||||||
|
const handed = calls.filter((call) => call.method === "uploadCompleted");
|
||||||
|
expect(handed).toHaveLength(1);
|
||||||
|
const input = handed[0]?.input as { tempPath: string; sha256: string; assetId: string };
|
||||||
|
expect(input.assetId).toBe(assetId);
|
||||||
|
expect(input.sha256).toBe(sha256(bytes));
|
||||||
|
expect(input.tempPath.startsWith(path.join(mediaRoot, b.spaceId, "uploads"))).toBe(true);
|
||||||
|
expect(await readFile(input.tempPath, "utf8")).toBe("0123456789");
|
||||||
|
const asset = await repos.getAssetRecord(b.spaceId, assetId);
|
||||||
|
expect(asset.storage).toBe("uploaded");
|
||||||
|
expect(asset.storagePath).toBe(`${b.spaceId}/uploads/${upload.id}/file`);
|
||||||
|
// Completing again is idempotent and does not hand the file over twice.
|
||||||
|
await procedure(client, "uploads/complete")({ uploadId: upload.id });
|
||||||
|
expect(calls.filter((call) => call.method === "uploadCompleted")).toHaveLength(1);
|
||||||
|
expect((await putChunk(upload.id, 0, bytes.slice(0, 4), owner)).status).toBe(409);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("opens uploads through the RPC only for items that wait for their original", async () => {
|
||||||
|
const client = rpc(actorOf(b));
|
||||||
|
const assetId = await remoteAsset(b, 102);
|
||||||
|
const input = { assetId, sizeBytes: 10, sha256: "a".repeat(64), fileName: "clip.mp4" };
|
||||||
|
const first = (await procedure(client, "uploads/begin")(input)) as { id: string };
|
||||||
|
const resumed = (await procedure(client, "uploads/begin")(input)) as { id: string };
|
||||||
|
expect(resumed.id).toBe(first.id);
|
||||||
|
const referenced = { ...input, assetId: seededB.assetId };
|
||||||
|
expect(await rpcCode(procedure(client, "uploads/begin")(referenced))).toBe("CONFLICT");
|
||||||
|
const tooLarge = { ...input, sizeBytes: 21 * 1024 * 1024 * 1024 };
|
||||||
|
expect(await rpcCode(procedure(client, "uploads/begin")(tooLarge))).toBe("BAD_REQUEST");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("fails an upload whose bytes do not match the announced checksum", async () => {
|
||||||
|
const assetId = await remoteAsset(b, 103);
|
||||||
|
const bytes = new TextEncoder().encode("abcdef");
|
||||||
|
const upload = await repos.beginUpload(b.spaceId, {
|
||||||
|
assetId,
|
||||||
|
userId: b.userId,
|
||||||
|
sizeBytes: bytes.length,
|
||||||
|
sha256: "f".repeat(64),
|
||||||
|
fileName: "clip.mp4",
|
||||||
|
chunkBytes: 4,
|
||||||
|
});
|
||||||
|
const owner = actorOf(b);
|
||||||
|
await putChunk(upload.id, 0, bytes.slice(0, 4), owner);
|
||||||
|
await putChunk(upload.id, 1, bytes.slice(4), owner);
|
||||||
|
const client = rpc(owner);
|
||||||
|
expect(await rpcCode(procedure(client, "uploads/complete")({ uploadId: upload.id }))).toBe(
|
||||||
|
"BAD_REQUEST",
|
||||||
|
);
|
||||||
|
const status = (await procedure(client, "uploads/status")({ uploadId: upload.id })) as {
|
||||||
|
status: string;
|
||||||
|
};
|
||||||
|
expect(status.status).toBe("failed");
|
||||||
|
const dir = path.join(mediaRoot, b.spaceId, "uploads", upload.id);
|
||||||
|
await expect(stat(dir)).rejects.toMatchObject({ code: "ENOENT" });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("expires an idle upload and starts a fresh one", async () => {
|
||||||
|
const assetId = await remoteAsset(b, 104);
|
||||||
|
const input = { assetId, sizeBytes: 10, sha256: "b".repeat(64), fileName: "clip.mp4" };
|
||||||
|
const first = (await procedure(rpc(actorOf(b)), "uploads/begin")(input)) as { id: string };
|
||||||
|
setup({ now: () => new Date(Date.now() + MEDIA_UPLOAD_IDLE_MS + 60_000) });
|
||||||
|
try {
|
||||||
|
const response = await putChunk(first.id, 0, new Uint8Array(10), actorOf(b));
|
||||||
|
expect(response.status).toBe(409);
|
||||||
|
const status = (await procedure(
|
||||||
|
rpc(actorOf(b)),
|
||||||
|
"uploads/status",
|
||||||
|
)({
|
||||||
|
uploadId: first.id,
|
||||||
|
})) as { status: string };
|
||||||
|
expect(status.status).toBe("failed");
|
||||||
|
const fresh = (await procedure(rpc(actorOf(b)), "uploads/begin")(input)) as { id: string };
|
||||||
|
expect(fresh.id).not.toBe(first.id);
|
||||||
|
} finally {
|
||||||
|
setup();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("drops a chunk that does not arrive in time", async () => {
|
||||||
|
const assetId = await remoteAsset(b, 105);
|
||||||
|
const upload = await repos.beginUpload(b.spaceId, {
|
||||||
|
assetId,
|
||||||
|
userId: b.userId,
|
||||||
|
sizeBytes: 10,
|
||||||
|
sha256: "c".repeat(64),
|
||||||
|
fileName: "clip.mp4",
|
||||||
|
});
|
||||||
|
setup({ chunkTimeoutMs: 50 });
|
||||||
|
try {
|
||||||
|
const stalled = new ReadableStream<Uint8Array>({
|
||||||
|
start(controller) {
|
||||||
|
controller.enqueue(new Uint8Array(2));
|
||||||
|
},
|
||||||
|
});
|
||||||
|
currentActor = actorOf(b);
|
||||||
|
const response = await app.request(`/api/media/uploads/${upload.id}/chunks/0`, {
|
||||||
|
method: "PUT",
|
||||||
|
body: stalled,
|
||||||
|
duplex: "half",
|
||||||
|
} as RequestInit);
|
||||||
|
expect(response.status).toBe(408);
|
||||||
|
const status = await repos.uploadStatus(b.spaceId, upload.id);
|
||||||
|
expect(status.receivedChunks).toEqual([]);
|
||||||
|
} finally {
|
||||||
|
setup();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("streams render files only inside the caller's space and render folder", async () => {
|
||||||
|
const renderDir = path.join(mediaRoot, a.spaceId, "renders", seededA.renderId);
|
||||||
|
await mkdir(renderDir, { recursive: true });
|
||||||
|
const video = path.join(renderDir, "video.mp4");
|
||||||
|
await writeFile(video, "0123456789");
|
||||||
|
renderOutputs.set(seededA.renderId, video);
|
||||||
|
const url = `/api/media/renders/${seededA.renderId}/files/video.mp4`;
|
||||||
|
|
||||||
|
currentActor = actorOf(a);
|
||||||
|
const full = await app.request(url);
|
||||||
|
expect(full.status).toBe(200);
|
||||||
|
expect(await full.text()).toBe("0123456789");
|
||||||
|
const ranged = await app.request(url, { headers: { range: "bytes=2-4" } });
|
||||||
|
expect(ranged.status).toBe(206);
|
||||||
|
expect(ranged.headers.get("content-range")).toBe("bytes 2-4/10");
|
||||||
|
expect(await ranged.text()).toBe("234");
|
||||||
|
const download = await app.request(`${url}?download=1`);
|
||||||
|
expect(download.headers.get("content-disposition")).toContain("attachment");
|
||||||
|
|
||||||
|
// A render that has not passed its gate never leaves as a download.
|
||||||
|
const queuedVideo = path.join(mediaRoot, a.spaceId, "renders", seededA.queuedRenderId, "v.mp4");
|
||||||
|
await mkdir(path.dirname(queuedVideo), { recursive: true });
|
||||||
|
await writeFile(queuedVideo, "x");
|
||||||
|
renderOutputs.set(seededA.queuedRenderId, queuedVideo);
|
||||||
|
const queuedUrl = `/api/media/renders/${seededA.queuedRenderId}/files/v.mp4`;
|
||||||
|
expect((await app.request(queuedUrl)).status).toBe(200);
|
||||||
|
expect((await app.request(`${queuedUrl}?download=1`)).status).toBe(404);
|
||||||
|
|
||||||
|
// A path the service reports outside the render folder is refused.
|
||||||
|
const outside = path.join(mediaRoot, a.spaceId, "video.mp4");
|
||||||
|
await writeFile(outside, "secret");
|
||||||
|
renderOutputs.set(seededA.renderId, outside);
|
||||||
|
expect((await app.request(url)).status).toBe(404);
|
||||||
|
renderOutputs.set(seededA.renderId, video);
|
||||||
|
|
||||||
|
currentActor = actorOf(b);
|
||||||
|
expect((await app.request(url)).status).toBe(404);
|
||||||
|
expect((await app.request(`/api/media/assets/${seededA.assetId}/thumb`)).status).toBe(404);
|
||||||
|
expect((await app.request(`/api/media/bundles/${seededA.publicationId}`)).status).toBe(404);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("refuses a folder source to members who do not own the deployment", async () => {
|
||||||
|
const folder = await mkdtemp(path.join(tmpdir(), "media-footage-"));
|
||||||
|
try {
|
||||||
|
const input = { label: "Footage", path: folder };
|
||||||
|
const member = procedure(rpc(actorOf(b)), "sources/createFolder")(input);
|
||||||
|
expect(await rpcCode(member)).toBe("FORBIDDEN");
|
||||||
|
const created = (await procedure(rpc(actorOf(b, true)), "sources/createFolder")(input)) as {
|
||||||
|
id: string;
|
||||||
|
kind: string;
|
||||||
|
};
|
||||||
|
expect(created.kind).toBe("folder");
|
||||||
|
const row = await db.prisma.mediaSource.findUniqueOrThrow({ where: { id: created.id } });
|
||||||
|
expect(row.ownerUserId).toBe(b.userId);
|
||||||
|
} finally {
|
||||||
|
await rm(folder, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
333
apps/api/src/media.test.ts
Normal file
333
apps/api/src/media.test.ts
Normal file
@ -0,0 +1,333 @@
|
|||||||
|
import { mkdir, mkdtemp, rm, symlink } from "node:fs/promises";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import path from "node:path";
|
||||||
|
import { createRouterClient, ORPCError } from "@orpc/server";
|
||||||
|
import type { Actor } from "@rakazo/contracts";
|
||||||
|
import { MediaToolInputSchemas, mediaRpcShapes } from "@rakazo/contracts";
|
||||||
|
import type { MediaRepos, PrismaClient } from "@rakazo/db";
|
||||||
|
import { InvalidMediaInputError, IsolationError, MediaStateConflictError } from "@rakazo/db";
|
||||||
|
import { PathOutsideRootError } from "@rakazo/media/node";
|
||||||
|
import { Hono } from "hono";
|
||||||
|
import { afterAll, beforeAll, describe, expect, it } from "vitest";
|
||||||
|
import {
|
||||||
|
createMediaRouter,
|
||||||
|
MEDIA_NOT_CONFIGURED,
|
||||||
|
mediaRpcError,
|
||||||
|
unconfiguredCapabilities,
|
||||||
|
validateMediaFolder,
|
||||||
|
} from "./media.js";
|
||||||
|
import { mountMediaHttpRoutes, parseByteRange } from "./media-http.js";
|
||||||
|
import type { MediaSampleIds } from "./media-test-fakes.js";
|
||||||
|
import { fakeMediaServices, mediaProcedurePaths, sampleMediaInputs } from "./media-test-fakes.js";
|
||||||
|
import { expectedChunkBytes, uploadChunkCount } from "./media-uploads.js";
|
||||||
|
|
||||||
|
const member: Actor = {
|
||||||
|
userId: "user-member",
|
||||||
|
spaceId: "space-a",
|
||||||
|
email: "member@rakazo.test",
|
||||||
|
isDeploymentOwner: false,
|
||||||
|
};
|
||||||
|
const owner: Actor = { ...member, userId: "user-owner", isDeploymentOwner: true };
|
||||||
|
|
||||||
|
const ids = Object.fromEntries(
|
||||||
|
[
|
||||||
|
"sourceId",
|
||||||
|
"assetId",
|
||||||
|
"secondAssetId",
|
||||||
|
"segmentId",
|
||||||
|
"trackId",
|
||||||
|
"personId",
|
||||||
|
"markId",
|
||||||
|
"collectionId",
|
||||||
|
"brandKitId",
|
||||||
|
"projectId",
|
||||||
|
"planId",
|
||||||
|
"renderId",
|
||||||
|
"publicationId",
|
||||||
|
"mappingId",
|
||||||
|
"uploadId",
|
||||||
|
].map((key) => [key, `id-${key.toLowerCase()}`]),
|
||||||
|
) as unknown as MediaSampleIds;
|
||||||
|
const kit = {
|
||||||
|
name: "Kit",
|
||||||
|
colors: {
|
||||||
|
primary: "#101010",
|
||||||
|
secondary: "#202020",
|
||||||
|
accent: "#303030",
|
||||||
|
text: "#f0f0f0",
|
||||||
|
background: "#000000",
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
function client(router: ReturnType<typeof createMediaRouter>, actor: Actor | null) {
|
||||||
|
return createRouterClient(router, { context: { actor } });
|
||||||
|
}
|
||||||
|
|
||||||
|
function procedure(target: unknown, rpcPath: string) {
|
||||||
|
let node: unknown = target;
|
||||||
|
for (const key of rpcPath.split("/")) node = (node as Record<string, unknown>)[key];
|
||||||
|
return node as (input: unknown) => Promise<unknown>;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function rpcCode(promise: Promise<unknown>): Promise<string> {
|
||||||
|
try {
|
||||||
|
await promise;
|
||||||
|
} catch (error) {
|
||||||
|
if (error instanceof ORPCError) return error.code;
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
return "OK";
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Repositories that fail the test when reached. */
|
||||||
|
function untouchedRepos(): MediaRepos {
|
||||||
|
return new Proxy({} as MediaRepos, {
|
||||||
|
get: (_target, key) => () => {
|
||||||
|
throw new Error(`repository ${String(key)} must not be called`);
|
||||||
|
},
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
const prismaStub = {} as PrismaClient;
|
||||||
|
|
||||||
|
describe("media router without a configured sidecar", () => {
|
||||||
|
const router = createMediaRouter({ prisma: prismaStub, services: null, dataDir: "/data" });
|
||||||
|
const inputs = sampleMediaInputs(ids, kit);
|
||||||
|
|
||||||
|
it("has a sample input for every media procedure", () => {
|
||||||
|
expect(Object.keys(inputs).sort()).toEqual(mediaProcedurePaths().sort());
|
||||||
|
});
|
||||||
|
|
||||||
|
it("reports media as not configured so every surface keeps it hidden", async () => {
|
||||||
|
const capabilities = await client(router, member).capabilities({});
|
||||||
|
expect(capabilities).toEqual(unconfiguredCapabilities());
|
||||||
|
expect(capabilities.configured).toBe(false);
|
||||||
|
expect(
|
||||||
|
Object.values(capabilities.operations).every(
|
||||||
|
(op) => !(op as { available: boolean }).available,
|
||||||
|
),
|
||||||
|
).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("refuses every other procedure", async () => {
|
||||||
|
for (const [rpcPath, sample] of Object.entries(inputs)) {
|
||||||
|
if (rpcPath === "capabilities") continue;
|
||||||
|
const promise = procedure(client(router, member), rpcPath)(sample.input);
|
||||||
|
await expect(promise, rpcPath).rejects.toMatchObject({
|
||||||
|
code: "PRECONDITION_FAILED",
|
||||||
|
message: MEDIA_NOT_CONFIGURED,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("requires a signed-in member for every procedure", async () => {
|
||||||
|
for (const [rpcPath, sample] of Object.entries(inputs)) {
|
||||||
|
const code = await rpcCode(procedure(client(router, null), rpcPath)(sample.input));
|
||||||
|
expect(code, rpcPath).toBe("UNAUTHORIZED");
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("answers the byte routes with 404 and anonymous callers with 401", async () => {
|
||||||
|
const app = new Hono();
|
||||||
|
let actor: Actor | null = member;
|
||||||
|
mountMediaHttpRoutes(app, { services: null }, async () => actor);
|
||||||
|
const requests = [
|
||||||
|
new Request("http://api/api/media/uploads/u1/chunks/0", { method: "PUT", body: "abcd" }),
|
||||||
|
new Request("http://api/api/media/renders/r1/files/video.mp4"),
|
||||||
|
new Request("http://api/api/media/assets/a1/thumb"),
|
||||||
|
new Request("http://api/api/media/assets/a1/proxy"),
|
||||||
|
new Request("http://api/api/media/bundles/p1"),
|
||||||
|
];
|
||||||
|
for (const request of requests) {
|
||||||
|
expect((await app.request(request.clone())).status, request.url).toBe(404);
|
||||||
|
}
|
||||||
|
actor = null;
|
||||||
|
for (const request of requests) {
|
||||||
|
expect((await app.request(request.clone())).status, request.url).toBe(401);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("media router permissions", () => {
|
||||||
|
let root: string;
|
||||||
|
|
||||||
|
beforeAll(async () => {
|
||||||
|
root = await mkdtemp(path.join(tmpdir(), "media-api-"));
|
||||||
|
});
|
||||||
|
|
||||||
|
afterAll(async () => {
|
||||||
|
await rm(root, { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("lets only the deployment owner add a folder source", async () => {
|
||||||
|
const created: unknown[] = [];
|
||||||
|
const repos = new Proxy({} as MediaRepos, {
|
||||||
|
get: (_target, key) => {
|
||||||
|
if (key === "createSource") {
|
||||||
|
return async (_spaceId: string, input: unknown) => {
|
||||||
|
created.push(input);
|
||||||
|
return {
|
||||||
|
id: "src-1",
|
||||||
|
kind: "folder",
|
||||||
|
label: "Footage",
|
||||||
|
status: "idle",
|
||||||
|
assetCount: 0,
|
||||||
|
lastSyncedAt: null,
|
||||||
|
error: null,
|
||||||
|
createdAt: "2026-10-01T00:00:00.000Z",
|
||||||
|
};
|
||||||
|
};
|
||||||
|
}
|
||||||
|
return () => {
|
||||||
|
throw new Error(`repository ${String(key)} must not be called`);
|
||||||
|
};
|
||||||
|
},
|
||||||
|
});
|
||||||
|
const { services, calls } = fakeMediaServices({ repos, mediaRoot: path.join(root, "media") });
|
||||||
|
const footage = path.join(root, "footage");
|
||||||
|
await mkdir(footage);
|
||||||
|
const router = createMediaRouter({
|
||||||
|
prisma: prismaStub,
|
||||||
|
services,
|
||||||
|
dataDir: path.join(root, "data"),
|
||||||
|
appRoot: path.join(root, "app"),
|
||||||
|
});
|
||||||
|
|
||||||
|
const denied = client(router, member).sources.createFolder({ label: "Footage", path: footage });
|
||||||
|
await expect(denied).rejects.toMatchObject({ code: "FORBIDDEN" });
|
||||||
|
expect(created).toEqual([]);
|
||||||
|
expect(calls).toEqual([]);
|
||||||
|
|
||||||
|
await client(router, owner).sources.createFolder({ label: "Footage", path: footage });
|
||||||
|
expect(created).toEqual([
|
||||||
|
{ kind: "folder", label: "Footage", ownerUserId: owner.userId, config: { path: footage } },
|
||||||
|
]);
|
||||||
|
expect(calls.map((call) => `${call.service}.${call.method}`)).toEqual(["ingest.requestSync"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("accepts only a real, specific folder apart from evOo's own directories", async () => {
|
||||||
|
const dataDir = path.join(root, "data");
|
||||||
|
const appRoot = path.join(root, "app");
|
||||||
|
const folder = path.join(root, "clips");
|
||||||
|
await mkdir(path.join(dataDir, "media"), { recursive: true });
|
||||||
|
await mkdir(path.join(appRoot, "packages"), { recursive: true });
|
||||||
|
await mkdir(folder, { recursive: true });
|
||||||
|
await symlink(folder, path.join(root, "clips-link"));
|
||||||
|
const forbidden = { dataDir, appRoot };
|
||||||
|
|
||||||
|
await expect(validateMediaFolder(folder, forbidden)).resolves.toBe(folder);
|
||||||
|
const rejected = [
|
||||||
|
"clips",
|
||||||
|
path.join(root, "missing"),
|
||||||
|
path.join(root, "clips-link"),
|
||||||
|
"/",
|
||||||
|
dataDir,
|
||||||
|
path.join(dataDir, "media"),
|
||||||
|
root,
|
||||||
|
path.join(appRoot, "packages"),
|
||||||
|
];
|
||||||
|
for (const candidate of rejected) {
|
||||||
|
await expect(validateMediaFolder(candidate, forbidden), candidate).rejects.toBeInstanceOf(
|
||||||
|
InvalidMediaInputError,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("consent stays with the signed-in user", () => {
|
||||||
|
it("never accepts consent or age fields from the agent's people tool", () => {
|
||||||
|
const schema = MediaToolInputSchemas.media_people;
|
||||||
|
expect(schema.shape.action.options).toEqual(["list", "update", "merge"]);
|
||||||
|
const parsed = schema.parse({
|
||||||
|
action: "update",
|
||||||
|
personId: "person-1",
|
||||||
|
consentArtifactId: "artifact-1",
|
||||||
|
consentRecordedAt: "2026-09-01T00:00:00.000Z",
|
||||||
|
ageVerificationMethod: "platform_verified",
|
||||||
|
ageVerifiedAt: "2026-09-01T00:00:00.000Z",
|
||||||
|
});
|
||||||
|
expect(Object.keys(parsed).sort()).toEqual(["action", "personId"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("drops consent fields sent to people.update", async () => {
|
||||||
|
const patches: unknown[] = [];
|
||||||
|
const repos = new Proxy({} as MediaRepos, {
|
||||||
|
get: (_target, key) => {
|
||||||
|
if (key === "updatePerson") {
|
||||||
|
return async (_spaceId: string, _personId: string, patch: unknown) => {
|
||||||
|
patches.push(patch);
|
||||||
|
return {};
|
||||||
|
};
|
||||||
|
}
|
||||||
|
return () => {
|
||||||
|
throw new Error(`repository ${String(key)} must not be called`);
|
||||||
|
};
|
||||||
|
},
|
||||||
|
});
|
||||||
|
const { services } = fakeMediaServices({ repos, mediaRoot: "/unused" });
|
||||||
|
const router = createMediaRouter({ prisma: prismaStub, services, dataDir: "/data" });
|
||||||
|
await procedure(
|
||||||
|
client(router, member),
|
||||||
|
"people/update",
|
||||||
|
)({
|
||||||
|
personId: "person-1",
|
||||||
|
displayName: "Person 1",
|
||||||
|
consentRecordedAt: "2026-09-01T00:00:00.000Z",
|
||||||
|
ageVerificationMethod: "platform_verified",
|
||||||
|
ageVerifiedAt: "2026-09-01T00:00:00.000Z",
|
||||||
|
}).catch(() => undefined);
|
||||||
|
expect(patches).toHaveLength(1);
|
||||||
|
expect(Object.entries(patches[0] as object).filter(([, value]) => value !== undefined)).toEqual(
|
||||||
|
[["displayName", "Person 1"]],
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("marks recordConsent as the only user-only write and keeps it out of the services", () => {
|
||||||
|
const people = mediaRpcShapes.people as Record<string, unknown>;
|
||||||
|
expect(Object.keys(people)).toContain("recordConsent");
|
||||||
|
const { services } = fakeMediaServices({ repos: untouchedRepos(), mediaRoot: "/unused" });
|
||||||
|
const serviceMethods = (["ingest", "privacy", "render", "marketing"] as const).flatMap((name) =>
|
||||||
|
Object.keys(services[name] as unknown as Record<string, unknown>),
|
||||||
|
);
|
||||||
|
expect(serviceMethods.filter((method) => /consent|age/i.test(method))).toEqual([]);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("media error mapping", () => {
|
||||||
|
it("never reveals whether a foreign id exists", () => {
|
||||||
|
expect((mediaRpcError(new IsolationError()) as ORPCError<string, unknown>).code).toBe(
|
||||||
|
"NOT_FOUND",
|
||||||
|
);
|
||||||
|
expect(
|
||||||
|
(mediaRpcError(new PathOutsideRootError("../x")) as ORPCError<string, unknown>).code,
|
||||||
|
).toBe("BAD_REQUEST");
|
||||||
|
expect(
|
||||||
|
(mediaRpcError(new MediaStateConflictError("busy")) as ORPCError<string, unknown>).code,
|
||||||
|
).toBe("CONFLICT");
|
||||||
|
const other = new Error("boom");
|
||||||
|
expect(mediaRpcError(other)).toBe(other);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("upload and streaming helpers", () => {
|
||||||
|
it("splits uploads into full chunks plus a shorter last one", () => {
|
||||||
|
const row = { sizeBytes: 10n, chunkBytes: 4 };
|
||||||
|
expect(uploadChunkCount(row)).toBe(3);
|
||||||
|
expect([0, 1, 2].map((index) => expectedChunkBytes(row, index))).toEqual([4, 4, 2]);
|
||||||
|
expect(uploadChunkCount({ sizeBytes: 8n, chunkBytes: 4 })).toBe(2);
|
||||||
|
expect(expectedChunkBytes({ sizeBytes: 8n, chunkBytes: 4 }, 1)).toBe(4);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("parses single byte ranges and refuses unsatisfiable ones", () => {
|
||||||
|
expect(parseByteRange(undefined, 100)).toBeNull();
|
||||||
|
expect(parseByteRange("bytes=0-9", 100)).toEqual({ start: 0, end: 9 });
|
||||||
|
expect(parseByteRange("bytes=90-", 100)).toEqual({ start: 90, end: 99 });
|
||||||
|
expect(parseByteRange("bytes=-10", 100)).toEqual({ start: 90, end: 99 });
|
||||||
|
expect(parseByteRange("bytes=50-500", 100)).toEqual({ start: 50, end: 99 });
|
||||||
|
expect(parseByteRange("bytes=0-1,5-6", 100)).toBeNull();
|
||||||
|
expect(parseByteRange("bytes=100-", 100)).toBe("unsatisfiable");
|
||||||
|
expect(parseByteRange("bytes=9-3", 100)).toBe("unsatisfiable");
|
||||||
|
expect(parseByteRange("items=0-1", 100)).toBe("unsatisfiable");
|
||||||
|
expect(parseByteRange("bytes=-0", 100)).toBe("unsatisfiable");
|
||||||
|
});
|
||||||
|
});
|
||||||
552
apps/api/src/media.ts
Normal file
552
apps/api/src/media.ts
Normal file
@ -0,0 +1,552 @@
|
|||||||
|
import { lstat, realpath, rm } from "node:fs/promises";
|
||||||
|
import path from "node:path";
|
||||||
|
import { fileURLToPath } from "node:url";
|
||||||
|
import { implement, ORPCError } from "@orpc/server";
|
||||||
|
import { UnsupportedMediaOperationError } from "@rakazo/adapter-kit";
|
||||||
|
import type { MediaServices } from "@rakazo/adapters";
|
||||||
|
import { recordMetric, resetTeaserModel, saveBrandKit, segmentFunnel } from "@rakazo/adapters";
|
||||||
|
import type { Actor, BrandKit, MediaCapabilities, MediaSourceKind } from "@rakazo/contracts";
|
||||||
|
import { appContract, MediaInferenceOperationSchema } from "@rakazo/contracts";
|
||||||
|
import type { PrismaClient } from "@rakazo/db";
|
||||||
|
import {
|
||||||
|
InvalidMediaInputError,
|
||||||
|
InvalidProjectWorkspacePathError,
|
||||||
|
IsolationError,
|
||||||
|
lockMediaKey,
|
||||||
|
MediaStateConflictError,
|
||||||
|
mapMediaSource,
|
||||||
|
resolveProjectWorkspaceDirectory,
|
||||||
|
} from "@rakazo/db";
|
||||||
|
import { summarizeProfile } from "@rakazo/media";
|
||||||
|
import { PathOutsideRootError, resolveInside } from "@rakazo/media/node";
|
||||||
|
import { beginMediaUpload, completeMediaUpload, UploadChunkError } from "./media-uploads.js";
|
||||||
|
|
||||||
|
export interface MediaRouterDeps {
|
||||||
|
prisma: PrismaClient;
|
||||||
|
/** Null when Media Studio is not configured: everything but capabilities refuses. */
|
||||||
|
services?: MediaServices | null;
|
||||||
|
/** DATA_DIR; a folder source may neither sit inside it nor contain it. */
|
||||||
|
dataDir: string;
|
||||||
|
/** Application install directory a folder source may not point into. */
|
||||||
|
appRoot?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export const MEDIA_NOT_CONFIGURED = "Media Studio is not configured";
|
||||||
|
|
||||||
|
const DEFAULT_APP_ROOT = fileURLToPath(new URL("../../../", import.meta.url));
|
||||||
|
|
||||||
|
/** Translate domain errors into RPC errors; everything else passes through. */
|
||||||
|
export function mediaRpcError(error: unknown): unknown {
|
||||||
|
if (error instanceof ORPCError) return error;
|
||||||
|
if (error instanceof IsolationError) return new ORPCError("NOT_FOUND");
|
||||||
|
if (
|
||||||
|
error instanceof InvalidMediaInputError ||
|
||||||
|
error instanceof InvalidProjectWorkspacePathError
|
||||||
|
) {
|
||||||
|
return new ORPCError("BAD_REQUEST", { message: error.message });
|
||||||
|
}
|
||||||
|
if (error instanceof PathOutsideRootError) {
|
||||||
|
return new ORPCError("BAD_REQUEST", { message: "That path is not allowed." });
|
||||||
|
}
|
||||||
|
if (error instanceof MediaStateConflictError) {
|
||||||
|
return new ORPCError("CONFLICT", { message: error.message });
|
||||||
|
}
|
||||||
|
if (error instanceof UploadChunkError) {
|
||||||
|
return new ORPCError(error.status === 409 ? "CONFLICT" : "BAD_REQUEST", {
|
||||||
|
message: error.message,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
if (error instanceof UnsupportedMediaOperationError) {
|
||||||
|
return new ORPCError("PRECONDITION_FAILED", { message: error.message });
|
||||||
|
}
|
||||||
|
return error;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function unconfiguredCapabilities(): MediaCapabilities {
|
||||||
|
const unavailable = {
|
||||||
|
available: false,
|
||||||
|
experimental: false,
|
||||||
|
model: null,
|
||||||
|
license: null,
|
||||||
|
commercialUse: null,
|
||||||
|
};
|
||||||
|
return {
|
||||||
|
configured: false,
|
||||||
|
operations: Object.fromEntries(
|
||||||
|
MediaInferenceOperationSchema.options.map((operation) => [operation, unavailable]),
|
||||||
|
) as MediaCapabilities["operations"],
|
||||||
|
accelerator: "none",
|
||||||
|
hardwareEncoders: [],
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function overlaps(a: string, b: string): boolean {
|
||||||
|
const inside = (root: string, candidate: string) => {
|
||||||
|
const relative = path.relative(root, candidate);
|
||||||
|
return relative === "" || (!relative.startsWith("..") && !path.isAbsolute(relative));
|
||||||
|
};
|
||||||
|
return inside(a, b) || inside(b, a);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function realOrResolved(target: string): Promise<string> {
|
||||||
|
return realpath(target).catch(() => path.resolve(target));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A host folder for a folder source: absolute, an existing directory, not a
|
||||||
|
* symlink, and apart from DATA_DIR and the application itself. Returns the real path.
|
||||||
|
*/
|
||||||
|
export async function validateMediaFolder(
|
||||||
|
input: string,
|
||||||
|
forbidden: { dataDir: string; appRoot: string },
|
||||||
|
): Promise<string> {
|
||||||
|
const trimmed = input.trim();
|
||||||
|
if (!path.isAbsolute(trimmed)) {
|
||||||
|
throw new InvalidMediaInputError("Use an absolute folder path.");
|
||||||
|
}
|
||||||
|
const info = await lstat(trimmed).catch(() => null);
|
||||||
|
if (!info) throw new InvalidMediaInputError("That folder does not exist or is not readable.");
|
||||||
|
if (info.isSymbolicLink())
|
||||||
|
throw new InvalidMediaInputError("Choose the folder, not a link to it.");
|
||||||
|
const resolved = await resolveProjectWorkspaceDirectory(trimmed);
|
||||||
|
if (path.parse(resolved).root === resolved) {
|
||||||
|
throw new InvalidMediaInputError("Choose a specific folder.");
|
||||||
|
}
|
||||||
|
const dataDir = await realOrResolved(forbidden.dataDir);
|
||||||
|
const appRoot = await realOrResolved(forbidden.appRoot);
|
||||||
|
const relativeToApp = path.relative(appRoot, resolved);
|
||||||
|
const insideApp =
|
||||||
|
relativeToApp === "" || (!relativeToApp.startsWith("..") && !path.isAbsolute(relativeToApp));
|
||||||
|
if (overlaps(dataDir, resolved) || insideApp) {
|
||||||
|
throw new InvalidMediaInputError("That folder belongs to evOo itself.");
|
||||||
|
}
|
||||||
|
return resolved;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Artifacts referenced from media records must belong to the caller's space. */
|
||||||
|
export async function assertSpaceArtifacts(
|
||||||
|
prisma: PrismaClient,
|
||||||
|
spaceId: string,
|
||||||
|
artifactIds: string[],
|
||||||
|
): Promise<void> {
|
||||||
|
const ids = [...new Set(artifactIds)];
|
||||||
|
if (ids.length === 0) return;
|
||||||
|
const owned = await prisma.artifact.count({ where: { spaceId, id: { in: ids } } });
|
||||||
|
if (owned !== ids.length) throw new IsolationError();
|
||||||
|
}
|
||||||
|
|
||||||
|
function brandKitArtifactIds(kit: BrandKit): string[] {
|
||||||
|
return [
|
||||||
|
...kit.fonts.map((font) => font.artifactId),
|
||||||
|
...kit.logoArtifactIds,
|
||||||
|
...(kit.lutArtifactId ? [kit.lutArtifactId] : []),
|
||||||
|
...kit.musicBeds.map((bed) => bed.artifactId),
|
||||||
|
];
|
||||||
|
}
|
||||||
|
|
||||||
|
async function removeMediaPath(mediaRoot: string, relative: string | null): Promise<void> {
|
||||||
|
if (!relative) return;
|
||||||
|
await rm(await resolveInside(mediaRoot, relative), { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
|
||||||
|
export function createMediaRouter(deps: MediaRouterDeps) {
|
||||||
|
const os = implement(appContract.media).$context<{ actor: Actor | null; signal?: AbortSignal }>();
|
||||||
|
const authed = os.use(async ({ context, next }) => {
|
||||||
|
if (!context.actor) throw new ORPCError("UNAUTHORIZED");
|
||||||
|
try {
|
||||||
|
return await next({ context: { ...context, actor: context.actor } });
|
||||||
|
} catch (error) {
|
||||||
|
throw mediaRpcError(error);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
const media = authed.use(async ({ next }) => {
|
||||||
|
if (!deps.services) {
|
||||||
|
throw new ORPCError("PRECONDITION_FAILED", { message: MEDIA_NOT_CONFIGURED });
|
||||||
|
}
|
||||||
|
return next({ context: { services: deps.services } });
|
||||||
|
});
|
||||||
|
const forbiddenRoots = { dataDir: deps.dataDir, appRoot: deps.appRoot ?? DEFAULT_APP_ROOT };
|
||||||
|
|
||||||
|
async function knownProfiles(services: MediaServices, profileIds: string[] | undefined) {
|
||||||
|
if (!profileIds) return;
|
||||||
|
const { profiles } = await services.deps.profiles();
|
||||||
|
const unknown = profileIds.filter((id) => !profiles.has(id));
|
||||||
|
if (unknown.length > 0) {
|
||||||
|
throw new InvalidMediaInputError(`Unknown profile: ${unknown.join(", ")}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return os.router({
|
||||||
|
capabilities: authed.capabilities.handler(async ({ context }) => {
|
||||||
|
const services = deps.services;
|
||||||
|
if (!services) return unconfiguredCapabilities();
|
||||||
|
const hardwareEncoders = await services.deps.renderer.hardwareEncoders().catch(() => []);
|
||||||
|
const inference = services.deps.inference;
|
||||||
|
if (!inference) return { ...unconfiguredCapabilities(), hardwareEncoders };
|
||||||
|
const capabilities = await inference
|
||||||
|
.capabilities(context.signal)
|
||||||
|
.catch(() => ({ ...unconfiguredCapabilities(), configured: true }));
|
||||||
|
return { ...capabilities, configured: true, hardwareEncoders };
|
||||||
|
}),
|
||||||
|
sources: {
|
||||||
|
list: media.sources.list.handler(({ context }) =>
|
||||||
|
context.services.deps.repos.listSources(context.actor.spaceId),
|
||||||
|
),
|
||||||
|
createFolder: media.sources.createFolder.handler(async ({ context, input }) => {
|
||||||
|
// A folder lives on the host, and the host is the deployment owner's machine.
|
||||||
|
if (!context.actor.isDeploymentOwner) throw new ORPCError("FORBIDDEN");
|
||||||
|
const folder = await validateMediaFolder(input.path, forbiddenRoots);
|
||||||
|
const { repos } = context.services.deps;
|
||||||
|
const source = await repos.createSource(context.actor.spaceId, {
|
||||||
|
kind: "folder",
|
||||||
|
label: input.label,
|
||||||
|
ownerUserId: context.actor.userId,
|
||||||
|
config: { path: folder },
|
||||||
|
});
|
||||||
|
await context.services.ingest.requestSync(context.actor, source.id);
|
||||||
|
return source;
|
||||||
|
}),
|
||||||
|
registerClient: media.sources.registerClient.handler(async ({ context, input }) => {
|
||||||
|
const { spaceId, userId } = context.actor;
|
||||||
|
const kind: MediaSourceKind = input.kind;
|
||||||
|
// One source per device: re-registering returns the existing one.
|
||||||
|
return deps.prisma.$transaction(async (tx) => {
|
||||||
|
await lockMediaKey(tx, `source:${spaceId}:${kind}:${input.deviceId}`);
|
||||||
|
const existing = await tx.mediaSource.findFirst({
|
||||||
|
where: { spaceId, kind, config: { path: ["deviceId"], equals: input.deviceId } },
|
||||||
|
orderBy: [{ createdAt: "asc" }, { id: "asc" }],
|
||||||
|
include: { _count: { select: { assets: true } } },
|
||||||
|
});
|
||||||
|
if (existing) return mapMediaSource(existing, existing._count.assets);
|
||||||
|
const created = await tx.mediaSource.create({
|
||||||
|
data: {
|
||||||
|
spaceId,
|
||||||
|
kind,
|
||||||
|
label: input.label,
|
||||||
|
ownerUserId: userId,
|
||||||
|
config: { deviceId: input.deviceId },
|
||||||
|
},
|
||||||
|
});
|
||||||
|
return mapMediaSource(created, 0);
|
||||||
|
});
|
||||||
|
}),
|
||||||
|
pushItems: media.sources.pushItems.handler(async ({ context, input }) => {
|
||||||
|
await context.services.deps.repos.getSource(context.actor.spaceId, input.sourceId);
|
||||||
|
return context.services.ingest.pushItems(context.actor, input);
|
||||||
|
}),
|
||||||
|
sync: media.sources.sync.handler(async ({ context, input }) => {
|
||||||
|
await context.services.deps.repos.getSource(context.actor.spaceId, input.sourceId);
|
||||||
|
await context.services.ingest.requestSync(context.actor, input.sourceId);
|
||||||
|
return { ok: true as const };
|
||||||
|
}),
|
||||||
|
remove: media.sources.remove.handler(async ({ context, input }) => {
|
||||||
|
await context.services.deps.repos.getSource(context.actor.spaceId, input.sourceId);
|
||||||
|
await context.services.ingest.removeSource(context.actor, input.sourceId);
|
||||||
|
return { ok: true as const };
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
assets: {
|
||||||
|
search: media.assets.search.handler(({ context, input }) =>
|
||||||
|
context.services.deps.repos.searchAssets(context.actor.spaceId, input),
|
||||||
|
),
|
||||||
|
get: media.assets.get.handler(({ context, input }) =>
|
||||||
|
context.services.deps.repos.getAsset(context.actor.spaceId, input.assetId),
|
||||||
|
),
|
||||||
|
setPaidOnly: media.assets.setPaidOnly.handler(async ({ context, input }) => {
|
||||||
|
await context.services.deps.repos.setSegmentsPaidOnly(
|
||||||
|
context.actor.spaceId,
|
||||||
|
input.segmentIds,
|
||||||
|
input.paidOnly,
|
||||||
|
);
|
||||||
|
return { ok: true as const };
|
||||||
|
}),
|
||||||
|
requestUpload: media.assets.requestUpload.handler(async ({ context, input }) => ({
|
||||||
|
awaiting: await context.services.deps.repos.requestUpload(
|
||||||
|
context.actor.spaceId,
|
||||||
|
input.assetIds,
|
||||||
|
),
|
||||||
|
})),
|
||||||
|
},
|
||||||
|
uploads: {
|
||||||
|
begin: media.uploads.begin.handler(({ context, input }) =>
|
||||||
|
beginMediaUpload(context.services, context.actor, input),
|
||||||
|
),
|
||||||
|
status: media.uploads.status.handler(({ context, input }) =>
|
||||||
|
context.services.deps.repos.uploadStatus(context.actor.spaceId, input.uploadId),
|
||||||
|
),
|
||||||
|
complete: media.uploads.complete.handler(({ context, input }) =>
|
||||||
|
completeMediaUpload(context.services, context.actor, input.uploadId),
|
||||||
|
),
|
||||||
|
},
|
||||||
|
collections: {
|
||||||
|
list: media.collections.list.handler(({ context, input }) =>
|
||||||
|
context.services.deps.repos.listCollections(context.actor.spaceId, input),
|
||||||
|
),
|
||||||
|
get: media.collections.get.handler(({ context, input }) =>
|
||||||
|
context.services.deps.repos.getCollection(context.actor.spaceId, input.collectionId),
|
||||||
|
),
|
||||||
|
rename: media.collections.rename.handler(({ context, input }) =>
|
||||||
|
context.services.deps.repos.renameCollection(
|
||||||
|
context.actor.spaceId,
|
||||||
|
input.collectionId,
|
||||||
|
input.title,
|
||||||
|
),
|
||||||
|
),
|
||||||
|
merge: media.collections.merge.handler(({ context, input }) =>
|
||||||
|
context.services.deps.repos.mergeCollections(context.actor.spaceId, input.collectionIds),
|
||||||
|
),
|
||||||
|
split: media.collections.split.handler(({ context, input }) =>
|
||||||
|
context.services.deps.repos.splitCollection(
|
||||||
|
context.actor.spaceId,
|
||||||
|
input.collectionId,
|
||||||
|
input.atAssetId,
|
||||||
|
),
|
||||||
|
),
|
||||||
|
createManual: media.collections.createManual.handler(({ context, input }) =>
|
||||||
|
context.services.deps.repos.createManualCollection(
|
||||||
|
context.actor.spaceId,
|
||||||
|
input.title,
|
||||||
|
input.assetIds,
|
||||||
|
),
|
||||||
|
),
|
||||||
|
recluster: media.collections.recluster.handler(async ({ context }) => {
|
||||||
|
await context.services.ingest.recluster(context.actor);
|
||||||
|
return { ok: true as const };
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
people: {
|
||||||
|
list: media.people.list.handler(({ context }) =>
|
||||||
|
context.services.deps.repos.listPeople(context.actor.spaceId),
|
||||||
|
),
|
||||||
|
update: media.people.update.handler(({ context, input }) =>
|
||||||
|
context.services.deps.repos.updatePerson(context.actor.spaceId, input.personId, {
|
||||||
|
displayName: input.displayName,
|
||||||
|
isSelf: input.isSelf,
|
||||||
|
confirmed: input.confirmed,
|
||||||
|
anonymizeByDefault: input.anonymizeByDefault,
|
||||||
|
}),
|
||||||
|
),
|
||||||
|
merge: media.people.merge.handler(({ context, input }) =>
|
||||||
|
context.services.deps.repos.mergePeople(context.actor.spaceId, input.personIds),
|
||||||
|
),
|
||||||
|
assignTrack: media.people.assignTrack.handler(({ context, input }) =>
|
||||||
|
context.services.deps.repos.assignTrack(
|
||||||
|
context.actor.spaceId,
|
||||||
|
input.trackId,
|
||||||
|
input.personId,
|
||||||
|
),
|
||||||
|
),
|
||||||
|
// The only consent write path: the signed-in member is the recorder.
|
||||||
|
recordConsent: media.people.recordConsent.handler(async ({ context, input }) => {
|
||||||
|
await assertSpaceArtifacts(
|
||||||
|
deps.prisma,
|
||||||
|
context.actor.spaceId,
|
||||||
|
input.consentArtifactId ? [input.consentArtifactId] : [],
|
||||||
|
);
|
||||||
|
return context.services.deps.repos.recordConsent(
|
||||||
|
{ spaceId: context.actor.spaceId, recordedByUserId: context.actor.userId },
|
||||||
|
input,
|
||||||
|
);
|
||||||
|
}),
|
||||||
|
remove: media.people.remove.handler(async ({ context, input }) => {
|
||||||
|
await context.services.deps.repos.getPerson(context.actor.spaceId, input.personId);
|
||||||
|
await context.services.privacy.removePerson(context.actor, input.personId);
|
||||||
|
return { ok: true as const };
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
marks: {
|
||||||
|
list: media.marks.list.handler(({ context, input }) =>
|
||||||
|
context.services.deps.repos.listMarks(context.actor.spaceId, {
|
||||||
|
...(input.personId ? { personId: input.personId } : {}),
|
||||||
|
...(input.assetId ? { assetId: input.assetId } : {}),
|
||||||
|
}),
|
||||||
|
),
|
||||||
|
create: media.marks.create.handler(async ({ context, input }) => {
|
||||||
|
const { repos } = context.services.deps;
|
||||||
|
await repos.getAssetRecord(context.actor.spaceId, input.assetId);
|
||||||
|
if (input.personId) await repos.getPerson(context.actor.spaceId, input.personId);
|
||||||
|
return context.services.privacy.createMark(context.actor, input);
|
||||||
|
}),
|
||||||
|
update: media.marks.update.handler(async ({ context, input }) => {
|
||||||
|
const { repos } = context.services.deps;
|
||||||
|
await repos.getMark(context.actor.spaceId, input.markId);
|
||||||
|
if (input.personId) await repos.getPerson(context.actor.spaceId, input.personId);
|
||||||
|
return context.services.privacy.updateMark(context.actor, input);
|
||||||
|
}),
|
||||||
|
remove: media.marks.remove.handler(async ({ context, input }) => {
|
||||||
|
const { repos, mediaRoot } = context.services.deps;
|
||||||
|
const { maskDir } = await repos.removeMark(context.actor.spaceId, input.markId);
|
||||||
|
await removeMediaPath(mediaRoot, maskDir);
|
||||||
|
return { ok: true as const };
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
brandKits: {
|
||||||
|
list: media.brandKits.list.handler(({ context }) =>
|
||||||
|
context.services.deps.repos.listBrandKits(context.actor.spaceId),
|
||||||
|
),
|
||||||
|
upsert: media.brandKits.upsert.handler(async ({ context, input }) => {
|
||||||
|
await assertSpaceArtifacts(
|
||||||
|
deps.prisma,
|
||||||
|
context.actor.spaceId,
|
||||||
|
brandKitArtifactIds(input.kit),
|
||||||
|
);
|
||||||
|
return saveBrandKit(context.services.deps, context.actor, input);
|
||||||
|
}),
|
||||||
|
remove: media.brandKits.remove.handler(async ({ context, input }) => {
|
||||||
|
await context.services.deps.repos.removeBrandKit(context.actor.spaceId, input.id);
|
||||||
|
return { ok: true as const };
|
||||||
|
}),
|
||||||
|
templates: media.brandKits.templates.handler(({ context }) =>
|
||||||
|
context.services.deps.motion.templates(),
|
||||||
|
),
|
||||||
|
introVariants: media.brandKits.introVariants.handler(async ({ context, input }) => {
|
||||||
|
await context.services.deps.repos.getBrandKit(context.actor.spaceId, input.id);
|
||||||
|
return context.services.marketing.introVariants(context.actor, {
|
||||||
|
brandKitId: input.id,
|
||||||
|
count: input.count,
|
||||||
|
});
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
profiles: {
|
||||||
|
list: media.profiles.list.handler(async ({ context }) => {
|
||||||
|
const { profiles } = await context.services.deps.profiles();
|
||||||
|
const now = context.services.deps.now();
|
||||||
|
return [...profiles.values()].map(({ profile, origin }) =>
|
||||||
|
summarizeProfile(profile, origin, now),
|
||||||
|
);
|
||||||
|
}),
|
||||||
|
get: media.profiles.get.handler(async ({ context, input }) => {
|
||||||
|
const { profiles } = await context.services.deps.profiles();
|
||||||
|
const entry = profiles.get(input.profileId);
|
||||||
|
if (!entry) throw new ORPCError("NOT_FOUND");
|
||||||
|
return entry.profile;
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
projects: {
|
||||||
|
list: media.projects.list.handler(({ context, input }) =>
|
||||||
|
context.services.deps.repos.listProjects(context.actor.spaceId, input),
|
||||||
|
),
|
||||||
|
get: media.projects.get.handler(({ context, input }) =>
|
||||||
|
context.services.deps.repos.getProject(context.actor.spaceId, input.projectId),
|
||||||
|
),
|
||||||
|
create: media.projects.create.handler(async ({ context, input }) => {
|
||||||
|
await knownProfiles(context.services, input.profileIds);
|
||||||
|
return context.services.deps.repos.createProject(
|
||||||
|
context.actor.spaceId,
|
||||||
|
context.actor.userId,
|
||||||
|
input,
|
||||||
|
);
|
||||||
|
}),
|
||||||
|
update: media.projects.update.handler(async ({ context, input }) => {
|
||||||
|
await knownProfiles(context.services, input.profileIds);
|
||||||
|
const { projectId, ...patch } = input;
|
||||||
|
return context.services.deps.repos.updateProject(context.actor.spaceId, projectId, patch);
|
||||||
|
}),
|
||||||
|
remove: media.projects.remove.handler(async ({ context, input }) => {
|
||||||
|
const { repos, mediaRoot } = context.services.deps;
|
||||||
|
const { spaceId } = context.actor;
|
||||||
|
await repos.getProjectRecord(spaceId, input.projectId);
|
||||||
|
const renders = await repos.listRenders(spaceId, input.projectId);
|
||||||
|
await repos.removeProject(spaceId, input.projectId);
|
||||||
|
for (const render of renders)
|
||||||
|
await removeMediaPath(mediaRoot, `${spaceId}/renders/${render.id}`);
|
||||||
|
return { ok: true as const };
|
||||||
|
}),
|
||||||
|
autoPlan: media.projects.autoPlan.handler(async ({ context, input }) => {
|
||||||
|
await context.services.deps.repos.getProjectRecord(context.actor.spaceId, input.projectId);
|
||||||
|
await knownProfiles(context.services, input.profileIds);
|
||||||
|
return context.services.marketing.autoPlan(context.actor, input);
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
plans: {
|
||||||
|
submit: media.plans.submit.handler(async ({ context, input }) => {
|
||||||
|
await context.services.deps.repos.getProjectRecord(context.actor.spaceId, input.projectId);
|
||||||
|
return context.services.marketing.submitPlan(context.actor, {
|
||||||
|
projectId: input.projectId,
|
||||||
|
edl: input.edl,
|
||||||
|
rationale: input.rationale,
|
||||||
|
createdBy: "user",
|
||||||
|
});
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
renders: {
|
||||||
|
start: media.renders.start.handler(async ({ context, input }) => {
|
||||||
|
await context.services.deps.repos.getPlan(context.actor.spaceId, input.planId);
|
||||||
|
return context.services.render.start(context.actor, input);
|
||||||
|
}),
|
||||||
|
get: media.renders.get.handler(({ context, input }) =>
|
||||||
|
context.services.deps.repos.getRender(context.actor.spaceId, input.renderId),
|
||||||
|
),
|
||||||
|
cancel: media.renders.cancel.handler(async ({ context, input }) => {
|
||||||
|
await context.services.deps.repos.getRender(context.actor.spaceId, input.renderId);
|
||||||
|
return context.services.render.cancel(context.actor, input.renderId);
|
||||||
|
}),
|
||||||
|
thumbnails: media.renders.thumbnails.handler(async ({ context, input }) => {
|
||||||
|
await context.services.deps.repos.getRender(context.actor.spaceId, input.renderId);
|
||||||
|
return context.services.marketing.thumbnails(context.actor, input);
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
marketing: {
|
||||||
|
copy: media.marketing.copy.handler(async ({ context, input }) => {
|
||||||
|
await context.services.deps.repos.getRender(context.actor.spaceId, input.renderId);
|
||||||
|
return context.services.marketing.copy(context.actor, input);
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
publications: {
|
||||||
|
list: media.publications.list.handler(({ context, input }) =>
|
||||||
|
context.services.deps.repos.listPublications(context.actor.spaceId, input),
|
||||||
|
),
|
||||||
|
create: media.publications.create.handler(async ({ context, input }) => {
|
||||||
|
await context.services.deps.repos.getRender(context.actor.spaceId, input.renderId);
|
||||||
|
return context.services.marketing.createPublication(context.actor, input);
|
||||||
|
}),
|
||||||
|
bundle: media.publications.bundle.handler(async ({ context, input }) => {
|
||||||
|
await context.services.deps.repos.getPublication(
|
||||||
|
context.actor.spaceId,
|
||||||
|
input.publicationId,
|
||||||
|
);
|
||||||
|
return context.services.marketing.bundle(context.actor, input.publicationId);
|
||||||
|
}),
|
||||||
|
markPublished: media.publications.markPublished.handler(({ context, input }) =>
|
||||||
|
context.services.deps.repos.markPublicationPublished(
|
||||||
|
context.actor.spaceId,
|
||||||
|
input.publicationId,
|
||||||
|
input.externalUrl,
|
||||||
|
context.services.deps.now(),
|
||||||
|
),
|
||||||
|
),
|
||||||
|
cancel: media.publications.cancel.handler(async ({ context, input }) => {
|
||||||
|
await context.services.deps.repos.getPublication(
|
||||||
|
context.actor.spaceId,
|
||||||
|
input.publicationId,
|
||||||
|
);
|
||||||
|
return context.services.marketing.cancelPublication(context.actor, input.publicationId);
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
metrics: {
|
||||||
|
mappings: media.metrics.mappings.handler(({ context }) =>
|
||||||
|
context.services.deps.repos.listMetricMappings(context.actor.spaceId),
|
||||||
|
),
|
||||||
|
saveMapping: media.metrics.saveMapping.handler(({ context, input }) =>
|
||||||
|
context.services.deps.repos.saveMetricMapping(context.actor.spaceId, input),
|
||||||
|
),
|
||||||
|
importCsv: media.metrics.importCsv.handler(async ({ context, input }) => {
|
||||||
|
await context.services.deps.repos.getMetricMapping(context.actor.spaceId, input.mappingId);
|
||||||
|
return context.services.marketing.importMetricsCsv(context.actor, input);
|
||||||
|
}),
|
||||||
|
record: media.metrics.record.handler(({ context, input }) =>
|
||||||
|
recordMetric(context.services.deps, context.actor, input),
|
||||||
|
),
|
||||||
|
funnel: media.metrics.funnel.handler(({ context, input }) =>
|
||||||
|
context.services.deps.repos.funnel(context.actor.spaceId, input.projectId),
|
||||||
|
),
|
||||||
|
segments: media.metrics.segments.handler(({ context, input }) =>
|
||||||
|
segmentFunnel(context.services.deps, context.actor, input.projectId),
|
||||||
|
),
|
||||||
|
resetTeaserModel: media.metrics.resetTeaserModel.handler(async ({ context }) => {
|
||||||
|
await resetTeaserModel(context.services.deps, context.actor);
|
||||||
|
return { ok: true as const };
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
});
|
||||||
|
}
|
||||||
@ -1,11 +1,44 @@
|
|||||||
import { mkdir, mkdtemp, writeFile } from "node:fs/promises";
|
import { mkdir, mkdtemp, realpath, writeFile } from "node:fs/promises";
|
||||||
import { tmpdir } from "node:os";
|
import { tmpdir } from "node:os";
|
||||||
import { join } from "node:path";
|
import { join } from "node:path";
|
||||||
|
import type * as db from "@rakazo/db";
|
||||||
|
import { SpaceLimitError } from "@rakazo/db";
|
||||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||||
import { getScanStatus, startOnboardingScan } from "./onboarding-scan.js";
|
import { getScanStatus, startOnboardingScan } from "./onboarding-scan.js";
|
||||||
|
|
||||||
const owner = { userId: "user-owner", isDeploymentOwner: true };
|
const createSpaceForMember = vi.hoisted(() => vi.fn());
|
||||||
const member = { userId: "user-member", isDeploymentOwner: false };
|
const ensureMainConversation = vi.hoisted(() => vi.fn());
|
||||||
|
vi.mock("@rakazo/db", async (original) => ({
|
||||||
|
...(await original<typeof db>()),
|
||||||
|
createSpaceForMember,
|
||||||
|
ensureMainConversation,
|
||||||
|
}));
|
||||||
|
|
||||||
|
const owner = { userId: "user-owner", spaceId: "space-owner", isDeploymentOwner: true };
|
||||||
|
const member = { userId: "user-member", spaceId: "space-member", isDeploymentOwner: false };
|
||||||
|
|
||||||
|
type SpaceUpdate = { where: { id: string }; data: { workspacePath: string } };
|
||||||
|
|
||||||
|
/** The spaces the scan reads and links; new ones come from `createSpaceForMember`. */
|
||||||
|
function fakeDatabase({ spaceLimit = Number.POSITIVE_INFINITY } = {}) {
|
||||||
|
const spaces: Array<{ id: string; name: string; workspacePath: string | null }> = [];
|
||||||
|
createSpaceForMember.mockImplementation(async (_prisma: unknown, input: { name: string }) => {
|
||||||
|
if (spaces.length >= spaceLimit) throw new SpaceLimitError();
|
||||||
|
const space = { id: `space-${spaces.length + 1}`, name: input.name, workspacePath: null };
|
||||||
|
spaces.push(space);
|
||||||
|
return { id: space.id, name: space.name };
|
||||||
|
});
|
||||||
|
const prisma = {
|
||||||
|
space: {
|
||||||
|
findMany: async () => spaces.filter((space) => space.workspacePath),
|
||||||
|
update: async ({ where, data }: SpaceUpdate) => {
|
||||||
|
const space = spaces.find((candidate) => candidate.id === where.id);
|
||||||
|
if (space) space.workspacePath = data.workspacePath;
|
||||||
|
},
|
||||||
|
},
|
||||||
|
} as unknown as db.PrismaClient;
|
||||||
|
return { prisma, spaces };
|
||||||
|
}
|
||||||
|
|
||||||
async function scanHomeWithOneFile(): Promise<string> {
|
async function scanHomeWithOneFile(): Promise<string> {
|
||||||
const home = await mkdtemp(join(tmpdir(), "evoo-scan-"));
|
const home = await mkdtemp(join(tmpdir(), "evoo-scan-"));
|
||||||
@ -14,6 +47,12 @@ async function scanHomeWithOneFile(): Promise<string> {
|
|||||||
return home;
|
return home;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function scanHomeWithRepos(...repos: string[]): Promise<string> {
|
||||||
|
const home = await scanHomeWithOneFile();
|
||||||
|
for (const repo of repos) await mkdir(join(home, repo, ".git"), { recursive: true });
|
||||||
|
return home;
|
||||||
|
}
|
||||||
|
|
||||||
async function completed(scanId: string, userId: string) {
|
async function completed(scanId: string, userId: string) {
|
||||||
return await vi.waitFor(
|
return await vi.waitFor(
|
||||||
async () => {
|
async () => {
|
||||||
@ -25,13 +64,22 @@ async function completed(scanId: string, userId: string) {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** The emulated runtime that the shared test setup pins keeps the scan simulated. */
|
||||||
|
function allowHostScan() {
|
||||||
|
vi.stubEnv("AGENT_RUNTIME", "");
|
||||||
|
vi.stubEnv("SANDBOX_PROVIDER", "");
|
||||||
|
}
|
||||||
|
|
||||||
describe("onboarding scan", () => {
|
describe("onboarding scan", () => {
|
||||||
afterEach(() => {
|
afterEach(() => {
|
||||||
vi.unstubAllEnvs();
|
vi.unstubAllEnvs();
|
||||||
|
createSpaceForMember.mockReset();
|
||||||
|
ensureMainConversation.mockReset();
|
||||||
});
|
});
|
||||||
|
|
||||||
it("hides a scan from every account except the one that started it", async () => {
|
it("hides a scan from every account except the one that started it", async () => {
|
||||||
const { scanId } = await startOnboardingScan({ actor: member }, {});
|
const { prisma } = fakeDatabase();
|
||||||
|
const { scanId } = await startOnboardingScan({ actor: member }, prisma);
|
||||||
await expect(getScanStatus(scanId, owner.userId)).rejects.toThrow(/Scan not found/);
|
await expect(getScanStatus(scanId, owner.userId)).rejects.toThrow(/Scan not found/);
|
||||||
await expect(getScanStatus(scanId, member.userId)).resolves.toMatchObject({
|
await expect(getScanStatus(scanId, member.userId)).resolves.toMatchObject({
|
||||||
status: expect.any(String),
|
status: expect.any(String),
|
||||||
@ -40,25 +88,79 @@ describe("onboarding scan", () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
it("never reads the host without a configured scan home", async () => {
|
it("never reads the host without a configured scan home", async () => {
|
||||||
const { scanId } = await startOnboardingScan({ actor: owner }, {});
|
const { prisma } = fakeDatabase();
|
||||||
|
const { scanId } = await startOnboardingScan({ actor: owner }, prisma);
|
||||||
const status = await completed(scanId, owner.userId);
|
const status = await completed(scanId, owner.userId);
|
||||||
expect(status.status).toBe("complete");
|
expect(status.status).toBe("complete");
|
||||||
expect(status.result).toMatchObject({ localRepos: [], fileSummary: { totalFiles: 0 } });
|
expect(status.result).toMatchObject({ localRepos: [], fileSummary: { totalFiles: 0 } });
|
||||||
});
|
});
|
||||||
|
|
||||||
it("lets only the deployment owner discover the configured scan home", async () => {
|
it("lets only the deployment owner discover the configured scan home", async () => {
|
||||||
// The shared test setup pins the emulated runtime, which also keeps the scan simulated.
|
allowHostScan();
|
||||||
vi.stubEnv("AGENT_RUNTIME", "");
|
const scanHome = await scanHomeWithRepos("Projects/app");
|
||||||
vi.stubEnv("SANDBOX_PROVIDER", "");
|
const { prisma } = fakeDatabase();
|
||||||
const scanHome = await scanHomeWithOneFile();
|
|
||||||
|
|
||||||
const asMember = await startOnboardingScan({ actor: member, scanHome }, {});
|
const asMember = await startOnboardingScan({ actor: member, scanHome }, prisma);
|
||||||
const memberStatus = await completed(asMember.scanId, member.userId);
|
const memberStatus = await completed(asMember.scanId, member.userId);
|
||||||
expect(memberStatus.result).toMatchObject({ fileSummary: { totalFiles: 0 } });
|
expect(memberStatus.result).toMatchObject({ fileSummary: { totalFiles: 0 } });
|
||||||
|
expect(createSpaceForMember).not.toHaveBeenCalled();
|
||||||
|
|
||||||
const asOwner = await startOnboardingScan({ actor: owner, scanHome }, {});
|
const asOwner = await startOnboardingScan({ actor: owner, scanHome }, prisma);
|
||||||
const ownerStatus = await completed(asOwner.scanId, owner.userId);
|
const ownerStatus = await completed(asOwner.scanId, owner.userId);
|
||||||
expect(ownerStatus.status).toBe("complete");
|
expect(ownerStatus.status).toBe("complete");
|
||||||
expect(ownerStatus.result).toMatchObject({ fileSummary: { totalFiles: 1 } });
|
expect(ownerStatus.result).toMatchObject({ fileSummary: { totalFiles: 1 } });
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("links each repository of the deployment owner to one project of its own", async () => {
|
||||||
|
allowHostScan();
|
||||||
|
const scanHome = await scanHomeWithRepos("Projects/app", ".config/nvim");
|
||||||
|
const app = await realpath(join(scanHome, "Projects", "app"));
|
||||||
|
const { prisma, spaces } = fakeDatabase();
|
||||||
|
|
||||||
|
const first = await startOnboardingScan({ actor: owner, scanHome }, prisma);
|
||||||
|
const firstStatus = await completed(first.scanId, owner.userId);
|
||||||
|
expect(firstStatus.status).toBe("complete");
|
||||||
|
expect(firstStatus.result).toMatchObject({
|
||||||
|
localRepos: [{ name: "app" }],
|
||||||
|
createdProjects: [{ spaceId: "space-1", name: "app", path: app }],
|
||||||
|
});
|
||||||
|
expect(createSpaceForMember).toHaveBeenCalledWith(prisma, {
|
||||||
|
currentSpaceId: owner.spaceId,
|
||||||
|
userId: owner.userId,
|
||||||
|
name: "app",
|
||||||
|
});
|
||||||
|
expect(spaces).toEqual([{ id: "space-1", name: "app", workspacePath: app }]);
|
||||||
|
// Each new project starts with its own main agent.
|
||||||
|
expect(ensureMainConversation).toHaveBeenCalledWith(prisma, owner.userId, "space-1");
|
||||||
|
|
||||||
|
// Scanning again finds the linked project instead of creating a second one.
|
||||||
|
const again = await startOnboardingScan({ actor: owner, scanHome }, prisma);
|
||||||
|
const againStatus = await completed(again.scanId, owner.userId);
|
||||||
|
expect(againStatus.result).toMatchObject({ createdProjects: [] });
|
||||||
|
expect(createSpaceForMember).toHaveBeenCalledTimes(1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("joins a running scan instead of linking its repositories twice", async () => {
|
||||||
|
allowHostScan();
|
||||||
|
const scanHome = await scanHomeWithRepos("Projects/app");
|
||||||
|
const { prisma, spaces } = fakeDatabase();
|
||||||
|
|
||||||
|
const first = await startOnboardingScan({ actor: owner, scanHome }, prisma);
|
||||||
|
const second = await startOnboardingScan({ actor: owner, scanHome }, prisma);
|
||||||
|
expect(second.scanId).toBe(first.scanId);
|
||||||
|
await completed(first.scanId, owner.userId);
|
||||||
|
expect(spaces).toHaveLength(1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("stops creating projects at the space limit and still completes", async () => {
|
||||||
|
allowHostScan();
|
||||||
|
const scanHome = await scanHomeWithRepos("Projects/a", "Projects/b", "Projects/c");
|
||||||
|
const { prisma } = fakeDatabase({ spaceLimit: 1 });
|
||||||
|
|
||||||
|
const { scanId } = await startOnboardingScan({ actor: owner, scanHome }, prisma);
|
||||||
|
const status = await completed(scanId, owner.userId);
|
||||||
|
expect(status.status).toBe("complete");
|
||||||
|
expect(status.result).toMatchObject({ createdProjects: [{ spaceId: "space-1" }] });
|
||||||
|
expect(createSpaceForMember).toHaveBeenCalledTimes(2);
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@ -2,8 +2,17 @@
|
|||||||
// Runs synchronously on the host machine (no sandbox needed).
|
// Runs synchronously on the host machine (no sandbox needed).
|
||||||
import { spawn } from "node:child_process";
|
import { spawn } from "node:child_process";
|
||||||
import { access, readdir, stat } from "node:fs/promises";
|
import { access, readdir, stat } from "node:fs/promises";
|
||||||
import { dirname, join } from "node:path";
|
import { basename, dirname, join } from "node:path";
|
||||||
import type { Actor } from "@rakazo/contracts";
|
import type { Actor } from "@rakazo/contracts";
|
||||||
|
import type { PrismaClient } from "@rakazo/db";
|
||||||
|
import {
|
||||||
|
createSpaceForMember,
|
||||||
|
ensureMainConversation,
|
||||||
|
InvalidSpaceNameError,
|
||||||
|
resolveProjectWorkspaceDirectory,
|
||||||
|
SpaceLimitError,
|
||||||
|
wireProjectWorkspace,
|
||||||
|
} from "@rakazo/db";
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Types
|
// Types
|
||||||
@ -54,10 +63,9 @@ interface ProfileSummary {
|
|||||||
}
|
}
|
||||||
|
|
||||||
interface CreatedProject {
|
interface CreatedProject {
|
||||||
id: string;
|
|
||||||
name: string;
|
|
||||||
spaceId: string;
|
spaceId: string;
|
||||||
description: string;
|
name: string;
|
||||||
|
path: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
interface ScanRecord {
|
interface ScanRecord {
|
||||||
@ -115,8 +123,10 @@ async function runSimulatedScan(record: ScanRecord) {
|
|||||||
record.completedAt = new Date();
|
record.completedAt = new Date();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
type ScanActor = Pick<Actor, "userId" | "spaceId" | "isDeploymentOwner">;
|
||||||
|
|
||||||
export interface OnboardingScanRequest {
|
export interface OnboardingScanRequest {
|
||||||
actor: Pick<Actor, "userId" | "isDeploymentOwner">;
|
actor: ScanActor;
|
||||||
botId?: string;
|
botId?: string;
|
||||||
/** Home-style directory to discover. Unset keeps the scan off the host filesystem. */
|
/** Home-style directory to discover. Unset keeps the scan off the host filesystem. */
|
||||||
scanHome?: string;
|
scanHome?: string;
|
||||||
@ -124,8 +134,12 @@ export interface OnboardingScanRequest {
|
|||||||
|
|
||||||
export async function startOnboardingScan(
|
export async function startOnboardingScan(
|
||||||
{ actor, botId, scanHome }: OnboardingScanRequest,
|
{ actor, botId, scanHome }: OnboardingScanRequest,
|
||||||
prisma: any,
|
prisma: PrismaClient,
|
||||||
): Promise<{ scanId: string }> {
|
): Promise<{ scanId: string }> {
|
||||||
|
// A repeated start (a remount, a retry) joins the running scan, so no repository is linked twice.
|
||||||
|
for (const running of scanRegistry.values()) {
|
||||||
|
if (running.ownerUserId === actor.userId && !running.completedAt) return { scanId: running.id };
|
||||||
|
}
|
||||||
const scanId = `scan_${Date.now()}_${Math.random().toString(36).slice(2, 8)}`;
|
const scanId = `scan_${Date.now()}_${Math.random().toString(36).slice(2, 8)}`;
|
||||||
const record: ScanRecord = {
|
const record: ScanRecord = {
|
||||||
id: scanId,
|
id: scanId,
|
||||||
@ -154,7 +168,7 @@ export async function startOnboardingScan(
|
|||||||
await runSimulatedScan(record);
|
await runSimulatedScan(record);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
const result = await executeFullScan(scanHome, botId ?? "", prisma);
|
const result = await executeFullScan(scanHome, actor, prisma);
|
||||||
record.status = "complete";
|
record.status = "complete";
|
||||||
record.progress = 100;
|
record.progress = 100;
|
||||||
record.message = "Scan complete — profile ready";
|
record.message = "Scan complete — profile ready";
|
||||||
@ -189,7 +203,11 @@ export async function getScanStatus(
|
|||||||
// Full scan pipeline
|
// Full scan pipeline
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
async function executeFullScan(homeDir: string, _botId: string, prisma: any): Promise<ScanResult> {
|
async function executeFullScan(
|
||||||
|
homeDir: string,
|
||||||
|
actor: ScanActor,
|
||||||
|
prisma: PrismaClient,
|
||||||
|
): Promise<ScanResult> {
|
||||||
// Phase 1: Local file scan
|
// Phase 1: Local file scan
|
||||||
const [fileSummary, projectFolders] = await scanFiles(homeDir);
|
const [fileSummary, projectFolders] = await scanFiles(homeDir);
|
||||||
|
|
||||||
@ -200,8 +218,8 @@ async function executeFullScan(homeDir: string, _botId: string, prisma: any): Pr
|
|||||||
// Phase 4: Profile compilation from findings
|
// Phase 4: Profile compilation from findings
|
||||||
const profileSummary = compileProfile(localRepos, fileSummary, projectFolders);
|
const profileSummary = compileProfile(localRepos, fileSummary, projectFolders);
|
||||||
|
|
||||||
// Phase 6: Project folder creation
|
// Phase 6: Projects for discovered repositories
|
||||||
const createdProjects = await createProjectFolders(localRepos, prisma);
|
const createdProjects = await createProjects(localRepos, actor, prisma);
|
||||||
|
|
||||||
// Phase 7: Sub-agent role creation
|
// Phase 7: Sub-agent role creation
|
||||||
const subAgentRoles = generateSubAgentRoles(profileSummary, localRepos);
|
const subAgentRoles = generateSubAgentRoles(profileSummary, localRepos);
|
||||||
@ -413,9 +431,9 @@ async function findGitDirectories(dir: string, depth: number = 0): Promise<strin
|
|||||||
if (entry.name === ".git") {
|
if (entry.name === ".git") {
|
||||||
results.push(dirname(fullPath));
|
results.push(dirname(fullPath));
|
||||||
} else if (
|
} else if (
|
||||||
|
// Dot folders hold tools and caches (shell and editor plugins), not the person's projects.
|
||||||
|
!entry.name.startsWith(".") &&
|
||||||
entry.name !== "node_modules" &&
|
entry.name !== "node_modules" &&
|
||||||
entry.name !== ".git" &&
|
|
||||||
entry.name !== ".venv" &&
|
|
||||||
entry.name !== "dist"
|
entry.name !== "dist"
|
||||||
) {
|
) {
|
||||||
results.push(...(await findGitDirectories(fullPath, depth + 1)));
|
results.push(...(await findGitDirectories(fullPath, depth + 1)));
|
||||||
@ -622,38 +640,45 @@ function generateBriefing(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Phase 5: Project folder creation
|
// Phase 5: Projects for discovered repositories
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
async function createProjectFolders(repos: RepoInfo[], prisma: any): Promise<CreatedProject[]> {
|
/** Give each discovered repository a project linked to its folder, unless one already links it. */
|
||||||
|
async function createProjects(
|
||||||
|
repos: RepoInfo[],
|
||||||
|
actor: ScanActor,
|
||||||
|
prisma: PrismaClient,
|
||||||
|
): Promise<CreatedProject[]> {
|
||||||
|
const linked = await prisma.space.findMany({
|
||||||
|
where: { workspacePath: { not: null }, memberships: { some: { userId: actor.userId } } },
|
||||||
|
select: { workspacePath: true },
|
||||||
|
});
|
||||||
|
const linkedFolders = new Set(linked.map((space) => space.workspacePath));
|
||||||
const created: CreatedProject[] = [];
|
const created: CreatedProject[] = [];
|
||||||
|
|
||||||
for (const repo of repos) {
|
for (const repo of repos) {
|
||||||
|
// Compare and link the real folder, so the next scan recognizes the projects of this one.
|
||||||
|
// Checking it before the project exists means a folder that cannot be linked leaves none.
|
||||||
|
const folder = await resolveProjectWorkspaceDirectory(repo.path).catch(() => null);
|
||||||
|
if (!folder || linkedFolders.has(folder)) continue;
|
||||||
|
let space: { id: string; name: string };
|
||||||
try {
|
try {
|
||||||
// Check if space already exists
|
space = await createSpaceForMember(prisma, {
|
||||||
const existing = await prisma.space.findFirst({ where: { title: repo.name } });
|
currentSpaceId: actor.spaceId,
|
||||||
if (existing) {
|
userId: actor.userId,
|
||||||
created.push({
|
name: basename(folder),
|
||||||
id: existing.id,
|
|
||||||
name: repo.name,
|
|
||||||
spaceId: existing.id,
|
|
||||||
description: repo.remote || repo.path,
|
|
||||||
});
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
// Create new space
|
|
||||||
const space = await prisma.space.create({
|
|
||||||
data: { title: repo.name, path: repo.path },
|
|
||||||
});
|
});
|
||||||
created.push({
|
} catch (error) {
|
||||||
id: space.id,
|
if (error instanceof SpaceLimitError) break;
|
||||||
name: repo.name,
|
if (error instanceof InvalidSpaceNameError) continue;
|
||||||
spaceId: space.id,
|
throw error;
|
||||||
description: repo.remote || repo.path,
|
|
||||||
});
|
|
||||||
} catch {
|
|
||||||
// Space might already exist or DB unavailable — skip
|
|
||||||
}
|
}
|
||||||
|
const workspace = await wireProjectWorkspace(prisma, {
|
||||||
|
spaceId: space.id,
|
||||||
|
workspacePath: folder,
|
||||||
|
});
|
||||||
|
await ensureMainConversation(prisma, actor.userId, space.id);
|
||||||
|
created.push({ spaceId: space.id, name: space.name, path: workspace.path });
|
||||||
}
|
}
|
||||||
|
|
||||||
return created;
|
return created;
|
||||||
|
|||||||
@ -79,7 +79,7 @@ async function requireBotThread(deps: OnboardingDeps, actor: Actor, botId: strin
|
|||||||
});
|
});
|
||||||
if (!bot?.thread) throw new IsolationError();
|
if (!bot?.thread) throw new IsolationError();
|
||||||
if (bot.kind === "main") {
|
if (bot.kind === "main") {
|
||||||
const conversation = await ensureMainConversation(deps.prisma, actor.userId);
|
const conversation = await ensureMainConversation(deps.prisma, actor.userId, bot.spaceId);
|
||||||
return { bot, thread: { id: conversation.threadId } };
|
return { bot, thread: { id: conversation.threadId } };
|
||||||
}
|
}
|
||||||
return { bot, thread: bot.thread };
|
return { bot, thread: bot.thread };
|
||||||
|
|||||||
@ -24,6 +24,7 @@ import type {
|
|||||||
ConnectorRegistry,
|
ConnectorRegistry,
|
||||||
EncryptedSecretStore,
|
EncryptedSecretStore,
|
||||||
IntegrationProviderSettings,
|
IntegrationProviderSettings,
|
||||||
|
MediaServices,
|
||||||
MemoryProviderResolver,
|
MemoryProviderResolver,
|
||||||
PiOAuthLogins,
|
PiOAuthLogins,
|
||||||
RemoteConnectorDependencies,
|
RemoteConnectorDependencies,
|
||||||
@ -88,7 +89,15 @@ import {
|
|||||||
verifyMcpInstall,
|
verifyMcpInstall,
|
||||||
} from "@rakazo/adapters";
|
} from "@rakazo/adapters";
|
||||||
import type { Auth } from "@rakazo/auth";
|
import type { Auth } from "@rakazo/auth";
|
||||||
import type { Actor, ComputerStatus, McpServer, Me, SpaceNavigation } from "@rakazo/contracts";
|
import type {
|
||||||
|
Actor,
|
||||||
|
ComputerStatus,
|
||||||
|
McpServer,
|
||||||
|
Me,
|
||||||
|
SpaceBot,
|
||||||
|
SpaceGroup,
|
||||||
|
SpaceNavigation,
|
||||||
|
} from "@rakazo/contracts";
|
||||||
import {
|
import {
|
||||||
appContract,
|
appContract,
|
||||||
IntegrationProviderIdSchema,
|
IntegrationProviderIdSchema,
|
||||||
@ -131,12 +140,14 @@ import {
|
|||||||
listSpaceUserBotIds,
|
listSpaceUserBotIds,
|
||||||
newestModelCredentialOrder,
|
newestModelCredentialOrder,
|
||||||
newestVoiceCredentialOrder,
|
newestVoiceCredentialOrder,
|
||||||
|
ownMainAgentsBlockingSpaceDeletion,
|
||||||
Prisma,
|
Prisma,
|
||||||
parseComputerMode,
|
parseComputerMode,
|
||||||
projectWorkspaceDto,
|
projectWorkspaceDto,
|
||||||
releaseSpaceDeletionClaim,
|
releaseSpaceDeletionClaim,
|
||||||
renewSpaceDeletionClaim,
|
renewSpaceDeletionClaim,
|
||||||
requireMembership,
|
requireMembership,
|
||||||
|
resolveProjectWorkspaceDirectory,
|
||||||
SPACE_DELETION_CLAIM_TIMEOUT_MS,
|
SPACE_DELETION_CLAIM_TIMEOUT_MS,
|
||||||
SpaceDeletionInProgressError,
|
SpaceDeletionInProgressError,
|
||||||
SpaceLimitError,
|
SpaceLimitError,
|
||||||
@ -166,6 +177,7 @@ import {
|
|||||||
renameMainConversation,
|
renameMainConversation,
|
||||||
} from "./main-conversation.js";
|
} from "./main-conversation.js";
|
||||||
import { buildMcpUpdateMaterial } from "./mcp-material.js";
|
import { buildMcpUpdateMaterial } from "./mcp-material.js";
|
||||||
|
import { createMediaRouter } from "./media.js";
|
||||||
import {
|
import {
|
||||||
disconnectMemoryProvider,
|
disconnectMemoryProvider,
|
||||||
getMemoryProviderConfig,
|
getMemoryProviderConfig,
|
||||||
@ -463,6 +475,8 @@ export interface RouterDeps {
|
|||||||
remoteConnectors?: RemoteConnectorDependencies;
|
remoteConnectors?: RemoteConnectorDependencies;
|
||||||
artifacts: ArtifactStore;
|
artifacts: ArtifactStore;
|
||||||
dataDir: string;
|
dataDir: string;
|
||||||
|
/** Null when no media sidecar (or scripted emulator) is configured; Media Studio then stays hidden. */
|
||||||
|
mediaServices?: MediaServices | null;
|
||||||
/** Present when the external messaging surface is enabled. */
|
/** Present when the external messaging surface is enabled. */
|
||||||
messaging?: { enabled: boolean; providers: string[]; openSignup: boolean };
|
messaging?: { enabled: boolean; providers: string[]; openSignup: boolean };
|
||||||
env: {
|
env: {
|
||||||
@ -490,6 +504,26 @@ export interface RouterDeps {
|
|||||||
* keep the deletion claim renewed forever and block stale-claim recovery. */
|
* keep the deletion claim renewed forever and block stale-claim recovery. */
|
||||||
const SPACE_TEARDOWN_TIMEOUT_MS = 120_000;
|
const SPACE_TEARDOWN_TIMEOUT_MS = 120_000;
|
||||||
|
|
||||||
|
async function removeGroupWithArtifacts(
|
||||||
|
deps: RouterDeps,
|
||||||
|
groupRepos: ReturnType<typeof createGroupRepos>,
|
||||||
|
actor: Actor,
|
||||||
|
groupId: string,
|
||||||
|
) {
|
||||||
|
const removed = await groupRepos.removeGroup(actor, groupId);
|
||||||
|
const cleanup = await Promise.allSettled(
|
||||||
|
removed.artifactStorageKeys.map((storageKey) =>
|
||||||
|
deps.artifacts.remove(
|
||||||
|
storageKey,
|
||||||
|
computerContext(actor, removed.contextBotId, `group-remove:${groupId}`),
|
||||||
|
),
|
||||||
|
),
|
||||||
|
);
|
||||||
|
for (const result of cleanup) {
|
||||||
|
if (result.status === "rejected") getLogger().error("group artifact cleanup", result.reason);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
function spaceTeardownTimeoutMs(): number {
|
function spaceTeardownTimeoutMs(): number {
|
||||||
const override = Number(process.env.SPACE_TEARDOWN_TIMEOUT_MS ?? "");
|
const override = Number(process.env.SPACE_TEARDOWN_TIMEOUT_MS ?? "");
|
||||||
return Number.isFinite(override) && override > 0 ? override : SPACE_TEARDOWN_TIMEOUT_MS;
|
return Number.isFinite(override) && override > 0 ? override : SPACE_TEARDOWN_TIMEOUT_MS;
|
||||||
@ -518,7 +552,9 @@ export function createRouter(deps: RouterDeps) {
|
|||||||
home: deps.home,
|
home: deps.home,
|
||||||
dataDir: deps.dataDir,
|
dataDir: deps.dataDir,
|
||||||
});
|
});
|
||||||
const agentSkills = createAgentSkillsService(deps.prisma);
|
const agentSkills = createAgentSkillsService(deps.prisma, {
|
||||||
|
mediaConfigured: Boolean(deps.mediaServices),
|
||||||
|
});
|
||||||
|
|
||||||
const authed = os.use(async ({ context, next }) => {
|
const authed = os.use(async ({ context, next }) => {
|
||||||
if (!context.actor) throw new ORPCError("UNAUTHORIZED");
|
if (!context.actor) throw new ORPCError("UNAUTHORIZED");
|
||||||
@ -526,6 +562,11 @@ export function createRouter(deps: RouterDeps) {
|
|||||||
});
|
});
|
||||||
|
|
||||||
return os.router({
|
return os.router({
|
||||||
|
media: createMediaRouter({
|
||||||
|
prisma: deps.prisma,
|
||||||
|
services: deps.mediaServices ?? null,
|
||||||
|
dataDir: deps.dataDir,
|
||||||
|
}),
|
||||||
aiConsent: {
|
aiConsent: {
|
||||||
status: authed.aiConsent.status.handler(({ context, input }) =>
|
status: authed.aiConsent.status.handler(({ context, input }) =>
|
||||||
aiConsentStatus(deps, context.actor, input),
|
aiConsentStatus(deps, context.actor, input),
|
||||||
@ -569,6 +610,22 @@ export function createRouter(deps: RouterDeps) {
|
|||||||
spaceNavigationDto(deps, context.actor, repos, groupRepos),
|
spaceNavigationDto(deps, context.actor, repos, groupRepos),
|
||||||
),
|
),
|
||||||
create: authed.spaces.create.handler(async ({ context, input }) => {
|
create: authed.spaces.create.handler(async ({ context, input }) => {
|
||||||
|
// A linked folder lives on the host, and the host is the deployment owner's machine.
|
||||||
|
if (input.workspacePath && !context.actor.isDeploymentOwner) {
|
||||||
|
throw new ORPCError("FORBIDDEN");
|
||||||
|
}
|
||||||
|
// Check the folder before the space exists, so a bad path leaves no space behind.
|
||||||
|
let workspacePath: string | null = null;
|
||||||
|
if (input.workspacePath) {
|
||||||
|
try {
|
||||||
|
workspacePath = await resolveProjectWorkspaceDirectory(input.workspacePath);
|
||||||
|
} catch (error) {
|
||||||
|
if (error instanceof InvalidProjectWorkspacePathError) {
|
||||||
|
throw new ORPCError("BAD_REQUEST", { message: error.message });
|
||||||
|
}
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
let space: { id: string; name: string };
|
let space: { id: string; name: string };
|
||||||
try {
|
try {
|
||||||
space = await createSpaceForMember(deps.prisma, {
|
space = await createSpaceForMember(deps.prisma, {
|
||||||
@ -582,20 +639,16 @@ export function createRouter(deps: RouterDeps) {
|
|||||||
}
|
}
|
||||||
throw error;
|
throw error;
|
||||||
}
|
}
|
||||||
let workspace = null;
|
const workspace = workspacePath
|
||||||
if (input.workspacePath) {
|
? await wireProjectWorkspace(deps.prisma, { spaceId: space.id, workspacePath })
|
||||||
try {
|
: null;
|
||||||
workspace = await wireProjectWorkspace(deps.prisma, {
|
// A project starts as a conversation with its own main agent.
|
||||||
spaceId: space.id,
|
await ensureMainConversation(deps.prisma, context.actor.userId, space.id);
|
||||||
workspacePath: input.workspacePath,
|
const spaceActor = await requireMembership(deps.prisma, context.actor.userId, space.id);
|
||||||
});
|
const [bots, groups] = await Promise.all([
|
||||||
} catch (error) {
|
repos.listBots(spaceActor),
|
||||||
if (error instanceof InvalidProjectWorkspacePathError) {
|
groupRepos.listGroups(spaceActor),
|
||||||
throw new ORPCError("BAD_REQUEST", { message: error.message });
|
]);
|
||||||
}
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return {
|
return {
|
||||||
id: space.id,
|
id: space.id,
|
||||||
name: space.name,
|
name: space.name,
|
||||||
@ -603,12 +656,26 @@ export function createRouter(deps: RouterDeps) {
|
|||||||
hasContent: false,
|
hasContent: false,
|
||||||
canDelete: true,
|
canDelete: true,
|
||||||
workspace,
|
workspace,
|
||||||
bots: [],
|
bots: bots.map(spaceBotDto),
|
||||||
groups: [],
|
groups: groups.map(spaceGroupDto),
|
||||||
externalConversations: [],
|
externalConversations: [],
|
||||||
botSections: [],
|
botSections: [],
|
||||||
};
|
};
|
||||||
}),
|
}),
|
||||||
|
ensureMain: authed.spaces.ensureMain.handler(async ({ context, input }) => {
|
||||||
|
const conversation = await ensureMainConversation(
|
||||||
|
deps.prisma,
|
||||||
|
context.actor.userId,
|
||||||
|
input.spaceId,
|
||||||
|
).catch((error: unknown) => {
|
||||||
|
throw mapSpaceLifecycleError(error);
|
||||||
|
});
|
||||||
|
return {
|
||||||
|
botId: conversation.leadBotId,
|
||||||
|
groupId: conversation.groupId,
|
||||||
|
spaceId: conversation.spaceId,
|
||||||
|
};
|
||||||
|
}),
|
||||||
remove: authed.spaces.remove.handler(async ({ context, input }) => {
|
remove: authed.spaces.remove.handler(async ({ context, input }) => {
|
||||||
let claimId: string | null = null;
|
let claimId: string | null = null;
|
||||||
let claimActive = false;
|
let claimActive = false;
|
||||||
@ -621,6 +688,41 @@ export function createRouter(deps: RouterDeps) {
|
|||||||
spaceId: input.spaceId,
|
spaceId: input.spaceId,
|
||||||
};
|
};
|
||||||
try {
|
try {
|
||||||
|
// The project's own main agent goes with the project; anything else must be
|
||||||
|
// cleared first.
|
||||||
|
const ownMain = await ownMainAgentsBlockingSpaceDeletion(deps.prisma, deleteInput);
|
||||||
|
if (ownMain.groupIds.length > 0 || ownMain.botIds.length > 0) {
|
||||||
|
const spaceActor = await requireMembership(
|
||||||
|
deps.prisma,
|
||||||
|
context.actor.userId,
|
||||||
|
input.spaceId,
|
||||||
|
);
|
||||||
|
for (const groupId of ownMain.groupIds) {
|
||||||
|
await removeGroupWithArtifacts(deps, groupRepos, spaceActor, groupId);
|
||||||
|
}
|
||||||
|
for (const botId of ownMain.botIds) {
|
||||||
|
const bot = await repos.getBot(spaceActor, botId, { includeArchived: true });
|
||||||
|
await destroyBot(
|
||||||
|
{
|
||||||
|
prisma: deps.prisma,
|
||||||
|
sandbox: deps.sandbox,
|
||||||
|
home: deps.home,
|
||||||
|
jobs: deps.jobs,
|
||||||
|
artifacts: deps.artifacts,
|
||||||
|
dataDir: deps.dataDir,
|
||||||
|
},
|
||||||
|
bot,
|
||||||
|
{
|
||||||
|
operationId: "spaces.remove",
|
||||||
|
traceId: "spaces.remove",
|
||||||
|
spaceId: input.spaceId,
|
||||||
|
userId: context.actor.userId,
|
||||||
|
signal: new AbortController().signal,
|
||||||
|
},
|
||||||
|
{ deleteMemories: true },
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
// Claim emptiness before external teardown. Bot and group creation
|
// Claim emptiness before external teardown. Bot and group creation
|
||||||
// take the same lifecycle lock and reject the Space until deletion
|
// take the same lifecycle lock and reject the Space until deletion
|
||||||
// finishes or this claim is released.
|
// finishes or this claim is released.
|
||||||
@ -1219,7 +1321,11 @@ export function createRouter(deps: RouterDeps) {
|
|||||||
},
|
},
|
||||||
});
|
});
|
||||||
if (existing.kind === "main" && input.name !== undefined && input.name !== existing.name) {
|
if (existing.kind === "main" && input.name !== undefined && input.name !== existing.name) {
|
||||||
await renameMainConversation(deps.prisma, context.actor.userId, input.name);
|
await renameMainConversation(
|
||||||
|
deps.prisma,
|
||||||
|
{ userId: context.actor.userId, spaceId: existing.spaceId },
|
||||||
|
input.name,
|
||||||
|
);
|
||||||
}
|
}
|
||||||
const bots = await repos.listBots(context.actor);
|
const bots = await repos.listBots(context.actor);
|
||||||
const bot = bots.find((b) => b.id === input.botId);
|
const bot = bots.find((b) => b.id === input.botId);
|
||||||
@ -1507,19 +1613,7 @@ export function createRouter(deps: RouterDeps) {
|
|||||||
input.groupId,
|
input.groupId,
|
||||||
"The main conversation can't be deleted.",
|
"The main conversation can't be deleted.",
|
||||||
);
|
);
|
||||||
const removed = await groupRepos.removeGroup(context.actor, input.groupId);
|
await removeGroupWithArtifacts(deps, groupRepos, context.actor, input.groupId);
|
||||||
const cleanup = await Promise.allSettled(
|
|
||||||
removed.artifactStorageKeys.map((storageKey) =>
|
|
||||||
deps.artifacts.remove(
|
|
||||||
storageKey,
|
|
||||||
computerContext(context.actor, removed.contextBotId, `group-remove:${input.groupId}`),
|
|
||||||
),
|
|
||||||
),
|
|
||||||
);
|
|
||||||
for (const result of cleanup) {
|
|
||||||
if (result.status === "rejected")
|
|
||||||
getLogger().error("group artifact cleanup", result.reason);
|
|
||||||
}
|
|
||||||
return { ok: true as const };
|
return { ok: true as const };
|
||||||
}),
|
}),
|
||||||
},
|
},
|
||||||
@ -2230,10 +2324,7 @@ export function createRouter(deps: RouterDeps) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
if (isProjectWorkspacePath(storedPath)) {
|
if (isProjectWorkspacePath(storedPath)) {
|
||||||
const workspaceRoot = await requireProjectWorkspaceRoot(
|
const workspaceRoot = await requireProjectWorkspaceRoot(deps.prisma, context.actor);
|
||||||
deps.prisma,
|
|
||||||
context.actor.spaceId,
|
|
||||||
);
|
|
||||||
const entries = await listProjectWorkspaceFiles(workspaceRoot, storedPath);
|
const entries = await listProjectWorkspaceFiles(workspaceRoot, storedPath);
|
||||||
return entries.map((entry) => ({
|
return entries.map((entry) => ({
|
||||||
...entry,
|
...entry,
|
||||||
@ -2281,10 +2372,7 @@ export function createRouter(deps: RouterDeps) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
if (isProjectWorkspacePath(storedPath)) {
|
if (isProjectWorkspacePath(storedPath)) {
|
||||||
const workspaceRoot = await requireProjectWorkspaceRoot(
|
const workspaceRoot = await requireProjectWorkspaceRoot(deps.prisma, context.actor);
|
||||||
deps.prisma,
|
|
||||||
context.actor.spaceId,
|
|
||||||
);
|
|
||||||
const bytes = await readProjectWorkspaceFile(workspaceRoot, storedPath, {
|
const bytes = await readProjectWorkspaceFile(workspaceRoot, storedPath, {
|
||||||
maxBytes: MAX_COMPUTER_TEXT_FILE_BYTES,
|
maxBytes: MAX_COMPUTER_TEXT_FILE_BYTES,
|
||||||
});
|
});
|
||||||
@ -4894,10 +4982,13 @@ function mapUpdaterError(error: unknown): never {
|
|||||||
|
|
||||||
async function requireProjectWorkspaceRoot(
|
async function requireProjectWorkspaceRoot(
|
||||||
prisma: Pick<PrismaClient, "space">,
|
prisma: Pick<PrismaClient, "space">,
|
||||||
spaceId: string,
|
actor: Actor,
|
||||||
): Promise<string> {
|
): Promise<string> {
|
||||||
|
// Only the deployment owner may link a host folder, so only they may browse one. This also
|
||||||
|
// closes folders that other members linked before linking was limited to the owner.
|
||||||
|
if (!actor.isDeploymentOwner) throw new ORPCError("FORBIDDEN");
|
||||||
const space = await prisma.space.findUnique({
|
const space = await prisma.space.findUnique({
|
||||||
where: { id: spaceId },
|
where: { id: actor.spaceId },
|
||||||
select: { workspacePath: true },
|
select: { workspacePath: true },
|
||||||
});
|
});
|
||||||
if (!space?.workspacePath) {
|
if (!space?.workspacePath) {
|
||||||
@ -4939,6 +5030,11 @@ async function spaceNavigationDto(
|
|||||||
orderBy: { createdAt: "asc" },
|
orderBy: { createdAt: "asc" },
|
||||||
});
|
});
|
||||||
const spaceIds = memberships.map((membership) => membership.spaceId);
|
const spaceIds = memberships.map((membership) => membership.spaceId);
|
||||||
|
const ownProjectMain = {
|
||||||
|
userId: actor.userId,
|
||||||
|
kind: "main",
|
||||||
|
space: { isDefault: false },
|
||||||
|
} as const;
|
||||||
const inactiveSpaceIds = spaceIds.filter((spaceId) => spaceId !== actor.spaceId);
|
const inactiveSpaceIds = spaceIds.filter((spaceId) => spaceId !== actor.spaceId);
|
||||||
const [
|
const [
|
||||||
currentBots,
|
currentBots,
|
||||||
@ -4959,13 +5055,14 @@ async function spaceNavigationDto(
|
|||||||
// Active-only navigation lists miss archived content in other spaces; count any
|
// Active-only navigation lists miss archived content in other spaces; count any
|
||||||
// bot/group in the actor's spaces (including another member's) so a shared
|
// bot/group in the actor's spaces (including another member's) so a shared
|
||||||
// non-empty space cannot look empty for onboarding redirects.
|
// non-empty space cannot look empty for onboarding redirects.
|
||||||
|
// The user's own main agent of a project is part of the project, not content to clear.
|
||||||
deps.prisma.bot.findMany({
|
deps.prisma.bot.findMany({
|
||||||
where: { spaceId: { in: spaceIds } },
|
where: { spaceId: { in: spaceIds }, NOT: ownProjectMain },
|
||||||
select: { spaceId: true },
|
select: { spaceId: true },
|
||||||
distinct: ["spaceId"],
|
distinct: ["spaceId"],
|
||||||
}),
|
}),
|
||||||
deps.prisma.chatGroup.findMany({
|
deps.prisma.chatGroup.findMany({
|
||||||
where: { spaceId: { in: spaceIds } },
|
where: { spaceId: { in: spaceIds }, NOT: ownProjectMain },
|
||||||
select: { spaceId: true },
|
select: { spaceId: true },
|
||||||
distinct: ["spaceId"],
|
distinct: ["spaceId"],
|
||||||
}),
|
}),
|
||||||
@ -5009,34 +5106,10 @@ async function spaceNavigationDto(
|
|||||||
memberships.length > 1 &&
|
memberships.length > 1 &&
|
||||||
!spacesWithContent.has(membership.spaceId) &&
|
!spacesWithContent.has(membership.spaceId) &&
|
||||||
(membership.space.deletingAt === null || membership.space.deletingAt < staleClaimBefore),
|
(membership.space.deletingAt === null || membership.space.deletingAt < staleClaimBefore),
|
||||||
workspace: projectWorkspaceDto(membership.space),
|
// A linked folder is on the owner's host; other members see neither its path nor remote.
|
||||||
bots: spaceBots.map((bot) => ({
|
workspace: actor.isDeploymentOwner ? projectWorkspaceDto(membership.space) : null,
|
||||||
id: bot.id,
|
bots: spaceBots.map(spaceBotDto),
|
||||||
spaceId: bot.spaceId,
|
groups: spaceGroups.map(spaceGroupDto),
|
||||||
kind: bot.kind,
|
|
||||||
name: bot.name,
|
|
||||||
title: bot.title,
|
|
||||||
color: bot.color,
|
|
||||||
notifyOnFinish: bot.notifyOnFinish,
|
|
||||||
pinned: bot.pinned,
|
|
||||||
sectionId: bot.sectionId,
|
|
||||||
unread: bot.unread,
|
|
||||||
preview: bot.preview,
|
|
||||||
status: bot.status,
|
|
||||||
updatedAt: bot.updatedAt,
|
|
||||||
})),
|
|
||||||
groups: spaceGroups.map((group) => ({
|
|
||||||
id: group.id,
|
|
||||||
spaceId: group.spaceId,
|
|
||||||
kind: group.kind,
|
|
||||||
name: group.name,
|
|
||||||
pinned: group.pinned,
|
|
||||||
sectionId: group.sectionId,
|
|
||||||
members: group.members,
|
|
||||||
preview: group.preview,
|
|
||||||
unread: group.unread,
|
|
||||||
updatedAt: group.updatedAt,
|
|
||||||
})),
|
|
||||||
externalConversations: externalConversations.filter(
|
externalConversations: externalConversations.filter(
|
||||||
(conversation) => conversation.spaceId === membership.spaceId,
|
(conversation) => conversation.spaceId === membership.spaceId,
|
||||||
),
|
),
|
||||||
@ -5047,6 +5120,39 @@ async function spaceNavigationDto(
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function spaceBotDto(bot: SpaceBot): SpaceBot {
|
||||||
|
return {
|
||||||
|
id: bot.id,
|
||||||
|
spaceId: bot.spaceId,
|
||||||
|
kind: bot.kind,
|
||||||
|
name: bot.name,
|
||||||
|
title: bot.title,
|
||||||
|
color: bot.color,
|
||||||
|
notifyOnFinish: bot.notifyOnFinish,
|
||||||
|
pinned: bot.pinned,
|
||||||
|
sectionId: bot.sectionId,
|
||||||
|
unread: bot.unread,
|
||||||
|
preview: bot.preview,
|
||||||
|
status: bot.status,
|
||||||
|
updatedAt: bot.updatedAt,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function spaceGroupDto(group: SpaceGroup): SpaceGroup {
|
||||||
|
return {
|
||||||
|
id: group.id,
|
||||||
|
spaceId: group.spaceId,
|
||||||
|
kind: group.kind,
|
||||||
|
name: group.name,
|
||||||
|
pinned: group.pinned,
|
||||||
|
sectionId: group.sectionId,
|
||||||
|
members: group.members,
|
||||||
|
preview: group.preview,
|
||||||
|
unread: group.unread,
|
||||||
|
updatedAt: group.updatedAt,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
function partitionBySpace<T extends { spaceId: string }>(rows: T[]): Map<string, T[]> {
|
function partitionBySpace<T extends { spaceId: string }>(rows: T[]): Map<string, T[]> {
|
||||||
const partitioned = new Map<string, T[]>();
|
const partitioned = new Map<string, T[]>();
|
||||||
for (const row of rows) {
|
for (const row of rows) {
|
||||||
|
|||||||
143
apps/api/src/thread-message-pages.postgres.test.ts
Normal file
143
apps/api/src/thread-message-pages.postgres.test.ts
Normal file
@ -0,0 +1,143 @@
|
|||||||
|
import type { MessageBlock } from "@rakazo/contracts";
|
||||||
|
import type { PrismaClient } from "@rakazo/db";
|
||||||
|
import { createDb, provisionMessagingIdentity } from "@rakazo/db";
|
||||||
|
import { afterAll, beforeAll, describe, expect, it } from "vitest";
|
||||||
|
import { loadMessagePage } from "./thread-message-pages.js";
|
||||||
|
|
||||||
|
const databaseUrl = process.env.DATABASE_URL;
|
||||||
|
const describePostgres =
|
||||||
|
process.env.VERIFY_DATABASE && databaseUrl ? describe.sequential : describe.skip;
|
||||||
|
|
||||||
|
describePostgres("loadMessagePage (PostgreSQL)", () => {
|
||||||
|
let prisma: PrismaClient;
|
||||||
|
let close: () => Promise<void>;
|
||||||
|
let identity: { userId: string; spaceId: string; botId: string; threadId: string };
|
||||||
|
let userRunId: string;
|
||||||
|
let peerRunId: string;
|
||||||
|
let seq = 0;
|
||||||
|
|
||||||
|
beforeAll(async () => {
|
||||||
|
const db = createDb(databaseUrl!);
|
||||||
|
prisma = db.prisma;
|
||||||
|
close = async () => {
|
||||||
|
await db.prisma.$disconnect();
|
||||||
|
await db.pool.end();
|
||||||
|
};
|
||||||
|
identity = await provisionMessagingIdentity(
|
||||||
|
prisma,
|
||||||
|
{ provider: "sendblue", address: `+1555${Date.now().toString().slice(-7)}` },
|
||||||
|
{ signupsEnabled: undefined, signupAllowlist: undefined },
|
||||||
|
);
|
||||||
|
const { spaceId, botId, threadId, userId } = identity;
|
||||||
|
const owner = { spaceId, botId, threadId, userId };
|
||||||
|
const run = async (trigger: string) => {
|
||||||
|
const task = await prisma.task.create({
|
||||||
|
data: { ...owner, prompt: "fixture", status: "completed" },
|
||||||
|
});
|
||||||
|
const created = await prisma.run.create({
|
||||||
|
data: { ...owner, taskId: task.id, status: "completed", trigger },
|
||||||
|
});
|
||||||
|
return created.id;
|
||||||
|
};
|
||||||
|
userRunId = await run("user_message");
|
||||||
|
peerRunId = await run("bot_message");
|
||||||
|
seq =
|
||||||
|
(
|
||||||
|
await prisma.message.aggregate({
|
||||||
|
where: { threadId: identity.threadId },
|
||||||
|
_max: { seq: true },
|
||||||
|
})
|
||||||
|
)._max.seq ?? 0;
|
||||||
|
});
|
||||||
|
|
||||||
|
afterAll(async () => {
|
||||||
|
if (identity) {
|
||||||
|
await prisma.messagingIdentity.deleteMany({ where: { userId: identity.userId } });
|
||||||
|
await prisma.organization.deleteMany({ where: { id: identity.spaceId } });
|
||||||
|
await prisma.user.deleteMany({ where: { id: identity.userId } });
|
||||||
|
}
|
||||||
|
await close?.();
|
||||||
|
});
|
||||||
|
|
||||||
|
async function message(id: string, runId: string | null, blocks: MessageBlock[]) {
|
||||||
|
seq += 1;
|
||||||
|
await prisma.message.create({
|
||||||
|
data: {
|
||||||
|
id: `${identity.threadId}-${id}`,
|
||||||
|
threadId: identity.threadId,
|
||||||
|
seq,
|
||||||
|
role: "bot",
|
||||||
|
runId,
|
||||||
|
blocks,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
return seq;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ids = (page: Awaited<ReturnType<typeof loadMessagePage>>) =>
|
||||||
|
page.messages.map((row) => row.id.slice(identity.threadId.length + 1));
|
||||||
|
|
||||||
|
it("fills a page with visible rows across a long stretch of hidden peer activity", async () => {
|
||||||
|
await message("older", userRunId, [{ kind: "text", text: "Older answer" }]);
|
||||||
|
await message("newer", userRunId, [{ kind: "text", text: "Newer answer" }]);
|
||||||
|
for (let index = 0; index < 30; index += 1) {
|
||||||
|
await message(`peer-steps-${index}`, peerRunId, [
|
||||||
|
{ kind: "steps", steps: [{ label: "Working", count: 1 }] },
|
||||||
|
]);
|
||||||
|
}
|
||||||
|
await message("peer-receipt", peerRunId, [
|
||||||
|
{ kind: "bot_message_received", fromBotId: "bot-2", fromBotName: "Coder", text: "Done." },
|
||||||
|
]);
|
||||||
|
await message("peer-reply", peerRunId, [{ kind: "text", text: "Peer summary" }]);
|
||||||
|
await message("orphan-run", "run-that-no-longer-exists", [{ kind: "text", text: "Kept" }]);
|
||||||
|
|
||||||
|
const first = await loadMessagePage(prisma, identity.threadId, undefined, 3);
|
||||||
|
expect(ids(first)).toEqual(["peer-receipt", "peer-reply", "orphan-run"]);
|
||||||
|
|
||||||
|
const second = await loadMessagePage(prisma, identity.threadId, first.olderCursor!, 2);
|
||||||
|
expect(ids(second)).toEqual(["older", "newer"]);
|
||||||
|
|
||||||
|
const peerView = await loadMessagePage(
|
||||||
|
prisma,
|
||||||
|
identity.threadId,
|
||||||
|
undefined,
|
||||||
|
3,
|
||||||
|
undefined,
|
||||||
|
true,
|
||||||
|
);
|
||||||
|
expect(ids(peerView)).toEqual(["peer-receipt", "peer-reply", "orphan-run"]);
|
||||||
|
const peerOlder = await loadMessagePage(
|
||||||
|
prisma,
|
||||||
|
identity.threadId,
|
||||||
|
peerView.olderCursor!,
|
||||||
|
1,
|
||||||
|
undefined,
|
||||||
|
true,
|
||||||
|
);
|
||||||
|
expect(ids(peerOlder)).toEqual(["peer-steps-29"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("skips a receipt-only page for web but returns it when receipts are displayed", async () => {
|
||||||
|
await message("before-receipts", userRunId, [{ kind: "text", text: "Answer" }]);
|
||||||
|
await message("sent", userRunId, [
|
||||||
|
{ kind: "bot_message_sent", toBotId: "bot-2", toBotName: "Coder", text: "Check this." },
|
||||||
|
]);
|
||||||
|
await message("received", peerRunId, [
|
||||||
|
{ kind: "bot_message_received", fromBotId: "bot-2", fromBotName: "Coder", text: "Done." },
|
||||||
|
]);
|
||||||
|
|
||||||
|
const web = await loadMessagePage(prisma, identity.threadId, undefined, 2);
|
||||||
|
expect(ids(web)).toEqual(["orphan-run", "before-receipts"]);
|
||||||
|
|
||||||
|
const mobile = await loadMessagePage(
|
||||||
|
prisma,
|
||||||
|
identity.threadId,
|
||||||
|
undefined,
|
||||||
|
2,
|
||||||
|
undefined,
|
||||||
|
false,
|
||||||
|
true,
|
||||||
|
);
|
||||||
|
expect(ids(mobile)).toEqual(["sent", "received"]);
|
||||||
|
});
|
||||||
|
});
|
||||||
@ -43,98 +43,6 @@ describe("thread message pages", () => {
|
|||||||
).toBe(false);
|
).toBe(false);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("keeps peer receipt rows when filtering peer-run output from pages", async () => {
|
|
||||||
const findMany = vi.fn(async () => [
|
|
||||||
{
|
|
||||||
id: "message-reply",
|
|
||||||
threadId: "thread-1",
|
|
||||||
seq: 3,
|
|
||||||
role: "bot",
|
|
||||||
blocks: [{ kind: "text", text: "Echoed peer reply" }],
|
|
||||||
botId: "bot-1",
|
|
||||||
replyToMessageId: null,
|
|
||||||
runId: "run-peer",
|
|
||||||
createdAt: new Date("2026-08-16T00:00:03.000Z"),
|
|
||||||
},
|
|
||||||
{
|
|
||||||
id: "message-received",
|
|
||||||
threadId: "thread-1",
|
|
||||||
seq: 2,
|
|
||||||
role: "user",
|
|
||||||
blocks: [
|
|
||||||
{
|
|
||||||
kind: "bot_message_received",
|
|
||||||
fromBotId: "bot-2",
|
|
||||||
fromBotName: "Coder",
|
|
||||||
text: "Done.",
|
|
||||||
},
|
|
||||||
],
|
|
||||||
botId: null,
|
|
||||||
replyToMessageId: null,
|
|
||||||
runId: "run-peer",
|
|
||||||
createdAt: new Date("2026-08-16T00:00:02.000Z"),
|
|
||||||
},
|
|
||||||
{
|
|
||||||
id: "message-user",
|
|
||||||
threadId: "thread-1",
|
|
||||||
seq: 1,
|
|
||||||
role: "bot",
|
|
||||||
blocks: [{ kind: "text", text: "Visible answer" }],
|
|
||||||
botId: "bot-1",
|
|
||||||
replyToMessageId: null,
|
|
||||||
runId: "run-user",
|
|
||||||
createdAt: new Date("2026-08-16T00:00:01.000Z"),
|
|
||||||
},
|
|
||||||
]);
|
|
||||||
const prisma = {
|
|
||||||
message: { findMany },
|
|
||||||
run: { findMany: vi.fn(async () => [{ id: "run-peer" }]) },
|
|
||||||
} as unknown as PrismaClient;
|
|
||||||
|
|
||||||
const page = await loadMessagePage(prisma, "thread-1", undefined, 3);
|
|
||||||
|
|
||||||
expect(page.messages.map((message) => message.id)).toEqual([
|
|
||||||
"message-user",
|
|
||||||
"message-received",
|
|
||||||
"message-reply",
|
|
||||||
]);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("filters peer-run activity when its receipt is outside the loaded page", async () => {
|
|
||||||
const findMany = vi.fn(async () => [
|
|
||||||
{
|
|
||||||
id: "message-peer",
|
|
||||||
threadId: "thread-1",
|
|
||||||
seq: 2,
|
|
||||||
role: "bot",
|
|
||||||
blocks: [{ kind: "steps", steps: [{ label: "Echoed peer reply", count: 1 }] }],
|
|
||||||
botId: "bot-1",
|
|
||||||
replyToMessageId: null,
|
|
||||||
runId: "run-peer",
|
|
||||||
createdAt: new Date("2026-08-16T00:00:02.000Z"),
|
|
||||||
},
|
|
||||||
{
|
|
||||||
id: "message-user",
|
|
||||||
threadId: "thread-1",
|
|
||||||
seq: 1,
|
|
||||||
role: "bot",
|
|
||||||
blocks: [{ kind: "text", text: "Visible answer" }],
|
|
||||||
botId: "bot-1",
|
|
||||||
replyToMessageId: null,
|
|
||||||
runId: "run-user",
|
|
||||||
createdAt: new Date("2026-08-16T00:00:01.000Z"),
|
|
||||||
},
|
|
||||||
]);
|
|
||||||
const prisma = {
|
|
||||||
message: { findMany },
|
|
||||||
run: { findMany: vi.fn(async () => [{ id: "run-peer" }]) },
|
|
||||||
} as unknown as PrismaClient;
|
|
||||||
|
|
||||||
const page = await loadMessagePage(prisma, "thread-1", undefined, 2);
|
|
||||||
|
|
||||||
expect(page.messages.map((message) => message.id)).toEqual(["message-user"]);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("keeps peer text around-page targets and omits peer activity", async () => {
|
it("keeps peer text around-page targets and omits peer activity", async () => {
|
||||||
const findMany = vi.fn(async () => [
|
const findMany = vi.fn(async () => [
|
||||||
{
|
{
|
||||||
@ -281,45 +189,6 @@ describe("thread message pages", () => {
|
|||||||
expect(page.messages.map((message) => message.id)).toEqual(["message-peer"]);
|
expect(page.messages.map((message) => message.id)).toEqual(["message-peer"]);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("scans past a page containing only hidden peer-run activity", async () => {
|
|
||||||
const row = (seq: number, runId: string, kind: "text" | "steps" = "text") => ({
|
|
||||||
id: `message-${seq}`,
|
|
||||||
threadId: "thread-1",
|
|
||||||
seq,
|
|
||||||
role: "bot",
|
|
||||||
blocks:
|
|
||||||
kind === "steps"
|
|
||||||
? [{ kind: "steps", steps: [{ label: String(seq), count: 1 }] }]
|
|
||||||
: [{ kind: "text", text: String(seq) }],
|
|
||||||
botId: "bot-1",
|
|
||||||
replyToMessageId: null,
|
|
||||||
runId,
|
|
||||||
createdAt: new Date("2026-08-16T00:00:00.000Z"),
|
|
||||||
});
|
|
||||||
const findMany = vi
|
|
||||||
.fn()
|
|
||||||
.mockResolvedValueOnce([
|
|
||||||
row(4, "run-peer", "steps"),
|
|
||||||
row(3, "run-peer", "steps"),
|
|
||||||
row(2, "run-peer", "steps"),
|
|
||||||
])
|
|
||||||
.mockResolvedValueOnce([row(1, "run-user")]);
|
|
||||||
const prisma = {
|
|
||||||
message: { findMany },
|
|
||||||
run: {
|
|
||||||
findMany: vi
|
|
||||||
.fn()
|
|
||||||
.mockResolvedValueOnce([{ id: "run-peer" }])
|
|
||||||
.mockResolvedValueOnce([]),
|
|
||||||
},
|
|
||||||
} as unknown as PrismaClient;
|
|
||||||
|
|
||||||
const page = await loadMessagePage(prisma, "thread-1", undefined, 2);
|
|
||||||
|
|
||||||
expect(page.messages.map((message) => message.id)).toEqual(["message-1"]);
|
|
||||||
expect(findMany).toHaveBeenCalledTimes(2);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("scans past a receipt-only page so web can reach older user-visible rows", async () => {
|
it("scans past a receipt-only page so web can reach older user-visible rows", async () => {
|
||||||
const receiptRows = [
|
const receiptRows = [
|
||||||
{
|
{
|
||||||
@ -383,21 +252,13 @@ describe("thread message pages", () => {
|
|||||||
createdAt: new Date("2026-08-16T00:00:01.000Z"),
|
createdAt: new Date("2026-08-16T00:00:01.000Z"),
|
||||||
},
|
},
|
||||||
];
|
];
|
||||||
const findMany = vi.fn().mockResolvedValueOnce(receiptRows).mockResolvedValueOnce(olderRows);
|
const queryRaw = vi.fn().mockResolvedValueOnce(receiptRows).mockResolvedValueOnce(olderRows);
|
||||||
const prisma = {
|
const prisma = { $queryRaw: queryRaw } as unknown as PrismaClient;
|
||||||
message: { findMany },
|
|
||||||
run: {
|
|
||||||
findMany: vi
|
|
||||||
.fn()
|
|
||||||
.mockResolvedValueOnce([{ id: "run-peer" }])
|
|
||||||
.mockResolvedValueOnce([]),
|
|
||||||
},
|
|
||||||
} as unknown as PrismaClient;
|
|
||||||
|
|
||||||
const page = await loadMessagePage(prisma, "thread-1", undefined, 2);
|
const page = await loadMessagePage(prisma, "thread-1", undefined, 2);
|
||||||
|
|
||||||
expect(page.messages.map((message) => message.id)).toEqual(["message-user"]);
|
expect(page.messages.map((message) => message.id)).toEqual(["message-user"]);
|
||||||
expect(findMany).toHaveBeenCalledTimes(2);
|
expect(queryRaw).toHaveBeenCalledTimes(2);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("returns a receipt-only page when the client displays peer receipts", async () => {
|
it("returns a receipt-only page when the client displays peer receipts", async () => {
|
||||||
@ -419,11 +280,8 @@ describe("thread message pages", () => {
|
|||||||
runId: "run-peer",
|
runId: "run-peer",
|
||||||
createdAt: new Date(`2026-08-16T00:00:0${seq}.000Z`),
|
createdAt: new Date(`2026-08-16T00:00:0${seq}.000Z`),
|
||||||
});
|
});
|
||||||
const findMany = vi.fn(async () => [receipt(3), receipt(2), receipt(1)]);
|
const queryRaw = vi.fn(async () => [receipt(3), receipt(2), receipt(1)]);
|
||||||
const prisma = {
|
const prisma = { $queryRaw: queryRaw } as unknown as PrismaClient;
|
||||||
message: { findMany },
|
|
||||||
run: { findMany: vi.fn(async () => [{ id: "run-peer" }]) },
|
|
||||||
} as unknown as PrismaClient;
|
|
||||||
|
|
||||||
const page = await loadMessagePage(prisma, "thread-1", undefined, 2, undefined, false, true);
|
const page = await loadMessagePage(prisma, "thread-1", undefined, 2, undefined, false, true);
|
||||||
|
|
||||||
@ -432,10 +290,10 @@ describe("thread message pages", () => {
|
|||||||
"message-receipt-3",
|
"message-receipt-3",
|
||||||
]);
|
]);
|
||||||
expect(page.olderCursor).toBe(2);
|
expect(page.olderCursor).toBe(2);
|
||||||
expect(findMany).toHaveBeenCalledTimes(1);
|
expect(queryRaw).toHaveBeenCalledTimes(1);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("queries before the cursor and returns an ascending bounded page", async () => {
|
it("queries the bot messages view before the cursor as an ascending bounded page", async () => {
|
||||||
const findMany = vi.fn(async () =>
|
const findMany = vi.fn(async () =>
|
||||||
[5, 4, 3].map((seq) => ({
|
[5, 4, 3].map((seq) => ({
|
||||||
id: `message-${seq}`,
|
id: `message-${seq}`,
|
||||||
@ -449,7 +307,7 @@ describe("thread message pages", () => {
|
|||||||
);
|
);
|
||||||
const prisma = { message: { findMany } } as unknown as PrismaClient;
|
const prisma = { message: { findMany } } as unknown as PrismaClient;
|
||||||
|
|
||||||
const page = await loadMessagePage(prisma, "thread-1", 6, 2);
|
const page = await loadMessagePage(prisma, "thread-1", 6, 2, undefined, true);
|
||||||
|
|
||||||
expect(findMany).toHaveBeenCalledWith({
|
expect(findMany).toHaveBeenCalledWith({
|
||||||
where: { threadId: "thread-1", seq: { lt: 6 } },
|
where: { threadId: "thread-1", seq: { lt: 6 } },
|
||||||
@ -461,7 +319,7 @@ describe("thread message pages", () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
it("ends pagination when the database returns no lookahead row", async () => {
|
it("ends pagination when the database returns no lookahead row", async () => {
|
||||||
const findMany = vi.fn(async () => [
|
const queryRaw = vi.fn(async () => [
|
||||||
{
|
{
|
||||||
id: "message-0",
|
id: "message-0",
|
||||||
threadId: "thread-1",
|
threadId: "thread-1",
|
||||||
@ -472,7 +330,7 @@ describe("thread message pages", () => {
|
|||||||
createdAt: new Date("2026-08-16T00:00:00.000Z"),
|
createdAt: new Date("2026-08-16T00:00:00.000Z"),
|
||||||
},
|
},
|
||||||
]);
|
]);
|
||||||
const prisma = { message: { findMany } } as unknown as PrismaClient;
|
const prisma = { $queryRaw: queryRaw } as unknown as PrismaClient;
|
||||||
|
|
||||||
const page = await loadMessagePage(prisma, "thread-1", 1, 2);
|
const page = await loadMessagePage(prisma, "thread-1", 1, 2);
|
||||||
|
|
||||||
|
|||||||
@ -1,9 +1,18 @@
|
|||||||
import type { MessageBlock, ThreadMessage, ThreadMessagePage } from "@rakazo/contracts";
|
import type { MessageBlock, ThreadMessage, ThreadMessagePage } from "@rakazo/contracts";
|
||||||
import { isPeerReceiptBlocks } from "@rakazo/core";
|
import { isPeerReceiptBlocks } from "@rakazo/core";
|
||||||
import type { Prisma, PrismaClient } from "@rakazo/db";
|
import type { PrismaClient } from "@rakazo/db";
|
||||||
|
import { Prisma } from "@rakazo/db";
|
||||||
|
|
||||||
type MessageDb = PrismaClient | Prisma.TransactionClient;
|
type MessageDb = PrismaClient | Prisma.TransactionClient;
|
||||||
|
|
||||||
|
/** Peer-run rows that stay in the normal transcript: receipts, ask cards and the bot's own reply. */
|
||||||
|
const PEER_VISIBLE_BLOCK_KINDS: readonly string[] = [
|
||||||
|
"bot_message_sent",
|
||||||
|
"bot_message_received",
|
||||||
|
"ask",
|
||||||
|
"text",
|
||||||
|
];
|
||||||
|
|
||||||
export async function loadMessagePage(
|
export async function loadMessagePage(
|
||||||
prisma: MessageDb,
|
prisma: MessageDb,
|
||||||
threadId: string,
|
threadId: string,
|
||||||
@ -47,37 +56,69 @@ export async function loadMessagePage(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
let cursor = before;
|
if (includePeerRuns) {
|
||||||
while (true) {
|
|
||||||
const rows = await prisma.message.findMany({
|
const rows = await prisma.message.findMany({
|
||||||
where: {
|
where: {
|
||||||
threadId,
|
threadId,
|
||||||
...(cursor === undefined ? {} : { seq: { lt: cursor } }),
|
...(before === undefined ? {} : { seq: { lt: before } }),
|
||||||
},
|
},
|
||||||
orderBy: { seq: "desc" },
|
orderBy: { seq: "desc" },
|
||||||
take: pageSize + 1,
|
take: pageSize + 1,
|
||||||
});
|
});
|
||||||
const hasOlder = rows.length > pageSize;
|
return toPage(threadId, rows, pageSize);
|
||||||
const pageRows = rows.slice(0, pageSize).reverse();
|
}
|
||||||
const visibleRows = includePeerRuns ? pageRows : await withoutPeerRunMessages(prisma, pageRows);
|
|
||||||
|
let cursor = before;
|
||||||
|
while (true) {
|
||||||
|
const rows = await visibleMessageRows(prisma, threadId, cursor, pageSize + 1);
|
||||||
|
const page = toPage(threadId, rows, pageSize);
|
||||||
// Web hides receipts client-side, so its receipt-only pages keep scanning.
|
// Web hides receipts client-side, so its receipt-only pages keep scanning.
|
||||||
// Mobile explicitly retains them and must receive each page for pagination.
|
// Mobile explicitly retains them and must receive each page for pagination.
|
||||||
const hasSubstantive = visibleRows.some(
|
const hasSubstantive = page.messages.some((message) => !isPeerReceiptBlocks(message.blocks));
|
||||||
(row) => !isPeerReceiptBlocks(row.blocks as MessageBlock[]),
|
if (hasSubstantive || includePeerReceipts || page.olderCursor === null) return page;
|
||||||
);
|
cursor = page.olderCursor;
|
||||||
if (hasSubstantive || includePeerReceipts || !hasOlder || includePeerRuns) {
|
|
||||||
return {
|
|
||||||
threadId,
|
|
||||||
messages: visibleRows.map(toThreadMessage),
|
|
||||||
olderCursor: hasOlder ? (pageRows[0]?.seq ?? null) : null,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
// TODO: only rescan when a raw page is entirely peer output. Consider a run relation if
|
|
||||||
// long peer-only histories make this path hot.
|
|
||||||
cursor = pageRows[0]?.seq;
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
type MessageRow = Parameters<typeof toThreadMessage>[0];
|
||||||
|
|
||||||
|
function toPage(threadId: string, rows: MessageRow[], pageSize: number): ThreadMessagePage {
|
||||||
|
const pageRows = rows.slice(0, pageSize).reverse();
|
||||||
|
return {
|
||||||
|
threadId,
|
||||||
|
messages: pageRows.map(toThreadMessage),
|
||||||
|
olderCursor: rows.length > pageSize ? (pageRows[0]?.seq ?? null) : null,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Newest-first transcript rows with peer-run activity excluded in the database, so a long
|
||||||
|
* peer-only stretch costs one query instead of a rescan per hidden page.
|
||||||
|
*/
|
||||||
|
function visibleMessageRows(
|
||||||
|
prisma: MessageDb,
|
||||||
|
threadId: string,
|
||||||
|
before: number | undefined,
|
||||||
|
limit: number,
|
||||||
|
): Promise<MessageRow[]> {
|
||||||
|
return prisma.$queryRaw<MessageRow[]>`
|
||||||
|
SELECT m.id, m."threadId", m.seq, m.role, m.blocks, m."botId", m."replyToMessageId",
|
||||||
|
m."replyQuote", m."runId", m."createdAt"
|
||||||
|
FROM messages m
|
||||||
|
LEFT JOIN runs r ON r.id = m."runId"
|
||||||
|
WHERE m."threadId" = ${threadId}
|
||||||
|
${before === undefined ? Prisma.empty : Prisma.sql`AND m.seq < ${before}`}
|
||||||
|
AND (r.trigger IS DISTINCT FROM 'bot_message' OR ${Prisma.join(
|
||||||
|
PEER_VISIBLE_BLOCK_KINDS.map(
|
||||||
|
(kind) => Prisma.sql`m.blocks @> ${JSON.stringify([{ kind }])}::jsonb`,
|
||||||
|
),
|
||||||
|
" OR ",
|
||||||
|
)})
|
||||||
|
ORDER BY m.seq DESC
|
||||||
|
LIMIT ${limit}
|
||||||
|
`;
|
||||||
|
}
|
||||||
|
|
||||||
export async function loadAllMessages(
|
export async function loadAllMessages(
|
||||||
prisma: PrismaClient,
|
prisma: PrismaClient,
|
||||||
threadId: string,
|
threadId: string,
|
||||||
@ -106,15 +147,8 @@ async function withoutPeerRunMessages<T extends { runId: string | null; blocks:
|
|||||||
const peerRunIds = new Set(peerRuns.map((run) => run.id));
|
const peerRunIds = new Set(peerRuns.map((run) => run.id));
|
||||||
return rows.filter((row) => {
|
return rows.filter((row) => {
|
||||||
if (!row.runId || !peerRunIds.has(row.runId)) return true;
|
if (!row.runId || !peerRunIds.has(row.runId)) return true;
|
||||||
// Keep peer receipts (chips), ask cards, and the bot's own text reply.
|
|
||||||
const blocks = row.blocks as MessageBlock[];
|
const blocks = row.blocks as MessageBlock[];
|
||||||
return blocks.some(
|
return blocks.some((block) => PEER_VISIBLE_BLOCK_KINDS.includes(block.kind));
|
||||||
(block) =>
|
|
||||||
block.kind === "bot_message_sent" ||
|
|
||||||
block.kind === "bot_message_received" ||
|
|
||||||
block.kind === "ask" ||
|
|
||||||
block.kind === "text",
|
|
||||||
);
|
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -159,10 +193,8 @@ export function shouldForwardPeerThreadEvent(event: {
|
|||||||
!!block &&
|
!!block &&
|
||||||
typeof block === "object" &&
|
typeof block === "object" &&
|
||||||
"kind" in block &&
|
"kind" in block &&
|
||||||
(block.kind === "bot_message_received" ||
|
typeof block.kind === "string" &&
|
||||||
block.kind === "bot_message_sent" ||
|
PEER_VISIBLE_BLOCK_KINDS.includes(block.kind),
|
||||||
block.kind === "ask" ||
|
|
||||||
block.kind === "text"),
|
|
||||||
)
|
)
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|||||||
@ -183,7 +183,7 @@ describe("threadSnapshot", () => {
|
|||||||
},
|
},
|
||||||
]);
|
]);
|
||||||
const tx = {
|
const tx = {
|
||||||
$queryRaw: vi.fn().mockResolvedValue([{ id: "thread-1" }]),
|
$queryRaw: threadQueries(),
|
||||||
message: { findMany: vi.fn().mockResolvedValue([]) },
|
message: { findMany: vi.fn().mockResolvedValue([]) },
|
||||||
event: {
|
event: {
|
||||||
findFirst: vi.fn().mockResolvedValue({ seq: 4 }),
|
findFirst: vi.fn().mockResolvedValue({ seq: 4 }),
|
||||||
@ -203,7 +203,10 @@ describe("threadSnapshot", () => {
|
|||||||
|
|
||||||
const snapshot = await threadSnapshot({ prisma }, target);
|
const snapshot = await threadSnapshot({ prisma }, target);
|
||||||
|
|
||||||
expect(tx.$queryRaw).toHaveBeenCalledOnce();
|
expect(tx.$queryRaw).toHaveBeenCalledWith(
|
||||||
|
expect.arrayContaining([expect.stringContaining("FOR SHARE")]),
|
||||||
|
"thread-1",
|
||||||
|
);
|
||||||
expect(findManyEvents).toHaveBeenCalledWith(
|
expect(findManyEvents).toHaveBeenCalledWith(
|
||||||
expect.objectContaining({
|
expect.objectContaining({
|
||||||
where: expect.objectContaining({
|
where: expect.objectContaining({
|
||||||
@ -245,7 +248,7 @@ describe("threadSnapshot", () => {
|
|||||||
const findManyEvents = vi.fn();
|
const findManyEvents = vi.fn();
|
||||||
const findFirstRun = botRunFindFirst([run]);
|
const findFirstRun = botRunFindFirst([run]);
|
||||||
const tx = {
|
const tx = {
|
||||||
$queryRaw: vi.fn().mockResolvedValue([{ id: "thread-1" }]),
|
$queryRaw: threadQueries(),
|
||||||
message: { findMany: vi.fn().mockResolvedValue([]) },
|
message: { findMany: vi.fn().mockResolvedValue([]) },
|
||||||
event: {
|
event: {
|
||||||
findFirst: vi.fn().mockResolvedValue(null),
|
findFirst: vi.fn().mockResolvedValue(null),
|
||||||
@ -330,7 +333,7 @@ describe("threadSnapshot", () => {
|
|||||||
prisma: {
|
prisma: {
|
||||||
$transaction: vi.fn(async (callback: (client: unknown) => unknown) =>
|
$transaction: vi.fn(async (callback: (client: unknown) => unknown) =>
|
||||||
callback({
|
callback({
|
||||||
$queryRaw: vi.fn().mockResolvedValue([{ id: "thread-1" }]),
|
$queryRaw: threadQueries(),
|
||||||
message: { findMany: vi.fn().mockResolvedValue([]) },
|
message: { findMany: vi.fn().mockResolvedValue([]) },
|
||||||
event: {
|
event: {
|
||||||
findFirst: vi.fn().mockResolvedValue(null),
|
findFirst: vi.fn().mockResolvedValue(null),
|
||||||
@ -385,7 +388,7 @@ describe("threadSnapshot", () => {
|
|||||||
};
|
};
|
||||||
const findFirstRun = botRunFindFirst([failed, completed]);
|
const findFirstRun = botRunFindFirst([failed, completed]);
|
||||||
const tx = {
|
const tx = {
|
||||||
$queryRaw: vi.fn().mockResolvedValue([{ id: "thread-1" }]),
|
$queryRaw: threadQueries(),
|
||||||
message: { findMany: vi.fn().mockResolvedValue([]) },
|
message: { findMany: vi.fn().mockResolvedValue([]) },
|
||||||
event: {
|
event: {
|
||||||
findFirst: vi.fn().mockResolvedValue(null),
|
findFirst: vi.fn().mockResolvedValue(null),
|
||||||
@ -421,7 +424,7 @@ describe("threadSnapshot", () => {
|
|||||||
const findManyEvents = vi.fn();
|
const findManyEvents = vi.fn();
|
||||||
const findFirstRun = botRunFindFirst([]);
|
const findFirstRun = botRunFindFirst([]);
|
||||||
const tx = {
|
const tx = {
|
||||||
$queryRaw: vi.fn().mockResolvedValue([{ id: "thread-1" }]),
|
$queryRaw: threadQueries(),
|
||||||
message: { findMany: vi.fn().mockResolvedValue([]) },
|
message: { findMany: vi.fn().mockResolvedValue([]) },
|
||||||
event: {
|
event: {
|
||||||
findFirst: vi.fn().mockResolvedValue(null),
|
findFirst: vi.fn().mockResolvedValue(null),
|
||||||
@ -925,9 +928,16 @@ function groupRunFindMany(input: { active?: unknown[]; terminals?: unknown[] })
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** The snapshot's thread lock finds the thread; its message page query finds no rows. */
|
||||||
|
function threadQueries() {
|
||||||
|
return vi.fn(async (sql: TemplateStringsArray) =>
|
||||||
|
sql.join("").includes("FOR SHARE") ? [{ id: "thread-1" }] : [],
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
function groupPrisma(findManyRuns: ReturnType<typeof groupRunFindMany>) {
|
function groupPrisma(findManyRuns: ReturnType<typeof groupRunFindMany>) {
|
||||||
const tx = {
|
const tx = {
|
||||||
$queryRaw: vi.fn().mockResolvedValue([{ id: "thread-1" }]),
|
$queryRaw: threadQueries(),
|
||||||
message: { findMany: vi.fn().mockResolvedValue([]) },
|
message: { findMany: vi.fn().mockResolvedValue([]) },
|
||||||
event: {
|
event: {
|
||||||
findFirst: vi.fn().mockResolvedValue(null),
|
findFirst: vi.fn().mockResolvedValue(null),
|
||||||
|
|||||||
@ -1,96 +0,0 @@
|
|||||||
import { TRADING_BOT_PROXY_PATH } from "@rakazo/core";
|
|
||||||
import { Hono } from "hono";
|
|
||||||
import { describe, expect, it, vi } from "vitest";
|
|
||||||
import { mountTradingBotStatus } from "./trading-bot-proxy.js";
|
|
||||||
|
|
||||||
function proxyApp(options: {
|
|
||||||
baseUrl?: string;
|
|
||||||
authorize?: (request: Request) => Promise<boolean>;
|
|
||||||
fetchImpl?: typeof fetch;
|
|
||||||
}) {
|
|
||||||
const app = new Hono();
|
|
||||||
mountTradingBotStatus(app, {
|
|
||||||
baseUrl: options.baseUrl,
|
|
||||||
authorize: options.authorize ?? (async () => true),
|
|
||||||
fetchImpl: options.fetchImpl,
|
|
||||||
});
|
|
||||||
return app;
|
|
||||||
}
|
|
||||||
|
|
||||||
describe("trading bot status proxy", () => {
|
|
||||||
it("stays idle when TRADING_BOT_URL is unset and does not call upstream", async () => {
|
|
||||||
const fetchImpl = vi.fn();
|
|
||||||
const response = await proxyApp({ fetchImpl }).request(TRADING_BOT_PROXY_PATH);
|
|
||||||
expect(response.status).toBe(200);
|
|
||||||
expect(await response.json()).toEqual({ configured: false });
|
|
||||||
expect(fetchImpl).not.toHaveBeenCalled();
|
|
||||||
});
|
|
||||||
|
|
||||||
it("requires a session before it proxies a configured origin", async () => {
|
|
||||||
const fetchImpl = vi.fn();
|
|
||||||
const response = await proxyApp({
|
|
||||||
baseUrl: "http://bot.test:8000",
|
|
||||||
authorize: async () => false,
|
|
||||||
fetchImpl,
|
|
||||||
}).request(TRADING_BOT_PROXY_PATH, { headers: { cookie: "session=secret" } });
|
|
||||||
expect(response.status).toBe(401);
|
|
||||||
expect(fetchImpl).not.toHaveBeenCalled();
|
|
||||||
});
|
|
||||||
|
|
||||||
it("returns the mapped status and does not forward browser credentials", async () => {
|
|
||||||
const fetchImpl = vi.fn(
|
|
||||||
async () =>
|
|
||||||
new Response(
|
|
||||||
JSON.stringify({ mode: "LIVE", adapter_count: 2, last_error: "adapter timeout" }),
|
|
||||||
{ status: 200, headers: { "content-type": "application/json" } },
|
|
||||||
),
|
|
||||||
);
|
|
||||||
const response = await proxyApp({
|
|
||||||
baseUrl: "http://bot.test:8000",
|
|
||||||
fetchImpl,
|
|
||||||
}).request(TRADING_BOT_PROXY_PATH, {
|
|
||||||
headers: { cookie: "session=secret", authorization: "Bearer browser-token" },
|
|
||||||
});
|
|
||||||
expect(response.status).toBe(200);
|
|
||||||
expect(await response.json()).toEqual({
|
|
||||||
configured: true,
|
|
||||||
reachable: true,
|
|
||||||
status: {
|
|
||||||
mode: "live",
|
|
||||||
kind: "live",
|
|
||||||
adapterCount: 2,
|
|
||||||
lastError: "adapter timeout",
|
|
||||||
},
|
|
||||||
});
|
|
||||||
expect(fetchImpl).toHaveBeenCalledTimes(1);
|
|
||||||
const [url, init] = fetchImpl.mock.calls[0] as [string, RequestInit];
|
|
||||||
expect(url).toBe("http://bot.test:8000/bot/api/status");
|
|
||||||
const headers = new Headers(init.headers);
|
|
||||||
expect(headers.get("cookie")).toBeNull();
|
|
||||||
expect(headers.get("authorization")).toBeNull();
|
|
||||||
expect(init.redirect).toBe("error");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("hides an unmapped body and an upstream failure instead of claiming live", async () => {
|
|
||||||
const unmapped = vi.fn(
|
|
||||||
async () =>
|
|
||||||
new Response(JSON.stringify({ status: "ok" }), {
|
|
||||||
status: 200,
|
|
||||||
headers: { "content-type": "application/json" },
|
|
||||||
}),
|
|
||||||
);
|
|
||||||
const unknown = await proxyApp({
|
|
||||||
baseUrl: "http://bot.test:8000",
|
|
||||||
fetchImpl: unmapped,
|
|
||||||
}).request(TRADING_BOT_PROXY_PATH);
|
|
||||||
expect(await unknown.json()).toEqual({ configured: true, reachable: true, status: null });
|
|
||||||
|
|
||||||
const down = vi.fn(async () => new Response("nope", { status: 500 }));
|
|
||||||
const offline = await proxyApp({
|
|
||||||
baseUrl: "http://bot.test:8000",
|
|
||||||
fetchImpl: down,
|
|
||||||
}).request(TRADING_BOT_PROXY_PATH);
|
|
||||||
expect(offline.status).toBe(502);
|
|
||||||
expect(await offline.json()).toEqual({ configured: true, reachable: false });
|
|
||||||
});
|
|
||||||
});
|
|
||||||
@ -1,79 +0,0 @@
|
|||||||
import {
|
|
||||||
mapTradingBotStatus,
|
|
||||||
readBoundedJsonResponse,
|
|
||||||
TRADING_BOT_PROXY_PATH,
|
|
||||||
TRADING_BOT_STATUS_MAX_BYTES,
|
|
||||||
TRADING_BOT_STATUS_TIMEOUT_MS,
|
|
||||||
tradingBotStatusUrl,
|
|
||||||
} from "@rakazo/core";
|
|
||||||
import type { Hono } from "hono";
|
|
||||||
|
|
||||||
export function mountTradingBotStatus(
|
|
||||||
app: Hono,
|
|
||||||
options: {
|
|
||||||
/** Normalized http(s) origin. Unset leaves the route idle for SaaS builds. */
|
|
||||||
baseUrl: string | undefined;
|
|
||||||
authorize: (request: Request) => Promise<boolean>;
|
|
||||||
fetchImpl?: typeof fetch;
|
|
||||||
},
|
|
||||||
) {
|
|
||||||
const fetchImpl = options.fetchImpl ?? fetch;
|
|
||||||
app.get(TRADING_BOT_PROXY_PATH, async (c) => {
|
|
||||||
c.header("cache-control", "no-store");
|
|
||||||
if (!options.baseUrl) return c.json({ configured: false });
|
|
||||||
const allowed = await options.authorize(c.req.raw).catch(() => false);
|
|
||||||
if (!allowed) return c.json({ error: "Unauthorized" }, 401);
|
|
||||||
|
|
||||||
const timeout = linkAbort(c.req.raw.signal, TRADING_BOT_STATUS_TIMEOUT_MS);
|
|
||||||
try {
|
|
||||||
const response = await fetchImpl(tradingBotStatusUrl(options.baseUrl), {
|
|
||||||
method: "GET",
|
|
||||||
redirect: "error",
|
|
||||||
cache: "no-store",
|
|
||||||
signal: timeout.signal,
|
|
||||||
headers: { accept: "application/json" },
|
|
||||||
});
|
|
||||||
if (!response.ok) {
|
|
||||||
cancelBody(response);
|
|
||||||
return c.json({ configured: true, reachable: false }, 502);
|
|
||||||
}
|
|
||||||
const body = await readBoundedJsonResponse<unknown>(
|
|
||||||
response,
|
|
||||||
TRADING_BOT_STATUS_MAX_BYTES,
|
|
||||||
timeout.signal,
|
|
||||||
);
|
|
||||||
const status = mapTradingBotStatus(body);
|
|
||||||
return c.json({ configured: true, reachable: true, status: status ?? null });
|
|
||||||
} catch {
|
|
||||||
return c.json({ configured: true, reachable: false }, 502);
|
|
||||||
} finally {
|
|
||||||
timeout.cancel();
|
|
||||||
}
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
function linkAbort(
|
|
||||||
parent: AbortSignal,
|
|
||||||
timeoutMs: number,
|
|
||||||
): { signal: AbortSignal; cancel: () => void } {
|
|
||||||
const controller = new AbortController();
|
|
||||||
const timer = setTimeout(() => controller.abort(), timeoutMs);
|
|
||||||
const onParent = () => controller.abort();
|
|
||||||
if (parent.aborted) controller.abort();
|
|
||||||
else parent.addEventListener("abort", onParent);
|
|
||||||
return {
|
|
||||||
signal: controller.signal,
|
|
||||||
cancel: () => {
|
|
||||||
clearTimeout(timer);
|
|
||||||
parent.removeEventListener("abort", onParent);
|
|
||||||
},
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
function cancelBody(response: Response): void {
|
|
||||||
try {
|
|
||||||
void Promise.resolve(response.body?.cancel()).catch(() => undefined);
|
|
||||||
} catch {
|
|
||||||
// Best-effort. The caller still reports the bot as unreachable.
|
|
||||||
}
|
|
||||||
}
|
|
||||||
155
apps/desktop/e2e/browser-preview.spec.ts
Normal file
155
apps/desktop/e2e/browser-preview.spec.ts
Normal file
@ -0,0 +1,155 @@
|
|||||||
|
import http from "node:http";
|
||||||
|
import path from "node:path";
|
||||||
|
import { _electron as electron, expect, test } from "@playwright/test";
|
||||||
|
import type { RakazoDesktop, RakazoDesktopBrowserPreviewState } from "@rakazo/contracts";
|
||||||
|
|
||||||
|
const fixture = `<!doctype html>
|
||||||
|
<html lang="en">
|
||||||
|
<head><meta charset="utf-8"><title>Rakazo desktop browser</title></head>
|
||||||
|
<body><main>App with a browser panel</main></body>
|
||||||
|
</html>`;
|
||||||
|
|
||||||
|
type Bridge = NonNullable<RakazoDesktop["browserPreview"]>;
|
||||||
|
|
||||||
|
test("the in-app browser is a native view the app window lays over its panel", async () => {
|
||||||
|
const pages = http.createServer((req, res) => {
|
||||||
|
res.setHeader("content-type", "text/html");
|
||||||
|
if (req.url === "/second") {
|
||||||
|
res.end("<!doctype html><title>Second page</title><p>second</p>");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
res.end(`<!doctype html><title>First page</title>
|
||||||
|
<a id="next" href="/second">next</a>
|
||||||
|
<a id="popup" href="/second" target="_blank">popup</a>
|
||||||
|
<a id="file" href="file:///etc/hosts">file</a>`);
|
||||||
|
});
|
||||||
|
await new Promise<void>((resolve) => pages.listen(0, "127.0.0.1", resolve));
|
||||||
|
const address = pages.address();
|
||||||
|
if (!address || typeof address === "string") throw new Error("No fixture server");
|
||||||
|
const origin = `http://127.0.0.1:${address.port}`;
|
||||||
|
|
||||||
|
const app = await electron.launch({
|
||||||
|
args: ["."],
|
||||||
|
cwd: path.resolve(import.meta.dirname, ".."),
|
||||||
|
env: {
|
||||||
|
...process.env,
|
||||||
|
RAKAZO_WEB_URL: `data:text/html;charset=utf-8,${encodeURIComponent(fixture)}`,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
try {
|
||||||
|
const page = await app.firstWindow();
|
||||||
|
await expect(page.getByText("App with a browser panel")).toBeVisible();
|
||||||
|
|
||||||
|
const call = <K extends keyof Bridge>(method: K, ...args: unknown[]) =>
|
||||||
|
page.evaluate(
|
||||||
|
([name, values]) => {
|
||||||
|
const bridge = (window as typeof window & { rakazoDesktop?: RakazoDesktop }).rakazoDesktop
|
||||||
|
?.browserPreview as unknown as Record<
|
||||||
|
string,
|
||||||
|
(...input: unknown[]) => Promise<unknown>
|
||||||
|
>;
|
||||||
|
return bridge[name as string]!(...(values as unknown[]));
|
||||||
|
},
|
||||||
|
[method, args] as const,
|
||||||
|
);
|
||||||
|
const state = () => call("state") as Promise<RakazoDesktopBrowserPreviewState>;
|
||||||
|
const view = () =>
|
||||||
|
app.evaluate(({ BrowserWindow }) => {
|
||||||
|
const win = BrowserWindow.getAllWindows()[0]!;
|
||||||
|
const child = win.contentView.children[0] as
|
||||||
|
| (Electron.View & { webContents?: Electron.WebContents })
|
||||||
|
| undefined;
|
||||||
|
return child
|
||||||
|
? {
|
||||||
|
count: win.contentView.children.length,
|
||||||
|
visible: child.getVisible(),
|
||||||
|
bounds: child.getBounds(),
|
||||||
|
url: child.webContents?.getURL(),
|
||||||
|
partition: child.webContents?.session === win.webContents.session,
|
||||||
|
sandbox: child.webContents?.getLastWebPreferences()?.sandbox,
|
||||||
|
nodeIntegration: child.webContents?.getLastWebPreferences()?.nodeIntegration,
|
||||||
|
}
|
||||||
|
: null;
|
||||||
|
});
|
||||||
|
|
||||||
|
// Pages load in a native view in their own session, never the app's.
|
||||||
|
await call("navigate", `${origin}/`);
|
||||||
|
await expect.poll(async () => (await state()).title).toBe("First page");
|
||||||
|
expect(await view()).toMatchObject({
|
||||||
|
count: 1,
|
||||||
|
visible: false,
|
||||||
|
url: `${origin}/`,
|
||||||
|
partition: false,
|
||||||
|
sandbox: true,
|
||||||
|
nodeIntegration: false,
|
||||||
|
});
|
||||||
|
|
||||||
|
await call("show", { x: 400, y: 50, width: 300, height: 200 });
|
||||||
|
expect(await view()).toMatchObject({
|
||||||
|
visible: true,
|
||||||
|
bounds: { x: 400, y: 50, width: 300, height: 200 },
|
||||||
|
});
|
||||||
|
// Renderer bounds outside the window are clamped to it.
|
||||||
|
await call("show", { x: -10, y: -10, width: 100_000, height: 100_000 });
|
||||||
|
const [width, height] = await app.evaluate(({ BrowserWindow }) =>
|
||||||
|
BrowserWindow.getAllWindows()[0]!.getContentSize(),
|
||||||
|
);
|
||||||
|
expect((await view())?.bounds).toEqual({ x: 0, y: 0, width, height });
|
||||||
|
|
||||||
|
const contents = () =>
|
||||||
|
app.evaluate(({ BrowserWindow }, script) => {
|
||||||
|
const win = BrowserWindow.getAllWindows()[0]!;
|
||||||
|
const child = win.contentView.children[0] as Electron.View & {
|
||||||
|
webContents: Electron.WebContents;
|
||||||
|
};
|
||||||
|
return child.webContents.executeJavaScript(script, true);
|
||||||
|
}, "document.getElementById('next').click()");
|
||||||
|
await contents();
|
||||||
|
await expect.poll(async () => (await state()).title).toBe("Second page");
|
||||||
|
expect(await state()).toMatchObject({ canGoBack: true, canGoForward: false, error: null });
|
||||||
|
|
||||||
|
await call("back");
|
||||||
|
await expect.poll(async () => (await state()).title).toBe("First page");
|
||||||
|
expect(await state()).toMatchObject({ canGoForward: true });
|
||||||
|
await call("forward");
|
||||||
|
await expect.poll(async () => (await state()).url).toBe(`${origin}/second`);
|
||||||
|
|
||||||
|
// New-window links stay in the one view; file links never load.
|
||||||
|
await call("back");
|
||||||
|
await expect.poll(async () => (await state()).title).toBe("First page");
|
||||||
|
await app.evaluate(({ BrowserWindow }) => {
|
||||||
|
const child = BrowserWindow.getAllWindows()[0]!.contentView.children[0] as Electron.View & {
|
||||||
|
webContents: Electron.WebContents;
|
||||||
|
};
|
||||||
|
return child.webContents.executeJavaScript("document.getElementById('popup').click()", true);
|
||||||
|
});
|
||||||
|
await expect.poll(async () => (await state()).url).toBe(`${origin}/second`);
|
||||||
|
expect(await app.evaluate(({ BrowserWindow }) => BrowserWindow.getAllWindows().length)).toBe(1);
|
||||||
|
await call("back");
|
||||||
|
await expect.poll(async () => (await state()).title).toBe("First page");
|
||||||
|
await app.evaluate(({ BrowserWindow }) => {
|
||||||
|
const child = BrowserWindow.getAllWindows()[0]!.contentView.children[0] as Electron.View & {
|
||||||
|
webContents: Electron.WebContents;
|
||||||
|
};
|
||||||
|
return child.webContents.executeJavaScript("document.getElementById('file').click()", true);
|
||||||
|
});
|
||||||
|
await page.waitForTimeout(300);
|
||||||
|
expect((await state()).url).toBe(`${origin}/`);
|
||||||
|
|
||||||
|
await expect(call("navigate", "file:///etc/hosts")).rejects.toThrow(/http/);
|
||||||
|
|
||||||
|
// A dev server that is not running yet shows as an error, then recovers on reload.
|
||||||
|
pages.close();
|
||||||
|
await new Promise((resolve) => setTimeout(resolve, 100));
|
||||||
|
await call("navigate", `${origin}/gone`);
|
||||||
|
await expect.poll(async () => (await state()).error).toMatch(/ERR_CONNECTION_REFUSED/);
|
||||||
|
|
||||||
|
await call("hide");
|
||||||
|
expect((await view())?.visible).toBe(false);
|
||||||
|
} finally {
|
||||||
|
await app.close();
|
||||||
|
pages.closeAllConnections();
|
||||||
|
pages.close();
|
||||||
|
}
|
||||||
|
});
|
||||||
@ -42,6 +42,7 @@ test("launches with a narrow preload bridge and an isolated renderer", async ()
|
|||||||
});
|
});
|
||||||
|
|
||||||
expect(renderer.bridgeKeys).toEqual([
|
expect(renderer.bridgeKeys).toEqual([
|
||||||
|
"browserPreview",
|
||||||
"files",
|
"files",
|
||||||
"localSettings",
|
"localSettings",
|
||||||
"oauth",
|
"oauth",
|
||||||
|
|||||||
2
apps/desktop/native/photos-helper/.gitignore
vendored
Normal file
2
apps/desktop/native/photos-helper/.gitignore
vendored
Normal file
@ -0,0 +1,2 @@
|
|||||||
|
.build/
|
||||||
|
.swiftpm/
|
||||||
18
apps/desktop/native/photos-helper/Package.swift
Normal file
18
apps/desktop/native/photos-helper/Package.swift
Normal file
@ -0,0 +1,18 @@
|
|||||||
|
// swift-tools-version:5.9
|
||||||
|
import PackageDescription
|
||||||
|
|
||||||
|
// PhotosHelperCore holds everything that does not touch PhotoKit (argument parsing,
|
||||||
|
// cursors, JSON envelopes, the export path guard, hashing) so XCTest runs without
|
||||||
|
// Photos access. The executable target is the thin PhotoKit layer.
|
||||||
|
let package = Package(
|
||||||
|
name: "photos-helper",
|
||||||
|
platforms: [.macOS(.v13)],
|
||||||
|
products: [
|
||||||
|
.executable(name: "photos-helper", targets: ["photos-helper"])
|
||||||
|
],
|
||||||
|
targets: [
|
||||||
|
.target(name: "PhotosHelperCore"),
|
||||||
|
.executableTarget(name: "photos-helper", dependencies: ["PhotosHelperCore"]),
|
||||||
|
.testTarget(name: "PhotosHelperCoreTests", dependencies: ["PhotosHelperCore"]),
|
||||||
|
]
|
||||||
|
)
|
||||||
@ -0,0 +1,142 @@
|
|||||||
|
import Foundation
|
||||||
|
|
||||||
|
public enum ExportVariant: String, Equatable {
|
||||||
|
/// The untouched original resource, downloaded from iCloud when needed.
|
||||||
|
case original
|
||||||
|
/// A smaller rendition of the current version (720p video, 1280 px JPEG).
|
||||||
|
case proxy
|
||||||
|
}
|
||||||
|
|
||||||
|
public enum Command: Equatable {
|
||||||
|
/// `prompt: false` only reads the current status and never shows the system dialog.
|
||||||
|
case authorize(prompt: Bool)
|
||||||
|
case list(cursor: ListCursor?, limit: Int, since: Date?)
|
||||||
|
case asset(localIdentifier: String)
|
||||||
|
case export(localIdentifier: String, destination: String, variant: ExportVariant)
|
||||||
|
case changes(token: String)
|
||||||
|
}
|
||||||
|
|
||||||
|
public let usageText = """
|
||||||
|
usage: photos-helper authorize [--no-prompt]
|
||||||
|
photos-helper list [--cursor <opaque>] [--limit <1-1000>] [--since <iso8601>]
|
||||||
|
photos-helper asset <localIdentifier>
|
||||||
|
photos-helper export <localIdentifier> --dest <dir> [--original|--proxy]
|
||||||
|
photos-helper changes --token <opaque>
|
||||||
|
"""
|
||||||
|
|
||||||
|
/// Parses argv without the program name. Unknown flags, repeated flags and missing
|
||||||
|
/// values are usage errors so a typo never silently changes behaviour.
|
||||||
|
public func parseArguments(_ arguments: [String]) throws -> Command {
|
||||||
|
guard let name = arguments.first else { throw HelperError(.usage, usageText) }
|
||||||
|
var options = try OptionReader(Array(arguments.dropFirst()))
|
||||||
|
switch name {
|
||||||
|
case "authorize":
|
||||||
|
let noPrompt = options.flag("--no-prompt")
|
||||||
|
try options.finish(positionals: 0)
|
||||||
|
return .authorize(prompt: !noPrompt)
|
||||||
|
case "list":
|
||||||
|
let cursor = try options.value("--cursor").map(ListCursor.decode)
|
||||||
|
let limit = try options.value("--limit").map(parseLimit) ?? Limits.defaultPage
|
||||||
|
let since = try options.value("--since").map { value -> Date in
|
||||||
|
guard let date = parseTimestamp(value) else {
|
||||||
|
throw HelperError(.usage, "--since must be an ISO 8601 timestamp.")
|
||||||
|
}
|
||||||
|
return date
|
||||||
|
}
|
||||||
|
try options.finish(positionals: 0)
|
||||||
|
return .list(cursor: cursor, limit: limit, since: since)
|
||||||
|
case "asset":
|
||||||
|
try options.finish(positionals: 1)
|
||||||
|
let identifier = try validIdentifier(options.positionals[0])
|
||||||
|
return .asset(localIdentifier: identifier)
|
||||||
|
case "export":
|
||||||
|
guard let destination = options.value("--dest") else {
|
||||||
|
throw HelperError(.usage, "export needs --dest <dir>.")
|
||||||
|
}
|
||||||
|
let original = options.flag("--original")
|
||||||
|
let proxy = options.flag("--proxy")
|
||||||
|
if original && proxy { throw HelperError(.usage, "Choose --original or --proxy, not both.") }
|
||||||
|
try options.finish(positionals: 1)
|
||||||
|
let identifier = try validIdentifier(options.positionals[0])
|
||||||
|
return .export(
|
||||||
|
localIdentifier: identifier,
|
||||||
|
destination: destination,
|
||||||
|
variant: proxy ? .proxy : .original)
|
||||||
|
case "changes":
|
||||||
|
guard let token = options.value("--token"), !token.isEmpty else {
|
||||||
|
throw HelperError(.usage, "changes needs --token <opaque>.")
|
||||||
|
}
|
||||||
|
try options.finish(positionals: 0)
|
||||||
|
return .changes(token: token)
|
||||||
|
default:
|
||||||
|
throw HelperError(.usage, usageText)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseLimit(_ value: String) throws -> Int {
|
||||||
|
guard let limit = Int(value), (1...Limits.maxPage).contains(limit) else {
|
||||||
|
throw HelperError(.usage, "--limit must be between 1 and \(Limits.maxPage).")
|
||||||
|
}
|
||||||
|
return limit
|
||||||
|
}
|
||||||
|
|
||||||
|
func validIdentifier(_ value: String) throws -> String {
|
||||||
|
guard !value.isEmpty, value.utf16.count <= Limits.externalId, !value.hasPrefix("-"),
|
||||||
|
!value.unicodeScalars.contains(where: { CharacterSet.controlCharacters.contains($0) })
|
||||||
|
else { throw HelperError(.usage, "Invalid local identifier.") }
|
||||||
|
return value
|
||||||
|
}
|
||||||
|
|
||||||
|
private struct OptionReader {
|
||||||
|
private var values: [String: String] = [:]
|
||||||
|
private var flags: Set<String> = []
|
||||||
|
private var consumed: Set<String> = []
|
||||||
|
private(set) var positionals: [String] = []
|
||||||
|
|
||||||
|
private static let valueOptions: Set<String> = ["--cursor", "--limit", "--since", "--dest", "--token"]
|
||||||
|
private static let flagOptions: Set<String> = ["--no-prompt", "--original", "--proxy"]
|
||||||
|
|
||||||
|
init(_ arguments: [String]) throws {
|
||||||
|
var index = 0
|
||||||
|
while index < arguments.count {
|
||||||
|
let argument = arguments[index]
|
||||||
|
if Self.valueOptions.contains(argument) {
|
||||||
|
guard index + 1 < arguments.count else {
|
||||||
|
throw HelperError(.usage, "\(argument) needs a value.")
|
||||||
|
}
|
||||||
|
guard values[argument] == nil else { throw HelperError(.usage, "\(argument) given twice.") }
|
||||||
|
values[argument] = arguments[index + 1]
|
||||||
|
index += 2
|
||||||
|
} else if Self.flagOptions.contains(argument) {
|
||||||
|
guard !flags.contains(argument) else { throw HelperError(.usage, "\(argument) given twice.") }
|
||||||
|
flags.insert(argument)
|
||||||
|
index += 1
|
||||||
|
} else if argument.hasPrefix("--") {
|
||||||
|
throw HelperError(.usage, "Unknown option \(argument).")
|
||||||
|
} else {
|
||||||
|
positionals.append(argument)
|
||||||
|
index += 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
mutating func value(_ name: String) -> String? {
|
||||||
|
consumed.insert(name)
|
||||||
|
return values[name]
|
||||||
|
}
|
||||||
|
|
||||||
|
mutating func flag(_ name: String) -> Bool {
|
||||||
|
consumed.insert(name)
|
||||||
|
return flags.contains(name)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Rejects options that belong to another command and surplus positionals.
|
||||||
|
func finish(positionals expected: Int) throws {
|
||||||
|
for name in Set(values.keys).union(flags) where !consumed.contains(name) {
|
||||||
|
throw HelperError(.usage, "\(name) is not valid here.")
|
||||||
|
}
|
||||||
|
guard positionals.count == expected else {
|
||||||
|
throw HelperError(.usage, expected == 0 ? "Unexpected argument." : "Expected \(expected) argument.")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -0,0 +1,37 @@
|
|||||||
|
import Foundation
|
||||||
|
|
||||||
|
/// A failure the desktop app maps by `code`. Messages never contain library content
|
||||||
|
/// beyond the identifier the caller passed in.
|
||||||
|
public struct HelperError: Error, Equatable {
|
||||||
|
public enum Code: String, Codable, CaseIterable {
|
||||||
|
case usage
|
||||||
|
case notAuthorized = "not_authorized"
|
||||||
|
case notFound = "not_found"
|
||||||
|
case pathRejected = "path_rejected"
|
||||||
|
case destinationExists = "destination_exists"
|
||||||
|
case network
|
||||||
|
case exportFailed = "export_failed"
|
||||||
|
case `internal`
|
||||||
|
}
|
||||||
|
|
||||||
|
public let code: Code
|
||||||
|
public let message: String
|
||||||
|
|
||||||
|
public init(_ code: Code, _ message: String) {
|
||||||
|
self.code = code
|
||||||
|
self.message = message
|
||||||
|
}
|
||||||
|
|
||||||
|
/// sysexits(3) style codes so a shell user can tell failure classes apart.
|
||||||
|
public var exitCode: Int32 {
|
||||||
|
switch code {
|
||||||
|
case .usage: return 64
|
||||||
|
case .notFound: return 66
|
||||||
|
case .pathRejected, .destinationExists: return 73
|
||||||
|
case .exportFailed: return 74
|
||||||
|
case .network: return 75
|
||||||
|
case .notAuthorized: return 77
|
||||||
|
case .internal: return 70
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -0,0 +1,249 @@
|
|||||||
|
import Foundation
|
||||||
|
|
||||||
|
/// Mirrors `ClientMediaItemSchema` in packages/contracts/src/media.ts. Every key is
|
||||||
|
/// always present (nullable fields encode as `null`) because the schema has no optional keys.
|
||||||
|
public struct ClientMediaItem: Encodable, Equatable {
|
||||||
|
public enum Kind: String, Encodable {
|
||||||
|
case video
|
||||||
|
case photo
|
||||||
|
case livePhoto = "live_photo"
|
||||||
|
}
|
||||||
|
|
||||||
|
public var externalId: String
|
||||||
|
public var kind: Kind
|
||||||
|
public var capturedAt: String?
|
||||||
|
public var durationMs: Int?
|
||||||
|
public var width: Int?
|
||||||
|
public var height: Int?
|
||||||
|
public var fps: Double?
|
||||||
|
public var gpsLat: Double?
|
||||||
|
public var gpsLon: Double?
|
||||||
|
public var deviceModel: String?
|
||||||
|
public var albumNames: [String]
|
||||||
|
public var isFavorite: Bool
|
||||||
|
public var fileName: String?
|
||||||
|
|
||||||
|
public init(
|
||||||
|
externalId: String,
|
||||||
|
kind: Kind,
|
||||||
|
capturedAt: Date?,
|
||||||
|
durationSeconds: Double?,
|
||||||
|
width: Int?,
|
||||||
|
height: Int?,
|
||||||
|
latitude: Double?,
|
||||||
|
longitude: Double?,
|
||||||
|
albumNames: [String],
|
||||||
|
isFavorite: Bool,
|
||||||
|
fileName: String?
|
||||||
|
) {
|
||||||
|
self.externalId = externalId
|
||||||
|
self.kind = kind
|
||||||
|
self.capturedAt = capturedAt.map(formatTimestamp)
|
||||||
|
if let durationSeconds, durationSeconds.isFinite, durationSeconds > 0 {
|
||||||
|
durationMs = Int((durationSeconds * 1000).rounded())
|
||||||
|
} else {
|
||||||
|
durationMs = nil
|
||||||
|
}
|
||||||
|
self.width = width.flatMap { $0 > 0 ? $0 : nil }
|
||||||
|
self.height = height.flatMap { $0 > 0 ? $0 : nil }
|
||||||
|
// PhotoKit exposes neither frame rate nor camera model without loading the
|
||||||
|
// file; the server reads both with ffprobe after upload.
|
||||||
|
fps = nil
|
||||||
|
deviceModel = nil
|
||||||
|
let coordinate = roundedCoordinate(latitude: latitude, longitude: longitude)
|
||||||
|
gpsLat = coordinate?.latitude
|
||||||
|
gpsLon = coordinate?.longitude
|
||||||
|
self.albumNames = Array(
|
||||||
|
albumNames.map { clip($0, maxCharacters: Limits.albumName) }
|
||||||
|
.filter { !$0.isEmpty }
|
||||||
|
.prefix(Limits.albumCount))
|
||||||
|
self.isFavorite = isFavorite
|
||||||
|
self.fileName = fileName.map { clip($0, maxCharacters: Limits.fileName) }
|
||||||
|
}
|
||||||
|
|
||||||
|
enum CodingKeys: String, CodingKey {
|
||||||
|
case externalId, kind, capturedAt, durationMs, width, height, fps, gpsLat, gpsLon
|
||||||
|
case deviceModel, albumNames, isFavorite, fileName
|
||||||
|
}
|
||||||
|
|
||||||
|
public func encode(to encoder: Encoder) throws {
|
||||||
|
var container = encoder.container(keyedBy: CodingKeys.self)
|
||||||
|
try container.encode(externalId, forKey: .externalId)
|
||||||
|
try container.encode(kind, forKey: .kind)
|
||||||
|
// `encode` on an Optional writes `null`, unlike the synthesized `encodeIfPresent`.
|
||||||
|
try container.encode(capturedAt, forKey: .capturedAt)
|
||||||
|
try container.encode(durationMs, forKey: .durationMs)
|
||||||
|
try container.encode(width, forKey: .width)
|
||||||
|
try container.encode(height, forKey: .height)
|
||||||
|
try container.encode(fps, forKey: .fps)
|
||||||
|
try container.encode(gpsLat, forKey: .gpsLat)
|
||||||
|
try container.encode(gpsLon, forKey: .gpsLon)
|
||||||
|
try container.encode(deviceModel, forKey: .deviceModel)
|
||||||
|
try container.encode(albumNames, forKey: .albumNames)
|
||||||
|
try container.encode(isFavorite, forKey: .isFavorite)
|
||||||
|
try container.encode(fileName, forKey: .fileName)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Bounds from the contract schema, applied here so one odd asset cannot make a whole
|
||||||
|
/// page fail validation in the app.
|
||||||
|
public enum Limits {
|
||||||
|
public static let externalId = 200
|
||||||
|
public static let albumName = 200
|
||||||
|
public static let albumCount = 50
|
||||||
|
public static let fileName = 255
|
||||||
|
public static let defaultPage = 500
|
||||||
|
public static let maxPage = 1000
|
||||||
|
/// Above this many changed assets a full relisting is cheaper than one huge document.
|
||||||
|
public static let maxChanges = 5000
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Clips to `maxCharacters` UTF-16 units (what zod's `.max()` counts) without splitting
|
||||||
|
/// a character.
|
||||||
|
public func clip(_ value: String, maxCharacters: Int) -> String {
|
||||||
|
if value.utf16.count <= maxCharacters { return value }
|
||||||
|
var result = ""
|
||||||
|
var units = 0
|
||||||
|
for character in value {
|
||||||
|
units += character.utf16.count
|
||||||
|
if units > maxCharacters { break }
|
||||||
|
result.append(character)
|
||||||
|
}
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Two decimals is about 1 km, matching `roundCoordinate` in @rakazo/media. Precise
|
||||||
|
/// locations never leave the device.
|
||||||
|
public func roundedCoordinate(latitude: Double?, longitude: Double?) -> (
|
||||||
|
latitude: Double, longitude: Double
|
||||||
|
)? {
|
||||||
|
guard let latitude, let longitude, latitude.isFinite, longitude.isFinite,
|
||||||
|
(-90...90).contains(latitude), (-180...180).contains(longitude),
|
||||||
|
!(latitude == 0 && longitude == 0)
|
||||||
|
else { return nil }
|
||||||
|
return ((latitude * 100).rounded() / 100, (longitude * 100).rounded() / 100)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// RFC 3339 in UTC with milliseconds, accepted by zod's `datetime({ offset: true })`.
|
||||||
|
public func formatTimestamp(_ date: Date) -> String {
|
||||||
|
let formatter = ISO8601DateFormatter()
|
||||||
|
formatter.formatOptions = [.withInternetDateTime, .withFractionalSeconds]
|
||||||
|
formatter.timeZone = TimeZone(identifier: "UTC")
|
||||||
|
return formatter.string(from: date)
|
||||||
|
}
|
||||||
|
|
||||||
|
public func parseTimestamp(_ value: String) -> Date? {
|
||||||
|
let formatter = ISO8601DateFormatter()
|
||||||
|
formatter.formatOptions = [.withInternetDateTime, .withFractionalSeconds]
|
||||||
|
if let date = formatter.date(from: value) { return date }
|
||||||
|
formatter.formatOptions = [.withInternetDateTime]
|
||||||
|
return formatter.date(from: value)
|
||||||
|
}
|
||||||
|
|
||||||
|
public enum AuthorizationState: String, Encodable {
|
||||||
|
case authorized, limited, denied, restricted, notDetermined
|
||||||
|
}
|
||||||
|
|
||||||
|
public struct AuthorizationOutput: Encodable, Equatable {
|
||||||
|
public let status: AuthorizationState
|
||||||
|
public init(status: AuthorizationState) { self.status = status }
|
||||||
|
}
|
||||||
|
|
||||||
|
public struct ListOutput: Encodable {
|
||||||
|
public let items: [ClientMediaItem]
|
||||||
|
public let nextCursor: String?
|
||||||
|
/// Library change token captured before enumeration; `changes --token` resumes from it.
|
||||||
|
public let changeToken: String?
|
||||||
|
|
||||||
|
public init(items: [ClientMediaItem], nextCursor: String?, changeToken: String?) {
|
||||||
|
self.items = items
|
||||||
|
self.nextCursor = nextCursor
|
||||||
|
self.changeToken = changeToken
|
||||||
|
}
|
||||||
|
|
||||||
|
enum CodingKeys: String, CodingKey { case items, nextCursor, changeToken }
|
||||||
|
|
||||||
|
public func encode(to encoder: Encoder) throws {
|
||||||
|
var container = encoder.container(keyedBy: CodingKeys.self)
|
||||||
|
try container.encode(items, forKey: .items)
|
||||||
|
try container.encode(nextCursor, forKey: .nextCursor)
|
||||||
|
try container.encode(changeToken, forKey: .changeToken)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public struct AssetResourceInfo: Encodable, Equatable {
|
||||||
|
public let type: String
|
||||||
|
public let fileName: String
|
||||||
|
public let uti: String
|
||||||
|
public init(type: String, fileName: String, uti: String) {
|
||||||
|
self.type = type
|
||||||
|
self.fileName = fileName
|
||||||
|
self.uti = uti
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public struct AssetOutput: Encodable {
|
||||||
|
public let item: ClientMediaItem
|
||||||
|
public let modifiedAt: String?
|
||||||
|
public let hidden: Bool
|
||||||
|
public let resources: [AssetResourceInfo]
|
||||||
|
|
||||||
|
public init(item: ClientMediaItem, modifiedAt: Date?, hidden: Bool, resources: [AssetResourceInfo]) {
|
||||||
|
self.item = item
|
||||||
|
self.modifiedAt = modifiedAt.map(formatTimestamp)
|
||||||
|
self.hidden = hidden
|
||||||
|
self.resources = resources
|
||||||
|
}
|
||||||
|
|
||||||
|
enum CodingKeys: String, CodingKey { case item, modifiedAt, hidden, resources }
|
||||||
|
|
||||||
|
public func encode(to encoder: Encoder) throws {
|
||||||
|
var container = encoder.container(keyedBy: CodingKeys.self)
|
||||||
|
try container.encode(item, forKey: .item)
|
||||||
|
try container.encode(modifiedAt, forKey: .modifiedAt)
|
||||||
|
try container.encode(hidden, forKey: .hidden)
|
||||||
|
try container.encode(resources, forKey: .resources)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public struct ChangesOutput: Encodable {
|
||||||
|
/// Token to store for the next call; null only when a full relisting is required.
|
||||||
|
public let token: String?
|
||||||
|
public let changed: [ClientMediaItem]
|
||||||
|
public let deleted: [String]
|
||||||
|
public let fullResyncRequired: Bool
|
||||||
|
|
||||||
|
public init(token: String?, changed: [ClientMediaItem], deleted: [String], fullResyncRequired: Bool) {
|
||||||
|
self.token = token
|
||||||
|
self.changed = changed
|
||||||
|
self.deleted = deleted
|
||||||
|
self.fullResyncRequired = fullResyncRequired
|
||||||
|
}
|
||||||
|
|
||||||
|
public static func fullResync() -> ChangesOutput {
|
||||||
|
ChangesOutput(token: nil, changed: [], deleted: [], fullResyncRequired: true)
|
||||||
|
}
|
||||||
|
|
||||||
|
enum CodingKeys: String, CodingKey { case token, changed, deleted, fullResyncRequired }
|
||||||
|
|
||||||
|
public func encode(to encoder: Encoder) throws {
|
||||||
|
var container = encoder.container(keyedBy: CodingKeys.self)
|
||||||
|
try container.encode(token, forKey: .token)
|
||||||
|
try container.encode(changed, forKey: .changed)
|
||||||
|
try container.encode(deleted, forKey: .deleted)
|
||||||
|
try container.encode(fullResyncRequired, forKey: .fullResyncRequired)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public struct ExportOutput: Encodable, Equatable {
|
||||||
|
public let path: String
|
||||||
|
public let fileName: String
|
||||||
|
public let sizeBytes: Int64
|
||||||
|
public let sha256: String
|
||||||
|
public init(path: String, fileName: String, sizeBytes: Int64, sha256: String) {
|
||||||
|
self.path = path
|
||||||
|
self.fileName = fileName
|
||||||
|
self.sizeBytes = sizeBytes
|
||||||
|
self.sha256 = sha256
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -0,0 +1,56 @@
|
|||||||
|
import CryptoKit
|
||||||
|
import Foundation
|
||||||
|
|
||||||
|
/// stdout carries exactly one JSON document per successful command; stderr carries
|
||||||
|
/// JSON lines (`{"progress":…}` while exporting, `{"error":…}` on failure).
|
||||||
|
public func encodeJSON<T: Encodable>(_ value: T) throws -> String {
|
||||||
|
let encoder = JSONEncoder()
|
||||||
|
encoder.outputFormatting = [.sortedKeys, .withoutEscapingSlashes]
|
||||||
|
encoder.nonConformingFloatEncodingStrategy = .throw
|
||||||
|
let data = try encoder.encode(value)
|
||||||
|
guard let text = String(data: data, encoding: .utf8) else {
|
||||||
|
throw HelperError(.internal, "Output was not UTF-8.")
|
||||||
|
}
|
||||||
|
return text
|
||||||
|
}
|
||||||
|
|
||||||
|
private struct ErrorEnvelope: Encodable {
|
||||||
|
struct Body: Encodable {
|
||||||
|
let code: String
|
||||||
|
let message: String
|
||||||
|
}
|
||||||
|
let error: Body
|
||||||
|
}
|
||||||
|
|
||||||
|
public func errorLine(_ error: HelperError) -> String {
|
||||||
|
let envelope = ErrorEnvelope(error: .init(code: error.code.rawValue, message: error.message))
|
||||||
|
return (try? encodeJSON(envelope))
|
||||||
|
?? #"{"error":{"code":"internal","message":"Could not encode the error."}}"#
|
||||||
|
}
|
||||||
|
|
||||||
|
private struct ProgressLine: Encodable {
|
||||||
|
let progress: Double
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Progress clamped to 0…1 and rounded to three decimals so the line stays short.
|
||||||
|
public func progressLine(_ fraction: Double) -> String {
|
||||||
|
let value = fraction.isFinite ? min(1, max(0, fraction)) : 0
|
||||||
|
return (try? encodeJSON(ProgressLine(progress: (value * 1000).rounded() / 1000)))
|
||||||
|
?? #"{"progress":0}"#
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Streams the file through SHA-256 in 1 MiB reads so large videos never load fully.
|
||||||
|
public func sha256Hex(of url: URL) throws -> (hex: String, sizeBytes: Int64) {
|
||||||
|
let handle = try FileHandle(forReadingFrom: url)
|
||||||
|
defer { try? handle.close() }
|
||||||
|
var hasher = SHA256()
|
||||||
|
var size: Int64 = 0
|
||||||
|
while true {
|
||||||
|
let chunk = try handle.read(upToCount: 1 << 20) ?? Data()
|
||||||
|
if chunk.isEmpty { break }
|
||||||
|
hasher.update(data: chunk)
|
||||||
|
size += Int64(chunk.count)
|
||||||
|
}
|
||||||
|
let hex = hasher.finalize().map { String(format: "%02x", $0) }.joined()
|
||||||
|
return (hex, size)
|
||||||
|
}
|
||||||
@ -0,0 +1,131 @@
|
|||||||
|
import Foundation
|
||||||
|
|
||||||
|
/// Listing order: undated assets first, then creation date ascending, ties broken by
|
||||||
|
/// local identifier. PhotoKit can only sort by date, so ties are ordered here.
|
||||||
|
public struct SortKey: Comparable, Hashable {
|
||||||
|
/// Milliseconds since 1970; nil for assets without a creation date.
|
||||||
|
public let creationMillis: Int64?
|
||||||
|
public let localIdentifier: String
|
||||||
|
|
||||||
|
public init(creationMillis: Int64?, localIdentifier: String) {
|
||||||
|
self.creationMillis = creationMillis
|
||||||
|
self.localIdentifier = localIdentifier
|
||||||
|
}
|
||||||
|
|
||||||
|
public init(creationDate: Date?, localIdentifier: String) {
|
||||||
|
self.init(
|
||||||
|
creationMillis: creationDate.map { Int64(($0.timeIntervalSince1970 * 1000).rounded(.down)) },
|
||||||
|
localIdentifier: localIdentifier)
|
||||||
|
}
|
||||||
|
|
||||||
|
public static func < (lhs: SortKey, rhs: SortKey) -> Bool {
|
||||||
|
if lhs.creationMillis != rhs.creationMillis {
|
||||||
|
switch (lhs.creationMillis, rhs.creationMillis) {
|
||||||
|
case (nil, _): return true
|
||||||
|
case (_, nil): return false
|
||||||
|
case let (left?, right?): return left < right
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return lhs.localIdentifier < rhs.localIdentifier
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Opaque to the app: base64url of `{"v":1,"d":<millis|null>,"id":"<localIdentifier>"}`.
|
||||||
|
/// The cursor names the last item returned, so listing resumes after it even when
|
||||||
|
/// assets were added or removed in between.
|
||||||
|
public struct ListCursor: Equatable {
|
||||||
|
public let key: SortKey
|
||||||
|
|
||||||
|
public init(key: SortKey) { self.key = key }
|
||||||
|
|
||||||
|
private struct Payload: Codable {
|
||||||
|
let v: Int
|
||||||
|
let d: Int64?
|
||||||
|
let id: String
|
||||||
|
|
||||||
|
enum CodingKeys: String, CodingKey { case v, d, id }
|
||||||
|
|
||||||
|
func encode(to encoder: Encoder) throws {
|
||||||
|
var container = encoder.container(keyedBy: CodingKeys.self)
|
||||||
|
try container.encode(v, forKey: .v)
|
||||||
|
try container.encode(d, forKey: .d)
|
||||||
|
try container.encode(id, forKey: .id)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public func encoded() -> String {
|
||||||
|
let payload = Payload(v: 1, d: key.creationMillis, id: key.localIdentifier)
|
||||||
|
let encoder = JSONEncoder()
|
||||||
|
encoder.outputFormatting = [.sortedKeys]
|
||||||
|
// Encoding a struct of plain values cannot fail.
|
||||||
|
let data = (try? encoder.encode(payload)) ?? Data()
|
||||||
|
return base64URLEncode(data)
|
||||||
|
}
|
||||||
|
|
||||||
|
public static func decode(_ value: String) throws -> ListCursor {
|
||||||
|
guard value.count <= 2048, let data = base64URLDecode(value),
|
||||||
|
let payload = try? JSONDecoder().decode(Payload.self, from: data),
|
||||||
|
payload.v == 1, !payload.id.isEmpty
|
||||||
|
else { throw HelperError(.usage, "Invalid cursor.") }
|
||||||
|
return ListCursor(key: SortKey(creationMillis: payload.d, localIdentifier: payload.id))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
public func base64URLEncode(_ data: Data) -> String {
|
||||||
|
data.base64EncodedString()
|
||||||
|
.replacingOccurrences(of: "+", with: "-")
|
||||||
|
.replacingOccurrences(of: "/", with: "_")
|
||||||
|
.replacingOccurrences(of: "=", with: "")
|
||||||
|
}
|
||||||
|
|
||||||
|
public func base64URLDecode(_ value: String) -> Data? {
|
||||||
|
guard value.allSatisfy({ $0.isASCII && ($0.isLetter || $0.isNumber || $0 == "-" || $0 == "_") })
|
||||||
|
else { return nil }
|
||||||
|
var base64 = value.replacingOccurrences(of: "-", with: "+").replacingOccurrences(of: "_", with: "/")
|
||||||
|
while base64.count % 4 != 0 { base64.append("=") }
|
||||||
|
return Data(base64Encoded: base64)
|
||||||
|
}
|
||||||
|
|
||||||
|
public struct Page {
|
||||||
|
/// Positions in the source sequence, in listing order.
|
||||||
|
public let indices: [Int]
|
||||||
|
public let hasMore: Bool
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Selects the next page from a sequence sorted by creation date only (ties in any
|
||||||
|
/// order). Binary search finds the first candidate date; a whole same-date group is
|
||||||
|
/// collected before sorting so tie order never depends on PhotoKit.
|
||||||
|
public func nextPage(count: Int, key: (Int) -> SortKey, after cursor: ListCursor?, limit: Int)
|
||||||
|
-> Page
|
||||||
|
{
|
||||||
|
precondition(limit > 0)
|
||||||
|
var start = 0
|
||||||
|
if let cursor, let cursorMillis = cursor.key.creationMillis {
|
||||||
|
var low = 0
|
||||||
|
var high = count
|
||||||
|
while low < high {
|
||||||
|
let mid = (low + high) / 2
|
||||||
|
let millis = key(mid).creationMillis
|
||||||
|
if millis == nil || millis! < cursorMillis { low = mid + 1 } else { high = mid }
|
||||||
|
}
|
||||||
|
start = low
|
||||||
|
}
|
||||||
|
var candidates: [(index: Int, key: SortKey)] = []
|
||||||
|
var stoppedEarly = false
|
||||||
|
var index = start
|
||||||
|
while index < count {
|
||||||
|
let current = key(index)
|
||||||
|
if candidates.count >= limit, let last = candidates.last, current.creationMillis != last.key.creationMillis {
|
||||||
|
// Dates ascend, so everything from here on sorts after the collected group.
|
||||||
|
stoppedEarly = true
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if cursor == nil || cursor!.key < current {
|
||||||
|
candidates.append((index, current))
|
||||||
|
}
|
||||||
|
index += 1
|
||||||
|
}
|
||||||
|
candidates.sort { $0.key < $1.key }
|
||||||
|
let hasMore = stoppedEarly || candidates.count > limit
|
||||||
|
return Page(indices: candidates.prefix(limit).map(\.index), hasMore: hasMore)
|
||||||
|
}
|
||||||
@ -0,0 +1,72 @@
|
|||||||
|
import Foundation
|
||||||
|
|
||||||
|
/// Where an export is written: a final path and a sibling temporary path inside the
|
||||||
|
/// destination directory.
|
||||||
|
public struct ExportDestination: Equatable {
|
||||||
|
public let directory: URL
|
||||||
|
public let finalURL: URL
|
||||||
|
public let temporaryURL: URL
|
||||||
|
public let fileName: String
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Resolves the export target. The directory must be absolute, free of `..`, existing
|
||||||
|
/// and a directory after resolving symlinks (macOS temp dirs live behind /var). The file
|
||||||
|
/// name comes from Photos, so it is reduced to one safe component and the joined path is
|
||||||
|
/// checked to stay a direct child of the resolved directory. Existing files are never
|
||||||
|
/// overwritten.
|
||||||
|
public func resolveExportDestination(
|
||||||
|
directory: String,
|
||||||
|
fileName rawFileName: String,
|
||||||
|
temporarySuffix: String = UUID().uuidString,
|
||||||
|
fileManager: FileManager = .default
|
||||||
|
) throws -> ExportDestination {
|
||||||
|
guard directory.hasPrefix("/"), !directory.contains("\0") else {
|
||||||
|
throw HelperError(.pathRejected, "--dest must be an absolute directory path.")
|
||||||
|
}
|
||||||
|
let components = directory.split(separator: "/", omittingEmptySubsequences: true)
|
||||||
|
guard !components.contains(where: { $0 == ".." || $0 == "." }) else {
|
||||||
|
throw HelperError(.pathRejected, "--dest must not contain . or .. segments.")
|
||||||
|
}
|
||||||
|
let resolved = URL(fileURLWithPath: directory, isDirectory: true).resolvingSymlinksInPath()
|
||||||
|
.standardizedFileURL
|
||||||
|
var isDirectory: ObjCBool = false
|
||||||
|
guard fileManager.fileExists(atPath: resolved.path, isDirectory: &isDirectory), isDirectory.boolValue
|
||||||
|
else { throw HelperError(.pathRejected, "--dest must be an existing directory.") }
|
||||||
|
|
||||||
|
let fileName = sanitizeFileName(rawFileName)
|
||||||
|
let finalURL = resolved.appendingPathComponent(fileName, isDirectory: false).standardizedFileURL
|
||||||
|
let temporaryURL = resolved.appendingPathComponent(
|
||||||
|
".\(fileName).\(temporarySuffix).partial", isDirectory: false
|
||||||
|
).standardizedFileURL
|
||||||
|
for candidate in [finalURL, temporaryURL]
|
||||||
|
where candidate.deletingLastPathComponent().path != resolved.path {
|
||||||
|
throw HelperError(.pathRejected, "Export path left the destination directory.")
|
||||||
|
}
|
||||||
|
// lstat semantics: a dangling symlink at the final path also counts as existing.
|
||||||
|
if (try? fileManager.attributesOfItem(atPath: finalURL.path)) != nil {
|
||||||
|
throw HelperError(.destinationExists, "The destination file already exists.")
|
||||||
|
}
|
||||||
|
return ExportDestination(
|
||||||
|
directory: resolved, finalURL: finalURL, temporaryURL: temporaryURL, fileName: fileName)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One path component with no separators, control characters or leading dots, at most
|
||||||
|
/// 200 UTF-8 bytes with the extension kept.
|
||||||
|
public func sanitizeFileName(_ raw: String) -> String {
|
||||||
|
var cleaned = String(
|
||||||
|
raw.unicodeScalars.map { scalar -> Character in
|
||||||
|
if scalar == "/" || scalar == "\\" || scalar == ":" { return "_" }
|
||||||
|
if CharacterSet.controlCharacters.contains(scalar) { return "_" }
|
||||||
|
return Character(scalar)
|
||||||
|
})
|
||||||
|
while cleaned.hasPrefix(".") || cleaned.hasPrefix(" ") { cleaned.removeFirst() }
|
||||||
|
cleaned = cleaned.trimmingCharacters(in: .whitespaces)
|
||||||
|
if cleaned.isEmpty { return "asset" }
|
||||||
|
let maxBytes = 200
|
||||||
|
if cleaned.utf8.count <= maxBytes { return cleaned }
|
||||||
|
let ext = (cleaned as NSString).pathExtension
|
||||||
|
let suffix = ext.isEmpty || ext.utf8.count > 16 ? "" : ".\(ext)"
|
||||||
|
var stem = suffix.isEmpty ? cleaned : String(cleaned.dropLast(suffix.count))
|
||||||
|
while stem.utf8.count + suffix.utf8.count > maxBytes { stem.removeLast() }
|
||||||
|
return stem + suffix
|
||||||
|
}
|
||||||
@ -0,0 +1,148 @@
|
|||||||
|
import AVFoundation
|
||||||
|
import AppKit
|
||||||
|
import Foundation
|
||||||
|
import Photos
|
||||||
|
import PhotosHelperCore
|
||||||
|
|
||||||
|
/// Writes one asset into the destination directory: first to a hidden temporary file,
|
||||||
|
/// then renamed into place after hashing, so a cancelled export never leaves a file
|
||||||
|
/// that looks complete.
|
||||||
|
struct Exporter {
|
||||||
|
func export(
|
||||||
|
localIdentifier: String,
|
||||||
|
directory: String,
|
||||||
|
variant: ExportVariant,
|
||||||
|
progress: @escaping (Double) -> Void
|
||||||
|
) async throws -> ExportOutput {
|
||||||
|
guard
|
||||||
|
let asset = PHAsset.fetchAssets(withLocalIdentifiers: [localIdentifier], options: nil)
|
||||||
|
.firstObject
|
||||||
|
else { throw HelperError(.notFound, "No photo or video with that identifier.") }
|
||||||
|
guard let resource = originalResource(for: asset, variant: variant) else {
|
||||||
|
throw HelperError(.notFound, "The asset has no exportable resource.")
|
||||||
|
}
|
||||||
|
var fileName = resource.originalFilename
|
||||||
|
if variant == .proxy {
|
||||||
|
let stem = (fileName as NSString).deletingPathExtension
|
||||||
|
fileName = stem + (asset.mediaType == .video ? ".mp4" : ".jpg")
|
||||||
|
}
|
||||||
|
let destination = try resolveExportDestination(directory: directory, fileName: fileName)
|
||||||
|
let fileManager = FileManager.default
|
||||||
|
do {
|
||||||
|
switch (variant, asset.mediaType) {
|
||||||
|
case (.original, _):
|
||||||
|
try await writeResource(resource, to: destination.temporaryURL, progress: progress)
|
||||||
|
case (.proxy, .video):
|
||||||
|
try await writeVideoProxy(asset, to: destination.temporaryURL, progress: progress)
|
||||||
|
case (.proxy, _):
|
||||||
|
try await writeImageProxy(asset, to: destination.temporaryURL, progress: progress)
|
||||||
|
}
|
||||||
|
let digest = try sha256Hex(of: destination.temporaryURL)
|
||||||
|
// moveItem refuses an existing destination, so nothing is ever overwritten.
|
||||||
|
try fileManager.moveItem(at: destination.temporaryURL, to: destination.finalURL)
|
||||||
|
progress(1)
|
||||||
|
return ExportOutput(
|
||||||
|
path: destination.finalURL.path, fileName: destination.fileName,
|
||||||
|
sizeBytes: digest.sizeBytes, sha256: digest.hex)
|
||||||
|
} catch {
|
||||||
|
try? fileManager.removeItem(at: destination.temporaryURL)
|
||||||
|
if let helperError = error as? HelperError { throw helperError }
|
||||||
|
throw mapExportError(error)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Original bytes through PHAssetResourceManager; downloads from iCloud when the
|
||||||
|
/// library keeps only optimized copies on this Mac.
|
||||||
|
private func writeResource(
|
||||||
|
_ resource: PHAssetResource, to url: URL, progress: @escaping (Double) -> Void
|
||||||
|
) async throws {
|
||||||
|
let options = PHAssetResourceRequestOptions()
|
||||||
|
options.isNetworkAccessAllowed = true
|
||||||
|
options.progressHandler = { value in progress(value) }
|
||||||
|
try await withCheckedThrowingContinuation { (continuation: CheckedContinuation<Void, Error>) in
|
||||||
|
PHAssetResourceManager.default().writeData(for: resource, toFile: url, options: options) {
|
||||||
|
error in
|
||||||
|
if let error { continuation.resume(throwing: error) } else { continuation.resume() }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 720p H.264 MP4 of the current version, without source metadata.
|
||||||
|
private func writeVideoProxy(
|
||||||
|
_ asset: PHAsset, to url: URL, progress: @escaping (Double) -> Void
|
||||||
|
) async throws {
|
||||||
|
let options = PHVideoRequestOptions()
|
||||||
|
options.isNetworkAccessAllowed = true
|
||||||
|
options.deliveryMode = .highQualityFormat
|
||||||
|
options.version = .current
|
||||||
|
options.progressHandler = { value, _, _, _ in progress(value * 0.5) }
|
||||||
|
let session: AVAssetExportSession = try await withCheckedThrowingContinuation { continuation in
|
||||||
|
PHImageManager.default().requestExportSession(
|
||||||
|
forVideo: asset, options: options, exportPreset: AVAssetExportPreset1280x720
|
||||||
|
) { session, info in
|
||||||
|
if let session {
|
||||||
|
continuation.resume(returning: session)
|
||||||
|
} else {
|
||||||
|
continuation.resume(
|
||||||
|
throwing: (info?[PHImageErrorKey] as? Error)
|
||||||
|
?? HelperError(.exportFailed, "Could not prepare the video export."))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
session.outputURL = url
|
||||||
|
session.outputFileType = .mp4
|
||||||
|
session.shouldOptimizeForNetworkUse = true
|
||||||
|
session.metadata = []
|
||||||
|
session.metadataItemFilter = .forSharing()
|
||||||
|
await session.export()
|
||||||
|
guard session.status == .completed else {
|
||||||
|
throw session.error ?? HelperError(.exportFailed, "The video export did not finish.")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// JPEG of the current version, at most 1280 px on the long edge, without EXIF/GPS.
|
||||||
|
private func writeImageProxy(
|
||||||
|
_ asset: PHAsset, to url: URL, progress: @escaping (Double) -> Void
|
||||||
|
) async throws {
|
||||||
|
let options = PHImageRequestOptions()
|
||||||
|
options.isNetworkAccessAllowed = true
|
||||||
|
options.deliveryMode = .highQualityFormat
|
||||||
|
options.resizeMode = .exact
|
||||||
|
options.version = .current
|
||||||
|
options.isSynchronous = false
|
||||||
|
options.progressHandler = { value, _, _, _ in progress(value) }
|
||||||
|
// highQualityFormat calls the handler exactly once.
|
||||||
|
let image: NSImage = try await withCheckedThrowingContinuation { continuation in
|
||||||
|
PHImageManager.default().requestImage(
|
||||||
|
for: asset, targetSize: CGSize(width: 1280, height: 1280), contentMode: .aspectFit,
|
||||||
|
options: options
|
||||||
|
) { image, info in
|
||||||
|
if let image {
|
||||||
|
continuation.resume(returning: image)
|
||||||
|
} else {
|
||||||
|
continuation.resume(
|
||||||
|
throwing: (info?[PHImageErrorKey] as? Error)
|
||||||
|
?? HelperError(.exportFailed, "Could not render the photo."))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
guard let cgImage = image.cgImage(forProposedRect: nil, context: nil, hints: nil),
|
||||||
|
let data = NSBitmapImageRep(cgImage: cgImage).representation(
|
||||||
|
using: .jpeg, properties: [.compressionFactor: 0.85])
|
||||||
|
else { throw HelperError(.exportFailed, "Could not encode the photo.") }
|
||||||
|
try data.write(to: url, options: .withoutOverwriting)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// iCloud and network failures are retryable; everything else is an export failure.
|
||||||
|
func mapExportError(_ error: Error) -> HelperError {
|
||||||
|
if let photosError = error as? PHPhotosError,
|
||||||
|
photosError.code == .networkAccessRequired || photosError.code == .networkError
|
||||||
|
{
|
||||||
|
return HelperError(.network, "The original could not be downloaded from iCloud.")
|
||||||
|
}
|
||||||
|
if (error as NSError).domain == NSURLErrorDomain {
|
||||||
|
return HelperError(.network, "The original could not be downloaded from iCloud.")
|
||||||
|
}
|
||||||
|
return HelperError(.exportFailed, "The export failed.")
|
||||||
|
}
|
||||||
@ -0,0 +1,183 @@
|
|||||||
|
import Foundation
|
||||||
|
import Photos
|
||||||
|
import PhotosHelperCore
|
||||||
|
|
||||||
|
/// Read-only PhotoKit queries. Nothing here writes to the library.
|
||||||
|
struct Library {
|
||||||
|
private let library = PHPhotoLibrary.shared()
|
||||||
|
|
||||||
|
/// Images and videos only; audio and unknown media types are not part of Media Studio.
|
||||||
|
/// PhotoKit sorts by date only; the store places missing dates first, which is what
|
||||||
|
/// `SortKey` assumes, and `nextPage` orders ties itself.
|
||||||
|
private func fetchOptions(since: Date?) -> PHFetchOptions {
|
||||||
|
let options = PHFetchOptions()
|
||||||
|
options.includeHiddenAssets = false
|
||||||
|
var predicates = [
|
||||||
|
NSPredicate(
|
||||||
|
format: "mediaType == %ld || mediaType == %ld",
|
||||||
|
PHAssetMediaType.image.rawValue, PHAssetMediaType.video.rawValue)
|
||||||
|
]
|
||||||
|
if let since {
|
||||||
|
predicates.append(NSPredicate(format: "modificationDate >= %@", since as NSDate))
|
||||||
|
}
|
||||||
|
options.predicate = NSCompoundPredicate(andPredicateWithSubpredicates: predicates)
|
||||||
|
options.sortDescriptors = [NSSortDescriptor(key: "creationDate", ascending: true)]
|
||||||
|
return options
|
||||||
|
}
|
||||||
|
|
||||||
|
func list(after cursor: ListCursor?, limit: Int, since: Date?) throws -> ListOutput {
|
||||||
|
// Captured before enumeration so changes made while listing show up next time.
|
||||||
|
let changeToken = try encodeToken(library.currentChangeToken)
|
||||||
|
let result = PHAsset.fetchAssets(with: fetchOptions(since: since))
|
||||||
|
let page = nextPage(
|
||||||
|
count: result.count,
|
||||||
|
key: { index in
|
||||||
|
let asset = result.object(at: index)
|
||||||
|
return SortKey(creationDate: asset.creationDate, localIdentifier: asset.localIdentifier)
|
||||||
|
},
|
||||||
|
after: cursor,
|
||||||
|
limit: limit)
|
||||||
|
let assets = page.indices.map { result.object(at: $0) }
|
||||||
|
let items = assets.compactMap(makeItem(for:))
|
||||||
|
let nextCursor = page.hasMore
|
||||||
|
? assets.last.map {
|
||||||
|
ListCursor(key: SortKey(creationDate: $0.creationDate, localIdentifier: $0.localIdentifier))
|
||||||
|
.encoded()
|
||||||
|
}
|
||||||
|
: nil
|
||||||
|
return ListOutput(items: items, nextCursor: nextCursor, changeToken: changeToken)
|
||||||
|
}
|
||||||
|
|
||||||
|
func asset(_ localIdentifier: String) throws -> AssetOutput {
|
||||||
|
guard let asset = PHAsset.fetchAssets(withLocalIdentifiers: [localIdentifier], options: nil).firstObject,
|
||||||
|
let item = makeItem(for: asset)
|
||||||
|
else { throw HelperError(.notFound, "No photo or video with that identifier.") }
|
||||||
|
let resources = PHAssetResource.assetResources(for: asset).map {
|
||||||
|
AssetResourceInfo(
|
||||||
|
type: resourceTypeName($0.type), fileName: $0.originalFilename, uti: $0.uniformTypeIdentifier)
|
||||||
|
}
|
||||||
|
return AssetOutput(
|
||||||
|
item: item, modifiedAt: asset.modificationDate, hidden: asset.isHidden, resources: resources)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Persistent change history (macOS 13). An expired token or missing history asks the
|
||||||
|
/// caller for a full relisting instead of guessing.
|
||||||
|
func changes(since encodedToken: String) throws -> ChangesOutput {
|
||||||
|
guard let token = decodeToken(encodedToken) else { return .fullResync() }
|
||||||
|
let current = try encodeToken(library.currentChangeToken)
|
||||||
|
var changed = Set<String>()
|
||||||
|
var deleted = Set<String>()
|
||||||
|
do {
|
||||||
|
let result = try library.fetchPersistentChanges(since: token)
|
||||||
|
for change in result {
|
||||||
|
let details = try change.changeDetails(for: PHObjectType.asset)
|
||||||
|
changed.formUnion(details.insertedLocalIdentifiers)
|
||||||
|
changed.formUnion(details.updatedLocalIdentifiers)
|
||||||
|
deleted.formUnion(details.deletedLocalIdentifiers)
|
||||||
|
if changed.count + deleted.count > Limits.maxChanges { break }
|
||||||
|
}
|
||||||
|
} catch let error as PHPhotosError
|
||||||
|
where error.code == .persistentChangeTokenExpired
|
||||||
|
|| error.code == .persistentChangeDetailsUnavailable
|
||||||
|
{
|
||||||
|
return .fullResync()
|
||||||
|
} catch {
|
||||||
|
throw HelperError(.internal, "Could not read the Photos change history.")
|
||||||
|
}
|
||||||
|
if changed.count + deleted.count > Limits.maxChanges { return .fullResync() }
|
||||||
|
|
||||||
|
changed.subtract(deleted)
|
||||||
|
var items: [ClientMediaItem] = []
|
||||||
|
var found = Set<String>()
|
||||||
|
if !changed.isEmpty {
|
||||||
|
PHAsset.fetchAssets(withLocalIdentifiers: Array(changed), options: nil).enumerateObjects {
|
||||||
|
asset, _, _ in
|
||||||
|
found.insert(asset.localIdentifier)
|
||||||
|
if let item = makeItem(for: asset) { items.append(item) }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Assets that vanished between the history read and the fetch count as deleted.
|
||||||
|
deleted.formUnion(changed.subtracting(found))
|
||||||
|
items.sort { $0.externalId < $1.externalId }
|
||||||
|
return ChangesOutput(
|
||||||
|
token: current, changed: items, deleted: deleted.sorted(), fullResyncRequired: false)
|
||||||
|
}
|
||||||
|
|
||||||
|
private func encodeToken(_ token: PHPersistentChangeToken) throws -> String {
|
||||||
|
guard
|
||||||
|
let data = try? NSKeyedArchiver.archivedData(withRootObject: token, requiringSecureCoding: true)
|
||||||
|
else { throw HelperError(.internal, "Could not encode the Photos change token.") }
|
||||||
|
return base64URLEncode(data)
|
||||||
|
}
|
||||||
|
|
||||||
|
private func decodeToken(_ value: String) -> PHPersistentChangeToken? {
|
||||||
|
guard let data = base64URLDecode(value) else { return nil }
|
||||||
|
return try? NSKeyedUnarchiver.unarchivedObject(ofClass: PHPersistentChangeToken.self, from: data)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Builds the contract item. Returns nil for media types Media Studio does not handle
|
||||||
|
/// and for identifiers longer than the contract allows.
|
||||||
|
func makeItem(for asset: PHAsset) -> ClientMediaItem? {
|
||||||
|
let kind: ClientMediaItem.Kind
|
||||||
|
switch asset.mediaType {
|
||||||
|
case .video: kind = .video
|
||||||
|
case .image: kind = asset.mediaSubtypes.contains(.photoLive) ? .livePhoto : .photo
|
||||||
|
default: return nil
|
||||||
|
}
|
||||||
|
guard asset.localIdentifier.utf16.count <= Limits.externalId else { return nil }
|
||||||
|
var albums: [String] = []
|
||||||
|
PHAssetCollection.fetchAssetCollectionsContaining(asset, with: .album, options: nil)
|
||||||
|
.enumerateObjects { collection, _, stop in
|
||||||
|
if let title = collection.localizedTitle { albums.append(title) }
|
||||||
|
if albums.count >= Limits.albumCount { stop.pointee = true }
|
||||||
|
}
|
||||||
|
let coordinate = asset.location?.coordinate
|
||||||
|
return ClientMediaItem(
|
||||||
|
externalId: asset.localIdentifier,
|
||||||
|
kind: kind,
|
||||||
|
capturedAt: asset.creationDate,
|
||||||
|
durationSeconds: kind == .video ? asset.duration : nil,
|
||||||
|
width: asset.pixelWidth,
|
||||||
|
height: asset.pixelHeight,
|
||||||
|
latitude: coordinate?.latitude,
|
||||||
|
longitude: coordinate?.longitude,
|
||||||
|
albumNames: albums,
|
||||||
|
isFavorite: asset.isFavorite,
|
||||||
|
fileName: originalResource(for: asset, variant: .original)?.originalFilename)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The resource holding the untouched original: the video track for videos, the still
|
||||||
|
/// for photos and Live Photos (the paired motion clip is not exported).
|
||||||
|
func originalResource(for asset: PHAsset, variant: ExportVariant) -> PHAssetResource? {
|
||||||
|
let resources = PHAssetResource.assetResources(for: asset)
|
||||||
|
let preferred: [PHAssetResourceType]
|
||||||
|
switch (asset.mediaType, variant) {
|
||||||
|
case (.video, .original): preferred = [.video, .fullSizeVideo]
|
||||||
|
case (.video, .proxy): preferred = [.fullSizeVideo, .video]
|
||||||
|
case (_, .original): preferred = [.photo, .fullSizePhoto]
|
||||||
|
case (_, .proxy): preferred = [.fullSizePhoto, .photo]
|
||||||
|
}
|
||||||
|
for type in preferred {
|
||||||
|
if let match = resources.first(where: { $0.type == type }) { return match }
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func resourceTypeName(_ type: PHAssetResourceType) -> String {
|
||||||
|
switch type {
|
||||||
|
case .photo: return "photo"
|
||||||
|
case .video: return "video"
|
||||||
|
case .audio: return "audio"
|
||||||
|
case .alternatePhoto: return "alternate_photo"
|
||||||
|
case .fullSizePhoto: return "full_size_photo"
|
||||||
|
case .fullSizeVideo: return "full_size_video"
|
||||||
|
case .adjustmentData: return "adjustment_data"
|
||||||
|
case .adjustmentBasePhoto: return "adjustment_base_photo"
|
||||||
|
case .pairedVideo: return "paired_video"
|
||||||
|
case .fullSizePairedVideo: return "full_size_paired_video"
|
||||||
|
case .adjustmentBasePairedVideo: return "adjustment_base_paired_video"
|
||||||
|
case .adjustmentBaseVideo: return "adjustment_base_video"
|
||||||
|
@unknown default: return "other"
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -0,0 +1,103 @@
|
|||||||
|
import Foundation
|
||||||
|
import Photos
|
||||||
|
import PhotosHelperCore
|
||||||
|
|
||||||
|
/// Entry point. Exactly one JSON document goes to stdout on success; failures write
|
||||||
|
/// `{"error":{code,message}}` to stderr and exit non-zero (see HelperError.exitCode).
|
||||||
|
@main
|
||||||
|
struct PhotosHelper {
|
||||||
|
static func main() async {
|
||||||
|
do {
|
||||||
|
let command = try parseArguments(Array(CommandLine.arguments.dropFirst()))
|
||||||
|
let output = try await run(command)
|
||||||
|
FileHandle.standardOutput.write(Data((output + "\n").utf8))
|
||||||
|
exit(0)
|
||||||
|
} catch let error as HelperError {
|
||||||
|
fail(error)
|
||||||
|
} catch {
|
||||||
|
fail(HelperError(.internal, "Unexpected failure."))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
static func fail(_ error: HelperError) -> Never {
|
||||||
|
FileHandle.standardError.write(Data((errorLine(error) + "\n").utf8))
|
||||||
|
exit(error.exitCode)
|
||||||
|
}
|
||||||
|
|
||||||
|
static func run(_ command: Command) async throws -> String {
|
||||||
|
switch command {
|
||||||
|
case .authorize(let prompt):
|
||||||
|
let status = await authorize(prompt: prompt)
|
||||||
|
return try encodeJSON(AuthorizationOutput(status: status))
|
||||||
|
case .list(let cursor, let limit, let since):
|
||||||
|
try requireAccess()
|
||||||
|
return try encodeJSON(try Library().list(after: cursor, limit: limit, since: since))
|
||||||
|
case .asset(let localIdentifier):
|
||||||
|
try requireAccess()
|
||||||
|
return try encodeJSON(try Library().asset(localIdentifier))
|
||||||
|
case .export(let localIdentifier, let destination, let variant):
|
||||||
|
try requireAccess()
|
||||||
|
let progress = ProgressReporter()
|
||||||
|
// Local copies report no progress, so a heartbeat repeats the last value; the app
|
||||||
|
// treats a silent helper as stalled.
|
||||||
|
let heartbeat = Task {
|
||||||
|
while !Task.isCancelled {
|
||||||
|
try? await Task.sleep(nanoseconds: 15_000_000_000)
|
||||||
|
if !Task.isCancelled { progress.emit(nil) }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
defer { heartbeat.cancel() }
|
||||||
|
progress.emit(0)
|
||||||
|
let result = try await Exporter().export(
|
||||||
|
localIdentifier: localIdentifier, directory: destination, variant: variant,
|
||||||
|
progress: { fraction in progress.emit(fraction) })
|
||||||
|
return try encodeJSON(result)
|
||||||
|
case .changes(let token):
|
||||||
|
try requireAccess()
|
||||||
|
return try encodeJSON(try Library().changes(since: token))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `.readWrite` is the lowest PhotoKit level that can read assets: `.addOnly` cannot
|
||||||
|
/// enumerate or export anything. The helper never writes to the library.
|
||||||
|
static func authorize(prompt: Bool) async -> AuthorizationState {
|
||||||
|
var status = PHPhotoLibrary.authorizationStatus(for: .readWrite)
|
||||||
|
if prompt && status == .notDetermined {
|
||||||
|
status = await PHPhotoLibrary.requestAuthorization(for: .readWrite)
|
||||||
|
}
|
||||||
|
return state(status)
|
||||||
|
}
|
||||||
|
|
||||||
|
static func state(_ status: PHAuthorizationStatus) -> AuthorizationState {
|
||||||
|
switch status {
|
||||||
|
case .authorized: return .authorized
|
||||||
|
case .limited: return .limited
|
||||||
|
case .denied: return .denied
|
||||||
|
case .restricted: return .restricted
|
||||||
|
case .notDetermined: return .notDetermined
|
||||||
|
@unknown default: return .denied
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Serializes progress lines written from PhotoKit callback queues and the heartbeat.
|
||||||
|
final class ProgressReporter: @unchecked Sendable {
|
||||||
|
private let lock = NSLock()
|
||||||
|
private var last = 0.0
|
||||||
|
|
||||||
|
/// nil repeats the last reported value.
|
||||||
|
func emit(_ fraction: Double?) {
|
||||||
|
lock.lock()
|
||||||
|
defer { lock.unlock() }
|
||||||
|
if let fraction { last = max(last, fraction) }
|
||||||
|
FileHandle.standardError.write(Data((progressLine(last) + "\n").utf8))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Every command except `authorize` runs only with granted access and never prompts.
|
||||||
|
static func requireAccess() throws {
|
||||||
|
switch state(PHPhotoLibrary.authorizationStatus(for: .readWrite)) {
|
||||||
|
case .authorized, .limited: return
|
||||||
|
default: throw HelperError(.notAuthorized, "Photos access has not been granted.")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -0,0 +1,73 @@
|
|||||||
|
import XCTest
|
||||||
|
|
||||||
|
@testable import PhotosHelperCore
|
||||||
|
|
||||||
|
final class ArgumentsTests: XCTestCase {
|
||||||
|
func testAuthorizePromptsUnlessAskedNotTo() throws {
|
||||||
|
XCTAssertEqual(try parseArguments(["authorize"]), .authorize(prompt: true))
|
||||||
|
XCTAssertEqual(try parseArguments(["authorize", "--no-prompt"]), .authorize(prompt: false))
|
||||||
|
}
|
||||||
|
|
||||||
|
func testListDefaultsAndOptions() throws {
|
||||||
|
XCTAssertEqual(try parseArguments(["list"]), .list(cursor: nil, limit: 500, since: nil))
|
||||||
|
let cursor = ListCursor(key: SortKey(creationMillis: 1_700_000_000_000, localIdentifier: "A/L0/001"))
|
||||||
|
let parsed = try parseArguments([
|
||||||
|
"list", "--cursor", cursor.encoded(), "--limit", "25", "--since", "2024-01-02T03:04:05Z",
|
||||||
|
])
|
||||||
|
XCTAssertEqual(
|
||||||
|
parsed,
|
||||||
|
.list(cursor: cursor, limit: 25, since: Date(timeIntervalSince1970: 1_704_164_645)))
|
||||||
|
}
|
||||||
|
|
||||||
|
func testListRejectsBadValues() {
|
||||||
|
for arguments in [
|
||||||
|
["list", "--limit", "0"],
|
||||||
|
["list", "--limit", "1001"],
|
||||||
|
["list", "--limit", "ten"],
|
||||||
|
["list", "--limit"],
|
||||||
|
["list", "--since", "yesterday"],
|
||||||
|
["list", "--cursor", "not a cursor"],
|
||||||
|
["list", "--limit", "5", "--limit", "6"],
|
||||||
|
["list", "--dest", "/tmp"],
|
||||||
|
["list", "extra"],
|
||||||
|
["list", "--unknown"],
|
||||||
|
] {
|
||||||
|
XCTAssertThrowsError(try parseArguments(arguments), "\(arguments)") { error in
|
||||||
|
XCTAssertEqual((error as? HelperError)?.code, .usage)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func testAssetNeedsExactlyOneIdentifier() throws {
|
||||||
|
XCTAssertEqual(try parseArguments(["asset", "ABC/L0/001"]), .asset(localIdentifier: "ABC/L0/001"))
|
||||||
|
XCTAssertThrowsError(try parseArguments(["asset"]))
|
||||||
|
XCTAssertThrowsError(try parseArguments(["asset", "a", "b"]))
|
||||||
|
XCTAssertThrowsError(try parseArguments(["asset", String(repeating: "x", count: 201)]))
|
||||||
|
XCTAssertThrowsError(try parseArguments(["asset", "bad\nid"]))
|
||||||
|
}
|
||||||
|
|
||||||
|
func testExportVariants() throws {
|
||||||
|
XCTAssertEqual(
|
||||||
|
try parseArguments(["export", "ID", "--dest", "/tmp/out"]),
|
||||||
|
.export(localIdentifier: "ID", destination: "/tmp/out", variant: .original))
|
||||||
|
XCTAssertEqual(
|
||||||
|
try parseArguments(["export", "--dest", "/tmp/out", "ID", "--proxy"]),
|
||||||
|
.export(localIdentifier: "ID", destination: "/tmp/out", variant: .proxy))
|
||||||
|
XCTAssertThrowsError(try parseArguments(["export", "ID"]))
|
||||||
|
XCTAssertThrowsError(try parseArguments(["export", "ID", "--dest", "/tmp", "--original", "--proxy"]))
|
||||||
|
}
|
||||||
|
|
||||||
|
func testChangesNeedsToken() throws {
|
||||||
|
XCTAssertEqual(try parseArguments(["changes", "--token", "abc"]), .changes(token: "abc"))
|
||||||
|
XCTAssertThrowsError(try parseArguments(["changes"]))
|
||||||
|
XCTAssertThrowsError(try parseArguments(["changes", "--token", ""]))
|
||||||
|
}
|
||||||
|
|
||||||
|
func testUnknownCommandIsUsageError() {
|
||||||
|
XCTAssertThrowsError(try parseArguments([])) { error in
|
||||||
|
XCTAssertEqual((error as? HelperError)?.code, .usage)
|
||||||
|
XCTAssertEqual((error as? HelperError)?.exitCode, 64)
|
||||||
|
}
|
||||||
|
XCTAssertThrowsError(try parseArguments(["delete", "ID"]))
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -0,0 +1,92 @@
|
|||||||
|
import XCTest
|
||||||
|
|
||||||
|
@testable import PhotosHelperCore
|
||||||
|
|
||||||
|
final class OutputTests: XCTestCase {
|
||||||
|
private func object(_ json: String) throws -> [String: Any] {
|
||||||
|
try XCTUnwrap(JSONSerialization.jsonObject(with: Data(json.utf8)) as? [String: Any])
|
||||||
|
}
|
||||||
|
|
||||||
|
func testItemEncodesEveryContractKeyWithExplicitNulls() throws {
|
||||||
|
let item = ClientMediaItem(
|
||||||
|
externalId: "ID/L0/001", kind: .photo, capturedAt: nil, durationSeconds: nil, width: 0,
|
||||||
|
height: nil, latitude: nil, longitude: nil, albumNames: [], isFavorite: false, fileName: nil)
|
||||||
|
let json = try object(encodeJSON(item))
|
||||||
|
XCTAssertEqual(
|
||||||
|
Set(json.keys),
|
||||||
|
[
|
||||||
|
"externalId", "kind", "capturedAt", "durationMs", "width", "height", "fps", "gpsLat",
|
||||||
|
"gpsLon", "deviceModel", "albumNames", "isFavorite", "fileName",
|
||||||
|
])
|
||||||
|
for key in ["capturedAt", "durationMs", "width", "height", "fps", "gpsLat", "gpsLon", "deviceModel", "fileName"] {
|
||||||
|
XCTAssertTrue(json[key] is NSNull, key)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func testItemNormalizesValues() throws {
|
||||||
|
let item = ClientMediaItem(
|
||||||
|
externalId: "V/L0/001", kind: .livePhoto, capturedAt: Date(timeIntervalSince1970: 1_704_164_645.5),
|
||||||
|
durationSeconds: 12.3456, width: 1920, height: 1080, latitude: 52.520_08, longitude: 13.404_95,
|
||||||
|
albumNames: ["Trip", "", String(repeating: "a", count: 300)] + Array(repeating: "x", count: 60),
|
||||||
|
isFavorite: true, fileName: "IMG_0001.HEIC")
|
||||||
|
let json = try object(encodeJSON(item))
|
||||||
|
XCTAssertEqual(json["kind"] as? String, "live_photo")
|
||||||
|
XCTAssertEqual(json["capturedAt"] as? String, "2024-01-02T03:04:05.500Z")
|
||||||
|
XCTAssertEqual(json["durationMs"] as? Int, 12_346)
|
||||||
|
XCTAssertEqual(json["gpsLat"] as? Double, 52.52)
|
||||||
|
XCTAssertEqual(json["gpsLon"] as? Double, 13.4)
|
||||||
|
let albums = try XCTUnwrap(json["albumNames"] as? [String])
|
||||||
|
XCTAssertEqual(albums.count, 50)
|
||||||
|
XCTAssertEqual(albums[1].count, 200)
|
||||||
|
}
|
||||||
|
|
||||||
|
func testInvalidCoordinatesAreDropped() {
|
||||||
|
XCTAssertNil(roundedCoordinate(latitude: 91, longitude: 0.5))
|
||||||
|
XCTAssertNil(roundedCoordinate(latitude: .nan, longitude: 1))
|
||||||
|
XCTAssertNil(roundedCoordinate(latitude: 0, longitude: 0))
|
||||||
|
XCTAssertNil(roundedCoordinate(latitude: 1, longitude: nil))
|
||||||
|
}
|
||||||
|
|
||||||
|
func testClipCountsUTF16Units() {
|
||||||
|
XCTAssertEqual(clip("abc", maxCharacters: 5), "abc")
|
||||||
|
// Each emoji is two UTF-16 units; a half character is never kept.
|
||||||
|
XCTAssertEqual(clip("😀😀😀", maxCharacters: 5), "😀😀")
|
||||||
|
}
|
||||||
|
|
||||||
|
func testErrorAndProgressLinesAreSingleLineJSON() throws {
|
||||||
|
let line = errorLine(HelperError(.notFound, "No \"asset\"\nhere."))
|
||||||
|
XCTAssertFalse(line.contains("\n"))
|
||||||
|
let error = try XCTUnwrap(try object(line)["error"] as? [String: String])
|
||||||
|
XCTAssertEqual(error["code"], "not_found")
|
||||||
|
XCTAssertEqual(error["message"], "No \"asset\"\nhere.")
|
||||||
|
for (input, expected) in [(0.42424, 0.424), (7, 1), (-1, 0), (.nan, 0)] as [(Double, Double)] {
|
||||||
|
let progress = progressLine(input)
|
||||||
|
XCTAssertFalse(progress.contains("\n"))
|
||||||
|
XCTAssertEqual(try object(progress)["progress"] as? Double ?? -1, expected, accuracy: 1e-9)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func testListAndChangesKeepNullKeys() throws {
|
||||||
|
let list = try object(encodeJSON(ListOutput(items: [], nextCursor: nil, changeToken: nil)))
|
||||||
|
XCTAssertTrue(list["nextCursor"] is NSNull)
|
||||||
|
XCTAssertTrue(list["changeToken"] is NSNull)
|
||||||
|
let changes = try object(encodeJSON(ChangesOutput.fullResync()))
|
||||||
|
XCTAssertTrue(changes["token"] is NSNull)
|
||||||
|
XCTAssertEqual(changes["fullResyncRequired"] as? Bool, true)
|
||||||
|
}
|
||||||
|
|
||||||
|
func testExitCodesAreDistinctPerFailureClass() {
|
||||||
|
let codes = HelperError.Code.allCases.map { HelperError($0, "").exitCode }
|
||||||
|
XCTAssertFalse(codes.contains(0))
|
||||||
|
XCTAssertEqual(HelperError(.notAuthorized, "").exitCode, 77)
|
||||||
|
}
|
||||||
|
|
||||||
|
func testSHA256StreamsFiles() throws {
|
||||||
|
let url = FileManager.default.temporaryDirectory.appendingPathComponent(UUID().uuidString)
|
||||||
|
defer { try? FileManager.default.removeItem(at: url) }
|
||||||
|
try Data("abc".utf8).write(to: url)
|
||||||
|
let digest = try sha256Hex(of: url)
|
||||||
|
XCTAssertEqual(digest.hex, "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad")
|
||||||
|
XCTAssertEqual(digest.sizeBytes, 3)
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -0,0 +1,102 @@
|
|||||||
|
import XCTest
|
||||||
|
|
||||||
|
@testable import PhotosHelperCore
|
||||||
|
|
||||||
|
final class PagingTests: XCTestCase {
|
||||||
|
func testCursorRoundTripsAndIsURLSafe() throws {
|
||||||
|
for key in [
|
||||||
|
SortKey(creationMillis: 1_700_000_000_123, localIdentifier: "9F2A/L0/001"),
|
||||||
|
SortKey(creationMillis: nil, localIdentifier: "undated+/=?"),
|
||||||
|
] {
|
||||||
|
let encoded = ListCursor(key: key).encoded()
|
||||||
|
XCTAssertFalse(encoded.contains { "+/=".contains($0) }, encoded)
|
||||||
|
XCTAssertEqual(try ListCursor.decode(encoded).key, key)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func testCursorRejectsGarbage() {
|
||||||
|
for value in ["", "!!!", base64URLEncode(Data("{\"v\":2,\"d\":1,\"id\":\"x\"}".utf8)),
|
||||||
|
base64URLEncode(Data("{\"v\":1,\"d\":1,\"id\":\"\"}".utf8)), base64URLEncode(Data("[]".utf8))]
|
||||||
|
{
|
||||||
|
XCTAssertThrowsError(try ListCursor.decode(value), value)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func testSortKeyPutsUndatedFirstAndBreaksTiesByIdentifier() {
|
||||||
|
let keys = [
|
||||||
|
SortKey(creationMillis: 2, localIdentifier: "a"),
|
||||||
|
SortKey(creationMillis: 1, localIdentifier: "b"),
|
||||||
|
SortKey(creationMillis: nil, localIdentifier: "z"),
|
||||||
|
SortKey(creationMillis: 1, localIdentifier: "a"),
|
||||||
|
SortKey(creationMillis: nil, localIdentifier: "c"),
|
||||||
|
]
|
||||||
|
XCTAssertEqual(
|
||||||
|
keys.sorted().map { key -> String in
|
||||||
|
let date = key.creationMillis.map { millis in String(millis) } ?? "-"
|
||||||
|
return date + key.localIdentifier
|
||||||
|
},
|
||||||
|
["-c", "-z", "1a", "1b", "2a"])
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Simulates PhotoKit: sorted by date, ties in arbitrary (here reversed) order.
|
||||||
|
private func photoKitOrder(_ keys: [SortKey]) -> [SortKey] {
|
||||||
|
keys.sorted { lhs, rhs in
|
||||||
|
if lhs.creationMillis == rhs.creationMillis { return lhs.localIdentifier > rhs.localIdentifier }
|
||||||
|
return lhs < rhs
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private func listAll(_ source: [SortKey], limit: Int) -> [SortKey] {
|
||||||
|
var cursor: ListCursor?
|
||||||
|
var listed: [SortKey] = []
|
||||||
|
var pages = 0
|
||||||
|
while true {
|
||||||
|
let page = nextPage(count: source.count, key: { source[$0] }, after: cursor, limit: limit)
|
||||||
|
XCTAssertLessThanOrEqual(page.indices.count, limit)
|
||||||
|
listed += page.indices.map { source[$0] }
|
||||||
|
pages += 1
|
||||||
|
guard page.hasMore, let last = listed.last else { break }
|
||||||
|
// The cursor survives a round trip through its encoded form, like the app stores it.
|
||||||
|
cursor = try? ListCursor.decode(ListCursor(key: last).encoded())
|
||||||
|
if pages > source.count + 1 { XCTFail("paging did not terminate"); break }
|
||||||
|
}
|
||||||
|
return listed
|
||||||
|
}
|
||||||
|
|
||||||
|
func testPagesCoverEverythingOnceInStableOrder() {
|
||||||
|
var keys: [SortKey] = []
|
||||||
|
for index in 0..<57 {
|
||||||
|
// Many equal timestamps (bursts) and a few undated assets.
|
||||||
|
let millis: Int64? = index % 11 == 0 ? nil : Int64(index / 4)
|
||||||
|
keys.append(SortKey(creationMillis: millis, localIdentifier: String(format: "id-%03d", index)))
|
||||||
|
}
|
||||||
|
let source = photoKitOrder(keys)
|
||||||
|
for limit in [1, 3, 4, 10, 500] {
|
||||||
|
XCTAssertEqual(listAll(source, limit: limit), keys.sorted(), "limit \(limit)")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func testEmptyLibraryHasNoMorePages() {
|
||||||
|
let page = nextPage(count: 0, key: { _ in fatalError() }, after: nil, limit: 10)
|
||||||
|
XCTAssertEqual(page.indices, [])
|
||||||
|
XCTAssertFalse(page.hasMore)
|
||||||
|
}
|
||||||
|
|
||||||
|
func testCursorAfterLastItemReturnsNothing() {
|
||||||
|
let source = [SortKey(creationMillis: 1, localIdentifier: "a")]
|
||||||
|
let page = nextPage(
|
||||||
|
count: 1, key: { source[$0] }, after: ListCursor(key: source[0]), limit: 10)
|
||||||
|
XCTAssertEqual(page.indices, [])
|
||||||
|
XCTAssertFalse(page.hasMore)
|
||||||
|
}
|
||||||
|
|
||||||
|
func testDeletedCursorItemStillResumesAfterIt() {
|
||||||
|
let source = [
|
||||||
|
SortKey(creationMillis: 1, localIdentifier: "a"),
|
||||||
|
SortKey(creationMillis: 3, localIdentifier: "c"),
|
||||||
|
]
|
||||||
|
let gone = ListCursor(key: SortKey(creationMillis: 2, localIdentifier: "b"))
|
||||||
|
let page = nextPage(count: 2, key: { source[$0] }, after: gone, limit: 10)
|
||||||
|
XCTAssertEqual(page.indices, [1])
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -0,0 +1,83 @@
|
|||||||
|
import XCTest
|
||||||
|
|
||||||
|
@testable import PhotosHelperCore
|
||||||
|
|
||||||
|
final class PathGuardTests: XCTestCase {
|
||||||
|
private var directory: URL!
|
||||||
|
|
||||||
|
override func setUpWithError() throws {
|
||||||
|
directory = FileManager.default.temporaryDirectory.appendingPathComponent(
|
||||||
|
"photos-helper-\(UUID().uuidString)", isDirectory: true)
|
||||||
|
try FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true)
|
||||||
|
}
|
||||||
|
|
||||||
|
override func tearDownWithError() throws {
|
||||||
|
try? FileManager.default.removeItem(at: directory)
|
||||||
|
}
|
||||||
|
|
||||||
|
private func resolved() -> URL { directory.resolvingSymlinksInPath().standardizedFileURL }
|
||||||
|
|
||||||
|
func testWritesDirectChildrenOfTheResolvedDirectory() throws {
|
||||||
|
let target = try resolveExportDestination(
|
||||||
|
directory: directory.path, fileName: "IMG_0001.MOV", temporarySuffix: "t")
|
||||||
|
XCTAssertEqual(target.finalURL.deletingLastPathComponent().path, resolved().path)
|
||||||
|
XCTAssertEqual(target.finalURL.lastPathComponent, "IMG_0001.MOV")
|
||||||
|
XCTAssertEqual(target.temporaryURL.lastPathComponent, ".IMG_0001.MOV.t.partial")
|
||||||
|
}
|
||||||
|
|
||||||
|
func testRejectsRelativeAndTraversingDirectories() {
|
||||||
|
for path in ["relative/dir", "\(directory.path)/../x", "\(directory.path)/./", "/a/../b", ""] {
|
||||||
|
XCTAssertThrowsError(
|
||||||
|
try resolveExportDestination(directory: path, fileName: "a.jpg"), path
|
||||||
|
) { error in
|
||||||
|
XCTAssertEqual((error as? HelperError)?.code, .pathRejected)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func testRejectsMissingDirectoryAndFiles() throws {
|
||||||
|
XCTAssertThrowsError(
|
||||||
|
try resolveExportDestination(directory: directory.path + "/missing", fileName: "a.jpg"))
|
||||||
|
let file = directory.appendingPathComponent("file")
|
||||||
|
try Data().write(to: file)
|
||||||
|
XCTAssertThrowsError(try resolveExportDestination(directory: file.path, fileName: "a.jpg"))
|
||||||
|
}
|
||||||
|
|
||||||
|
func testHostileFileNamesStayInside() throws {
|
||||||
|
for name in ["../../etc/passwd", "/etc/passwd", "..", ".hidden", "a/b\\c:d", "\u{0}x", " "] {
|
||||||
|
let target = try resolveExportDestination(directory: directory.path, fileName: name)
|
||||||
|
XCTAssertEqual(target.finalURL.deletingLastPathComponent().path, resolved().path, name)
|
||||||
|
XCTAssertFalse(target.fileName.hasPrefix("."), name)
|
||||||
|
XCTAssertFalse(target.fileName.contains("/"), name)
|
||||||
|
}
|
||||||
|
XCTAssertEqual(sanitizeFileName("../../etc/passwd"), "_.._etc_passwd")
|
||||||
|
XCTAssertEqual(sanitizeFileName(""), "asset")
|
||||||
|
}
|
||||||
|
|
||||||
|
func testLongNamesKeepTheirExtension() {
|
||||||
|
let name = sanitizeFileName(String(repeating: "é", count: 300) + ".mov")
|
||||||
|
XCTAssertLessThanOrEqual(name.utf8.count, 200)
|
||||||
|
XCTAssertTrue(name.hasSuffix(".mov"))
|
||||||
|
}
|
||||||
|
|
||||||
|
func testNeverOverwritesAnExistingFileOrSymlink() throws {
|
||||||
|
try Data().write(to: directory.appendingPathComponent("taken.jpg"))
|
||||||
|
XCTAssertThrowsError(try resolveExportDestination(directory: directory.path, fileName: "taken.jpg")) {
|
||||||
|
error in XCTAssertEqual((error as? HelperError)?.code, .destinationExists)
|
||||||
|
}
|
||||||
|
try FileManager.default.createSymbolicLink(
|
||||||
|
atPath: directory.appendingPathComponent("link.jpg").path, withDestinationPath: "/nonexistent")
|
||||||
|
XCTAssertThrowsError(try resolveExportDestination(directory: directory.path, fileName: "link.jpg"))
|
||||||
|
}
|
||||||
|
|
||||||
|
func testSymlinkedDirectoryResolvesToItsTarget() throws {
|
||||||
|
let link = directory.appendingPathComponent("link", isDirectory: true)
|
||||||
|
let real = directory.appendingPathComponent("real", isDirectory: true)
|
||||||
|
try FileManager.default.createDirectory(at: real, withIntermediateDirectories: false)
|
||||||
|
try FileManager.default.createSymbolicLink(at: link, withDestinationURL: real)
|
||||||
|
let target = try resolveExportDestination(directory: link.path, fileName: "a.jpg")
|
||||||
|
XCTAssertEqual(
|
||||||
|
target.finalURL.deletingLastPathComponent().path,
|
||||||
|
real.resolvingSymlinksInPath().standardizedFileURL.path)
|
||||||
|
}
|
||||||
|
}
|
||||||
16
apps/desktop/native/photos-helper/entitlements.mac.plist
Normal file
16
apps/desktop/native/photos-helper/entitlements.mac.plist
Normal file
@ -0,0 +1,16 @@
|
|||||||
|
<?xml version="1.0" encoding="UTF-8"?>
|
||||||
|
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||||
|
<plist version="1.0">
|
||||||
|
<dict>
|
||||||
|
<!-- electron-builder's default hardened-runtime entitlements for Electron. -->
|
||||||
|
<key>com.apple.security.cs.allow-jit</key>
|
||||||
|
<true/>
|
||||||
|
<key>com.apple.security.cs.allow-unsigned-executable-memory</key>
|
||||||
|
<true/>
|
||||||
|
<key>com.apple.security.cs.disable-library-validation</key>
|
||||||
|
<true/>
|
||||||
|
<!-- Hardened runtime blocks Photos without this, for the app and the bundled photos-helper. -->
|
||||||
|
<key>com.apple.security.personal-information.photos-library</key>
|
||||||
|
<true/>
|
||||||
|
</dict>
|
||||||
|
</plist>
|
||||||
@ -16,6 +16,8 @@
|
|||||||
"release": "pnpm --filter @rakazo/web build && pnpm build && electron-builder --publish never"
|
"release": "pnpm --filter @rakazo/web build && pnpm build && electron-builder --publish never"
|
||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
|
"@orpc/client": "1.15.0",
|
||||||
|
"@orpc/contract": "1.15.0",
|
||||||
"@rakazo/contracts": "workspace:*",
|
"@rakazo/contracts": "workspace:*",
|
||||||
"electron-updater": "6.8.9"
|
"electron-updater": "6.8.9"
|
||||||
},
|
},
|
||||||
@ -61,7 +63,18 @@
|
|||||||
"dmg",
|
"dmg",
|
||||||
"zip"
|
"zip"
|
||||||
],
|
],
|
||||||
"notarize": true
|
"notarize": true,
|
||||||
|
"entitlements": "native/photos-helper/entitlements.mac.plist",
|
||||||
|
"entitlementsInherit": "native/photos-helper/entitlements.mac.plist",
|
||||||
|
"extendInfo": {
|
||||||
|
"NSPhotoLibraryUsageDescription": "Rakazo adds your photos and videos to the media library on your Rakazo server."
|
||||||
|
},
|
||||||
|
"extraResources": [
|
||||||
|
{
|
||||||
|
"from": "native/photos-helper/.build/apple/Products/Release/photos-helper",
|
||||||
|
"to": "bin/photos-helper"
|
||||||
|
}
|
||||||
|
]
|
||||||
},
|
},
|
||||||
"win": {
|
"win": {
|
||||||
"icon": "assets/icon.ico",
|
"icon": "assets/icon.ico",
|
||||||
|
|||||||
96
apps/desktop/src/browser-preview-policy.test.ts
Normal file
96
apps/desktop/src/browser-preview-policy.test.ts
Normal file
@ -0,0 +1,96 @@
|
|||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
import {
|
||||||
|
previewBounds,
|
||||||
|
previewLoadError,
|
||||||
|
previewShortcut,
|
||||||
|
previewUrl,
|
||||||
|
} from "./browser-preview-policy.js";
|
||||||
|
|
||||||
|
describe("previewUrl", () => {
|
||||||
|
it("accepts only http(s) without credentials", () => {
|
||||||
|
expect(previewUrl("http://localhost:5173")).toBe("http://localhost:5173/");
|
||||||
|
expect(previewUrl("https://example.com/a?b=1")).toBe("https://example.com/a?b=1");
|
||||||
|
for (const value of [
|
||||||
|
"file:///etc/hosts",
|
||||||
|
"javascript:alert(1)",
|
||||||
|
"chrome://settings",
|
||||||
|
"https://user:fake-password@example.com",
|
||||||
|
"not a url",
|
||||||
|
42,
|
||||||
|
`https://example.com/${"a".repeat(9000)}`,
|
||||||
|
]) {
|
||||||
|
expect(previewUrl(value)).toBeNull();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("previewBounds", () => {
|
||||||
|
const content = { width: 1200, height: 800 };
|
||||||
|
|
||||||
|
it("rounds and applies the app zoom factor", () => {
|
||||||
|
expect(previewBounds({ x: 800.4, y: 40.6, width: 399.5, height: 700 }, content)).toEqual({
|
||||||
|
x: 800,
|
||||||
|
y: 41,
|
||||||
|
width: 400,
|
||||||
|
height: 700,
|
||||||
|
});
|
||||||
|
expect(previewBounds({ x: 100, y: 10, width: 200, height: 100 }, content, 1.25)).toEqual({
|
||||||
|
x: 125,
|
||||||
|
y: 13,
|
||||||
|
width: 250,
|
||||||
|
height: 125,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("clamps to the window and rejects empty or malformed rectangles", () => {
|
||||||
|
expect(previewBounds({ x: -50, y: -10, width: 5000, height: 5000 }, content)).toEqual({
|
||||||
|
x: 0,
|
||||||
|
y: 0,
|
||||||
|
width: 1200,
|
||||||
|
height: 800,
|
||||||
|
});
|
||||||
|
expect(previewBounds({ x: 1300, y: 0, width: 100, height: 100 }, content)).toBeNull();
|
||||||
|
expect(previewBounds({ x: 0, y: 0, width: 0, height: 100 }, content)).toBeNull();
|
||||||
|
expect(previewBounds({ x: "1", y: 0, width: 1, height: 1 }, content)).toBeNull();
|
||||||
|
expect(previewBounds({ x: Number.NaN, y: 0, width: 1, height: 1 }, content)).toBeNull();
|
||||||
|
expect(previewBounds(null, content)).toBeNull();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("previewShortcut", () => {
|
||||||
|
const key = (overrides: Partial<Parameters<typeof previewShortcut>[0]>) => ({
|
||||||
|
type: "keyDown",
|
||||||
|
key: "r",
|
||||||
|
control: false,
|
||||||
|
meta: false,
|
||||||
|
alt: false,
|
||||||
|
shift: false,
|
||||||
|
...overrides,
|
||||||
|
});
|
||||||
|
|
||||||
|
it("maps platform browser shortcuts", () => {
|
||||||
|
expect(previewShortcut(key({ meta: true }), "darwin")).toBe("reload");
|
||||||
|
expect(previewShortcut(key({ meta: true, shift: true, key: "R" }), "darwin")).toBe(
|
||||||
|
"hard-reload",
|
||||||
|
);
|
||||||
|
expect(previewShortcut(key({ control: true }), "linux")).toBe("reload");
|
||||||
|
expect(previewShortcut(key({ meta: true, key: "[" }), "darwin")).toBe("back");
|
||||||
|
expect(previewShortcut(key({ alt: true, key: "ArrowRight" }), "win32")).toBe("forward");
|
||||||
|
expect(previewShortcut(key({ key: "F12" }), "linux")).toBe("devtools");
|
||||||
|
expect(previewShortcut(key({ key: "F5" }), "win32")).toBe("reload");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("leaves typing alone", () => {
|
||||||
|
expect(previewShortcut(key({}), "darwin")).toBeNull();
|
||||||
|
expect(previewShortcut(key({ control: true }), "darwin")).toBeNull();
|
||||||
|
expect(previewShortcut(key({ type: "keyUp", meta: true }), "darwin")).toBeNull();
|
||||||
|
expect(previewShortcut(key({ key: "ArrowLeft" }), "linux")).toBeNull();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("previewLoadError", () => {
|
||||||
|
it("ignores superseded navigations", () => {
|
||||||
|
expect(previewLoadError(-3, "ERR_ABORTED")).toBeNull();
|
||||||
|
expect(previewLoadError(-102, "ERR_CONNECTION_REFUSED")).toBe("ERR_CONNECTION_REFUSED");
|
||||||
|
});
|
||||||
|
});
|
||||||
76
apps/desktop/src/browser-preview-policy.ts
Normal file
76
apps/desktop/src/browser-preview-policy.ts
Normal file
@ -0,0 +1,76 @@
|
|||||||
|
import type { RakazoDesktopBrowserBounds } from "@rakazo/contracts";
|
||||||
|
|
||||||
|
/** Only plain http(s) pages load in the in-app browser; no files, schemes or URL credentials. */
|
||||||
|
export function previewUrl(value: unknown): string | null {
|
||||||
|
if (typeof value !== "string" || value.length > 8192) return null;
|
||||||
|
try {
|
||||||
|
const url = new URL(value);
|
||||||
|
if (url.protocol !== "http:" && url.protocol !== "https:") return null;
|
||||||
|
if (url.username || url.password) return null;
|
||||||
|
return url.toString();
|
||||||
|
} catch {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Renderer-reported CSS-pixel bounds, converted to window DIPs and clamped to the content area
|
||||||
|
* so a compromised renderer cannot place the view outside the panel it reserved.
|
||||||
|
*/
|
||||||
|
export function previewBounds(
|
||||||
|
value: unknown,
|
||||||
|
content: { width: number; height: number },
|
||||||
|
zoomFactor = 1,
|
||||||
|
): RakazoDesktopBrowserBounds | null {
|
||||||
|
if (!value || typeof value !== "object") return null;
|
||||||
|
const raw = value as Record<string, unknown>;
|
||||||
|
const numbers = [raw.x, raw.y, raw.width, raw.height];
|
||||||
|
if (!numbers.every((n) => typeof n === "number" && Number.isFinite(n))) return null;
|
||||||
|
const scale = Number.isFinite(zoomFactor) && zoomFactor > 0 ? zoomFactor : 1;
|
||||||
|
const [x, y, width, height] = (numbers as number[]).map((n) => Math.round(n * scale)) as [
|
||||||
|
number,
|
||||||
|
number,
|
||||||
|
number,
|
||||||
|
number,
|
||||||
|
];
|
||||||
|
const left = Math.min(Math.max(0, x), content.width);
|
||||||
|
const top = Math.min(Math.max(0, y), content.height);
|
||||||
|
const right = Math.min(Math.max(left, x + width), content.width);
|
||||||
|
const bottom = Math.min(Math.max(top, y + height), content.height);
|
||||||
|
if (right - left < 1 || bottom - top < 1) return null;
|
||||||
|
return { x: left, y: top, width: right - left, height: bottom - top };
|
||||||
|
}
|
||||||
|
|
||||||
|
export type PreviewShortcut = "reload" | "hard-reload" | "back" | "forward" | "devtools";
|
||||||
|
|
||||||
|
/** Browser keyboard shortcuts while focus is inside the page. */
|
||||||
|
export function previewShortcut(
|
||||||
|
input: {
|
||||||
|
type: string;
|
||||||
|
key: string;
|
||||||
|
control: boolean;
|
||||||
|
meta: boolean;
|
||||||
|
alt: boolean;
|
||||||
|
shift: boolean;
|
||||||
|
},
|
||||||
|
platform: string,
|
||||||
|
): PreviewShortcut | null {
|
||||||
|
if (input.type !== "keyDown") return null;
|
||||||
|
const mod = platform === "darwin" ? input.meta : input.control;
|
||||||
|
const key = input.key.length === 1 ? input.key.toLowerCase() : input.key;
|
||||||
|
if (key === "F5") return input.shift ? "hard-reload" : "reload";
|
||||||
|
if (key === "F12") return "devtools";
|
||||||
|
if (mod && key === "r") return input.shift ? "hard-reload" : "reload";
|
||||||
|
if (mod && input.alt && key === "i") return "devtools";
|
||||||
|
if (platform === "darwin" ? input.meta && !input.alt : input.alt && !input.control) {
|
||||||
|
if (key === (platform === "darwin" ? "[" : "ArrowLeft")) return "back";
|
||||||
|
if (key === (platform === "darwin" ? "]" : "ArrowRight")) return "forward";
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Load failures worth showing; an aborted load (-3) is a superseded navigation, not an error. */
|
||||||
|
export function previewLoadError(errorCode: number, description: string): string | null {
|
||||||
|
if (errorCode === -3 || errorCode === 0) return null;
|
||||||
|
return description || `Load failed (${errorCode})`;
|
||||||
|
}
|
||||||
189
apps/desktop/src/browser-preview.ts
Normal file
189
apps/desktop/src/browser-preview.ts
Normal file
@ -0,0 +1,189 @@
|
|||||||
|
import type { RakazoDesktopBrowserPreviewState } from "@rakazo/contracts";
|
||||||
|
import { type BrowserWindow, WebContentsView } from "electron";
|
||||||
|
import {
|
||||||
|
previewBounds,
|
||||||
|
previewLoadError,
|
||||||
|
previewShortcut,
|
||||||
|
previewUrl,
|
||||||
|
} from "./browser-preview-policy.js";
|
||||||
|
|
||||||
|
/** Pages share one persistent profile, separate from the app's own session and cookies. */
|
||||||
|
const PREVIEW_PARTITION = "persist:rakazo-browser";
|
||||||
|
export const PREVIEW_STATE_CHANNEL = "desktop.browserPreview.state";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The in-app browser's native page: one WebContentsView laid over the panel the app window
|
||||||
|
* reserves. It belongs to one app window at a time and is torn down with it.
|
||||||
|
*/
|
||||||
|
export class BrowserPreviewController {
|
||||||
|
private host: BrowserWindow | null = null;
|
||||||
|
private view: WebContentsView | null = null;
|
||||||
|
private error: string | null = null;
|
||||||
|
|
||||||
|
state(win: BrowserWindow): RakazoDesktopBrowserPreviewState {
|
||||||
|
const contents = this.host === win ? this.view?.webContents : undefined;
|
||||||
|
if (!contents || contents.isDestroyed()) {
|
||||||
|
return {
|
||||||
|
url: "",
|
||||||
|
title: "",
|
||||||
|
loading: false,
|
||||||
|
canGoBack: false,
|
||||||
|
canGoForward: false,
|
||||||
|
error: null,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
const url = contents.getURL();
|
||||||
|
return {
|
||||||
|
url,
|
||||||
|
title: contents.getTitle() === url ? "" : contents.getTitle(),
|
||||||
|
loading: contents.isLoading(),
|
||||||
|
canGoBack: contents.navigationHistory.canGoBack(),
|
||||||
|
canGoForward: contents.navigationHistory.canGoForward(),
|
||||||
|
error: this.error,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
show(win: BrowserWindow, rawBounds: unknown) {
|
||||||
|
const view = this.viewFor(win);
|
||||||
|
const [width, height] = win.getContentSize();
|
||||||
|
const bounds = previewBounds(
|
||||||
|
rawBounds,
|
||||||
|
{ width: width ?? 0, height: height ?? 0 },
|
||||||
|
win.webContents.getZoomFactor(),
|
||||||
|
);
|
||||||
|
if (!bounds) {
|
||||||
|
view.setVisible(false);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
view.setBounds(bounds);
|
||||||
|
view.setVisible(true);
|
||||||
|
}
|
||||||
|
|
||||||
|
hide(win: BrowserWindow) {
|
||||||
|
if (this.host === win) this.view?.setVisible(false);
|
||||||
|
}
|
||||||
|
|
||||||
|
async navigate(win: BrowserWindow, rawUrl: unknown) {
|
||||||
|
const url = previewUrl(rawUrl);
|
||||||
|
if (!url) throw new Error("Only http(s) pages can open here.");
|
||||||
|
this.error = null;
|
||||||
|
// Failures surface through did-fail-load as page state, not as a rejected call.
|
||||||
|
await this.viewFor(win)
|
||||||
|
.webContents.loadURL(url)
|
||||||
|
.catch(() => undefined);
|
||||||
|
}
|
||||||
|
|
||||||
|
back(win: BrowserWindow) {
|
||||||
|
const history = this.contentsFor(win)?.navigationHistory;
|
||||||
|
if (history?.canGoBack()) history.goBack();
|
||||||
|
}
|
||||||
|
|
||||||
|
forward(win: BrowserWindow) {
|
||||||
|
const history = this.contentsFor(win)?.navigationHistory;
|
||||||
|
if (history?.canGoForward()) history.goForward();
|
||||||
|
}
|
||||||
|
|
||||||
|
reload(win: BrowserWindow, ignoreCache = false) {
|
||||||
|
const contents = this.contentsFor(win);
|
||||||
|
if (!contents?.getURL()) return;
|
||||||
|
this.error = null;
|
||||||
|
if (ignoreCache) contents.reloadIgnoringCache();
|
||||||
|
else contents.reload();
|
||||||
|
}
|
||||||
|
|
||||||
|
stop(win: BrowserWindow) {
|
||||||
|
this.contentsFor(win)?.stop();
|
||||||
|
}
|
||||||
|
|
||||||
|
openDevTools(win: BrowserWindow) {
|
||||||
|
const contents = this.contentsFor(win);
|
||||||
|
if (contents?.getURL()) contents.openDevTools({ mode: "detach" });
|
||||||
|
}
|
||||||
|
|
||||||
|
private contentsFor(win: BrowserWindow) {
|
||||||
|
const contents = this.host === win ? this.view?.webContents : undefined;
|
||||||
|
return contents && !contents.isDestroyed() ? contents : undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
private viewFor(win: BrowserWindow): WebContentsView {
|
||||||
|
if (this.host === win && this.view && !this.view.webContents.isDestroyed()) return this.view;
|
||||||
|
this.destroy();
|
||||||
|
const view = new WebContentsView({
|
||||||
|
webPreferences: {
|
||||||
|
partition: PREVIEW_PARTITION,
|
||||||
|
sandbox: true,
|
||||||
|
contextIsolation: true,
|
||||||
|
nodeIntegration: false,
|
||||||
|
webSecurity: true,
|
||||||
|
spellcheck: false,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
view.setVisible(false);
|
||||||
|
const contents = view.webContents;
|
||||||
|
const publish = () => {
|
||||||
|
if (!win.isDestroyed()) win.webContents.send(PREVIEW_STATE_CHANNEL, this.state(win));
|
||||||
|
};
|
||||||
|
// One tab: links that want a new window open here instead.
|
||||||
|
contents.setWindowOpenHandler(({ url }) => {
|
||||||
|
const target = previewUrl(url);
|
||||||
|
if (target) void contents.loadURL(target).catch(() => undefined);
|
||||||
|
return { action: "deny" };
|
||||||
|
});
|
||||||
|
contents.on("will-navigate", (event, url) => {
|
||||||
|
if (!previewUrl(url)) event.preventDefault();
|
||||||
|
});
|
||||||
|
contents.on("will-redirect", (event, url) => {
|
||||||
|
if (!previewUrl(url)) event.preventDefault();
|
||||||
|
});
|
||||||
|
contents.on("did-start-loading", publish);
|
||||||
|
contents.on("did-stop-loading", publish);
|
||||||
|
contents.on("did-navigate", () => {
|
||||||
|
this.error = null;
|
||||||
|
publish();
|
||||||
|
});
|
||||||
|
contents.on("did-navigate-in-page", publish);
|
||||||
|
contents.on("page-title-updated", publish);
|
||||||
|
contents.on("did-fail-load", (_event, code, description, _url, isMainFrame) => {
|
||||||
|
if (!isMainFrame) return;
|
||||||
|
const error = previewLoadError(code, description);
|
||||||
|
if (error === null) return;
|
||||||
|
this.error = error;
|
||||||
|
publish();
|
||||||
|
});
|
||||||
|
contents.on("render-process-gone", (_event, details) => {
|
||||||
|
this.error = details.reason === "clean-exit" ? null : "The page stopped responding.";
|
||||||
|
publish();
|
||||||
|
});
|
||||||
|
contents.on("before-input-event", (event, input) => {
|
||||||
|
const shortcut = previewShortcut(input, process.platform);
|
||||||
|
if (!shortcut) return;
|
||||||
|
event.preventDefault();
|
||||||
|
if (shortcut === "reload" || shortcut === "hard-reload") {
|
||||||
|
this.reload(win, shortcut === "hard-reload");
|
||||||
|
} else if (shortcut === "back") {
|
||||||
|
this.back(win);
|
||||||
|
} else if (shortcut === "forward") {
|
||||||
|
this.forward(win);
|
||||||
|
} else {
|
||||||
|
this.openDevTools(win);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
win.contentView.addChildView(view);
|
||||||
|
win.once("closed", () => {
|
||||||
|
if (this.host === win) this.destroy();
|
||||||
|
});
|
||||||
|
this.host = win;
|
||||||
|
this.view = view;
|
||||||
|
return view;
|
||||||
|
}
|
||||||
|
|
||||||
|
private destroy() {
|
||||||
|
const { host, view } = this;
|
||||||
|
this.host = null;
|
||||||
|
this.view = null;
|
||||||
|
this.error = null;
|
||||||
|
if (!view) return;
|
||||||
|
if (host && !host.isDestroyed()) host.contentView.removeChildView(view);
|
||||||
|
if (!view.webContents.isDestroyed()) view.webContents.close();
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -2,7 +2,10 @@ import { randomBytes } from "node:crypto";
|
|||||||
import { existsSync } from "node:fs";
|
import { existsSync } from "node:fs";
|
||||||
import { readFile, stat, writeFile } from "node:fs/promises";
|
import { readFile, stat, writeFile } from "node:fs/promises";
|
||||||
import path from "node:path";
|
import path from "node:path";
|
||||||
import type { DesktopReachability, DesktopSetup } from "@rakazo/contracts";
|
import { createORPCClient } from "@orpc/client";
|
||||||
|
import { RPCLink } from "@orpc/client/fetch";
|
||||||
|
import type { ContractRouterClient } from "@orpc/contract";
|
||||||
|
import type { AppContract, DesktopReachability, DesktopSetup } from "@rakazo/contracts";
|
||||||
import { LOCAL_SETTINGS_PAGE } from "@rakazo/contracts/local-settings";
|
import { LOCAL_SETTINGS_PAGE } from "@rakazo/contracts/local-settings";
|
||||||
import {
|
import {
|
||||||
app,
|
app,
|
||||||
@ -21,6 +24,7 @@ import {
|
|||||||
LAUNCH_CHECK_DELAY_MS,
|
LAUNCH_CHECK_DELAY_MS,
|
||||||
} from "./auto-update.js";
|
} from "./auto-update.js";
|
||||||
import { openBrowserAuth } from "./browser-auth.js";
|
import { openBrowserAuth } from "./browser-auth.js";
|
||||||
|
import { BrowserPreviewController } from "./browser-preview.js";
|
||||||
import { DOCKER_INSTALL_LINKS, isDesktopSetupLink, runDocker } from "./docker-cli.js";
|
import { DOCKER_INSTALL_LINKS, isDesktopSetupLink, runDocker } from "./docker-cli.js";
|
||||||
import { requestLocalSettings } from "./local-settings.js";
|
import { requestLocalSettings } from "./local-settings.js";
|
||||||
import {
|
import {
|
||||||
@ -31,6 +35,14 @@ import {
|
|||||||
stackDir,
|
stackDir,
|
||||||
stackResourceDir,
|
stackResourceDir,
|
||||||
} from "./local-stack.js";
|
} from "./local-stack.js";
|
||||||
|
import type { MediaIpcEvent } from "./media-ipc.js";
|
||||||
|
import { registerMediaIpc } from "./media-ipc.js";
|
||||||
|
import { createPhotosHelper, resolvePhotosHelperBinary } from "./media-photos-helper.js";
|
||||||
|
import {
|
||||||
|
createChunkUploader,
|
||||||
|
createPhotosSyncFileStore,
|
||||||
|
createPhotosSyncRunner,
|
||||||
|
} from "./media-photos-sync.js";
|
||||||
import { oauthCallbackFrom } from "./oauth-callback.js";
|
import { oauthCallbackFrom } from "./oauth-callback.js";
|
||||||
import {
|
import {
|
||||||
bundledRendererCandidates,
|
bundledRendererCandidates,
|
||||||
@ -1129,7 +1141,90 @@ app.whenReady().then(async () => {
|
|||||||
);
|
);
|
||||||
},
|
},
|
||||||
);
|
);
|
||||||
|
const photosBinary = resolvePhotosHelperBinary({
|
||||||
|
platform: process.platform,
|
||||||
|
packaged: app.isPackaged,
|
||||||
|
resourcesPath: process.resourcesPath,
|
||||||
|
appPath: app.getAppPath(),
|
||||||
|
exists: existsSync,
|
||||||
|
});
|
||||||
|
const photosHelper = photosBinary ? createPhotosHelper(photosBinary) : null;
|
||||||
|
const mediaIpc = registerMediaIpc({
|
||||||
|
ipcMain,
|
||||||
|
platform: process.platform,
|
||||||
|
helper: photosHelper,
|
||||||
|
isTrusted: (raw: MediaIpcEvent) => {
|
||||||
|
const event = raw as Electron.IpcMainInvokeEvent;
|
||||||
|
return fromMainWindow(event) && event.senderFrame === event.sender.mainFrame;
|
||||||
|
},
|
||||||
|
runSync: photosHelper
|
||||||
|
? createPhotosSyncRunner({
|
||||||
|
helper: photosHelper,
|
||||||
|
store: createPhotosSyncFileStore(userDataDir),
|
||||||
|
tempRoot: app.getPath("temp"),
|
||||||
|
label: "Apple Photos",
|
||||||
|
connect: async (spaceId) => {
|
||||||
|
if (currentTargetUrl === null) throw new Error("No server is open.");
|
||||||
|
const origin = new URL(currentTargetUrl).origin;
|
||||||
|
const target = (await resolveSessionForTarget(currentTargetUrl)).value;
|
||||||
|
const fetchWithSession = (input: string | Request, init?: RequestInit) =>
|
||||||
|
target.fetch(input, {
|
||||||
|
...init,
|
||||||
|
credentials: "include",
|
||||||
|
bypassCustomProtocolHandlers: true,
|
||||||
|
});
|
||||||
|
const api: ContractRouterClient<AppContract> = createORPCClient(
|
||||||
|
new RPCLink({
|
||||||
|
url: `${origin}/rpc`,
|
||||||
|
fetch: (request, init) => {
|
||||||
|
const headers = new Headers(request.headers);
|
||||||
|
if (spaceId) headers.set("x-rakazo-space-id", spaceId);
|
||||||
|
return fetchWithSession(new Request(request, { headers }), init);
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
return {
|
||||||
|
origin,
|
||||||
|
api,
|
||||||
|
uploadChunk: createChunkUploader({ origin, spaceId, fetch: fetchWithSession }),
|
||||||
|
};
|
||||||
|
},
|
||||||
|
})
|
||||||
|
: async () => {
|
||||||
|
throw new Error("Apple Photos is unavailable.");
|
||||||
|
},
|
||||||
|
});
|
||||||
|
app.on("before-quit", mediaIpc.cancel);
|
||||||
ipcMain.handle("desktop.platform", () => process.platform);
|
ipcMain.handle("desktop.platform", () => process.platform);
|
||||||
|
const browserPreview = new BrowserPreviewController();
|
||||||
|
// The in-app browser answers only the app window's own top frame, never a page inside it.
|
||||||
|
const previewHost = (event: Electron.IpcMainInvokeEvent) => {
|
||||||
|
if (!fromMainWindow(event) || event.senderFrame !== event.sender.mainFrame) {
|
||||||
|
throw new Error("The in-app browser is not available here.");
|
||||||
|
}
|
||||||
|
return mainWindow!;
|
||||||
|
};
|
||||||
|
ipcMain.handle("desktop.browserPreview.state", (event) =>
|
||||||
|
browserPreview.state(previewHost(event)),
|
||||||
|
);
|
||||||
|
ipcMain.handle("desktop.browserPreview.show", (event, bounds: unknown) =>
|
||||||
|
browserPreview.show(previewHost(event), bounds),
|
||||||
|
);
|
||||||
|
ipcMain.handle("desktop.browserPreview.hide", (event) => browserPreview.hide(previewHost(event)));
|
||||||
|
ipcMain.handle("desktop.browserPreview.navigate", (event, url: unknown) =>
|
||||||
|
browserPreview.navigate(previewHost(event), url),
|
||||||
|
);
|
||||||
|
ipcMain.handle("desktop.browserPreview.back", (event) => browserPreview.back(previewHost(event)));
|
||||||
|
ipcMain.handle("desktop.browserPreview.forward", (event) =>
|
||||||
|
browserPreview.forward(previewHost(event)),
|
||||||
|
);
|
||||||
|
ipcMain.handle("desktop.browserPreview.reload", (event) =>
|
||||||
|
browserPreview.reload(previewHost(event)),
|
||||||
|
);
|
||||||
|
ipcMain.handle("desktop.browserPreview.stop", (event) => browserPreview.stop(previewHost(event)));
|
||||||
|
ipcMain.handle("desktop.browserPreview.openDevTools", (event) =>
|
||||||
|
browserPreview.openDevTools(previewHost(event)),
|
||||||
|
);
|
||||||
ipcMain.handle("desktop.files.pickDirectory", async (event) => {
|
ipcMain.handle("desktop.files.pickDirectory", async (event) => {
|
||||||
if (!fromMainWindow(event)) return null;
|
if (!fromMainWindow(event)) return null;
|
||||||
const result = await dialog.showOpenDialog(windowFrom(event)!, {
|
const result = await dialog.showOpenDialog(windowFrom(event)!, {
|
||||||
|
|||||||
161
apps/desktop/src/media-ipc.test.ts
Normal file
161
apps/desktop/src/media-ipc.test.ts
Normal file
@ -0,0 +1,161 @@
|
|||||||
|
import { describe, expect, it, vi } from "vitest";
|
||||||
|
import type { MediaIpcEvent, MediaIpcOptions } from "./media-ipc.js";
|
||||||
|
import {
|
||||||
|
MEDIA_PHOTOS_CHANNELS,
|
||||||
|
outcomeForError,
|
||||||
|
parseSyncRequest,
|
||||||
|
registerMediaIpc,
|
||||||
|
} from "./media-ipc.js";
|
||||||
|
import type { PhotosHelper } from "./media-photos-helper.js";
|
||||||
|
import { PhotosHelperError } from "./media-photos-helper.js";
|
||||||
|
import type { PhotosSyncResult } from "./media-photos-sync.js";
|
||||||
|
import { PhotosSyncError } from "./media-photos-sync.js";
|
||||||
|
|
||||||
|
type Listener = (event: MediaIpcEvent, ...args: unknown[]) => unknown;
|
||||||
|
|
||||||
|
const result: PhotosSyncResult = {
|
||||||
|
indexed: 2,
|
||||||
|
created: 2,
|
||||||
|
updated: 0,
|
||||||
|
unchanged: 0,
|
||||||
|
uploaded: 0,
|
||||||
|
fullListing: true,
|
||||||
|
};
|
||||||
|
|
||||||
|
function setup(overrides: Partial<MediaIpcOptions> = {}) {
|
||||||
|
const handlers = new Map<string, Listener>();
|
||||||
|
const helper = {
|
||||||
|
authorize: vi.fn(async () => "authorized" as const),
|
||||||
|
} as unknown as PhotosHelper;
|
||||||
|
const options: MediaIpcOptions = {
|
||||||
|
ipcMain: { handle: (channel, listener) => handlers.set(channel, listener) },
|
||||||
|
platform: "darwin",
|
||||||
|
helper,
|
||||||
|
isTrusted: () => true,
|
||||||
|
runSync: vi.fn(async () => result),
|
||||||
|
...overrides,
|
||||||
|
};
|
||||||
|
const registration = registerMediaIpc(options);
|
||||||
|
const sent: [string, unknown][] = [];
|
||||||
|
const event: MediaIpcEvent = {
|
||||||
|
sender: { send: (channel, payload) => sent.push([channel, payload]), isDestroyed: () => false },
|
||||||
|
};
|
||||||
|
const call = (channel: string, ...args: unknown[]) => handlers.get(channel)!(event, ...args);
|
||||||
|
return { handlers, helper, options, registration, call, sent };
|
||||||
|
}
|
||||||
|
|
||||||
|
describe("media IPC on platforms without Apple Photos", () => {
|
||||||
|
it.each(["linux", "win32"])(
|
||||||
|
"reports %s as unavailable and rejects sync and cancel",
|
||||||
|
async (platform) => {
|
||||||
|
const { call, options, handlers } = setup({ platform });
|
||||||
|
expect([...handlers.keys()].sort()).toEqual(
|
||||||
|
[
|
||||||
|
MEDIA_PHOTOS_CHANNELS.cancel,
|
||||||
|
MEDIA_PHOTOS_CHANNELS.status,
|
||||||
|
MEDIA_PHOTOS_CHANNELS.sync,
|
||||||
|
].sort(),
|
||||||
|
);
|
||||||
|
await expect(call(MEDIA_PHOTOS_CHANNELS.status)).resolves.toEqual({ available: false });
|
||||||
|
expect(() => call(MEDIA_PHOTOS_CHANNELS.sync, { spaceId: null })).toThrow(/macOS/);
|
||||||
|
expect(() => call(MEDIA_PHOTOS_CHANNELS.cancel)).toThrow(/macOS/);
|
||||||
|
expect(options.runSync).not.toHaveBeenCalled();
|
||||||
|
expect(options.helper!.authorize).not.toHaveBeenCalled();
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
it("treats a macOS build without the helper as unavailable", async () => {
|
||||||
|
const { call } = setup({ helper: null });
|
||||||
|
await expect(call(MEDIA_PHOTOS_CHANNELS.status)).resolves.toEqual({ available: false });
|
||||||
|
expect(() => call(MEDIA_PHOTOS_CHANNELS.sync, {})).toThrow(/macOS/);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("treats a helper that cannot start as unavailable", async () => {
|
||||||
|
const { call, helper } = setup();
|
||||||
|
vi.mocked(helper.authorize).mockRejectedValueOnce(new PhotosHelperError("unavailable", "x"));
|
||||||
|
await expect(call(MEDIA_PHOTOS_CHANNELS.status)).resolves.toEqual({ available: false });
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("media IPC on macOS", () => {
|
||||||
|
it("reads status without prompting", async () => {
|
||||||
|
const { call, helper } = setup();
|
||||||
|
await expect(call(MEDIA_PHOTOS_CHANNELS.status)).resolves.toEqual({
|
||||||
|
available: true,
|
||||||
|
authorization: "authorized",
|
||||||
|
syncing: false,
|
||||||
|
});
|
||||||
|
expect(helper.authorize).toHaveBeenCalledWith(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("runs one sync at a time, forwards progress and resolves with the outcome", async () => {
|
||||||
|
let finish: (value: PhotosSyncResult) => void = () => {};
|
||||||
|
const runSync = vi.fn(
|
||||||
|
({ onProgress }: Parameters<MediaIpcOptions["runSync"]>[0]) =>
|
||||||
|
new Promise<PhotosSyncResult>((resolve) => {
|
||||||
|
onProgress({ phase: "indexing", indexed: 1, uploadsDone: 0, uploadsTotal: 0 });
|
||||||
|
finish = resolve;
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
const { call, sent } = setup({ runSync });
|
||||||
|
const first = call(MEDIA_PHOTOS_CHANNELS.sync, { spaceId: "space_1" });
|
||||||
|
const second = call(MEDIA_PHOTOS_CHANNELS.sync, { spaceId: "space_1" });
|
||||||
|
expect(second).toBe(first);
|
||||||
|
expect(runSync).toHaveBeenCalledTimes(1);
|
||||||
|
expect(runSync.mock.calls[0]?.[0].spaceId).toBe("space_1");
|
||||||
|
await expect(call(MEDIA_PHOTOS_CHANNELS.status)).resolves.toMatchObject({ syncing: true });
|
||||||
|
expect(sent).toEqual([
|
||||||
|
[
|
||||||
|
MEDIA_PHOTOS_CHANNELS.progress,
|
||||||
|
{ phase: "indexing", indexed: 1, uploadsDone: 0, uploadsTotal: 0 },
|
||||||
|
],
|
||||||
|
]);
|
||||||
|
finish(result);
|
||||||
|
await expect(first).resolves.toEqual({ ok: true, result });
|
||||||
|
await expect(call(MEDIA_PHOTOS_CHANNELS.status)).resolves.toMatchObject({ syncing: false });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("cancels the running sync", async () => {
|
||||||
|
const runSync = vi.fn(
|
||||||
|
({ signal }: Parameters<MediaIpcOptions["runSync"]>[0]) =>
|
||||||
|
new Promise<PhotosSyncResult>((_resolve, reject) => {
|
||||||
|
signal.addEventListener("abort", () =>
|
||||||
|
reject(new PhotosSyncError("cancelled", "Sync cancelled.")),
|
||||||
|
);
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
const { call } = setup({ runSync });
|
||||||
|
const running = call(MEDIA_PHOTOS_CHANNELS.sync, {});
|
||||||
|
await call(MEDIA_PHOTOS_CHANNELS.cancel);
|
||||||
|
await expect(running).resolves.toEqual({
|
||||||
|
ok: false,
|
||||||
|
code: "cancelled",
|
||||||
|
message: "Sync cancelled.",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects untrusted senders and malformed requests", async () => {
|
||||||
|
const { call } = setup({ isTrusted: () => false });
|
||||||
|
expect(() => call(MEDIA_PHOTOS_CHANNELS.sync, {})).toThrow("Not allowed.");
|
||||||
|
await expect(call(MEDIA_PHOTOS_CHANNELS.status)).rejects.toThrow("Not allowed.");
|
||||||
|
expect(() => parseSyncRequest(null)).toThrow();
|
||||||
|
expect(() => parseSyncRequest({ spaceId: "../x" })).toThrow();
|
||||||
|
expect(() => parseSyncRequest({ spaceId: 4 })).toThrow();
|
||||||
|
expect(parseSyncRequest({})).toEqual({ spaceId: null });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("never forwards unexpected error text to the renderer", () => {
|
||||||
|
expect(outcomeForError(new Error("/Users/example/secret path"))).toEqual({
|
||||||
|
ok: false,
|
||||||
|
code: "failed",
|
||||||
|
message: "Apple Photos sync failed.",
|
||||||
|
});
|
||||||
|
expect(
|
||||||
|
outcomeForError(
|
||||||
|
new PhotosHelperError("not_authorized", "Photos access has not been granted."),
|
||||||
|
),
|
||||||
|
).toMatchObject({
|
||||||
|
code: "not_authorized",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
126
apps/desktop/src/media-ipc.ts
Normal file
126
apps/desktop/src/media-ipc.ts
Normal file
@ -0,0 +1,126 @@
|
|||||||
|
import type {
|
||||||
|
PhotosSourceStatus,
|
||||||
|
PhotosSyncOutcome,
|
||||||
|
PhotosSyncProgress,
|
||||||
|
PhotosSyncResult,
|
||||||
|
RakazoDesktopMediaPhotos,
|
||||||
|
} from "@rakazo/contracts";
|
||||||
|
import type { PhotosHelper } from "./media-photos-helper.js";
|
||||||
|
import { PhotosHelperError } from "./media-photos-helper.js";
|
||||||
|
import { PhotosSyncError } from "./media-photos-sync.js";
|
||||||
|
|
||||||
|
/*
|
||||||
|
* IPC for the Apple Photos source. The renderer only expresses intent (sync, cancel)
|
||||||
|
* and renders state; the main process owns the helper, the session and the upload.
|
||||||
|
*/
|
||||||
|
|
||||||
|
export const MEDIA_PHOTOS_CHANNELS = {
|
||||||
|
status: "desktop.media.photos.status",
|
||||||
|
sync: "desktop.media.photos.sync",
|
||||||
|
cancel: "desktop.media.photos.cancel",
|
||||||
|
progress: "desktop.media.photos.progress",
|
||||||
|
} as const;
|
||||||
|
|
||||||
|
export type { PhotosSourceStatus, PhotosSyncOutcome, RakazoDesktopMediaPhotos };
|
||||||
|
|
||||||
|
export interface MediaIpcEvent {
|
||||||
|
sender: { send(channel: string, payload: unknown): void; isDestroyed(): boolean };
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface MediaIpcMain {
|
||||||
|
handle(channel: string, listener: (event: MediaIpcEvent, ...args: unknown[]) => unknown): void;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface MediaIpcOptions {
|
||||||
|
ipcMain: MediaIpcMain;
|
||||||
|
platform: string;
|
||||||
|
/** Null when the helper binary is missing (unbuilt dev checkout). */
|
||||||
|
helper: PhotosHelper | null;
|
||||||
|
/** Same gate as the other app-window handlers: main window, main frame. */
|
||||||
|
isTrusted: (event: MediaIpcEvent) => boolean;
|
||||||
|
runSync: (request: {
|
||||||
|
spaceId: string | null;
|
||||||
|
signal: AbortSignal;
|
||||||
|
onProgress: (progress: PhotosSyncProgress) => void;
|
||||||
|
}) => Promise<PhotosSyncResult>;
|
||||||
|
}
|
||||||
|
|
||||||
|
const UNAVAILABLE = "Apple Photos is only available in the macOS app.";
|
||||||
|
const SPACE_ID = /^[A-Za-z0-9_-]{1,200}$/;
|
||||||
|
|
||||||
|
export function parseSyncRequest(value: unknown): { spaceId: string | null } {
|
||||||
|
if (typeof value !== "object" || value === null) throw new Error("Invalid sync request.");
|
||||||
|
const spaceId = (value as { spaceId?: unknown }).spaceId ?? null;
|
||||||
|
if (spaceId !== null && (typeof spaceId !== "string" || !SPACE_ID.test(spaceId))) {
|
||||||
|
throw new Error("Invalid sync request.");
|
||||||
|
}
|
||||||
|
return { spaceId };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Registers the handlers; `cancel()` stops a running sync, e.g. before quit. */
|
||||||
|
export function registerMediaIpc(options: MediaIpcOptions): { cancel: () => void } {
|
||||||
|
const available = options.platform === "darwin" && options.helper !== null;
|
||||||
|
let running: { controller: AbortController; done: Promise<PhotosSyncOutcome> } | null = null;
|
||||||
|
|
||||||
|
const requireTrusted = (event: MediaIpcEvent) => {
|
||||||
|
if (!options.isTrusted(event)) throw new Error("Not allowed.");
|
||||||
|
};
|
||||||
|
|
||||||
|
options.ipcMain.handle(MEDIA_PHOTOS_CHANNELS.status, async (event) => {
|
||||||
|
requireTrusted(event);
|
||||||
|
if (!available) return { available: false } satisfies PhotosSourceStatus;
|
||||||
|
try {
|
||||||
|
const authorization = await options.helper!.authorize(false);
|
||||||
|
return { available: true, authorization, syncing: running !== null };
|
||||||
|
} catch (error) {
|
||||||
|
if (error instanceof PhotosHelperError && error.code === "unavailable") {
|
||||||
|
return { available: false } satisfies PhotosSourceStatus;
|
||||||
|
}
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
options.ipcMain.handle(MEDIA_PHOTOS_CHANNELS.sync, (event, request: unknown) => {
|
||||||
|
requireTrusted(event);
|
||||||
|
if (!available) throw new Error(UNAVAILABLE);
|
||||||
|
const { spaceId } = parseSyncRequest(request);
|
||||||
|
// A second click joins the running sync instead of starting a parallel one.
|
||||||
|
if (running) return running.done;
|
||||||
|
const controller = new AbortController();
|
||||||
|
const done = options
|
||||||
|
.runSync({
|
||||||
|
spaceId,
|
||||||
|
signal: controller.signal,
|
||||||
|
onProgress: (progress) => {
|
||||||
|
if (!event.sender.isDestroyed()) {
|
||||||
|
event.sender.send(MEDIA_PHOTOS_CHANNELS.progress, progress);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
})
|
||||||
|
.then(
|
||||||
|
(result): PhotosSyncOutcome => ({ ok: true, result }),
|
||||||
|
(error: unknown): PhotosSyncOutcome => outcomeForError(error),
|
||||||
|
)
|
||||||
|
.finally(() => {
|
||||||
|
if (running?.controller === controller) running = null;
|
||||||
|
});
|
||||||
|
running = { controller, done };
|
||||||
|
return done;
|
||||||
|
});
|
||||||
|
|
||||||
|
options.ipcMain.handle(MEDIA_PHOTOS_CHANNELS.cancel, (event) => {
|
||||||
|
requireTrusted(event);
|
||||||
|
if (!available) throw new Error(UNAVAILABLE);
|
||||||
|
running?.controller.abort();
|
||||||
|
});
|
||||||
|
|
||||||
|
return { cancel: () => running?.controller.abort() };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Only our own fixed messages reach the renderer; anything else is generic. */
|
||||||
|
export function outcomeForError(error: unknown): PhotosSyncOutcome {
|
||||||
|
if (error instanceof PhotosSyncError || error instanceof PhotosHelperError) {
|
||||||
|
return { ok: false, code: error.code, message: error.message };
|
||||||
|
}
|
||||||
|
return { ok: false, code: "failed", message: "Apple Photos sync failed." };
|
||||||
|
}
|
||||||
437
apps/desktop/src/media-photos-helper.test.ts
Normal file
437
apps/desktop/src/media-photos-helper.test.ts
Normal file
@ -0,0 +1,437 @@
|
|||||||
|
import { EventEmitter } from "node:events";
|
||||||
|
import { mkdtemp, rm, symlink, writeFile } from "node:fs/promises";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import path from "node:path";
|
||||||
|
import { PassThrough } from "node:stream";
|
||||||
|
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||||
|
import type { HelperProcess, SpawnHelper } from "./media-photos-helper.js";
|
||||||
|
import {
|
||||||
|
changesArgs,
|
||||||
|
createPhotosHelper,
|
||||||
|
exportArgs,
|
||||||
|
helperEnv,
|
||||||
|
listArgs,
|
||||||
|
PhotosHelperError,
|
||||||
|
parseAssetOutput,
|
||||||
|
parseAuthorizeOutput,
|
||||||
|
parseChangesOutput,
|
||||||
|
parseExportOutput,
|
||||||
|
parseHelperError,
|
||||||
|
parseListOutput,
|
||||||
|
parseProgressLine,
|
||||||
|
photosHelperCandidates,
|
||||||
|
resolvePhotosHelperBinary,
|
||||||
|
runPhotosHelper,
|
||||||
|
} from "./media-photos-helper.js";
|
||||||
|
|
||||||
|
const item = (overrides: Record<string, unknown> = {}) => ({
|
||||||
|
externalId: "9F2A61B0-0000-4000-8000-000000000001/L0/001",
|
||||||
|
kind: "video",
|
||||||
|
capturedAt: "2024-01-02T03:04:05.500Z",
|
||||||
|
durationMs: 12_346,
|
||||||
|
width: 1920,
|
||||||
|
height: 1080,
|
||||||
|
fps: null,
|
||||||
|
gpsLat: 52.52,
|
||||||
|
gpsLon: 13.4,
|
||||||
|
deviceModel: null,
|
||||||
|
albumNames: ["Trip"],
|
||||||
|
isFavorite: true,
|
||||||
|
fileName: "IMG_0001.MOV",
|
||||||
|
...overrides,
|
||||||
|
});
|
||||||
|
|
||||||
|
const listJson = (items: unknown[], nextCursor: unknown = null, changeToken: unknown = "tok") =>
|
||||||
|
JSON.stringify({ items, nextCursor, changeToken });
|
||||||
|
|
||||||
|
function expectCode(run: () => unknown, code: string) {
|
||||||
|
try {
|
||||||
|
run();
|
||||||
|
} catch (error) {
|
||||||
|
expect(error).toBeInstanceOf(PhotosHelperError);
|
||||||
|
expect((error as PhotosHelperError).code).toBe(code);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
throw new Error("expected a PhotosHelperError");
|
||||||
|
}
|
||||||
|
|
||||||
|
describe("photos helper output parsing", () => {
|
||||||
|
it("accepts a list page that matches the shared item contract", () => {
|
||||||
|
const page = parseListOutput(
|
||||||
|
listJson([item(), item({ kind: "photo", durationMs: null })], "c1"),
|
||||||
|
2,
|
||||||
|
);
|
||||||
|
expect(page.nextCursor).toBe("c1");
|
||||||
|
expect(page.changeToken).toBe("tok");
|
||||||
|
expect(page.items).toHaveLength(2);
|
||||||
|
expect(page.items[0]).toEqual(item());
|
||||||
|
});
|
||||||
|
|
||||||
|
it("accepts the last page of an empty library", () => {
|
||||||
|
expect(parseListOutput(listJson([], null, null), 500)).toEqual({
|
||||||
|
items: [],
|
||||||
|
nextCursor: null,
|
||||||
|
changeToken: null,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each([
|
||||||
|
["not JSON", "items: []"],
|
||||||
|
["an array document", "[]"],
|
||||||
|
["truncated JSON", '{"items":[{"externalId":'],
|
||||||
|
["missing items", JSON.stringify({ nextCursor: null, changeToken: null })],
|
||||||
|
["an item missing a key", listJson([{ ...item(), width: undefined }])],
|
||||||
|
["an unknown kind", listJson([item({ kind: "audio" })])],
|
||||||
|
["out-of-range GPS", listJson([item({ gpsLat: 91 })])],
|
||||||
|
["a non-UTC-parsable date", listJson([item({ capturedAt: "yesterday" })])],
|
||||||
|
["a negative duration", listJson([item({ durationMs: -1 })])],
|
||||||
|
["an empty external id", listJson([item({ externalId: "" })])],
|
||||||
|
["an empty cursor", listJson([item()], "")],
|
||||||
|
["a numeric cursor", listJson([item()], 3)],
|
||||||
|
["too many items", listJson([item(), item(), item()])],
|
||||||
|
])("rejects %s", (_name, text) => {
|
||||||
|
expectCode(() => parseListOutput(text, 2), "invalid_output");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("parses authorization states and rejects unknown ones", () => {
|
||||||
|
expect(parseAuthorizeOutput('{"status":"limited"}')).toBe("limited");
|
||||||
|
expect(parseAuthorizeOutput('{"status":"notDetermined"}')).toBe("notDetermined");
|
||||||
|
expectCode(() => parseAuthorizeOutput('{"status":"maybe"}'), "invalid_output");
|
||||||
|
expectCode(() => parseAuthorizeOutput(""), "invalid_output");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("parses asset details", () => {
|
||||||
|
const detail = parseAssetOutput(
|
||||||
|
JSON.stringify({
|
||||||
|
item: item(),
|
||||||
|
modifiedAt: "2024-02-01T00:00:00.000Z",
|
||||||
|
hidden: false,
|
||||||
|
resources: [{ type: "video", fileName: "IMG_0001.MOV", uti: "com.apple.quicktime-movie" }],
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
expect(detail.resources[0]?.type).toBe("video");
|
||||||
|
expectCode(
|
||||||
|
() => parseAssetOutput(JSON.stringify({ item: item(), modifiedAt: null, resources: [] })),
|
||||||
|
"invalid_output",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("parses changes and requires a token unless a full relisting is needed", () => {
|
||||||
|
expect(
|
||||||
|
parseChangesOutput(
|
||||||
|
JSON.stringify({
|
||||||
|
token: "t2",
|
||||||
|
changed: [item()],
|
||||||
|
deleted: ["x/L0/001"],
|
||||||
|
fullResyncRequired: false,
|
||||||
|
}),
|
||||||
|
),
|
||||||
|
).toMatchObject({ token: "t2", deleted: ["x/L0/001"], fullResyncRequired: false });
|
||||||
|
expect(
|
||||||
|
parseChangesOutput(
|
||||||
|
JSON.stringify({ token: null, changed: [], deleted: [], fullResyncRequired: true }),
|
||||||
|
).fullResyncRequired,
|
||||||
|
).toBe(true);
|
||||||
|
expectCode(
|
||||||
|
() =>
|
||||||
|
parseChangesOutput(
|
||||||
|
JSON.stringify({ token: null, changed: [], deleted: [], fullResyncRequired: false }),
|
||||||
|
),
|
||||||
|
"invalid_output",
|
||||||
|
);
|
||||||
|
expectCode(
|
||||||
|
() =>
|
||||||
|
parseChangesOutput(
|
||||||
|
JSON.stringify({ token: "t", changed: [], deleted: [7], fullResyncRequired: false }),
|
||||||
|
),
|
||||||
|
"invalid_output",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("maps helper error lines to codes", () => {
|
||||||
|
const stderr =
|
||||||
|
'{"progress":0.5}\n{"error":{"code":"not_authorized","message":"Photos access has not been granted."}}\n';
|
||||||
|
const error = parseHelperError(stderr, 77);
|
||||||
|
expect(error.code).toBe("not_authorized");
|
||||||
|
expect(error.message).toBe("Photos access has not been granted.");
|
||||||
|
expect(parseHelperError('{"error":{"code":"internal","message":"x"}}', 70).code).toBe("failed");
|
||||||
|
expect(parseHelperError("Segmentation fault", 139).message).toContain("139");
|
||||||
|
expect(parseHelperError('{"error":"oops"}', 1).code).toBe("failed");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("reads progress lines only", () => {
|
||||||
|
expect(parseProgressLine('{"progress":0.42}')).toBe(0.42);
|
||||||
|
expect(parseProgressLine('{"progress":2}')).toBeNull();
|
||||||
|
expect(parseProgressLine('{"error":{"code":"usage"}}')).toBeNull();
|
||||||
|
expect(parseProgressLine('{"progress":')).toBeNull();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("photos helper export result", () => {
|
||||||
|
let dir: string;
|
||||||
|
let other: string;
|
||||||
|
|
||||||
|
beforeEach(async () => {
|
||||||
|
dir = await mkdtemp(path.join(tmpdir(), "rakazo-photos-export-"));
|
||||||
|
other = await mkdtemp(path.join(tmpdir(), "rakazo-photos-other-"));
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(async () => {
|
||||||
|
await rm(dir, { recursive: true, force: true });
|
||||||
|
await rm(other, { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = (file: string, fileName = path.basename(file)) =>
|
||||||
|
JSON.stringify({ path: file, fileName, sizeBytes: 3, sha256: "a".repeat(64) });
|
||||||
|
|
||||||
|
it("accepts a file directly inside the export folder", async () => {
|
||||||
|
const file = path.join(dir, "IMG_0001.MOV");
|
||||||
|
await writeFile(file, "abc");
|
||||||
|
await expect(parseExportOutput(result(file), dir)).resolves.toMatchObject({
|
||||||
|
fileName: "IMG_0001.MOV",
|
||||||
|
sizeBytes: 3,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects files outside the export folder, including through symlinks", async () => {
|
||||||
|
const outside = path.join(other, "secret.txt");
|
||||||
|
await writeFile(outside, "abc");
|
||||||
|
await expect(parseExportOutput(result(outside), dir)).rejects.toMatchObject({
|
||||||
|
code: "path_rejected",
|
||||||
|
});
|
||||||
|
const link = path.join(dir, "link.MOV");
|
||||||
|
await symlink(outside, link);
|
||||||
|
await expect(parseExportOutput(result(link), dir)).rejects.toMatchObject({
|
||||||
|
code: "path_rejected",
|
||||||
|
});
|
||||||
|
const nested = path.join(dir, "..", path.basename(other), "secret.txt");
|
||||||
|
await expect(parseExportOutput(result(nested), dir)).rejects.toMatchObject({
|
||||||
|
code: "path_rejected",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects malformed results", async () => {
|
||||||
|
const file = path.join(dir, "a.jpg");
|
||||||
|
await writeFile(file, "abc");
|
||||||
|
for (const text of [
|
||||||
|
JSON.stringify({ path: file, fileName: "a.jpg", sizeBytes: 3, sha256: "XYZ" }),
|
||||||
|
JSON.stringify({ path: "a.jpg", fileName: "a.jpg", sizeBytes: 3, sha256: "a".repeat(64) }),
|
||||||
|
JSON.stringify({ path: file, fileName: "a.jpg", sizeBytes: -1, sha256: "a".repeat(64) }),
|
||||||
|
JSON.stringify({
|
||||||
|
path: path.join(dir, "missing.jpg"),
|
||||||
|
fileName: "missing.jpg",
|
||||||
|
sizeBytes: 3,
|
||||||
|
sha256: "a".repeat(64),
|
||||||
|
}),
|
||||||
|
]) {
|
||||||
|
await expect(parseExportOutput(text, dir)).rejects.toMatchObject({ code: "invalid_output" });
|
||||||
|
}
|
||||||
|
await expect(parseExportOutput(result(file, "b.jpg"), dir)).rejects.toMatchObject({
|
||||||
|
code: "path_rejected",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("photos helper argv", () => {
|
||||||
|
it("builds fixed argv arrays", () => {
|
||||||
|
expect(listArgs({ cursor: null, limit: 500 })).toEqual(["list", "--limit", "500"]);
|
||||||
|
expect(listArgs({ cursor: "abc_-", limit: 10, since: "2024-01-01T00:00:00Z" })).toEqual([
|
||||||
|
"list",
|
||||||
|
"--limit",
|
||||||
|
"10",
|
||||||
|
"--cursor",
|
||||||
|
"abc_-",
|
||||||
|
"--since",
|
||||||
|
"2024-01-01T00:00:00Z",
|
||||||
|
]);
|
||||||
|
expect(exportArgs("ID/L0/001", "/tmp/rakazo-photos-1")).toEqual([
|
||||||
|
"export",
|
||||||
|
"ID/L0/001",
|
||||||
|
"--dest",
|
||||||
|
"/tmp/rakazo-photos-1",
|
||||||
|
"--original",
|
||||||
|
]);
|
||||||
|
expect(changesArgs("tok")).toEqual(["changes", "--token", "tok"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps shell syntax as inert data and rejects option-like values", () => {
|
||||||
|
// No shell runs, so metacharacters stay one literal argument.
|
||||||
|
expect(exportArgs("$(rm -rf ~);`x`", "/tmp/a b")[1]).toBe("$(rm -rf ~);`x`");
|
||||||
|
expectCode(() => exportArgs("--dest", "/tmp"), "usage");
|
||||||
|
expectCode(() => listArgs({ cursor: "--limit", limit: 1 }), "usage");
|
||||||
|
expectCode(() => changesArgs("a\nb"), "usage");
|
||||||
|
expectCode(() => listArgs({ cursor: null, limit: 501 }), "usage");
|
||||||
|
expectCode(() => exportArgs("ID", "relative/dir"), "path_rejected");
|
||||||
|
expectCode(() => exportArgs("ID", "/tmp/../etc"), "path_rejected");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("passes only the environment PhotoKit needs", () => {
|
||||||
|
expect(
|
||||||
|
helperEnv({ HOME: "/Users/example", TMPDIR: "/tmp", OPENAI_API_KEY: "secret", PATH: "/bin" }),
|
||||||
|
).toEqual({ HOME: "/Users/example", TMPDIR: "/tmp" });
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("photos helper binary", () => {
|
||||||
|
it("is only available on macOS", () => {
|
||||||
|
const options = {
|
||||||
|
packaged: true,
|
||||||
|
resourcesPath: "/App/Resources",
|
||||||
|
appPath: "/app",
|
||||||
|
exists: () => true,
|
||||||
|
};
|
||||||
|
expect(resolvePhotosHelperBinary({ ...options, platform: "linux" })).toBeNull();
|
||||||
|
expect(resolvePhotosHelperBinary({ ...options, platform: "win32" })).toBeNull();
|
||||||
|
expect(resolvePhotosHelperBinary({ ...options, platform: "darwin" })).toBe(
|
||||||
|
"/App/Resources/bin/photos-helper",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("falls back to the development build and reports a missing build", () => {
|
||||||
|
const options = {
|
||||||
|
platform: "darwin",
|
||||||
|
packaged: false,
|
||||||
|
resourcesPath: "/r",
|
||||||
|
appPath: "/repo/apps/desktop",
|
||||||
|
};
|
||||||
|
expect(photosHelperCandidates(options)).toEqual([
|
||||||
|
"/repo/apps/desktop/native/photos-helper/.build/apple/Products/Release/photos-helper",
|
||||||
|
"/repo/apps/desktop/native/photos-helper/.build/release/photos-helper",
|
||||||
|
]);
|
||||||
|
expect(
|
||||||
|
resolvePhotosHelperBinary({
|
||||||
|
...options,
|
||||||
|
exists: (file) => file.endsWith(".build/release/photos-helper"),
|
||||||
|
}),
|
||||||
|
).toBe("/repo/apps/desktop/native/photos-helper/.build/release/photos-helper");
|
||||||
|
expect(resolvePhotosHelperBinary({ ...options, exists: () => false })).toBeNull();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
class FakeProcess extends EventEmitter implements HelperProcess {
|
||||||
|
stdout = new PassThrough();
|
||||||
|
stderr = new PassThrough();
|
||||||
|
kill = vi.fn(() => {
|
||||||
|
queueMicrotask(() => this.emit("close", null));
|
||||||
|
return true;
|
||||||
|
});
|
||||||
|
exit(code: number, stdout = "", stderr = "") {
|
||||||
|
if (stdout) this.stdout.write(stdout);
|
||||||
|
if (stderr) this.stderr.write(stderr);
|
||||||
|
setImmediate(() => this.emit("close", code));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function fakeSpawn() {
|
||||||
|
const processes: FakeProcess[] = [];
|
||||||
|
const spawn = vi.fn<SpawnHelper>(() => {
|
||||||
|
const child = new FakeProcess();
|
||||||
|
processes.push(child);
|
||||||
|
return child;
|
||||||
|
});
|
||||||
|
return { spawn, processes };
|
||||||
|
}
|
||||||
|
|
||||||
|
describe("photos helper process", () => {
|
||||||
|
it("spawns the binary with argv and no shell", async () => {
|
||||||
|
const { spawn, processes } = fakeSpawn();
|
||||||
|
const helper = createPhotosHelper("/App/Resources/bin/photos-helper", spawn);
|
||||||
|
const pending = helper.list({ cursor: null, limit: 2 });
|
||||||
|
processes[0]!.exit(0, listJson([item()]));
|
||||||
|
await expect(pending).resolves.toMatchObject({ items: [item()] });
|
||||||
|
expect(spawn).toHaveBeenCalledWith(
|
||||||
|
"/App/Resources/bin/photos-helper",
|
||||||
|
["list", "--limit", "2"],
|
||||||
|
expect.objectContaining({ shell: false, stdio: ["ignore", "pipe", "pipe"] }),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("maps a failing exit to the helper's error code", async () => {
|
||||||
|
const { spawn, processes } = fakeSpawn();
|
||||||
|
const pending = createPhotosHelper("/bin/photos-helper", spawn).asset("ID/L0/001");
|
||||||
|
processes[0]!.exit(
|
||||||
|
66,
|
||||||
|
"",
|
||||||
|
'{"error":{"code":"not_found","message":"No photo or video with that identifier."}}\n',
|
||||||
|
);
|
||||||
|
await expect(pending).rejects.toMatchObject({ code: "not_found" });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("reports export progress and validates the result", async () => {
|
||||||
|
const dir = await mkdtemp(path.join(tmpdir(), "rakazo-photos-run-"));
|
||||||
|
try {
|
||||||
|
const file = path.join(dir, "IMG_0001.MOV");
|
||||||
|
await writeFile(file, "abc");
|
||||||
|
const { spawn, processes } = fakeSpawn();
|
||||||
|
const progress: number[] = [];
|
||||||
|
const pending = createPhotosHelper("/bin/photos-helper", spawn).exportOriginal("ID", dir, {
|
||||||
|
onProgress: (value) => progress.push(value),
|
||||||
|
});
|
||||||
|
const child = processes[0]!;
|
||||||
|
child.stderr.write('{"progress":0}\n{"progress":0.5');
|
||||||
|
child.stderr.write('}\nnoise\n{"progress":1}\n');
|
||||||
|
child.exit(
|
||||||
|
0,
|
||||||
|
JSON.stringify({
|
||||||
|
path: file,
|
||||||
|
fileName: "IMG_0001.MOV",
|
||||||
|
sizeBytes: 3,
|
||||||
|
sha256: "b".repeat(64),
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
await expect(pending).resolves.toMatchObject({ sizeBytes: 3 });
|
||||||
|
expect(progress).toEqual([0, 0.5, 1]);
|
||||||
|
} finally {
|
||||||
|
await rm(dir, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("kills the helper on abort and on timeout", async () => {
|
||||||
|
const { spawn, processes } = fakeSpawn();
|
||||||
|
const controller = new AbortController();
|
||||||
|
const aborted = runPhotosHelper("/bin/h", ["list"], {
|
||||||
|
spawn,
|
||||||
|
timeoutMs: 60_000,
|
||||||
|
signal: controller.signal,
|
||||||
|
});
|
||||||
|
controller.abort();
|
||||||
|
await expect(aborted).rejects.toMatchObject({ code: "cancelled" });
|
||||||
|
expect(processes[0]!.kill).toHaveBeenCalledWith("SIGTERM");
|
||||||
|
|
||||||
|
const timedOut = runPhotosHelper("/bin/h", ["list"], { spawn, timeoutMs: 5 });
|
||||||
|
await expect(timedOut).rejects.toMatchObject({ code: "timeout" });
|
||||||
|
expect(processes[1]!.kill).toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("fails a silent export through the idle limit", async () => {
|
||||||
|
const { spawn, processes } = fakeSpawn();
|
||||||
|
const pending = runPhotosHelper("/bin/h", ["export"], {
|
||||||
|
spawn,
|
||||||
|
timeoutMs: 60_000,
|
||||||
|
idleTimeoutMs: 5,
|
||||||
|
});
|
||||||
|
await expect(pending).rejects.toMatchObject({ code: "timeout" });
|
||||||
|
expect(processes[0]!.kill).toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("does not start when already aborted and reports spawn failures", async () => {
|
||||||
|
const { spawn, processes } = fakeSpawn();
|
||||||
|
const controller = new AbortController();
|
||||||
|
controller.abort();
|
||||||
|
await expect(
|
||||||
|
runPhotosHelper("/bin/h", ["list"], { spawn, timeoutMs: 1000, signal: controller.signal }),
|
||||||
|
).rejects.toMatchObject({ code: "cancelled" });
|
||||||
|
expect(spawn).not.toHaveBeenCalled();
|
||||||
|
|
||||||
|
const pending = runPhotosHelper("/bin/h", ["list"], { spawn, timeoutMs: 1000 });
|
||||||
|
processes[0]!.emit("error", new Error("ENOENT"));
|
||||||
|
await expect(pending).rejects.toMatchObject({ code: "unavailable" });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects oversized output", async () => {
|
||||||
|
const { spawn, processes } = fakeSpawn();
|
||||||
|
const pending = runPhotosHelper("/bin/h", ["list"], { spawn, timeoutMs: 1000 });
|
||||||
|
processes[0]!.stdout.write(Buffer.alloc(33 * 1024 * 1024));
|
||||||
|
await expect(pending).rejects.toMatchObject({ code: "invalid_output" });
|
||||||
|
});
|
||||||
|
});
|
||||||
512
apps/desktop/src/media-photos-helper.ts
Normal file
512
apps/desktop/src/media-photos-helper.ts
Normal file
@ -0,0 +1,512 @@
|
|||||||
|
import { spawn as nodeSpawn } from "node:child_process";
|
||||||
|
import { realpath } from "node:fs/promises";
|
||||||
|
import path from "node:path";
|
||||||
|
import type { Readable } from "node:stream";
|
||||||
|
import type { ClientMediaItem, PhotosAuthorization } from "@rakazo/contracts";
|
||||||
|
import { ClientMediaItemSchema } from "@rakazo/contracts";
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Client for the macOS `photos-helper` CLI (apps/desktop/native/photos-helper). The
|
||||||
|
* helper runs with a fixed argv and no shell; every byte it prints is validated before
|
||||||
|
* the app uses it, so a malformed or unexpected answer fails instead of being guessed at.
|
||||||
|
*/
|
||||||
|
|
||||||
|
export type { PhotosAuthorization };
|
||||||
|
|
||||||
|
export interface PhotosListPage {
|
||||||
|
items: ClientMediaItem[];
|
||||||
|
nextCursor: string | null;
|
||||||
|
/** Library change token from before this page was read. */
|
||||||
|
changeToken: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface PhotosAssetDetail {
|
||||||
|
item: ClientMediaItem;
|
||||||
|
modifiedAt: string | null;
|
||||||
|
hidden: boolean;
|
||||||
|
resources: { type: string; fileName: string; uti: string }[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface PhotosChanges {
|
||||||
|
token: string | null;
|
||||||
|
changed: ClientMediaItem[];
|
||||||
|
deleted: string[];
|
||||||
|
fullResyncRequired: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface PhotosExport {
|
||||||
|
path: string;
|
||||||
|
fileName: string;
|
||||||
|
sizeBytes: number;
|
||||||
|
sha256: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export type PhotosHelperErrorCode =
|
||||||
|
| "unavailable"
|
||||||
|
| "usage"
|
||||||
|
| "not_authorized"
|
||||||
|
| "not_found"
|
||||||
|
| "path_rejected"
|
||||||
|
| "destination_exists"
|
||||||
|
| "network"
|
||||||
|
| "export_failed"
|
||||||
|
| "invalid_output"
|
||||||
|
| "timeout"
|
||||||
|
| "cancelled"
|
||||||
|
| "failed";
|
||||||
|
|
||||||
|
const HELPER_CODES = new Set<PhotosHelperErrorCode>([
|
||||||
|
"usage",
|
||||||
|
"not_authorized",
|
||||||
|
"not_found",
|
||||||
|
"path_rejected",
|
||||||
|
"destination_exists",
|
||||||
|
"network",
|
||||||
|
"export_failed",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export class PhotosHelperError extends Error {
|
||||||
|
constructor(
|
||||||
|
readonly code: PhotosHelperErrorCode,
|
||||||
|
message: string,
|
||||||
|
) {
|
||||||
|
super(message);
|
||||||
|
this.name = "PhotosHelperError";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export const PHOTOS_HELPER_NAME = "photos-helper";
|
||||||
|
/** Matches the contract's batch limit for `media.sources.pushItems`. */
|
||||||
|
export const PHOTOS_MAX_PAGE = 500;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Packaged apps ship the universal helper in Resources/bin; development uses the
|
||||||
|
* SwiftPM output (universal `--arch` build first, then the host-arch build).
|
||||||
|
*/
|
||||||
|
export function photosHelperCandidates(options: {
|
||||||
|
packaged: boolean;
|
||||||
|
resourcesPath: string;
|
||||||
|
appPath: string;
|
||||||
|
}): string[] {
|
||||||
|
if (options.packaged) return [path.join(options.resourcesPath, "bin", PHOTOS_HELPER_NAME)];
|
||||||
|
const build = path.join(options.appPath, "native", "photos-helper", ".build");
|
||||||
|
return [
|
||||||
|
path.join(build, "apple", "Products", "Release", PHOTOS_HELPER_NAME),
|
||||||
|
path.join(build, "release", PHOTOS_HELPER_NAME),
|
||||||
|
];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Null off macOS or when the helper was not built; the Apple Photos source is then hidden. */
|
||||||
|
export function resolvePhotosHelperBinary(options: {
|
||||||
|
platform: string;
|
||||||
|
packaged: boolean;
|
||||||
|
resourcesPath: string;
|
||||||
|
appPath: string;
|
||||||
|
exists: (file: string) => boolean;
|
||||||
|
}): string | null {
|
||||||
|
if (options.platform !== "darwin") return null;
|
||||||
|
return photosHelperCandidates(options).find((candidate) => options.exists(candidate)) ?? null;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// argv
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
export function authorizeArgs(prompt: boolean): string[] {
|
||||||
|
return prompt ? ["authorize"] : ["authorize", "--no-prompt"];
|
||||||
|
}
|
||||||
|
|
||||||
|
export function listArgs(request: {
|
||||||
|
cursor: string | null;
|
||||||
|
limit: number;
|
||||||
|
since?: string | null;
|
||||||
|
}): string[] {
|
||||||
|
if (!Number.isInteger(request.limit) || request.limit < 1 || request.limit > PHOTOS_MAX_PAGE) {
|
||||||
|
throw new PhotosHelperError("usage", "List limit is out of range.");
|
||||||
|
}
|
||||||
|
const args = ["list", "--limit", String(request.limit)];
|
||||||
|
if (request.cursor !== null) args.push("--cursor", opaqueArgument(request.cursor));
|
||||||
|
if (request.since) args.push("--since", opaqueArgument(request.since));
|
||||||
|
return args;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function assetArgs(localIdentifier: string): string[] {
|
||||||
|
return ["asset", identifierArgument(localIdentifier)];
|
||||||
|
}
|
||||||
|
|
||||||
|
export function exportArgs(localIdentifier: string, destination: string): string[] {
|
||||||
|
if (!path.isAbsolute(destination) || destination.split(/[\\/]/).includes("..")) {
|
||||||
|
throw new PhotosHelperError("path_rejected", "Export directory must be absolute.");
|
||||||
|
}
|
||||||
|
return ["export", identifierArgument(localIdentifier), "--dest", destination, "--original"];
|
||||||
|
}
|
||||||
|
|
||||||
|
export function changesArgs(token: string): string[] {
|
||||||
|
return ["changes", "--token", opaqueArgument(token)];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Identifiers come from the helper itself; a leading dash would read as an option. */
|
||||||
|
function identifierArgument(value: string): string {
|
||||||
|
if (value === "" || value.length > 200 || value.startsWith("-") || hasControl(value)) {
|
||||||
|
throw new PhotosHelperError("usage", "Invalid Photos identifier.");
|
||||||
|
}
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
|
||||||
|
function opaqueArgument(value: string): string {
|
||||||
|
if (value === "" || value.length > 4096 || value.startsWith("-") || hasControl(value)) {
|
||||||
|
throw new PhotosHelperError("usage", "Invalid helper argument.");
|
||||||
|
}
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
|
||||||
|
function hasControl(value: string): boolean {
|
||||||
|
for (const char of value) {
|
||||||
|
const code = char.codePointAt(0) ?? 0;
|
||||||
|
if (code < 0x20 || code === 0x7f) return true;
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Output parsing
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
function invalid(what: string): never {
|
||||||
|
throw new PhotosHelperError("invalid_output", `Photos helper returned invalid ${what}.`);
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseDocument(text: string): Record<string, unknown> {
|
||||||
|
let value: unknown;
|
||||||
|
try {
|
||||||
|
value = JSON.parse(text);
|
||||||
|
} catch {
|
||||||
|
invalid("JSON");
|
||||||
|
}
|
||||||
|
if (typeof value !== "object" || value === null || Array.isArray(value)) invalid("JSON");
|
||||||
|
return value as Record<string, unknown>;
|
||||||
|
}
|
||||||
|
|
||||||
|
function nullableString(value: unknown, what: string, max = 4096): string | null {
|
||||||
|
if (value === null) return null;
|
||||||
|
if (typeof value !== "string" || value === "" || value.length > max) invalid(what);
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Each item must satisfy the shared contract schema exactly as the server will parse it. */
|
||||||
|
function parseItems(value: unknown, what: string): ClientMediaItem[] {
|
||||||
|
if (!Array.isArray(value) || value.length > 10_000) invalid(what);
|
||||||
|
return value.map((entry) => {
|
||||||
|
if (typeof entry !== "object" || entry === null) invalid(what);
|
||||||
|
const parsed = ClientMediaItemSchema.safeParse(entry);
|
||||||
|
if (!parsed.success) invalid(what);
|
||||||
|
return parsed.data;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
const AUTHORIZATION = new Set<PhotosAuthorization>([
|
||||||
|
"authorized",
|
||||||
|
"limited",
|
||||||
|
"denied",
|
||||||
|
"restricted",
|
||||||
|
"notDetermined",
|
||||||
|
]);
|
||||||
|
|
||||||
|
export function parseAuthorizeOutput(text: string): PhotosAuthorization {
|
||||||
|
const status = parseDocument(text).status;
|
||||||
|
if (typeof status !== "string" || !AUTHORIZATION.has(status as PhotosAuthorization)) {
|
||||||
|
invalid("authorization status");
|
||||||
|
}
|
||||||
|
return status as PhotosAuthorization;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function parseListOutput(text: string, limit: number): PhotosListPage {
|
||||||
|
const document = parseDocument(text);
|
||||||
|
const items = parseItems(document.items, "list items");
|
||||||
|
if (items.length > limit) invalid("list page size");
|
||||||
|
const nextCursor = nullableString(document.nextCursor, "list cursor");
|
||||||
|
const changeToken = nullableString(document.changeToken, "change token");
|
||||||
|
return { items, nextCursor, changeToken };
|
||||||
|
}
|
||||||
|
|
||||||
|
export function parseAssetOutput(text: string): PhotosAssetDetail {
|
||||||
|
const document = parseDocument(text);
|
||||||
|
const [item] = parseItems([document.item], "asset");
|
||||||
|
const modifiedAt = nullableString(document.modifiedAt, "asset date", 64);
|
||||||
|
if (modifiedAt !== null && Number.isNaN(Date.parse(modifiedAt))) invalid("asset date");
|
||||||
|
if (typeof document.hidden !== "boolean") invalid("asset");
|
||||||
|
if (!Array.isArray(document.resources) || document.resources.length > 50) invalid("resources");
|
||||||
|
const resources = document.resources.map((resource) => {
|
||||||
|
if (typeof resource !== "object" || resource === null) invalid("resources");
|
||||||
|
const { type, fileName, uti } = resource as Record<string, unknown>;
|
||||||
|
if (typeof type !== "string" || typeof fileName !== "string" || typeof uti !== "string") {
|
||||||
|
invalid("resources");
|
||||||
|
}
|
||||||
|
return { type, fileName, uti };
|
||||||
|
});
|
||||||
|
return { item: item!, modifiedAt, hidden: document.hidden, resources };
|
||||||
|
}
|
||||||
|
|
||||||
|
export function parseChangesOutput(text: string): PhotosChanges {
|
||||||
|
const document = parseDocument(text);
|
||||||
|
const token = nullableString(document.token, "change token");
|
||||||
|
if (typeof document.fullResyncRequired !== "boolean") invalid("changes");
|
||||||
|
const changed = parseItems(document.changed, "changed items");
|
||||||
|
if (!Array.isArray(document.deleted) || document.deleted.length > 10_000) invalid("changes");
|
||||||
|
const deleted = document.deleted.map((id) => {
|
||||||
|
if (typeof id !== "string" || id === "" || id.length > 200) invalid("deleted ids");
|
||||||
|
return id;
|
||||||
|
});
|
||||||
|
if (!document.fullResyncRequired && token === null) invalid("change token");
|
||||||
|
return { token, changed, deleted, fullResyncRequired: document.fullResyncRequired };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Besides the shape, the file must be a direct child of the directory the app chose,
|
||||||
|
* after resolving symlinks, so a confused helper cannot point the upload elsewhere.
|
||||||
|
*/
|
||||||
|
export async function parseExportOutput(text: string, destination: string): Promise<PhotosExport> {
|
||||||
|
const document = parseDocument(text);
|
||||||
|
const { path: file, fileName, sizeBytes, sha256 } = document;
|
||||||
|
if (
|
||||||
|
typeof file !== "string" ||
|
||||||
|
!path.isAbsolute(file) ||
|
||||||
|
typeof fileName !== "string" ||
|
||||||
|
fileName === "" ||
|
||||||
|
typeof sizeBytes !== "number" ||
|
||||||
|
!Number.isSafeInteger(sizeBytes) ||
|
||||||
|
sizeBytes < 0 ||
|
||||||
|
typeof sha256 !== "string" ||
|
||||||
|
!/^[0-9a-f]{64}$/.test(sha256)
|
||||||
|
) {
|
||||||
|
invalid("export result");
|
||||||
|
}
|
||||||
|
let real: string;
|
||||||
|
let realDestination: string;
|
||||||
|
try {
|
||||||
|
[real, realDestination] = await Promise.all([realpath(file), realpath(destination)]);
|
||||||
|
} catch {
|
||||||
|
invalid("export path");
|
||||||
|
}
|
||||||
|
if (path.dirname(real) !== realDestination || path.basename(real) !== fileName) {
|
||||||
|
throw new PhotosHelperError("path_rejected", "Photos helper wrote outside the export folder.");
|
||||||
|
}
|
||||||
|
return { path: real, fileName, sizeBytes, sha256 };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The last `{"error":{code,message}}` line on stderr; unknown codes become `failed`. */
|
||||||
|
export function parseHelperError(stderr: string, exitCode: number | null): PhotosHelperError {
|
||||||
|
const lines = stderr.split("\n").reverse();
|
||||||
|
for (const line of lines) {
|
||||||
|
if (!line.startsWith("{")) continue;
|
||||||
|
try {
|
||||||
|
const error = (JSON.parse(line) as { error?: unknown }).error as
|
||||||
|
| { code?: unknown; message?: unknown }
|
||||||
|
| undefined;
|
||||||
|
if (typeof error?.code !== "string") continue;
|
||||||
|
const code = HELPER_CODES.has(error.code as PhotosHelperErrorCode)
|
||||||
|
? (error.code as PhotosHelperErrorCode)
|
||||||
|
: "failed";
|
||||||
|
const message =
|
||||||
|
typeof error.message === "string" && error.message.length <= 500
|
||||||
|
? error.message
|
||||||
|
: "Photos helper failed.";
|
||||||
|
return new PhotosHelperError(code, message);
|
||||||
|
} catch {}
|
||||||
|
}
|
||||||
|
return new PhotosHelperError("failed", `Photos helper exited with status ${exitCode ?? "?"}.`);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** `{"progress":0.42}` lines on stderr; anything else is ignored. */
|
||||||
|
export function parseProgressLine(line: string): number | null {
|
||||||
|
if (!line.startsWith('{"progress"')) return null;
|
||||||
|
try {
|
||||||
|
const value = (JSON.parse(line) as { progress?: unknown }).progress;
|
||||||
|
return typeof value === "number" && value >= 0 && value <= 1 ? value : null;
|
||||||
|
} catch {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Process
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/** The subset of ChildProcess the runner needs, so tests can supply a fake. */
|
||||||
|
export interface HelperProcess {
|
||||||
|
stdout: Readable | null;
|
||||||
|
stderr: Readable | null;
|
||||||
|
kill(signal?: NodeJS.Signals): boolean;
|
||||||
|
on(event: "error", listener: (error: Error) => void): unknown;
|
||||||
|
on(event: "close", listener: (code: number | null) => void): unknown;
|
||||||
|
}
|
||||||
|
|
||||||
|
export type SpawnHelper = (
|
||||||
|
binary: string,
|
||||||
|
args: string[],
|
||||||
|
options: { shell: false; env: Record<string, string>; stdio: ["ignore", "pipe", "pipe"] },
|
||||||
|
) => HelperProcess;
|
||||||
|
|
||||||
|
export interface RunOptions {
|
||||||
|
signal?: AbortSignal;
|
||||||
|
/** Hard limit for the whole run. */
|
||||||
|
timeoutMs: number;
|
||||||
|
/** Fails when stderr progress stops for this long (exports only). */
|
||||||
|
idleTimeoutMs?: number;
|
||||||
|
onProgress?: (fraction: number) => void;
|
||||||
|
}
|
||||||
|
|
||||||
|
const MAX_STDOUT_BYTES = 32 * 1024 * 1024;
|
||||||
|
const MAX_STDERR_BYTES = 1024 * 1024;
|
||||||
|
const ENV_ALLOWLIST = ["HOME", "USER", "LOGNAME", "TMPDIR", "LANG", "LC_ALL"];
|
||||||
|
|
||||||
|
/** PhotoKit needs the user's home; nothing else from the app environment is passed on. */
|
||||||
|
export function helperEnv(env: NodeJS.ProcessEnv): Record<string, string> {
|
||||||
|
const result: Record<string, string> = {};
|
||||||
|
for (const name of ENV_ALLOWLIST) {
|
||||||
|
const value = env[name];
|
||||||
|
if (value) result[name] = value;
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
export const defaultSpawnHelper: SpawnHelper = (binary, args, options) =>
|
||||||
|
nodeSpawn(binary, args, options);
|
||||||
|
|
||||||
|
export function runPhotosHelper(
|
||||||
|
binary: string,
|
||||||
|
args: string[],
|
||||||
|
options: RunOptions & { spawn?: SpawnHelper; env?: NodeJS.ProcessEnv },
|
||||||
|
): Promise<string> {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
if (options.signal?.aborted) {
|
||||||
|
reject(new PhotosHelperError("cancelled", "Cancelled."));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const child = (options.spawn ?? defaultSpawnHelper)(binary, args, {
|
||||||
|
shell: false,
|
||||||
|
env: helperEnv(options.env ?? process.env),
|
||||||
|
stdio: ["ignore", "pipe", "pipe"],
|
||||||
|
});
|
||||||
|
const stdout: Buffer[] = [];
|
||||||
|
let stdoutBytes = 0;
|
||||||
|
let stderr = "";
|
||||||
|
let pendingLine = "";
|
||||||
|
let settled = false;
|
||||||
|
let idleTimer: NodeJS.Timeout | undefined;
|
||||||
|
|
||||||
|
const finish = (error: PhotosHelperError | null, output = "") => {
|
||||||
|
if (settled) return;
|
||||||
|
settled = true;
|
||||||
|
clearTimeout(timer);
|
||||||
|
clearTimeout(idleTimer);
|
||||||
|
options.signal?.removeEventListener("abort", abort);
|
||||||
|
if (error) reject(error);
|
||||||
|
else resolve(output);
|
||||||
|
};
|
||||||
|
const stop = (error: PhotosHelperError) => {
|
||||||
|
child.kill("SIGTERM");
|
||||||
|
finish(error);
|
||||||
|
};
|
||||||
|
const abort = () => stop(new PhotosHelperError("cancelled", "Cancelled."));
|
||||||
|
const timer = setTimeout(
|
||||||
|
() => stop(new PhotosHelperError("timeout", "Photos helper timed out.")),
|
||||||
|
options.timeoutMs,
|
||||||
|
);
|
||||||
|
timer.unref?.();
|
||||||
|
const armIdle = () => {
|
||||||
|
if (options.idleTimeoutMs === undefined) return;
|
||||||
|
clearTimeout(idleTimer);
|
||||||
|
idleTimer = setTimeout(
|
||||||
|
() => stop(new PhotosHelperError("timeout", "Photos export stalled.")),
|
||||||
|
options.idleTimeoutMs,
|
||||||
|
);
|
||||||
|
idleTimer.unref?.();
|
||||||
|
};
|
||||||
|
armIdle();
|
||||||
|
options.signal?.addEventListener("abort", abort, { once: true });
|
||||||
|
|
||||||
|
child.stdout?.on("data", (chunk: Buffer) => {
|
||||||
|
stdoutBytes += chunk.byteLength;
|
||||||
|
if (stdoutBytes > MAX_STDOUT_BYTES) {
|
||||||
|
stop(new PhotosHelperError("invalid_output", "Photos helper output is too large."));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
stdout.push(chunk);
|
||||||
|
});
|
||||||
|
child.stderr?.on("data", (chunk: Buffer) => {
|
||||||
|
const text = chunk.toString("utf8");
|
||||||
|
if (stderr.length < MAX_STDERR_BYTES) stderr += text;
|
||||||
|
const lines = (pendingLine + text).split("\n");
|
||||||
|
pendingLine = lines.pop() ?? "";
|
||||||
|
for (const line of lines) {
|
||||||
|
const progress = parseProgressLine(line.trim());
|
||||||
|
if (progress !== null) {
|
||||||
|
armIdle();
|
||||||
|
options.onProgress?.(progress);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
child.on("error", () => {
|
||||||
|
finish(new PhotosHelperError("unavailable", "Photos helper could not be started."));
|
||||||
|
});
|
||||||
|
child.on("close", (code) => {
|
||||||
|
if (code === 0) finish(null, Buffer.concat(stdout).toString("utf8"));
|
||||||
|
else finish(parseHelperError(stderr + pendingLine, code));
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
const QUERY_TIMEOUT_MS = 2 * 60_000;
|
||||||
|
/** The system permission dialog waits on the person. */
|
||||||
|
const AUTHORIZE_TIMEOUT_MS = 10 * 60_000;
|
||||||
|
/** Large iCloud videos take long; a stalled download fails sooner through the idle limit. */
|
||||||
|
const EXPORT_TIMEOUT_MS = 2 * 60 * 60_000;
|
||||||
|
const EXPORT_IDLE_TIMEOUT_MS = 5 * 60_000;
|
||||||
|
|
||||||
|
export interface PhotosHelper {
|
||||||
|
authorize(prompt: boolean, signal?: AbortSignal): Promise<PhotosAuthorization>;
|
||||||
|
list(
|
||||||
|
request: { cursor: string | null; limit: number; since?: string | null },
|
||||||
|
signal?: AbortSignal,
|
||||||
|
): Promise<PhotosListPage>;
|
||||||
|
asset(localIdentifier: string, signal?: AbortSignal): Promise<PhotosAssetDetail>;
|
||||||
|
exportOriginal(
|
||||||
|
localIdentifier: string,
|
||||||
|
destination: string,
|
||||||
|
options?: { signal?: AbortSignal; onProgress?: (fraction: number) => void },
|
||||||
|
): Promise<PhotosExport>;
|
||||||
|
changes(token: string, signal?: AbortSignal): Promise<PhotosChanges>;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function createPhotosHelper(binary: string, spawn?: SpawnHelper): PhotosHelper {
|
||||||
|
const run = (args: string[], options: RunOptions) =>
|
||||||
|
runPhotosHelper(binary, args, { ...options, spawn });
|
||||||
|
return {
|
||||||
|
async authorize(prompt, signal) {
|
||||||
|
const timeoutMs = prompt ? AUTHORIZE_TIMEOUT_MS : QUERY_TIMEOUT_MS;
|
||||||
|
return parseAuthorizeOutput(await run(authorizeArgs(prompt), { signal, timeoutMs }));
|
||||||
|
},
|
||||||
|
async list(request, signal) {
|
||||||
|
const output = await run(listArgs(request), { signal, timeoutMs: QUERY_TIMEOUT_MS });
|
||||||
|
return parseListOutput(output, request.limit);
|
||||||
|
},
|
||||||
|
async asset(localIdentifier, signal) {
|
||||||
|
const output = await run(assetArgs(localIdentifier), { signal, timeoutMs: QUERY_TIMEOUT_MS });
|
||||||
|
return parseAssetOutput(output);
|
||||||
|
},
|
||||||
|
async exportOriginal(localIdentifier, destination, options = {}) {
|
||||||
|
const output = await run(exportArgs(localIdentifier, destination), {
|
||||||
|
signal: options.signal,
|
||||||
|
onProgress: options.onProgress,
|
||||||
|
timeoutMs: EXPORT_TIMEOUT_MS,
|
||||||
|
idleTimeoutMs: EXPORT_IDLE_TIMEOUT_MS,
|
||||||
|
});
|
||||||
|
return parseExportOutput(output, destination);
|
||||||
|
},
|
||||||
|
async changes(token, signal) {
|
||||||
|
const output = await run(changesArgs(token), { signal, timeoutMs: QUERY_TIMEOUT_MS });
|
||||||
|
return parseChangesOutput(output);
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
655
apps/desktop/src/media-photos-sync.test.ts
Normal file
655
apps/desktop/src/media-photos-sync.test.ts
Normal file
@ -0,0 +1,655 @@
|
|||||||
|
import { createHash } from "node:crypto";
|
||||||
|
import { mkdtemp, readdir, readFile, rm, writeFile } from "node:fs/promises";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import path from "node:path";
|
||||||
|
import type { ClientMediaItem, MediaAsset, MediaUpload } from "@rakazo/contracts";
|
||||||
|
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||||
|
import type { PhotosHelper } from "./media-photos-helper.js";
|
||||||
|
import type { MediaSyncApi, PhotosSyncState, PhotosSyncStore } from "./media-photos-sync.js";
|
||||||
|
import {
|
||||||
|
createChunkUploader,
|
||||||
|
createPhotosSyncFileStore,
|
||||||
|
parsePhotosSyncState,
|
||||||
|
runPool,
|
||||||
|
syncApplePhotos,
|
||||||
|
} from "./media-photos-sync.js";
|
||||||
|
|
||||||
|
const sha = (bytes: string | Buffer) => createHash("sha256").update(bytes).digest("hex");
|
||||||
|
|
||||||
|
function libraryItem(index: number): ClientMediaItem {
|
||||||
|
return {
|
||||||
|
externalId: `ASSET-${String(index).padStart(5, "0")}/L0/001`,
|
||||||
|
kind: index % 3 === 0 ? "video" : "photo",
|
||||||
|
capturedAt: new Date(Date.UTC(2024, 0, 1) + index * 60_000).toISOString(),
|
||||||
|
durationMs: index % 3 === 0 ? 5000 : null,
|
||||||
|
width: 1920,
|
||||||
|
height: 1080,
|
||||||
|
fps: null,
|
||||||
|
gpsLat: null,
|
||||||
|
gpsLon: null,
|
||||||
|
deviceModel: null,
|
||||||
|
albumNames: [],
|
||||||
|
isFavorite: false,
|
||||||
|
fileName: `IMG_${index}.JPG`,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Pages a synthetic library the way the Swift helper does: opaque cursor = next index. */
|
||||||
|
function fakeHelper(options: { size: number; contents?: (id: string) => string }) {
|
||||||
|
const library = Array.from({ length: options.size }, (_, index) => libraryItem(index));
|
||||||
|
const exports: string[] = [];
|
||||||
|
let active = 0;
|
||||||
|
const stats = { maxActiveExports: 0 };
|
||||||
|
const helper: PhotosHelper & { library: ClientMediaItem[] } = {
|
||||||
|
library,
|
||||||
|
authorize: vi.fn(async () => "authorized" as const),
|
||||||
|
list: vi.fn(async ({ cursor, limit }) => {
|
||||||
|
const start = cursor === null ? 0 : Number(cursor);
|
||||||
|
const items = library.slice(start, start + limit);
|
||||||
|
const next = start + limit < library.length ? String(start + limit) : null;
|
||||||
|
return { items, nextCursor: next, changeToken: `token-${library.length}` };
|
||||||
|
}),
|
||||||
|
asset: vi.fn(),
|
||||||
|
changes: vi.fn(),
|
||||||
|
exportOriginal: vi.fn(async (id: string, destination: string, opts) => {
|
||||||
|
active += 1;
|
||||||
|
stats.maxActiveExports = Math.max(stats.maxActiveExports, active);
|
||||||
|
try {
|
||||||
|
await new Promise((resolve) => setTimeout(resolve, 5));
|
||||||
|
opts?.signal?.throwIfAborted();
|
||||||
|
const contents = options.contents?.(id) ?? `original bytes of ${id}`;
|
||||||
|
const file = path.join(destination, "IMG.MOV");
|
||||||
|
await writeFile(file, contents);
|
||||||
|
exports.push(destination);
|
||||||
|
return {
|
||||||
|
path: file,
|
||||||
|
fileName: "IMG.MOV",
|
||||||
|
sizeBytes: Buffer.byteLength(contents),
|
||||||
|
sha256: sha(contents),
|
||||||
|
};
|
||||||
|
} finally {
|
||||||
|
active -= 1;
|
||||||
|
}
|
||||||
|
}),
|
||||||
|
};
|
||||||
|
return { helper, exports, stats };
|
||||||
|
}
|
||||||
|
|
||||||
|
function memoryStore(initial?: PhotosSyncState) {
|
||||||
|
let state: PhotosSyncState = initial ?? { version: 1, deviceId: "device-1", targets: {} };
|
||||||
|
const writes: PhotosSyncState[] = [];
|
||||||
|
const store: PhotosSyncStore = {
|
||||||
|
read: async () => JSON.parse(JSON.stringify(state)) as PhotosSyncState,
|
||||||
|
write: async (next) => {
|
||||||
|
state = JSON.parse(JSON.stringify(next)) as PhotosSyncState;
|
||||||
|
writes.push(state);
|
||||||
|
},
|
||||||
|
};
|
||||||
|
return { store, writes, current: () => state };
|
||||||
|
}
|
||||||
|
|
||||||
|
function asset(id: string, externalId: string, ingestState: MediaAsset["ingestState"]) {
|
||||||
|
return {
|
||||||
|
id,
|
||||||
|
sourceId: "source-1",
|
||||||
|
externalId,
|
||||||
|
kind: "video",
|
||||||
|
storage: "remote",
|
||||||
|
capturedAt: null,
|
||||||
|
durationMs: null,
|
||||||
|
width: null,
|
||||||
|
height: null,
|
||||||
|
fps: null,
|
||||||
|
gpsLat: null,
|
||||||
|
gpsLon: null,
|
||||||
|
deviceModel: null,
|
||||||
|
albumNames: [],
|
||||||
|
isFavorite: false,
|
||||||
|
ingestState,
|
||||||
|
hasProxy: false,
|
||||||
|
hasThumbnail: false,
|
||||||
|
qualityScore: null,
|
||||||
|
createdAt: "2024-01-01T00:00:00.000Z",
|
||||||
|
} as MediaAsset;
|
||||||
|
}
|
||||||
|
|
||||||
|
function fakeApi(
|
||||||
|
options: {
|
||||||
|
assets?: MediaAsset[];
|
||||||
|
chunkBytes?: number;
|
||||||
|
uploads?: Record<string, MediaUpload>;
|
||||||
|
} = {},
|
||||||
|
) {
|
||||||
|
const uploads = new Map(Object.entries(options.uploads ?? {}));
|
||||||
|
const pushes: {
|
||||||
|
items: number;
|
||||||
|
syncToken: string | null;
|
||||||
|
complete: boolean;
|
||||||
|
deleted?: number;
|
||||||
|
}[] = [];
|
||||||
|
const chunkBytes = options.chunkBytes ?? 4;
|
||||||
|
const api = {
|
||||||
|
media: {
|
||||||
|
sources: {
|
||||||
|
registerClient: vi.fn(async (input: { deviceId: string }) => ({
|
||||||
|
id: "source-1",
|
||||||
|
kind: "apple_photos" as const,
|
||||||
|
label: "Apple Photos",
|
||||||
|
status: "idle" as const,
|
||||||
|
assetCount: 0,
|
||||||
|
lastSyncedAt: null,
|
||||||
|
error: null,
|
||||||
|
createdAt: "2024-01-01T00:00:00.000Z",
|
||||||
|
deviceId: input.deviceId,
|
||||||
|
})),
|
||||||
|
pushItems: vi.fn(
|
||||||
|
async (input: {
|
||||||
|
items: unknown[];
|
||||||
|
deletedExternalIds?: string[];
|
||||||
|
syncToken: string | null;
|
||||||
|
complete: boolean;
|
||||||
|
}) => {
|
||||||
|
const deleted = input.deletedExternalIds?.length ?? 0;
|
||||||
|
expect(input.items.length > 0 || deleted > 0 || input.complete).toBe(true);
|
||||||
|
expect(input.items.length).toBeLessThanOrEqual(500);
|
||||||
|
pushes.push({
|
||||||
|
items: input.items.length,
|
||||||
|
syncToken: input.syncToken,
|
||||||
|
complete: input.complete,
|
||||||
|
...(deleted > 0 ? { deleted } : {}),
|
||||||
|
});
|
||||||
|
return { created: input.items.length, updated: 0, unchanged: 0, deleted };
|
||||||
|
},
|
||||||
|
),
|
||||||
|
},
|
||||||
|
assets: {
|
||||||
|
search: vi.fn(async (input: { cursor?: string | null }) => {
|
||||||
|
const all = options.assets ?? [];
|
||||||
|
// Two pages to exercise pagination.
|
||||||
|
const half = Math.ceil(all.length / 2);
|
||||||
|
return input.cursor === "p2"
|
||||||
|
? { items: all.slice(half), nextCursor: null }
|
||||||
|
: { items: all.slice(0, half), nextCursor: all.length > 1 ? "p2" : null };
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
uploads: {
|
||||||
|
begin: vi.fn(async (input: { assetId: string; sizeBytes: number }) => {
|
||||||
|
const upload: MediaUpload = {
|
||||||
|
id: `upload-${input.assetId}`,
|
||||||
|
assetId: input.assetId,
|
||||||
|
sizeBytes: input.sizeBytes,
|
||||||
|
chunkBytes,
|
||||||
|
receivedChunks: [],
|
||||||
|
status: "open",
|
||||||
|
};
|
||||||
|
uploads.set(upload.id, upload);
|
||||||
|
return upload;
|
||||||
|
}),
|
||||||
|
status: vi.fn(async (input: { uploadId: string }) => {
|
||||||
|
const upload = uploads.get(input.uploadId);
|
||||||
|
if (!upload) throw Object.assign(new Error("Not found"), { code: "NOT_FOUND" });
|
||||||
|
return upload;
|
||||||
|
}),
|
||||||
|
complete: vi.fn(async (input: { uploadId: string }) => {
|
||||||
|
const upload = uploads.get(input.uploadId)!;
|
||||||
|
return { ...upload, status: "complete" as const };
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
};
|
||||||
|
return { api: api as unknown as MediaSyncApi & typeof api, pushes };
|
||||||
|
}
|
||||||
|
|
||||||
|
let tempRoot: string;
|
||||||
|
|
||||||
|
beforeEach(async () => {
|
||||||
|
tempRoot = await mkdtemp(path.join(tmpdir(), "rakazo-photos-sync-"));
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(async () => {
|
||||||
|
await rm(tempRoot, { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
const baseOptions = () => ({
|
||||||
|
target: "https://rakazo.example.com#space_1",
|
||||||
|
label: "Apple Photos",
|
||||||
|
tempRoot,
|
||||||
|
signal: new AbortController().signal,
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("Apple Photos metadata sync", () => {
|
||||||
|
it("registers once, pushes batches of at most 500 and completes the listing", async () => {
|
||||||
|
const { helper } = fakeHelper({ size: 1203 });
|
||||||
|
const { api, pushes } = fakeApi();
|
||||||
|
const { store, current } = memoryStore();
|
||||||
|
const chunks = vi.fn();
|
||||||
|
|
||||||
|
const result = await syncApplePhotos({
|
||||||
|
...baseOptions(),
|
||||||
|
helper,
|
||||||
|
api,
|
||||||
|
store,
|
||||||
|
uploadChunk: chunks,
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(api.media.sources.registerClient).toHaveBeenCalledWith(
|
||||||
|
{ kind: "apple_photos", label: "Apple Photos", deviceId: "device-1" },
|
||||||
|
expect.anything(),
|
||||||
|
);
|
||||||
|
expect(pushes).toEqual([
|
||||||
|
{ items: 500, syncToken: null, complete: false },
|
||||||
|
{ items: 500, syncToken: null, complete: false },
|
||||||
|
{ items: 203, syncToken: "token-1203", complete: true },
|
||||||
|
]);
|
||||||
|
expect(result).toMatchObject({ indexed: 1203, created: 1203, uploaded: 0, fullListing: true });
|
||||||
|
expect(current().targets["https://rakazo.example.com#space_1"]).toEqual({
|
||||||
|
sourceId: "source-1",
|
||||||
|
listing: null,
|
||||||
|
changeToken: "token-1203",
|
||||||
|
uploads: {},
|
||||||
|
});
|
||||||
|
|
||||||
|
// The next run reads changes only and does not register again.
|
||||||
|
vi.mocked(helper.changes).mockResolvedValueOnce({
|
||||||
|
token: "token-next",
|
||||||
|
changed: [libraryItem(1), libraryItem(2)],
|
||||||
|
deleted: [],
|
||||||
|
fullResyncRequired: false,
|
||||||
|
});
|
||||||
|
const second = await syncApplePhotos({
|
||||||
|
...baseOptions(),
|
||||||
|
helper,
|
||||||
|
api,
|
||||||
|
store,
|
||||||
|
uploadChunk: chunks,
|
||||||
|
});
|
||||||
|
expect(api.media.sources.registerClient).toHaveBeenCalledTimes(1);
|
||||||
|
expect(helper.changes).toHaveBeenCalledWith("token-1203", expect.anything());
|
||||||
|
expect(pushes.at(-1)).toEqual({ items: 2, syncToken: "token-next", complete: false });
|
||||||
|
expect(second.fullListing).toBe(false);
|
||||||
|
expect(current().targets["https://rakazo.example.com#space_1"]?.changeToken).toBe("token-next");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("persists the cursor after every batch and resumes an interrupted listing", async () => {
|
||||||
|
const { helper } = fakeHelper({ size: 1100 });
|
||||||
|
const { api, pushes } = fakeApi();
|
||||||
|
const { store, writes, current } = memoryStore();
|
||||||
|
vi.mocked(api.media.sources.pushItems).mockImplementationOnce(async (input) => {
|
||||||
|
pushes.push({
|
||||||
|
items: input.items.length,
|
||||||
|
syncToken: input.syncToken,
|
||||||
|
complete: input.complete,
|
||||||
|
});
|
||||||
|
return { created: input.items.length, updated: 0, unchanged: 0 };
|
||||||
|
});
|
||||||
|
vi.mocked(api.media.sources.pushItems).mockRejectedValueOnce(new Error("network down"));
|
||||||
|
|
||||||
|
await expect(
|
||||||
|
syncApplePhotos({ ...baseOptions(), helper, api, store, uploadChunk: vi.fn() }),
|
||||||
|
).rejects.toThrow("network down");
|
||||||
|
const interrupted = current().targets["https://rakazo.example.com#space_1"]!;
|
||||||
|
expect(interrupted.listing).toMatchObject({ cursor: "500", token: "token-1100" });
|
||||||
|
expect(interrupted.listing?.lastItem?.externalId).toBe(libraryItem(499).externalId);
|
||||||
|
expect(writes.length).toBeGreaterThan(1);
|
||||||
|
|
||||||
|
await syncApplePhotos({ ...baseOptions(), helper, api, store, uploadChunk: vi.fn() });
|
||||||
|
expect(vi.mocked(helper.list).mock.calls.at(-2)?.[0]).toMatchObject({ cursor: "500" });
|
||||||
|
expect(pushes.slice(1)).toEqual([
|
||||||
|
{ items: 500, syncToken: null, complete: false },
|
||||||
|
{ items: 100, syncToken: "token-1100", complete: true },
|
||||||
|
]);
|
||||||
|
expect(current().targets["https://rakazo.example.com#space_1"]).toMatchObject({
|
||||||
|
listing: null,
|
||||||
|
changeToken: "token-1100",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("closes a resumed listing whose last page came back empty", async () => {
|
||||||
|
const { helper } = fakeHelper({ size: 0 });
|
||||||
|
const { api, pushes } = fakeApi();
|
||||||
|
const last = libraryItem(7);
|
||||||
|
const { store } = memoryStore({
|
||||||
|
version: 1,
|
||||||
|
deviceId: "device-1",
|
||||||
|
targets: {
|
||||||
|
"https://rakazo.example.com#space_1": {
|
||||||
|
sourceId: "source-1",
|
||||||
|
listing: { cursor: "500", token: "token-a", lastItem: last },
|
||||||
|
changeToken: null,
|
||||||
|
uploads: {},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
});
|
||||||
|
await syncApplePhotos({ ...baseOptions(), helper, api, store, uploadChunk: vi.fn() });
|
||||||
|
expect(pushes).toEqual([{ items: 0, syncToken: "token-a", complete: true }]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("pushes device deletions without relisting", async () => {
|
||||||
|
const { helper } = fakeHelper({ size: 3 });
|
||||||
|
const { api, pushes } = fakeApi();
|
||||||
|
const { store } = memoryStore({
|
||||||
|
version: 1,
|
||||||
|
deviceId: "device-1",
|
||||||
|
targets: {
|
||||||
|
"https://rakazo.example.com#space_1": {
|
||||||
|
sourceId: "source-1",
|
||||||
|
listing: null,
|
||||||
|
changeToken: "old",
|
||||||
|
uploads: {},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
});
|
||||||
|
vi.mocked(helper.changes).mockResolvedValueOnce({
|
||||||
|
token: "new",
|
||||||
|
changed: [],
|
||||||
|
deleted: ["gone/L0/001"],
|
||||||
|
fullResyncRequired: false,
|
||||||
|
});
|
||||||
|
const result = await syncApplePhotos({
|
||||||
|
...baseOptions(),
|
||||||
|
helper,
|
||||||
|
api,
|
||||||
|
store,
|
||||||
|
uploadChunk: vi.fn(),
|
||||||
|
});
|
||||||
|
expect(result.fullListing).toBe(false);
|
||||||
|
expect(pushes).toEqual([{ items: 0, syncToken: "new", complete: false, deleted: 1 }]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("registers again when the server no longer knows the source", async () => {
|
||||||
|
const { helper } = fakeHelper({ size: 2 });
|
||||||
|
const { api } = fakeApi();
|
||||||
|
const { store } = memoryStore({
|
||||||
|
version: 1,
|
||||||
|
deviceId: "device-1",
|
||||||
|
targets: {
|
||||||
|
"https://rakazo.example.com#space_1": {
|
||||||
|
sourceId: "deleted-source",
|
||||||
|
listing: { cursor: null, token: null, lastItem: null },
|
||||||
|
changeToken: null,
|
||||||
|
uploads: {},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
});
|
||||||
|
vi.mocked(api.media.sources.pushItems).mockRejectedValueOnce(
|
||||||
|
Object.assign(new Error("Not found"), { code: "NOT_FOUND" }),
|
||||||
|
);
|
||||||
|
await syncApplePhotos({ ...baseOptions(), helper, api, store, uploadChunk: vi.fn() });
|
||||||
|
expect(api.media.sources.registerClient).toHaveBeenCalledTimes(1);
|
||||||
|
expect(vi.mocked(api.media.sources.pushItems).mock.calls.at(-1)?.[0].sourceId).toBe("source-1");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("stops before anything is sent when Photos access is denied", async () => {
|
||||||
|
const { helper } = fakeHelper({ size: 2 });
|
||||||
|
vi.mocked(helper.authorize).mockResolvedValueOnce("denied");
|
||||||
|
const { api } = fakeApi();
|
||||||
|
await expect(
|
||||||
|
syncApplePhotos({
|
||||||
|
...baseOptions(),
|
||||||
|
helper,
|
||||||
|
api,
|
||||||
|
store: memoryStore().store,
|
||||||
|
uploadChunk: vi.fn(),
|
||||||
|
}),
|
||||||
|
).rejects.toMatchObject({ code: "not_authorized" });
|
||||||
|
expect(api.media.sources.registerClient).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("Apple Photos original upload", () => {
|
||||||
|
const awaitingAssets = [
|
||||||
|
asset("a1", "ASSET-00001/L0/001", "awaiting_upload"),
|
||||||
|
asset("a2", "ASSET-00002/L0/001", "proxied"),
|
||||||
|
asset("a3", "ASSET-00003/L0/001", "awaiting_upload"),
|
||||||
|
asset("a4", "ASSET-00004/L0/001", "awaiting_upload"),
|
||||||
|
];
|
||||||
|
|
||||||
|
it("uploads every awaiting original in chunks with two at a time and cleans up", async () => {
|
||||||
|
const { helper, exports, stats } = fakeHelper({ size: 1 });
|
||||||
|
const { api } = fakeApi({ assets: awaitingAssets, chunkBytes: 8 });
|
||||||
|
const sent = new Map<string, Buffer[]>();
|
||||||
|
const uploadChunk = vi.fn(async (uploadId: string, index: number, bytes: Uint8Array) => {
|
||||||
|
const parts = sent.get(uploadId) ?? [];
|
||||||
|
parts[index] = Buffer.from(bytes);
|
||||||
|
sent.set(uploadId, parts);
|
||||||
|
});
|
||||||
|
const progress = vi.fn();
|
||||||
|
|
||||||
|
const result = await syncApplePhotos({
|
||||||
|
...baseOptions(),
|
||||||
|
helper,
|
||||||
|
api,
|
||||||
|
store: memoryStore().store,
|
||||||
|
uploadChunk,
|
||||||
|
onProgress: progress,
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(result.uploaded).toBe(3);
|
||||||
|
expect(
|
||||||
|
vi
|
||||||
|
.mocked(helper.exportOriginal)
|
||||||
|
.mock.calls.map((call) => call[0])
|
||||||
|
.sort(),
|
||||||
|
).toEqual(["ASSET-00001/L0/001", "ASSET-00003/L0/001", "ASSET-00004/L0/001"]);
|
||||||
|
expect(stats.maxActiveExports).toBe(2);
|
||||||
|
expect(Buffer.concat(sent.get("upload-a1")!).toString()).toBe(
|
||||||
|
"original bytes of ASSET-00001/L0/001",
|
||||||
|
);
|
||||||
|
expect(sent.get("upload-a1")).toHaveLength(Math.ceil(36 / 8));
|
||||||
|
expect(api.media.uploads.begin).toHaveBeenCalledWith(
|
||||||
|
expect.objectContaining({
|
||||||
|
assetId: "a1",
|
||||||
|
sizeBytes: 36,
|
||||||
|
sha256: sha("original bytes of ASSET-00001/L0/001"),
|
||||||
|
}),
|
||||||
|
expect.anything(),
|
||||||
|
);
|
||||||
|
expect(api.media.uploads.complete).toHaveBeenCalledTimes(3);
|
||||||
|
expect(progress).toHaveBeenLastCalledWith({
|
||||||
|
phase: "uploading",
|
||||||
|
indexed: 1,
|
||||||
|
uploadsDone: 3,
|
||||||
|
uploadsTotal: 3,
|
||||||
|
});
|
||||||
|
for (const dir of exports) await expect(readdir(dir)).rejects.toThrow();
|
||||||
|
await expect(readdir(tempRoot)).resolves.toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("resumes an open upload and sends only the missing chunks", async () => {
|
||||||
|
const { helper } = fakeHelper({ size: 1 });
|
||||||
|
const contents = "original bytes of ASSET-00001/L0/001";
|
||||||
|
const { api } = fakeApi({
|
||||||
|
assets: [awaitingAssets[0]!],
|
||||||
|
uploads: {
|
||||||
|
"upload-old": {
|
||||||
|
id: "upload-old",
|
||||||
|
assetId: "a1",
|
||||||
|
sizeBytes: Buffer.byteLength(contents),
|
||||||
|
chunkBytes: 10,
|
||||||
|
receivedChunks: [0, 2],
|
||||||
|
status: "open",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
});
|
||||||
|
const { store, current } = memoryStore({
|
||||||
|
version: 1,
|
||||||
|
deviceId: "device-1",
|
||||||
|
targets: {
|
||||||
|
"https://rakazo.example.com#space_1": {
|
||||||
|
sourceId: "source-1",
|
||||||
|
listing: null,
|
||||||
|
changeToken: "tok",
|
||||||
|
uploads: { a1: "upload-old" },
|
||||||
|
},
|
||||||
|
},
|
||||||
|
});
|
||||||
|
vi.mocked(helper.changes).mockResolvedValueOnce({
|
||||||
|
token: "tok",
|
||||||
|
changed: [],
|
||||||
|
deleted: [],
|
||||||
|
fullResyncRequired: false,
|
||||||
|
});
|
||||||
|
const uploadChunk = vi.fn(async () => {});
|
||||||
|
|
||||||
|
await syncApplePhotos({ ...baseOptions(), helper, api, store, uploadChunk });
|
||||||
|
|
||||||
|
expect(api.media.uploads.begin).not.toHaveBeenCalled();
|
||||||
|
expect(
|
||||||
|
uploadChunk.mock.calls.map((call) => [
|
||||||
|
call[0],
|
||||||
|
call[1],
|
||||||
|
Buffer.from(call[2] as Uint8Array).toString(),
|
||||||
|
]),
|
||||||
|
).toEqual([
|
||||||
|
["upload-old", 1, contents.slice(10, 20)],
|
||||||
|
["upload-old", 3, contents.slice(30)],
|
||||||
|
]);
|
||||||
|
expect(current().targets["https://rakazo.example.com#space_1"]?.uploads).toEqual({});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("aborts on a checksum mismatch without starting an upload", async () => {
|
||||||
|
const { helper } = fakeHelper({ size: 1 });
|
||||||
|
vi.mocked(helper.exportOriginal).mockImplementationOnce(async (_id, destination) => {
|
||||||
|
const file = path.join(destination, "IMG.MOV");
|
||||||
|
await writeFile(file, "tampered");
|
||||||
|
return { path: file, fileName: "IMG.MOV", sizeBytes: 8, sha256: sha("original") };
|
||||||
|
});
|
||||||
|
const { api } = fakeApi({ assets: [awaitingAssets[0]!] });
|
||||||
|
const uploadChunk = vi.fn();
|
||||||
|
|
||||||
|
await expect(
|
||||||
|
syncApplePhotos({ ...baseOptions(), helper, api, store: memoryStore().store, uploadChunk }),
|
||||||
|
).rejects.toMatchObject({ code: "integrity" });
|
||||||
|
expect(api.media.uploads.begin).not.toHaveBeenCalled();
|
||||||
|
expect(uploadChunk).not.toHaveBeenCalled();
|
||||||
|
await expect(readdir(tempRoot)).resolves.toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("cancels a running upload", async () => {
|
||||||
|
const { helper } = fakeHelper({ size: 1 });
|
||||||
|
const { api } = fakeApi({ assets: [awaitingAssets[0]!], chunkBytes: 4 });
|
||||||
|
const controller = new AbortController();
|
||||||
|
const uploadChunk = vi.fn(
|
||||||
|
async (_id: string, index: number, _bytes: Uint8Array, signal: AbortSignal) => {
|
||||||
|
if (index === 1) controller.abort();
|
||||||
|
signal.throwIfAborted();
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
await expect(
|
||||||
|
syncApplePhotos({
|
||||||
|
...baseOptions(),
|
||||||
|
signal: controller.signal,
|
||||||
|
helper,
|
||||||
|
api,
|
||||||
|
store: memoryStore().store,
|
||||||
|
uploadChunk,
|
||||||
|
}),
|
||||||
|
).rejects.toMatchObject({ code: "cancelled" });
|
||||||
|
expect(uploadChunk).toHaveBeenCalledTimes(2);
|
||||||
|
expect(api.media.uploads.complete).not.toHaveBeenCalled();
|
||||||
|
await expect(readdir(tempRoot)).resolves.toEqual([]);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("Apple Photos sync state file", () => {
|
||||||
|
it("keeps a stable device id across launches", async () => {
|
||||||
|
const dir = await mkdtemp(path.join(tmpdir(), "rakazo-photos-state-"));
|
||||||
|
try {
|
||||||
|
const store = createPhotosSyncFileStore(dir, () => "device-a");
|
||||||
|
const first = await store.read();
|
||||||
|
expect(first.deviceId).toBe("device-a");
|
||||||
|
first.targets.x = { sourceId: "s", listing: null, changeToken: "t", uploads: {} };
|
||||||
|
await store.write(first);
|
||||||
|
const reopened = await createPhotosSyncFileStore(dir, () => "device-b").read();
|
||||||
|
expect(reopened).toEqual(first);
|
||||||
|
expect(await readFile(path.join(dir, "media-photos-sync.json"), "utf8")).toContain(
|
||||||
|
"device-a",
|
||||||
|
);
|
||||||
|
} finally {
|
||||||
|
await rm(dir, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("drops unreadable state and malformed targets", () => {
|
||||||
|
expect(parsePhotosSyncState("{")).toBeNull();
|
||||||
|
expect(
|
||||||
|
parsePhotosSyncState(JSON.stringify({ version: 2, deviceId: "d", targets: {} })),
|
||||||
|
).toBeNull();
|
||||||
|
expect(
|
||||||
|
parsePhotosSyncState(
|
||||||
|
JSON.stringify({
|
||||||
|
version: 1,
|
||||||
|
deviceId: "d",
|
||||||
|
targets: {
|
||||||
|
good: { sourceId: "s", listing: null, changeToken: null, uploads: {} },
|
||||||
|
bad: { sourceId: 4, listing: null, changeToken: null, uploads: {} },
|
||||||
|
badListing: { sourceId: "s", listing: { cursor: 3 }, changeToken: null, uploads: {} },
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
)?.targets,
|
||||||
|
).toEqual({ good: { sourceId: "s", listing: null, changeToken: null, uploads: {} } });
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("chunk uploader", () => {
|
||||||
|
it("PUTs raw bytes with the session and space", async () => {
|
||||||
|
const fetch = vi.fn(async () => new Response(null, { status: 204 }));
|
||||||
|
const upload = createChunkUploader({
|
||||||
|
origin: "https://rakazo.example.com",
|
||||||
|
spaceId: "space_1",
|
||||||
|
fetch,
|
||||||
|
});
|
||||||
|
const signal = new AbortController().signal;
|
||||||
|
await upload("up/1", 3, new Uint8Array([1, 2, 3]), signal);
|
||||||
|
expect(fetch).toHaveBeenCalledWith(
|
||||||
|
"https://rakazo.example.com/api/media/uploads/up%2F1/chunks/3",
|
||||||
|
expect.objectContaining({
|
||||||
|
method: "PUT",
|
||||||
|
credentials: "include",
|
||||||
|
redirect: "error",
|
||||||
|
signal,
|
||||||
|
headers: { "content-type": "application/octet-stream", "x-rakazo-space-id": "space_1" },
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects failed chunks and non-origin targets", async () => {
|
||||||
|
const fetch = vi.fn(async () => new Response(null, { status: 409 }));
|
||||||
|
const upload = createChunkUploader({
|
||||||
|
origin: "https://rakazo.example.com",
|
||||||
|
spaceId: null,
|
||||||
|
fetch,
|
||||||
|
});
|
||||||
|
await expect(
|
||||||
|
upload("u", 0, new Uint8Array(), new AbortController().signal),
|
||||||
|
).rejects.toMatchObject({
|
||||||
|
code: "upload_failed",
|
||||||
|
});
|
||||||
|
expect(() =>
|
||||||
|
createChunkUploader({ origin: "https://rakazo.example.com/rpc", spaceId: null, fetch }),
|
||||||
|
).toThrow();
|
||||||
|
expect(() => createChunkUploader({ origin: "file:///etc", spaceId: null, fetch })).toThrow();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("runPool", () => {
|
||||||
|
it("bounds concurrency and reports the first failure", async () => {
|
||||||
|
let active = 0;
|
||||||
|
let max = 0;
|
||||||
|
const seen: number[] = [];
|
||||||
|
await expect(
|
||||||
|
runPool([1, 2, 3, 4, 5], 2, new AbortController().signal, async (item, signal) => {
|
||||||
|
active += 1;
|
||||||
|
max = Math.max(max, active);
|
||||||
|
await new Promise((resolve) => setTimeout(resolve, 5));
|
||||||
|
active -= 1;
|
||||||
|
if (item === 2) throw new Error("boom");
|
||||||
|
signal.throwIfAborted();
|
||||||
|
seen.push(item);
|
||||||
|
}),
|
||||||
|
).rejects.toThrow("boom");
|
||||||
|
expect(max).toBe(2);
|
||||||
|
expect(seen).not.toContain(4);
|
||||||
|
});
|
||||||
|
});
|
||||||
581
apps/desktop/src/media-photos-sync.ts
Normal file
581
apps/desktop/src/media-photos-sync.ts
Normal file
@ -0,0 +1,581 @@
|
|||||||
|
import { createHash, randomUUID } from "node:crypto";
|
||||||
|
import { createReadStream } from "node:fs";
|
||||||
|
import { mkdtemp, open, rm, stat } from "node:fs/promises";
|
||||||
|
import path from "node:path";
|
||||||
|
import type {
|
||||||
|
ClientMediaItem,
|
||||||
|
MediaAsset,
|
||||||
|
MediaUpload,
|
||||||
|
mediaRpcShapes,
|
||||||
|
PhotosSyncProgress,
|
||||||
|
PhotosSyncResult,
|
||||||
|
} from "@rakazo/contracts";
|
||||||
|
import type { PhotosHelper } from "./media-photos-helper.js";
|
||||||
|
import { PHOTOS_MAX_PAGE } from "./media-photos-helper.js";
|
||||||
|
import { readPrivateFile, writePrivateFile } from "./setup-store.js";
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Apple Photos is a client-pushed source: the API may run on another machine, so the
|
||||||
|
* app never sends a local path. It pushes metadata in batches and uploads the originals
|
||||||
|
* the server asks for (`awaiting_upload`) in resumable raw-byte chunks.
|
||||||
|
*/
|
||||||
|
|
||||||
|
type Shapes = typeof mediaRpcShapes;
|
||||||
|
type Shape = { input: { _input: unknown }; output: { _output: unknown } };
|
||||||
|
/** One oRPC procedure as `ContractRouterClient<AppContract>` exposes it. */
|
||||||
|
type Procedure<S extends Shape> = (
|
||||||
|
input: S["input"]["_input"],
|
||||||
|
options?: { signal?: AbortSignal },
|
||||||
|
) => Promise<S["output"]["_output"]>;
|
||||||
|
|
||||||
|
/** The slice of the app RPC client the sync needs; `rpc` from a contract client satisfies it. */
|
||||||
|
export interface MediaSyncApi {
|
||||||
|
media: {
|
||||||
|
sources: {
|
||||||
|
registerClient: Procedure<Shapes["sources"]["registerClient"]>;
|
||||||
|
pushItems: Procedure<Shapes["sources"]["pushItems"]>;
|
||||||
|
};
|
||||||
|
assets: { search: Procedure<Shapes["assets"]["search"]> };
|
||||||
|
uploads: {
|
||||||
|
begin: Procedure<Shapes["uploads"]["begin"]>;
|
||||||
|
status: Procedure<Shapes["uploads"]["status"]>;
|
||||||
|
complete: Procedure<Shapes["uploads"]["complete"]>;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Sends one raw chunk to `PUT /api/media/uploads/:uploadId/chunks/:index`. */
|
||||||
|
export type UploadChunk = (
|
||||||
|
uploadId: string,
|
||||||
|
index: number,
|
||||||
|
bytes: Uint8Array<ArrayBuffer>,
|
||||||
|
signal: AbortSignal,
|
||||||
|
) => Promise<void>;
|
||||||
|
|
||||||
|
export type PhotosSyncErrorCode = "not_authorized" | "cancelled" | "integrity" | "upload_failed";
|
||||||
|
|
||||||
|
export class PhotosSyncError extends Error {
|
||||||
|
constructor(
|
||||||
|
readonly code: PhotosSyncErrorCode,
|
||||||
|
message: string,
|
||||||
|
) {
|
||||||
|
super(message);
|
||||||
|
this.name = "PhotosSyncError";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Persistent state
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
export interface PhotosSyncTargetState {
|
||||||
|
sourceId: string;
|
||||||
|
/** A full listing in progress: where to resume, the token from its start and the last pushed item. */
|
||||||
|
listing: { cursor: string | null; token: string | null; lastItem: ClientMediaItem | null } | null;
|
||||||
|
/** Set after a complete listing; later syncs only read changes since it. */
|
||||||
|
changeToken: string | null;
|
||||||
|
/** Open uploads by asset id so an interrupted upload resumes instead of restarting. */
|
||||||
|
uploads: Record<string, string>;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface PhotosSyncState {
|
||||||
|
version: 1;
|
||||||
|
/** Stable per installation; the server keys the Apple Photos source by it. */
|
||||||
|
deviceId: string;
|
||||||
|
/** Keyed by server origin and space, since each has its own source. */
|
||||||
|
targets: Record<string, PhotosSyncTargetState>;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface PhotosSyncStore {
|
||||||
|
read(): Promise<PhotosSyncState>;
|
||||||
|
write(state: PhotosSyncState): Promise<void>;
|
||||||
|
}
|
||||||
|
|
||||||
|
export const PHOTOS_SYNC_FILE_NAME = "media-photos-sync.json";
|
||||||
|
const MAX_STATE_BYTES = 4 * 1024 * 1024;
|
||||||
|
|
||||||
|
/** Kept next to the desktop setup file, private to the user and written atomically. */
|
||||||
|
export function createPhotosSyncFileStore(
|
||||||
|
userDataDir: string,
|
||||||
|
newDeviceId: () => string = randomUUID,
|
||||||
|
): PhotosSyncStore {
|
||||||
|
const file = path.join(userDataDir, PHOTOS_SYNC_FILE_NAME);
|
||||||
|
return {
|
||||||
|
async read() {
|
||||||
|
const raw = await readPrivateFile(file, MAX_STATE_BYTES);
|
||||||
|
return (raw === null ? null : parsePhotosSyncState(raw)) ?? emptyState(newDeviceId());
|
||||||
|
},
|
||||||
|
write: (state) => writePrivateFile(file, `${JSON.stringify(state)}\n`),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function emptyState(deviceId: string): PhotosSyncState {
|
||||||
|
return { version: 1, deviceId, targets: {} };
|
||||||
|
}
|
||||||
|
|
||||||
|
const isRecord = (value: unknown): value is Record<string, unknown> =>
|
||||||
|
typeof value === "object" && value !== null && !Array.isArray(value);
|
||||||
|
const isNullableString = (value: unknown) => value === null || typeof value === "string";
|
||||||
|
|
||||||
|
/** Unreadable state starts over: the server deduplicates by device and external id. */
|
||||||
|
export function parsePhotosSyncState(raw: string): PhotosSyncState | null {
|
||||||
|
let value: unknown;
|
||||||
|
try {
|
||||||
|
value = JSON.parse(raw);
|
||||||
|
} catch {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
if (
|
||||||
|
!isRecord(value) ||
|
||||||
|
value.version !== 1 ||
|
||||||
|
typeof value.deviceId !== "string" ||
|
||||||
|
value.deviceId === "" ||
|
||||||
|
value.deviceId.length > 200 ||
|
||||||
|
!isRecord(value.targets)
|
||||||
|
) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
const targets: Record<string, PhotosSyncTargetState> = {};
|
||||||
|
for (const [key, target] of Object.entries(value.targets)) {
|
||||||
|
if (
|
||||||
|
!isRecord(target) ||
|
||||||
|
typeof target.sourceId !== "string" ||
|
||||||
|
!isNullableString(target.changeToken) ||
|
||||||
|
!isRecord(target.uploads) ||
|
||||||
|
!Object.values(target.uploads).every((id) => typeof id === "string")
|
||||||
|
) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const listing = target.listing;
|
||||||
|
const validListing =
|
||||||
|
listing === null ||
|
||||||
|
(isRecord(listing) &&
|
||||||
|
isNullableString(listing.cursor) &&
|
||||||
|
isNullableString(listing.token) &&
|
||||||
|
(listing.lastItem === null || isRecord(listing.lastItem)));
|
||||||
|
if (!validListing) continue;
|
||||||
|
targets[key] = target as unknown as PhotosSyncTargetState;
|
||||||
|
}
|
||||||
|
return { version: 1, deviceId: value.deviceId, targets };
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Chunk upload over HTTP
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Chunk PUTs ride the same signed-in session as the app window: the injected `fetch`
|
||||||
|
* must carry that session's cookies (Electron `session.fetch` with credentials).
|
||||||
|
*/
|
||||||
|
export function createChunkUploader(options: {
|
||||||
|
origin: string;
|
||||||
|
spaceId: string | null;
|
||||||
|
fetch: (url: string, init: RequestInit) => Promise<Response>;
|
||||||
|
}): UploadChunk {
|
||||||
|
const origin = new URL(options.origin);
|
||||||
|
if (!["http:", "https:"].includes(origin.protocol) || origin.origin !== options.origin) {
|
||||||
|
throw new Error("Uploads need the server origin.");
|
||||||
|
}
|
||||||
|
return async (uploadId, index, bytes, signal) => {
|
||||||
|
if (!Number.isSafeInteger(index) || index < 0) throw new Error("Invalid chunk index.");
|
||||||
|
const headers: Record<string, string> = { "content-type": "application/octet-stream" };
|
||||||
|
if (options.spaceId) headers["x-rakazo-space-id"] = options.spaceId;
|
||||||
|
const response = await options.fetch(
|
||||||
|
`${origin.origin}/api/media/uploads/${encodeURIComponent(uploadId)}/chunks/${index}`,
|
||||||
|
{
|
||||||
|
method: "PUT",
|
||||||
|
headers,
|
||||||
|
body: bytes,
|
||||||
|
credentials: "include",
|
||||||
|
redirect: "error",
|
||||||
|
signal,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
if (!response.ok) {
|
||||||
|
throw new PhotosSyncError(
|
||||||
|
"upload_failed",
|
||||||
|
`The server rejected an upload (${response.status}).`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Sync
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
export type { PhotosSyncProgress, PhotosSyncResult };
|
||||||
|
|
||||||
|
export interface PhotosSyncOptions {
|
||||||
|
helper: PhotosHelper;
|
||||||
|
api: MediaSyncApi;
|
||||||
|
uploadChunk: UploadChunk;
|
||||||
|
store: PhotosSyncStore;
|
||||||
|
/** Server origin plus space, e.g. `https://rakazo.example.com#space_1`. */
|
||||||
|
target: string;
|
||||||
|
/** Source label shown in the library, e.g. the Mac's name or "Apple Photos". */
|
||||||
|
label: string;
|
||||||
|
/** Exports land in a fresh folder below this and are deleted after upload. */
|
||||||
|
tempRoot: string;
|
||||||
|
signal: AbortSignal;
|
||||||
|
onProgress?: (progress: PhotosSyncProgress) => void;
|
||||||
|
/** At most 500, the contract's batch limit. */
|
||||||
|
batchSize?: number;
|
||||||
|
concurrency?: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
const MAX_SYNC_TOKEN_LENGTH = 2000;
|
||||||
|
|
||||||
|
export async function syncApplePhotos(options: PhotosSyncOptions): Promise<PhotosSyncResult> {
|
||||||
|
const { helper, api, signal } = options;
|
||||||
|
const batchSize = Math.min(Math.max(1, options.batchSize ?? PHOTOS_MAX_PAGE), PHOTOS_MAX_PAGE);
|
||||||
|
const result: PhotosSyncResult = {
|
||||||
|
indexed: 0,
|
||||||
|
created: 0,
|
||||||
|
updated: 0,
|
||||||
|
unchanged: 0,
|
||||||
|
uploaded: 0,
|
||||||
|
fullListing: false,
|
||||||
|
};
|
||||||
|
const progress = { uploadsDone: 0, uploadsTotal: 0 };
|
||||||
|
const report = (phase: PhotosSyncProgress["phase"]) =>
|
||||||
|
options.onProgress?.({ phase, indexed: result.indexed, ...progress });
|
||||||
|
|
||||||
|
const state = await options.store.read();
|
||||||
|
// Writes are serialized so concurrent upload workers never interleave partial state.
|
||||||
|
let saving = Promise.resolve();
|
||||||
|
const save = () => {
|
||||||
|
const snapshot = JSON.parse(JSON.stringify(state)) as PhotosSyncState;
|
||||||
|
saving = saving.then(() => options.store.write(snapshot));
|
||||||
|
return saving;
|
||||||
|
};
|
||||||
|
|
||||||
|
try {
|
||||||
|
report("authorizing");
|
||||||
|
const authorization = await helper.authorize(true, signal);
|
||||||
|
if (authorization !== "authorized" && authorization !== "limited") {
|
||||||
|
throw new PhotosSyncError("not_authorized", "Photos access was not granted.");
|
||||||
|
}
|
||||||
|
|
||||||
|
const ensureTarget = async (): Promise<PhotosSyncTargetState> => {
|
||||||
|
const existing = state.targets[options.target];
|
||||||
|
if (existing) return existing;
|
||||||
|
const source = await api.media.sources.registerClient(
|
||||||
|
{ kind: "apple_photos", label: options.label, deviceId: state.deviceId },
|
||||||
|
{ signal },
|
||||||
|
);
|
||||||
|
const created: PhotosSyncTargetState = {
|
||||||
|
sourceId: source.id,
|
||||||
|
listing: null,
|
||||||
|
changeToken: null,
|
||||||
|
uploads: {},
|
||||||
|
};
|
||||||
|
state.targets[options.target] = created;
|
||||||
|
await save();
|
||||||
|
return created;
|
||||||
|
};
|
||||||
|
|
||||||
|
const push = async (
|
||||||
|
target: PhotosSyncTargetState,
|
||||||
|
items: ClientMediaItem[],
|
||||||
|
syncToken: string | null,
|
||||||
|
complete: boolean,
|
||||||
|
deletedExternalIds: string[] = [],
|
||||||
|
) => {
|
||||||
|
const counts = await api.media.sources.pushItems(
|
||||||
|
{
|
||||||
|
sourceId: target.sourceId,
|
||||||
|
items,
|
||||||
|
deletedExternalIds,
|
||||||
|
syncToken:
|
||||||
|
syncToken !== null && syncToken.length <= MAX_SYNC_TOKEN_LENGTH ? syncToken : null,
|
||||||
|
complete,
|
||||||
|
},
|
||||||
|
{ signal },
|
||||||
|
);
|
||||||
|
result.indexed += items.length;
|
||||||
|
result.created += counts.created;
|
||||||
|
result.updated += counts.updated;
|
||||||
|
result.unchanged += counts.unchanged;
|
||||||
|
report("indexing");
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Full listing, resumable page by page; the last push says `complete`. */
|
||||||
|
const listAll = async (target: PhotosSyncTargetState) => {
|
||||||
|
result.fullListing = true;
|
||||||
|
target.listing ??= { cursor: null, token: null, lastItem: null };
|
||||||
|
const listing = target.listing;
|
||||||
|
while (true) {
|
||||||
|
const page = await helper.list({ cursor: listing.cursor, limit: batchSize }, signal);
|
||||||
|
if (listing.cursor === null) listing.token = page.changeToken;
|
||||||
|
const done = page.nextCursor === null;
|
||||||
|
const items = page.items;
|
||||||
|
// An empty final page still closes the listing.
|
||||||
|
if (items.length > 0 || done) await push(target, items, done ? listing.token : null, done);
|
||||||
|
if (done) {
|
||||||
|
target.changeToken = listing.token;
|
||||||
|
target.listing = null;
|
||||||
|
await save();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
listing.cursor = page.nextCursor;
|
||||||
|
listing.lastItem = items.at(-1) ?? listing.lastItem;
|
||||||
|
await save();
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const syncMetadata = async () => {
|
||||||
|
const target = await ensureTarget();
|
||||||
|
if (target.listing !== null || target.changeToken === null) {
|
||||||
|
await listAll(target);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const changes = await helper.changes(target.changeToken, signal);
|
||||||
|
if (changes.fullResyncRequired || changes.token === null) {
|
||||||
|
target.listing = null;
|
||||||
|
await listAll(target);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const batches = Math.max(
|
||||||
|
Math.ceil(changes.changed.length / batchSize),
|
||||||
|
Math.ceil(changes.deleted.length / DELETED_BATCH_SIZE),
|
||||||
|
);
|
||||||
|
for (let index = 0; index < batches; index++) {
|
||||||
|
const batch = changes.changed.slice(index * batchSize, (index + 1) * batchSize);
|
||||||
|
const deleted = changes.deleted.slice(
|
||||||
|
index * DELETED_BATCH_SIZE,
|
||||||
|
(index + 1) * DELETED_BATCH_SIZE,
|
||||||
|
);
|
||||||
|
const last = index === batches - 1;
|
||||||
|
await push(target, batch, last ? changes.token : null, false, deleted);
|
||||||
|
}
|
||||||
|
target.changeToken = changes.token;
|
||||||
|
await save();
|
||||||
|
};
|
||||||
|
|
||||||
|
report("indexing");
|
||||||
|
try {
|
||||||
|
await syncMetadata();
|
||||||
|
} catch (error) {
|
||||||
|
// The source was removed on the server: register again and list from scratch.
|
||||||
|
if (!isNotFound(error) || signal.aborted) throw error;
|
||||||
|
delete state.targets[options.target];
|
||||||
|
await save();
|
||||||
|
await syncMetadata();
|
||||||
|
}
|
||||||
|
|
||||||
|
const target = state.targets[options.target]!;
|
||||||
|
const awaiting = await awaitingUploads(api, target.sourceId, signal);
|
||||||
|
progress.uploadsTotal = awaiting.length;
|
||||||
|
// Forget uploads for assets the server no longer waits for.
|
||||||
|
const waiting = new Set(awaiting.map((asset) => asset.id));
|
||||||
|
for (const assetId of Object.keys(target.uploads)) {
|
||||||
|
if (!waiting.has(assetId)) delete target.uploads[assetId];
|
||||||
|
}
|
||||||
|
await save();
|
||||||
|
report("uploading");
|
||||||
|
await runPool(awaiting, options.concurrency ?? 2, signal, async (asset, workerSignal) => {
|
||||||
|
await uploadAsset({ ...options, signal: workerSignal }, target, asset, save);
|
||||||
|
result.uploaded += 1;
|
||||||
|
progress.uploadsDone += 1;
|
||||||
|
report("uploading");
|
||||||
|
});
|
||||||
|
return result;
|
||||||
|
} catch (error) {
|
||||||
|
if (signal.aborted) throw new PhotosSyncError("cancelled", "Sync cancelled.");
|
||||||
|
throw error;
|
||||||
|
} finally {
|
||||||
|
await saving.catch(() => undefined);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Binds the sync to the server and space the request names. `connect` builds the RPC
|
||||||
|
* client and chunk uploader on the signed-in session for that space.
|
||||||
|
*/
|
||||||
|
export function createPhotosSyncRunner(deps: {
|
||||||
|
helper: PhotosHelper;
|
||||||
|
store: PhotosSyncStore;
|
||||||
|
tempRoot: string;
|
||||||
|
label: string;
|
||||||
|
connect: (
|
||||||
|
spaceId: string | null,
|
||||||
|
) => Promise<{ origin: string; api: MediaSyncApi; uploadChunk: UploadChunk }>;
|
||||||
|
}) {
|
||||||
|
return async (request: {
|
||||||
|
spaceId: string | null;
|
||||||
|
signal: AbortSignal;
|
||||||
|
onProgress: (progress: PhotosSyncProgress) => void;
|
||||||
|
}): Promise<PhotosSyncResult> => {
|
||||||
|
const { origin, api, uploadChunk } = await deps.connect(request.spaceId);
|
||||||
|
return syncApplePhotos({
|
||||||
|
helper: deps.helper,
|
||||||
|
store: deps.store,
|
||||||
|
tempRoot: deps.tempRoot,
|
||||||
|
label: deps.label,
|
||||||
|
api,
|
||||||
|
uploadChunk,
|
||||||
|
target: `${origin}#${request.spaceId ?? "default"}`,
|
||||||
|
signal: request.signal,
|
||||||
|
onProgress: request.onProgress,
|
||||||
|
});
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function isNotFound(error: unknown): boolean {
|
||||||
|
return isRecord(error) && error.code === "NOT_FOUND";
|
||||||
|
}
|
||||||
|
|
||||||
|
interface AwaitingAsset {
|
||||||
|
id: string;
|
||||||
|
externalId: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Assets of this source the server is waiting for, mapped back to Photos items. */
|
||||||
|
const DELETED_BATCH_SIZE = 5000;
|
||||||
|
|
||||||
|
export async function awaitingUploads(
|
||||||
|
api: MediaSyncApi,
|
||||||
|
sourceId: string,
|
||||||
|
signal: AbortSignal,
|
||||||
|
): Promise<AwaitingAsset[]> {
|
||||||
|
const awaiting: AwaitingAsset[] = [];
|
||||||
|
let cursor: string | null = null;
|
||||||
|
do {
|
||||||
|
const page: { items: MediaAsset[]; nextCursor: string | null } = await api.media.assets.search(
|
||||||
|
{ sourceId, ingestStates: ["awaiting_upload"], cursor, limit: PHOTOS_MAX_PAGE },
|
||||||
|
{ signal },
|
||||||
|
);
|
||||||
|
for (const asset of page.items) {
|
||||||
|
if (asset.ingestState !== "awaiting_upload" || asset.sourceId !== sourceId) continue;
|
||||||
|
awaiting.push({ id: asset.id, externalId: asset.externalId });
|
||||||
|
}
|
||||||
|
cursor = page.nextCursor;
|
||||||
|
} while (cursor !== null);
|
||||||
|
return awaiting;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function uploadAsset(
|
||||||
|
options: PhotosSyncOptions,
|
||||||
|
target: PhotosSyncTargetState,
|
||||||
|
asset: AwaitingAsset,
|
||||||
|
save: () => Promise<void>,
|
||||||
|
): Promise<void> {
|
||||||
|
const { api, signal } = options;
|
||||||
|
const directory = await mkdtemp(path.join(options.tempRoot, "rakazo-photos-"));
|
||||||
|
try {
|
||||||
|
const exported = await options.helper.exportOriginal(asset.externalId, directory, { signal });
|
||||||
|
// The helper's hash is checked against the bytes on disk before anything is sent.
|
||||||
|
const local = await hashFile(exported.path, signal);
|
||||||
|
if (local.sha256 !== exported.sha256 || local.sizeBytes !== exported.sizeBytes) {
|
||||||
|
throw new PhotosSyncError("integrity", "The exported file does not match its checksum.");
|
||||||
|
}
|
||||||
|
|
||||||
|
let upload: MediaUpload | null = null;
|
||||||
|
const previous = target.uploads[asset.id];
|
||||||
|
if (previous) {
|
||||||
|
const status = await api.media.uploads
|
||||||
|
.status({ uploadId: previous }, { signal })
|
||||||
|
.catch((error: unknown) => {
|
||||||
|
if (isNotFound(error)) return null;
|
||||||
|
throw error;
|
||||||
|
});
|
||||||
|
if (status?.status === "complete") {
|
||||||
|
delete target.uploads[asset.id];
|
||||||
|
await save();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (status?.status === "open" && status.sizeBytes === local.sizeBytes) upload = status;
|
||||||
|
}
|
||||||
|
if (upload === null) {
|
||||||
|
upload = await api.media.uploads.begin(
|
||||||
|
{
|
||||||
|
assetId: asset.id,
|
||||||
|
sizeBytes: local.sizeBytes,
|
||||||
|
sha256: local.sha256,
|
||||||
|
fileName: exported.fileName.slice(0, 255),
|
||||||
|
},
|
||||||
|
{ signal },
|
||||||
|
);
|
||||||
|
target.uploads[asset.id] = upload.id;
|
||||||
|
await save();
|
||||||
|
}
|
||||||
|
|
||||||
|
const received = new Set(upload.receivedChunks);
|
||||||
|
const chunkCount = Math.ceil(local.sizeBytes / upload.chunkBytes);
|
||||||
|
const file = await open(exported.path, "r");
|
||||||
|
try {
|
||||||
|
for (let index = 0; index < chunkCount; index += 1) {
|
||||||
|
if (received.has(index)) continue;
|
||||||
|
signal.throwIfAborted();
|
||||||
|
const offset = index * upload.chunkBytes;
|
||||||
|
const length = Math.min(upload.chunkBytes, local.sizeBytes - offset);
|
||||||
|
const bytes = Buffer.alloc(length);
|
||||||
|
const { bytesRead } = await file.read(bytes, 0, length, offset);
|
||||||
|
if (bytesRead !== length) {
|
||||||
|
throw new PhotosSyncError("integrity", "The exported file changed during upload.");
|
||||||
|
}
|
||||||
|
await options.uploadChunk(upload.id, index, bytes, signal);
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
await file.close();
|
||||||
|
}
|
||||||
|
const completed = await api.media.uploads.complete({ uploadId: upload.id }, { signal });
|
||||||
|
if (completed.status !== "complete") {
|
||||||
|
throw new PhotosSyncError("upload_failed", "The server did not accept the upload.");
|
||||||
|
}
|
||||||
|
delete target.uploads[asset.id];
|
||||||
|
await save();
|
||||||
|
} finally {
|
||||||
|
await rm(directory, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function hashFile(
|
||||||
|
file: string,
|
||||||
|
signal?: AbortSignal,
|
||||||
|
): Promise<{ sha256: string; sizeBytes: number }> {
|
||||||
|
const hash = createHash("sha256");
|
||||||
|
for await (const chunk of createReadStream(file, { signal })) hash.update(chunk as Buffer);
|
||||||
|
return { sha256: hash.digest("hex"), sizeBytes: (await stat(file)).size };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Runs `work` with at most `limit` items in flight. The first failure aborts the
|
||||||
|
* others and is the error reported.
|
||||||
|
*/
|
||||||
|
export async function runPool<T>(
|
||||||
|
items: T[],
|
||||||
|
limit: number,
|
||||||
|
signal: AbortSignal,
|
||||||
|
work: (item: T, signal: AbortSignal) => Promise<void>,
|
||||||
|
): Promise<void> {
|
||||||
|
const controller = new AbortController();
|
||||||
|
const forward = () => controller.abort(signal.reason);
|
||||||
|
signal.addEventListener("abort", forward, { once: true });
|
||||||
|
if (signal.aborted) forward();
|
||||||
|
let next = 0;
|
||||||
|
let failed = false;
|
||||||
|
let failure: unknown;
|
||||||
|
const worker = async () => {
|
||||||
|
while (!failed && !controller.signal.aborted && next < items.length) {
|
||||||
|
const item = items[next++]!;
|
||||||
|
try {
|
||||||
|
await work(item, controller.signal);
|
||||||
|
} catch (error) {
|
||||||
|
if (!failed) {
|
||||||
|
failed = true;
|
||||||
|
failure = error;
|
||||||
|
controller.abort(error);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
try {
|
||||||
|
await Promise.all(Array.from({ length: Math.min(Math.max(1, limit), items.length) }, worker));
|
||||||
|
} finally {
|
||||||
|
signal.removeEventListener("abort", forward);
|
||||||
|
}
|
||||||
|
if (failed) throw failure;
|
||||||
|
signal.throwIfAborted();
|
||||||
|
}
|
||||||
@ -9,6 +9,18 @@ contextBridge.exposeInMainWorld("rakazoDesktop", {
|
|||||||
request: (pathname, body) =>
|
request: (pathname, body) =>
|
||||||
ipcRenderer.invoke("desktop.localSettings.request", pathname, body),
|
ipcRenderer.invoke("desktop.localSettings.request", pathname, body),
|
||||||
},
|
},
|
||||||
|
media: {
|
||||||
|
photos: {
|
||||||
|
status: () => ipcRenderer.invoke("desktop.media.photos.status"),
|
||||||
|
sync: (request) => ipcRenderer.invoke("desktop.media.photos.sync", request),
|
||||||
|
cancel: () => ipcRenderer.invoke("desktop.media.photos.cancel"),
|
||||||
|
onProgress: (listener) => {
|
||||||
|
const handler = (_event, progress) => listener(progress);
|
||||||
|
ipcRenderer.on("desktop.media.photos.progress", handler);
|
||||||
|
return () => ipcRenderer.off("desktop.media.photos.progress", handler);
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
window: {
|
window: {
|
||||||
close: () => ipcRenderer.invoke("desktop.window.close"),
|
close: () => ipcRenderer.invoke("desktop.window.close"),
|
||||||
minimize: () => ipcRenderer.invoke("desktop.window.minimize"),
|
minimize: () => ipcRenderer.invoke("desktop.window.minimize"),
|
||||||
@ -21,6 +33,22 @@ contextBridge.exposeInMainWorld("rakazoDesktop", {
|
|||||||
download: () => ipcRenderer.invoke("desktop.update.download"),
|
download: () => ipcRenderer.invoke("desktop.update.download"),
|
||||||
install: () => ipcRenderer.invoke("desktop.update.install"),
|
install: () => ipcRenderer.invoke("desktop.update.install"),
|
||||||
},
|
},
|
||||||
|
browserPreview: {
|
||||||
|
state: () => ipcRenderer.invoke("desktop.browserPreview.state"),
|
||||||
|
show: (bounds) => ipcRenderer.invoke("desktop.browserPreview.show", bounds),
|
||||||
|
hide: () => ipcRenderer.invoke("desktop.browserPreview.hide"),
|
||||||
|
navigate: (url) => ipcRenderer.invoke("desktop.browserPreview.navigate", url),
|
||||||
|
back: () => ipcRenderer.invoke("desktop.browserPreview.back"),
|
||||||
|
forward: () => ipcRenderer.invoke("desktop.browserPreview.forward"),
|
||||||
|
reload: () => ipcRenderer.invoke("desktop.browserPreview.reload"),
|
||||||
|
stop: () => ipcRenderer.invoke("desktop.browserPreview.stop"),
|
||||||
|
openDevTools: () => ipcRenderer.invoke("desktop.browserPreview.openDevTools"),
|
||||||
|
onState: (listener) => {
|
||||||
|
const handler = (_event, state) => listener(state);
|
||||||
|
ipcRenderer.on("desktop.browserPreview.state", handler);
|
||||||
|
return () => ipcRenderer.off("desktop.browserPreview.state", handler);
|
||||||
|
},
|
||||||
|
},
|
||||||
oauth: {
|
oauth: {
|
||||||
open: (url) => ipcRenderer.invoke("desktop.oauth.open", url),
|
open: (url) => ipcRenderer.invoke("desktop.oauth.open", url),
|
||||||
cancel: (url) => ipcRenderer.invoke("desktop.oauth.cancel", url),
|
cancel: (url) => ipcRenderer.invoke("desktop.oauth.cancel", url),
|
||||||
|
|||||||
@ -32,8 +32,10 @@ describe("desktop preload bridge", () => {
|
|||||||
expect(globalName).toBe("rakazoDesktop");
|
expect(globalName).toBe("rakazoDesktop");
|
||||||
expect(bridge.platform).toBe("linux");
|
expect(bridge.platform).toBe("linux");
|
||||||
expect(Object.keys(bridge).sort()).toEqual([
|
expect(Object.keys(bridge).sort()).toEqual([
|
||||||
|
"browserPreview",
|
||||||
"files",
|
"files",
|
||||||
"localSettings",
|
"localSettings",
|
||||||
|
"media",
|
||||||
"oauth",
|
"oauth",
|
||||||
"platform",
|
"platform",
|
||||||
"update",
|
"update",
|
||||||
@ -46,6 +48,24 @@ describe("desktop preload bridge", () => {
|
|||||||
"toggleMaximize",
|
"toggleMaximize",
|
||||||
]);
|
]);
|
||||||
expect(Object.keys(bridge.update).sort()).toEqual(["check", "download", "install", "state"]);
|
expect(Object.keys(bridge.update).sort()).toEqual(["check", "download", "install", "state"]);
|
||||||
|
expect(Object.keys(bridge.media?.photos ?? {}).sort()).toEqual([
|
||||||
|
"cancel",
|
||||||
|
"onProgress",
|
||||||
|
"status",
|
||||||
|
"sync",
|
||||||
|
]);
|
||||||
|
expect(Object.keys(bridge.browserPreview ?? {}).sort()).toEqual([
|
||||||
|
"back",
|
||||||
|
"forward",
|
||||||
|
"hide",
|
||||||
|
"navigate",
|
||||||
|
"onState",
|
||||||
|
"openDevTools",
|
||||||
|
"reload",
|
||||||
|
"show",
|
||||||
|
"state",
|
||||||
|
"stop",
|
||||||
|
]);
|
||||||
|
|
||||||
await bridge.files?.pickDirectory();
|
await bridge.files?.pickDirectory();
|
||||||
await bridge.oauth.open?.("https://provider.example.com/authorize");
|
await bridge.oauth.open?.("https://provider.example.com/authorize");
|
||||||
@ -58,6 +78,8 @@ describe("desktop preload bridge", () => {
|
|||||||
await bridge.update.check();
|
await bridge.update.check();
|
||||||
await bridge.update.download();
|
await bridge.update.download();
|
||||||
await bridge.update.install();
|
await bridge.update.install();
|
||||||
|
await bridge.browserPreview?.navigate("http://localhost:5173/");
|
||||||
|
await bridge.browserPreview?.show({ x: 1, y: 2, width: 3, height: 4 });
|
||||||
expect(invoke.mock.calls.map(([channel]) => channel)).toEqual([
|
expect(invoke.mock.calls.map(([channel]) => channel)).toEqual([
|
||||||
"desktop.files.pickDirectory",
|
"desktop.files.pickDirectory",
|
||||||
"desktop.oauth.open",
|
"desktop.oauth.open",
|
||||||
@ -70,6 +92,8 @@ describe("desktop preload bridge", () => {
|
|||||||
"desktop.update.check",
|
"desktop.update.check",
|
||||||
"desktop.update.download",
|
"desktop.update.download",
|
||||||
"desktop.update.install",
|
"desktop.update.install",
|
||||||
|
"desktop.browserPreview.navigate",
|
||||||
|
"desktop.browserPreview.show",
|
||||||
]);
|
]);
|
||||||
});
|
});
|
||||||
|
|
||||||
@ -77,8 +101,10 @@ describe("desktop preload bridge", () => {
|
|||||||
const { exposeInMainWorld } = runPreload("preload.cjs");
|
const { exposeInMainWorld } = runPreload("preload.cjs");
|
||||||
const [, bridge] = exposeInMainWorld.mock.calls[0] as [string, Record<string, unknown>];
|
const [, bridge] = exposeInMainWorld.mock.calls[0] as [string, Record<string, unknown>];
|
||||||
expect(Object.keys(bridge).sort()).toEqual([
|
expect(Object.keys(bridge).sort()).toEqual([
|
||||||
|
"browserPreview",
|
||||||
"files",
|
"files",
|
||||||
"localSettings",
|
"localSettings",
|
||||||
|
"media",
|
||||||
"oauth",
|
"oauth",
|
||||||
"platform",
|
"platform",
|
||||||
"update",
|
"update",
|
||||||
|
|||||||
@ -33,12 +33,21 @@
|
|||||||
[
|
[
|
||||||
"expo-image-picker",
|
"expo-image-picker",
|
||||||
{
|
{
|
||||||
"photosPermission": "Rakazo uses your photo library so you can attach images to a bot message.",
|
"photosPermission": "Rakazo uses your photo library so you can attach images and turn your photos and videos into projects.",
|
||||||
"cameraPermission": "Rakazo uses the camera so you can attach photos to a bot message.",
|
"cameraPermission": "Rakazo uses the camera so you can attach photos to a bot message.",
|
||||||
"microphonePermission": false
|
"microphonePermission": false
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"expo-sharing",
|
"expo-sharing",
|
||||||
|
[
|
||||||
|
"expo-media-library",
|
||||||
|
{
|
||||||
|
"photosPermission": "Rakazo uses your photo library so you can attach images and turn your photos and videos into projects.",
|
||||||
|
"savePhotosPermission": false,
|
||||||
|
"isAccessMediaLocationEnabled": true,
|
||||||
|
"granularPermissions": ["photo", "video"]
|
||||||
|
}
|
||||||
|
],
|
||||||
"./plugins/with-worklets-headers",
|
"./plugins/with-worklets-headers",
|
||||||
"./plugins/with-scene-lifecycle"
|
"./plugins/with-scene-lifecycle"
|
||||||
],
|
],
|
||||||
@ -52,7 +61,7 @@
|
|||||||
},
|
},
|
||||||
"infoPlist": {
|
"infoPlist": {
|
||||||
"NSCameraUsageDescription": "Rakazo uses the camera so you can attach photos to a bot message.",
|
"NSCameraUsageDescription": "Rakazo uses the camera so you can attach photos to a bot message.",
|
||||||
"NSPhotoLibraryUsageDescription": "Rakazo uses your photo library so you can attach images to a bot message.",
|
"NSPhotoLibraryUsageDescription": "Rakazo uses your photo library so you can attach images and turn your photos and videos into projects.",
|
||||||
"NSLocalNetworkUsageDescription": "Rakazo connects to a self-hosted Rakazo server on your local network.",
|
"NSLocalNetworkUsageDescription": "Rakazo connects to a self-hosted Rakazo server on your local network.",
|
||||||
"NSAppTransportSecurity": {
|
"NSAppTransportSecurity": {
|
||||||
"NSAllowsLocalNetworking": true
|
"NSAllowsLocalNetworking": true
|
||||||
|
|||||||
@ -13,10 +13,16 @@ import {
|
|||||||
configureForegroundNotifications,
|
configureForegroundNotifications,
|
||||||
resumeLiveNotifications,
|
resumeLiveNotifications,
|
||||||
} from "../lib/live-notifications";
|
} from "../lib/live-notifications";
|
||||||
|
import { useMediaNotificationRouting } from "../lib/media-api";
|
||||||
import { native, useResolvedAppearance } from "../lib/native";
|
import { native, useResolvedAppearance } from "../lib/native";
|
||||||
|
|
||||||
configureForegroundNotifications();
|
configureForegroundNotifications();
|
||||||
|
|
||||||
|
function MediaNotificationRouting() {
|
||||||
|
useMediaNotificationRouting();
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
export default function Layout() {
|
export default function Layout() {
|
||||||
const { t } = useI18n();
|
const { t } = useI18n();
|
||||||
const [ready, setReady] = useState(false);
|
const [ready, setReady] = useState(false);
|
||||||
@ -121,8 +127,12 @@ export default function Layout() {
|
|||||||
<Stack.Screen name="thread" options={{ title: t("Thread") }} />
|
<Stack.Screen name="thread" options={{ title: t("Thread") }} />
|
||||||
<Stack.Screen name="routine" options={{ title: t("Routine") }} />
|
<Stack.Screen name="routine" options={{ title: t("Routine") }} />
|
||||||
<Stack.Screen name="computer" options={{ title: t("Computer") }} />
|
<Stack.Screen name="computer" options={{ title: t("Computer") }} />
|
||||||
|
<Stack.Screen name="media" options={{ title: t("Media") }} />
|
||||||
|
<Stack.Screen name="media-session" options={{ title: "" }} />
|
||||||
|
<Stack.Screen name="media-project" options={{ title: "" }} />
|
||||||
</Stack>
|
</Stack>
|
||||||
<ComputerUpdateProgress />
|
<ComputerUpdateProgress />
|
||||||
|
<MediaNotificationRouting />
|
||||||
</ThemeProvider>
|
</ThemeProvider>
|
||||||
</AvatarStyleProvider>
|
</AvatarStyleProvider>
|
||||||
) : (
|
) : (
|
||||||
|
|||||||
@ -44,6 +44,7 @@ import {
|
|||||||
openPromotedNotificationSettings,
|
openPromotedNotificationSettings,
|
||||||
setLiveNotificationSettings,
|
setLiveNotificationSettings,
|
||||||
} from "../lib/live-notifications";
|
} from "../lib/live-notifications";
|
||||||
|
import { useMediaAvailable } from "../lib/media-api";
|
||||||
import { presentMessageActionSheet } from "../lib/message-action-sheet";
|
import { presentMessageActionSheet } from "../lib/message-action-sheet";
|
||||||
import { native, useResolvedAppearance, useThemedStyles } from "../lib/native";
|
import { native, useResolvedAppearance, useThemedStyles } from "../lib/native";
|
||||||
import { registerPushToken } from "../lib/push";
|
import { registerPushToken } from "../lib/push";
|
||||||
@ -55,6 +56,7 @@ export default function Account() {
|
|||||||
const { t, locale } = useI18n();
|
const { t, locale } = useI18n();
|
||||||
const colorScheme = useResolvedAppearance();
|
const colorScheme = useResolvedAppearance();
|
||||||
const router = useRouter();
|
const router = useRouter();
|
||||||
|
const mediaAvailable = useMediaAvailable();
|
||||||
const { focus } = useLocalSearchParams<{ focus?: string }>();
|
const { focus } = useLocalSearchParams<{ focus?: string }>();
|
||||||
const [me, setMe] = useState<MobileMe | null>(null);
|
const [me, setMe] = useState<MobileMe | null>(null);
|
||||||
const [password, setPassword] = useState("");
|
const [password, setPassword] = useState("");
|
||||||
@ -424,6 +426,18 @@ export default function Account() {
|
|||||||
<Text style={styles.chevron}>›</Text>
|
<Text style={styles.chevron}>›</Text>
|
||||||
</Pressable>
|
</Pressable>
|
||||||
|
|
||||||
|
{mediaAvailable ? (
|
||||||
|
<Pressable
|
||||||
|
accessibilityRole="button"
|
||||||
|
disabled={pending}
|
||||||
|
onPress={() => router.push("/media")}
|
||||||
|
style={({ pressed }) => [styles.settingsButton, pressed && styles.pressed]}
|
||||||
|
>
|
||||||
|
<Text style={styles.settingsTitle}>{t("Media")}</Text>
|
||||||
|
<Text style={styles.chevron}>›</Text>
|
||||||
|
</Pressable>
|
||||||
|
) : null}
|
||||||
|
|
||||||
<Pressable
|
<Pressable
|
||||||
accessibilityRole="button"
|
accessibilityRole="button"
|
||||||
disabled={pending}
|
disabled={pending}
|
||||||
|
|||||||
344
apps/mobile/app/media-project.tsx
Normal file
344
apps/mobile/app/media-project.tsx
Normal file
@ -0,0 +1,344 @@
|
|||||||
|
import type {
|
||||||
|
MediaAsset,
|
||||||
|
MediaCapabilities,
|
||||||
|
MediaCollection,
|
||||||
|
MediaEditPlan,
|
||||||
|
MediaProject,
|
||||||
|
MediaRender,
|
||||||
|
PlatformProfileSummary,
|
||||||
|
} from "@rakazo/contracts";
|
||||||
|
import { Stack, useLocalSearchParams } from "expo-router";
|
||||||
|
import { useVideoPlayer, VideoView } from "expo-video";
|
||||||
|
import { useCallback, useEffect, useRef, useState } from "react";
|
||||||
|
import {
|
||||||
|
ActivityIndicator,
|
||||||
|
Alert,
|
||||||
|
Pressable,
|
||||||
|
ScrollView,
|
||||||
|
StyleSheet,
|
||||||
|
Text,
|
||||||
|
View,
|
||||||
|
} from "react-native";
|
||||||
|
import { mobileTokens } from "../lib/appearance";
|
||||||
|
import { useI18n } from "../lib/i18n";
|
||||||
|
import {
|
||||||
|
mediaRpc,
|
||||||
|
registerDeviceSource,
|
||||||
|
renderVideoSource,
|
||||||
|
shareRender,
|
||||||
|
uploadAssets,
|
||||||
|
} from "../lib/media-api";
|
||||||
|
import type { MediaVariant } from "../lib/media-project";
|
||||||
|
import {
|
||||||
|
blockers,
|
||||||
|
canApprove,
|
||||||
|
canShare,
|
||||||
|
MEDIA_POLL_MS,
|
||||||
|
projectReadiness,
|
||||||
|
projectVariants,
|
||||||
|
shouldPollProject,
|
||||||
|
variantState,
|
||||||
|
videoOutput,
|
||||||
|
} from "../lib/media-project";
|
||||||
|
import { native, useThemedStyles } from "../lib/native";
|
||||||
|
|
||||||
|
type ProjectData = { project: MediaProject; plans: MediaEditPlan[]; renders: MediaRender[] };
|
||||||
|
|
||||||
|
function message(reason: unknown): string {
|
||||||
|
return reason instanceof Error ? reason.message : String(reason);
|
||||||
|
}
|
||||||
|
|
||||||
|
export default function MediaProjectScreen() {
|
||||||
|
const styles = useThemedStyles(createStyles);
|
||||||
|
const { t } = useI18n();
|
||||||
|
const { projectId, upload } = useLocalSearchParams<{ projectId: string; upload?: string }>();
|
||||||
|
const [data, setData] = useState<ProjectData | null>(null);
|
||||||
|
const [assets, setAssets] = useState<MediaAsset[]>([]);
|
||||||
|
const [profiles, setProfiles] = useState<PlatformProfileSummary[]>([]);
|
||||||
|
const [analysis, setAnalysis] = useState(true);
|
||||||
|
const [deviceSourceId, setDeviceSourceId] = useState<string | null>(null);
|
||||||
|
const [uploadProgress, setUploadProgress] = useState<number | null>(null);
|
||||||
|
const [busy, setBusy] = useState<string | null>(null);
|
||||||
|
const [error, setError] = useState<string | null>(null);
|
||||||
|
const uploadStarted = useRef(false);
|
||||||
|
const planned = useRef(false);
|
||||||
|
const uploadAbort = useRef<AbortController | null>(null);
|
||||||
|
|
||||||
|
const load = useCallback(async () => {
|
||||||
|
if (!projectId) return;
|
||||||
|
const next = await mediaRpc<ProjectData>("projects/get", { projectId });
|
||||||
|
const collections = await Promise.all(
|
||||||
|
next.project.collectionIds.map((collectionId) =>
|
||||||
|
mediaRpc<{ collection: MediaCollection; assets: MediaAsset[] }>("collections/get", {
|
||||||
|
collectionId,
|
||||||
|
}),
|
||||||
|
),
|
||||||
|
);
|
||||||
|
const byId = new Map<string, MediaAsset>();
|
||||||
|
for (const entry of collections) for (const asset of entry.assets) byId.set(asset.id, asset);
|
||||||
|
setAssets([...byId.values()]);
|
||||||
|
setData(next);
|
||||||
|
}, [projectId]);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
void load().catch((reason) => setError(message(reason)));
|
||||||
|
void mediaRpc<PlatformProfileSummary[]>("profiles/list")
|
||||||
|
.then(setProfiles)
|
||||||
|
.catch(() => undefined);
|
||||||
|
void registerDeviceSource()
|
||||||
|
.then((source) => setDeviceSourceId(source.id))
|
||||||
|
.catch(() => undefined);
|
||||||
|
void mediaRpc<MediaCapabilities>("capabilities")
|
||||||
|
.then((capabilities) => setAnalysis(capabilities.configured))
|
||||||
|
.catch(() => undefined);
|
||||||
|
return () => uploadAbort.current?.abort();
|
||||||
|
}, [load]);
|
||||||
|
|
||||||
|
const readiness = projectReadiness(assets, analysis, deviceSourceId);
|
||||||
|
const uploading = uploadProgress !== null;
|
||||||
|
|
||||||
|
const startUpload = useCallback(async () => {
|
||||||
|
if (!deviceSourceId) return;
|
||||||
|
uploadStarted.current = true;
|
||||||
|
const controller = new AbortController();
|
||||||
|
uploadAbort.current = controller;
|
||||||
|
setUploadProgress(0);
|
||||||
|
try {
|
||||||
|
const failed = await uploadAssets(
|
||||||
|
assets,
|
||||||
|
deviceSourceId,
|
||||||
|
setUploadProgress,
|
||||||
|
controller.signal,
|
||||||
|
);
|
||||||
|
if (failed.length > 0) {
|
||||||
|
Alert.alert(t("{count} items could not be uploaded.", { count: failed.length }));
|
||||||
|
}
|
||||||
|
} catch (reason) {
|
||||||
|
if (!controller.signal.aborted) setError(message(reason));
|
||||||
|
} finally {
|
||||||
|
if (!controller.signal.aborted) {
|
||||||
|
setUploadProgress(null);
|
||||||
|
await load().catch(() => undefined);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}, [assets, deviceSourceId, load, t]);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (upload === "1" && readiness === "uploading" && !uploadStarted.current) {
|
||||||
|
void startUpload();
|
||||||
|
}
|
||||||
|
}, [upload, readiness, startUpload]);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (!data || readiness !== "ready" || data.plans.length > 0 || planned.current) return;
|
||||||
|
planned.current = true;
|
||||||
|
void (async () => {
|
||||||
|
try {
|
||||||
|
const plans = await mediaRpc<MediaEditPlan[]>(
|
||||||
|
"projects/autoPlan",
|
||||||
|
{ projectId: data.project.id },
|
||||||
|
{ timeoutMs: 120_000 },
|
||||||
|
);
|
||||||
|
for (const plan of plans) {
|
||||||
|
await mediaRpc("renders/start", { planId: plan.id, quality: "draft" });
|
||||||
|
}
|
||||||
|
await load();
|
||||||
|
} catch (reason) {
|
||||||
|
setError(message(reason));
|
||||||
|
}
|
||||||
|
})();
|
||||||
|
}, [data, readiness, load]);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (!data || uploading) return;
|
||||||
|
const waiting = data.plans.length === 0 && readiness === "processing";
|
||||||
|
if (!waiting && !shouldPollProject(data.renders)) return;
|
||||||
|
const timer = setTimeout(() => void load().catch(() => undefined), MEDIA_POLL_MS);
|
||||||
|
return () => clearTimeout(timer);
|
||||||
|
}, [data, readiness, uploading, load]);
|
||||||
|
|
||||||
|
async function act(variant: MediaVariant, action: () => Promise<unknown>) {
|
||||||
|
setBusy(variant.profileId);
|
||||||
|
try {
|
||||||
|
await action();
|
||||||
|
await load();
|
||||||
|
} catch (reason) {
|
||||||
|
Alert.alert(message(reason));
|
||||||
|
} finally {
|
||||||
|
setBusy(null);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const variants = data
|
||||||
|
? projectVariants({
|
||||||
|
profileIds: data.project.profileIds,
|
||||||
|
plans: data.plans,
|
||||||
|
renders: data.renders,
|
||||||
|
profiles,
|
||||||
|
})
|
||||||
|
: [];
|
||||||
|
|
||||||
|
return (
|
||||||
|
<ScrollView
|
||||||
|
style={styles.screen}
|
||||||
|
contentContainerStyle={styles.content}
|
||||||
|
contentInsetAdjustmentBehavior="automatic"
|
||||||
|
>
|
||||||
|
<Stack.Screen options={{ title: data?.project.title ?? "" }} />
|
||||||
|
{error ? <Text style={styles.error}>{error}</Text> : null}
|
||||||
|
{!data ? <ActivityIndicator color={native.secondaryLabel} /> : null}
|
||||||
|
{data && readiness === "uploading" ? (
|
||||||
|
uploading ? (
|
||||||
|
<View style={styles.progressRow}>
|
||||||
|
<ActivityIndicator color={native.secondaryLabel} />
|
||||||
|
<Text style={styles.status}>
|
||||||
|
{t("Uploading {percent}%", { percent: Math.round((uploadProgress ?? 0) * 100) })}
|
||||||
|
</Text>
|
||||||
|
</View>
|
||||||
|
) : (
|
||||||
|
<Pressable
|
||||||
|
accessibilityRole="button"
|
||||||
|
style={styles.primaryButton}
|
||||||
|
onPress={() => void startUpload()}
|
||||||
|
>
|
||||||
|
<Text style={styles.primaryLabel}>{t("Upload")}</Text>
|
||||||
|
</Pressable>
|
||||||
|
)
|
||||||
|
) : null}
|
||||||
|
{data && variants.length === 0 && readiness === "processing" ? (
|
||||||
|
<ActivityIndicator color={native.secondaryLabel} />
|
||||||
|
) : null}
|
||||||
|
{variants.map((variant) => (
|
||||||
|
<VariantCard
|
||||||
|
key={variant.profileId}
|
||||||
|
variant={variant}
|
||||||
|
busy={busy === variant.profileId}
|
||||||
|
onApprove={() =>
|
||||||
|
void act(variant, () =>
|
||||||
|
mediaRpc("renders/start", { planId: variant.plan.id, quality: "final" }),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
onShare={() =>
|
||||||
|
void act(variant, async () => {
|
||||||
|
if (variant.final) await shareRender(variant.final);
|
||||||
|
})
|
||||||
|
}
|
||||||
|
/>
|
||||||
|
))}
|
||||||
|
{variants.length > 0 ? <Text style={styles.note}>{t("Edit on web or desktop.")}</Text> : null}
|
||||||
|
</ScrollView>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function VariantCard({
|
||||||
|
variant,
|
||||||
|
busy,
|
||||||
|
onApprove,
|
||||||
|
onShare,
|
||||||
|
}: {
|
||||||
|
variant: MediaVariant;
|
||||||
|
busy: boolean;
|
||||||
|
onApprove: () => void;
|
||||||
|
onShare: () => void;
|
||||||
|
}) {
|
||||||
|
const styles = useThemedStyles(createStyles);
|
||||||
|
const { t } = useI18n();
|
||||||
|
const state = variantState(variant);
|
||||||
|
const progress = Math.round((variant.latest?.progress ?? 0) * 100);
|
||||||
|
const blocker = blockers(variant.latest)[0];
|
||||||
|
const status =
|
||||||
|
state === "rendering" || state === "approved_rendering"
|
||||||
|
? t("Rendering {percent}%", { percent: progress })
|
||||||
|
: state === "blocked"
|
||||||
|
? (blocker?.message ?? t("Blocked"))
|
||||||
|
: state === "failed"
|
||||||
|
? (variant.latest?.error ?? t("Failed"))
|
||||||
|
: null;
|
||||||
|
|
||||||
|
return (
|
||||||
|
<View style={styles.card}>
|
||||||
|
<Text style={styles.cardTitle}>{variant.profileName}</Text>
|
||||||
|
{variant.preview ? <VariantPlayer render={variant.preview} /> : null}
|
||||||
|
{status ? (
|
||||||
|
<Text style={state === "blocked" || state === "failed" ? styles.blocked : styles.status}>
|
||||||
|
{status}
|
||||||
|
</Text>
|
||||||
|
) : null}
|
||||||
|
{busy ? (
|
||||||
|
<ActivityIndicator color={native.secondaryLabel} />
|
||||||
|
) : canApprove(variant) || canShare(variant) ? (
|
||||||
|
<View style={styles.actions}>
|
||||||
|
{canApprove(variant) ? (
|
||||||
|
<Pressable accessibilityRole="button" style={styles.button} onPress={onApprove}>
|
||||||
|
<Text style={styles.buttonLabel}>{t("Approve")}</Text>
|
||||||
|
</Pressable>
|
||||||
|
) : null}
|
||||||
|
{canShare(variant) ? (
|
||||||
|
<Pressable accessibilityRole="button" style={styles.button} onPress={onShare}>
|
||||||
|
<Text style={styles.buttonLabel}>{t("Share")}</Text>
|
||||||
|
</Pressable>
|
||||||
|
) : null}
|
||||||
|
</View>
|
||||||
|
) : null}
|
||||||
|
</View>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function VariantPlayer({ render }: { render: MediaRender }) {
|
||||||
|
const renderId = render.id;
|
||||||
|
const fileName = videoOutput(render)?.fileName ?? null;
|
||||||
|
const [source, setSource] = useState<{ uri: string; headers: Record<string, string> } | null>(
|
||||||
|
null,
|
||||||
|
);
|
||||||
|
useEffect(() => {
|
||||||
|
if (!fileName) return;
|
||||||
|
let active = true;
|
||||||
|
void renderVideoSource(renderId, fileName).then((next) => {
|
||||||
|
if (active) setSource(next);
|
||||||
|
});
|
||||||
|
return () => {
|
||||||
|
active = false;
|
||||||
|
};
|
||||||
|
}, [renderId, fileName]);
|
||||||
|
return source ? <Player key={source.uri} source={source} /> : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function Player({ source }: { source: { uri: string; headers: Record<string, string> } }) {
|
||||||
|
const styles = useThemedStyles(createStyles);
|
||||||
|
const player = useVideoPlayer(source);
|
||||||
|
return <VideoView player={player} nativeControls contentFit="contain" style={styles.video} />;
|
||||||
|
}
|
||||||
|
|
||||||
|
function createStyles() {
|
||||||
|
const tokens = mobileTokens();
|
||||||
|
return StyleSheet.create({
|
||||||
|
screen: { flex: 1, backgroundColor: native.page },
|
||||||
|
content: { padding: 16, gap: 12 },
|
||||||
|
card: { padding: 14, borderRadius: 16, backgroundColor: native.fill, gap: 10 },
|
||||||
|
cardTitle: { color: native.label, fontSize: 16, fontWeight: "600" },
|
||||||
|
video: { width: "100%", aspectRatio: 9 / 16, maxHeight: 460, borderRadius: 12 },
|
||||||
|
status: { color: native.secondaryLabel, fontSize: 14 },
|
||||||
|
blocked: { color: tokens.destructive, fontSize: 14 },
|
||||||
|
progressRow: { flexDirection: "row", alignItems: "center", gap: 10 },
|
||||||
|
actions: { flexDirection: "row", gap: 8 },
|
||||||
|
button: {
|
||||||
|
minHeight: 42,
|
||||||
|
paddingHorizontal: 16,
|
||||||
|
borderRadius: 12,
|
||||||
|
backgroundColor: native.fillPressed,
|
||||||
|
alignItems: "center",
|
||||||
|
justifyContent: "center",
|
||||||
|
},
|
||||||
|
buttonLabel: { color: native.label, fontSize: 15, fontWeight: "600" },
|
||||||
|
primaryButton: {
|
||||||
|
minHeight: 48,
|
||||||
|
borderRadius: 12,
|
||||||
|
backgroundColor: tokens.primary,
|
||||||
|
alignItems: "center",
|
||||||
|
justifyContent: "center",
|
||||||
|
},
|
||||||
|
primaryLabel: { color: tokens.primaryForeground, fontSize: 16, fontWeight: "600" },
|
||||||
|
error: { color: tokens.destructive, fontSize: 14 },
|
||||||
|
note: { color: native.tertiaryLabel, fontSize: 13, textAlign: "center", marginTop: 4 },
|
||||||
|
});
|
||||||
|
}
|
||||||
189
apps/mobile/app/media-session.tsx
Normal file
189
apps/mobile/app/media-session.tsx
Normal file
@ -0,0 +1,189 @@
|
|||||||
|
import type {
|
||||||
|
MediaAsset,
|
||||||
|
MediaCollection,
|
||||||
|
MediaProject,
|
||||||
|
PlatformProfileSummary,
|
||||||
|
} from "@rakazo/contracts";
|
||||||
|
import { Stack, useLocalSearchParams, useRouter } from "expo-router";
|
||||||
|
import { useEffect, useState } from "react";
|
||||||
|
import {
|
||||||
|
ActivityIndicator,
|
||||||
|
Alert,
|
||||||
|
FlatList,
|
||||||
|
Image,
|
||||||
|
Pressable,
|
||||||
|
StyleSheet,
|
||||||
|
Text,
|
||||||
|
useWindowDimensions,
|
||||||
|
View,
|
||||||
|
} from "react-native";
|
||||||
|
import { SafeAreaView } from "react-native-safe-area-context";
|
||||||
|
import { NativeSymbol } from "../components/native-symbol";
|
||||||
|
import { authHeaders, currentApiBase } from "../lib/api";
|
||||||
|
import { mobileTokens } from "../lib/appearance";
|
||||||
|
import { useI18n } from "../lib/i18n";
|
||||||
|
import { mediaRpc } from "../lib/media-api";
|
||||||
|
import { assetThumbUrl, formatDuration } from "../lib/media-project";
|
||||||
|
import { presentMessageActionSheet } from "../lib/message-action-sheet";
|
||||||
|
import { native, useResolvedAppearance, useThemedStyles } from "../lib/native";
|
||||||
|
|
||||||
|
const COLUMNS = 3;
|
||||||
|
const GAP = 2;
|
||||||
|
|
||||||
|
export default function MediaSession() {
|
||||||
|
const styles = useThemedStyles(createStyles);
|
||||||
|
const { t } = useI18n();
|
||||||
|
const router = useRouter();
|
||||||
|
const appearance = useResolvedAppearance();
|
||||||
|
const { width } = useWindowDimensions();
|
||||||
|
const { collectionId } = useLocalSearchParams<{ collectionId: string }>();
|
||||||
|
const [collection, setCollection] = useState<MediaCollection | null>(null);
|
||||||
|
const [assets, setAssets] = useState<MediaAsset[]>([]);
|
||||||
|
const [headers, setHeaders] = useState<Record<string, string>>({});
|
||||||
|
const [creating, setCreating] = useState(false);
|
||||||
|
const [error, setError] = useState<string | null>(null);
|
||||||
|
const tile = (width - GAP * (COLUMNS - 1)) / COLUMNS;
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (!collectionId) return;
|
||||||
|
void authHeaders().then(setHeaders);
|
||||||
|
void mediaRpc<{ collection: MediaCollection; assets: MediaAsset[] }>("collections/get", {
|
||||||
|
collectionId,
|
||||||
|
})
|
||||||
|
.then((result) => {
|
||||||
|
setCollection(result.collection);
|
||||||
|
setAssets(result.assets);
|
||||||
|
})
|
||||||
|
.catch((reason) => setError(reason instanceof Error ? reason.message : String(reason)));
|
||||||
|
}, [collectionId]);
|
||||||
|
|
||||||
|
async function create(profileId: string) {
|
||||||
|
if (!collection) return;
|
||||||
|
setCreating(true);
|
||||||
|
try {
|
||||||
|
const project = await mediaRpc<MediaProject>("projects/create", {
|
||||||
|
title: collection.title.slice(0, 120) || t("Media"),
|
||||||
|
goal: "teaser",
|
||||||
|
collectionIds: [collection.id],
|
||||||
|
profileIds: [profileId],
|
||||||
|
});
|
||||||
|
router.replace({
|
||||||
|
pathname: "/media-project",
|
||||||
|
params: { projectId: project.id, upload: "1" },
|
||||||
|
});
|
||||||
|
} catch (reason) {
|
||||||
|
Alert.alert(reason instanceof Error ? reason.message : String(reason));
|
||||||
|
} finally {
|
||||||
|
setCreating(false);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function chooseProfile() {
|
||||||
|
try {
|
||||||
|
const profiles = await mediaRpc<PlatformProfileSummary[]>("profiles/list");
|
||||||
|
if (profiles.length === 1 && profiles[0]) {
|
||||||
|
await create(profiles[0].id);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
presentMessageActionSheet({
|
||||||
|
actions: profiles.map((profile) => ({
|
||||||
|
text: profile.name,
|
||||||
|
onPress: () => void create(profile.id),
|
||||||
|
})),
|
||||||
|
cancel: t("Cancel"),
|
||||||
|
more: t("More"),
|
||||||
|
colorScheme: appearance,
|
||||||
|
});
|
||||||
|
} catch (reason) {
|
||||||
|
Alert.alert(reason instanceof Error ? reason.message : String(reason));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return (
|
||||||
|
<View style={styles.screen}>
|
||||||
|
<Stack.Screen options={{ title: collection?.title ?? "" }} />
|
||||||
|
<FlatList
|
||||||
|
data={assets}
|
||||||
|
keyExtractor={(asset) => asset.id}
|
||||||
|
numColumns={COLUMNS}
|
||||||
|
columnWrapperStyle={{ gap: GAP }}
|
||||||
|
contentContainerStyle={{ gap: GAP }}
|
||||||
|
contentInsetAdjustmentBehavior="automatic"
|
||||||
|
ListHeaderComponent={error ? <Text style={styles.error}>{error}</Text> : null}
|
||||||
|
renderItem={({ item }) => (
|
||||||
|
<View style={[styles.tile, { width: tile, height: tile }]}>
|
||||||
|
{item.hasThumbnail ? (
|
||||||
|
<Image
|
||||||
|
accessibilityIgnoresInvertColors
|
||||||
|
source={{ uri: assetThumbUrl(currentApiBase(), item.id), headers }}
|
||||||
|
style={StyleSheet.absoluteFill}
|
||||||
|
/>
|
||||||
|
) : (
|
||||||
|
<NativeSymbol
|
||||||
|
ios={item.kind === "video" ? "video" : "photo"}
|
||||||
|
android={item.kind === "video" ? "videocam-outline" : "image-outline"}
|
||||||
|
color={native.tertiaryLabel}
|
||||||
|
size={22}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
{item.kind === "video" && item.durationMs ? (
|
||||||
|
<Text style={styles.duration}>{formatDuration(item.durationMs)}</Text>
|
||||||
|
) : null}
|
||||||
|
</View>
|
||||||
|
)}
|
||||||
|
/>
|
||||||
|
{collection ? (
|
||||||
|
<SafeAreaView edges={["bottom"]} style={styles.footer}>
|
||||||
|
<Pressable
|
||||||
|
accessibilityRole="button"
|
||||||
|
disabled={creating}
|
||||||
|
style={({ pressed }) => [styles.primaryButton, pressed && styles.pressed]}
|
||||||
|
onPress={() => void chooseProfile()}
|
||||||
|
>
|
||||||
|
{creating ? (
|
||||||
|
<ActivityIndicator color={mobileTokens().primaryForeground} />
|
||||||
|
) : (
|
||||||
|
<Text style={styles.primaryLabel}>{t("Upload and create project")}</Text>
|
||||||
|
)}
|
||||||
|
</Pressable>
|
||||||
|
</SafeAreaView>
|
||||||
|
) : null}
|
||||||
|
</View>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function createStyles() {
|
||||||
|
const tokens = mobileTokens();
|
||||||
|
return StyleSheet.create({
|
||||||
|
screen: { flex: 1, backgroundColor: native.page },
|
||||||
|
tile: {
|
||||||
|
backgroundColor: native.fill,
|
||||||
|
alignItems: "center",
|
||||||
|
justifyContent: "center",
|
||||||
|
overflow: "hidden",
|
||||||
|
},
|
||||||
|
duration: {
|
||||||
|
position: "absolute",
|
||||||
|
right: 6,
|
||||||
|
bottom: 4,
|
||||||
|
color: tokens.foreground,
|
||||||
|
fontSize: 12,
|
||||||
|
fontWeight: "600",
|
||||||
|
fontVariant: ["tabular-nums"],
|
||||||
|
textShadowColor: tokens.background,
|
||||||
|
textShadowRadius: 3,
|
||||||
|
},
|
||||||
|
footer: { paddingHorizontal: 16, paddingTop: 8, backgroundColor: native.page },
|
||||||
|
primaryButton: {
|
||||||
|
minHeight: 50,
|
||||||
|
borderRadius: 14,
|
||||||
|
backgroundColor: tokens.primary,
|
||||||
|
alignItems: "center",
|
||||||
|
justifyContent: "center",
|
||||||
|
marginBottom: 8,
|
||||||
|
},
|
||||||
|
pressed: { opacity: 0.8 },
|
||||||
|
primaryLabel: { color: tokens.primaryForeground, fontSize: 16, fontWeight: "600" },
|
||||||
|
error: { color: tokens.destructive, fontSize: 14, padding: 16 },
|
||||||
|
});
|
||||||
|
}
|
||||||
241
apps/mobile/app/media.tsx
Normal file
241
apps/mobile/app/media.tsx
Normal file
@ -0,0 +1,241 @@
|
|||||||
|
import type { MediaCollection, MediaProject } from "@rakazo/contracts";
|
||||||
|
import { Stack, useRouter } from "expo-router";
|
||||||
|
import { useCallback, useEffect, useState } from "react";
|
||||||
|
import {
|
||||||
|
ActivityIndicator,
|
||||||
|
Linking,
|
||||||
|
Pressable,
|
||||||
|
RefreshControl,
|
||||||
|
SectionList,
|
||||||
|
StyleSheet,
|
||||||
|
Text,
|
||||||
|
View,
|
||||||
|
} from "react-native";
|
||||||
|
import { NativeSymbol } from "../components/native-symbol";
|
||||||
|
import { mobileTokens } from "../lib/appearance";
|
||||||
|
import { useI18n } from "../lib/i18n";
|
||||||
|
import {
|
||||||
|
changeLimitedSelection,
|
||||||
|
mediaRpc,
|
||||||
|
readLibraryAccess,
|
||||||
|
requestLibraryAccess,
|
||||||
|
syncDeviceLibrary,
|
||||||
|
} from "../lib/media-api";
|
||||||
|
import type { LibraryAccess } from "../lib/media-project";
|
||||||
|
import { formatDuration } from "../lib/media-project";
|
||||||
|
import { native, useThemedStyles } from "../lib/native";
|
||||||
|
|
||||||
|
type Row =
|
||||||
|
| { kind: "project"; project: MediaProject }
|
||||||
|
| { kind: "session"; collection: MediaCollection };
|
||||||
|
|
||||||
|
export default function MediaLibrary() {
|
||||||
|
const styles = useThemedStyles(createStyles);
|
||||||
|
const { t } = useI18n();
|
||||||
|
const router = useRouter();
|
||||||
|
const [access, setAccess] = useState<LibraryAccess | null>(null);
|
||||||
|
const [sessions, setSessions] = useState<MediaCollection[]>([]);
|
||||||
|
const [projects, setProjects] = useState<MediaProject[]>([]);
|
||||||
|
const [syncing, setSyncing] = useState(false);
|
||||||
|
const [error, setError] = useState<string | null>(null);
|
||||||
|
|
||||||
|
const loadLists = useCallback(async () => {
|
||||||
|
const [sessionPage, projectPage] = await Promise.all([
|
||||||
|
mediaRpc<{ items: MediaCollection[] }>("collections/list", { kind: "session", limit: 200 }),
|
||||||
|
mediaRpc<{ items: MediaProject[] }>("projects/list", { limit: 50 }),
|
||||||
|
]);
|
||||||
|
setSessions(sessionPage.items);
|
||||||
|
setProjects(projectPage.items.filter((project) => project.status !== "archived"));
|
||||||
|
}, []);
|
||||||
|
|
||||||
|
const refresh = useCallback(
|
||||||
|
async (current: LibraryAccess | null) => {
|
||||||
|
setError(null);
|
||||||
|
try {
|
||||||
|
if (current === "all" || current === "limited") {
|
||||||
|
setSyncing(true);
|
||||||
|
await syncDeviceLibrary();
|
||||||
|
}
|
||||||
|
await loadLists();
|
||||||
|
} catch (reason) {
|
||||||
|
setError(reason instanceof Error ? reason.message : String(reason));
|
||||||
|
} finally {
|
||||||
|
setSyncing(false);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
[loadLists],
|
||||||
|
);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
void readLibraryAccess()
|
||||||
|
.catch(() => "denied" as const)
|
||||||
|
.then((current) => {
|
||||||
|
setAccess(current);
|
||||||
|
return refresh(current);
|
||||||
|
});
|
||||||
|
}, [refresh]);
|
||||||
|
|
||||||
|
async function allow() {
|
||||||
|
if (access === "blocked") {
|
||||||
|
await Linking.openSettings();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const next = await requestLibraryAccess();
|
||||||
|
setAccess(next);
|
||||||
|
await refresh(next);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function chooseItems() {
|
||||||
|
await changeLimitedSelection().catch(() => undefined);
|
||||||
|
await refresh(access);
|
||||||
|
}
|
||||||
|
|
||||||
|
const sections = [
|
||||||
|
...(projects.length
|
||||||
|
? [
|
||||||
|
{
|
||||||
|
title: t("Projects"),
|
||||||
|
data: projects.map((project): Row => ({ kind: "project", project })),
|
||||||
|
},
|
||||||
|
]
|
||||||
|
: []),
|
||||||
|
{
|
||||||
|
title: t("Sessions"),
|
||||||
|
data: sessions.map((collection): Row => ({ kind: "session", collection })),
|
||||||
|
},
|
||||||
|
];
|
||||||
|
const needsAccess = access !== null && access !== "all" && access !== "limited";
|
||||||
|
|
||||||
|
return (
|
||||||
|
<View style={styles.screen}>
|
||||||
|
<Stack.Screen
|
||||||
|
options={{
|
||||||
|
title: t("Media"),
|
||||||
|
headerRight: () =>
|
||||||
|
syncing ? (
|
||||||
|
<ActivityIndicator color={native.secondaryLabel} />
|
||||||
|
) : access === "limited" ? (
|
||||||
|
<Pressable
|
||||||
|
accessibilityRole="button"
|
||||||
|
accessibilityLabel={t("Choose photos and videos")}
|
||||||
|
hitSlop={10}
|
||||||
|
onPress={() => void chooseItems()}
|
||||||
|
>
|
||||||
|
<NativeSymbol ios="photo.badge.plus" android="images-outline" />
|
||||||
|
</Pressable>
|
||||||
|
) : null,
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
<SectionList
|
||||||
|
sections={sections}
|
||||||
|
keyExtractor={(row) => (row.kind === "project" ? row.project.id : row.collection.id)}
|
||||||
|
contentContainerStyle={styles.content}
|
||||||
|
contentInsetAdjustmentBehavior="automatic"
|
||||||
|
stickySectionHeadersEnabled={false}
|
||||||
|
refreshControl={
|
||||||
|
<RefreshControl refreshing={false} onRefresh={() => void refresh(access)} />
|
||||||
|
}
|
||||||
|
ListHeaderComponent={
|
||||||
|
needsAccess ? (
|
||||||
|
<Pressable
|
||||||
|
accessibilityRole="button"
|
||||||
|
style={styles.primaryButton}
|
||||||
|
onPress={() => void allow()}
|
||||||
|
>
|
||||||
|
<Text style={styles.primaryLabel}>
|
||||||
|
{access === "blocked" ? t("Open Settings") : t("Allow photo access")}
|
||||||
|
</Text>
|
||||||
|
</Pressable>
|
||||||
|
) : error ? (
|
||||||
|
<Text style={styles.error}>{error}</Text>
|
||||||
|
) : null
|
||||||
|
}
|
||||||
|
renderSectionHeader={({ section }) =>
|
||||||
|
// A lone sessions list needs no heading.
|
||||||
|
sections.length > 1 ? <Text style={styles.sectionTitle}>{section.title}</Text> : null
|
||||||
|
}
|
||||||
|
renderItem={({ item }) =>
|
||||||
|
item.kind === "project" ? (
|
||||||
|
<Pressable
|
||||||
|
accessibilityRole="button"
|
||||||
|
style={({ pressed }) => [styles.row, pressed && styles.rowPressed]}
|
||||||
|
onPress={() =>
|
||||||
|
router.push({ pathname: "/media-project", params: { projectId: item.project.id } })
|
||||||
|
}
|
||||||
|
>
|
||||||
|
<Text style={styles.rowTitle} numberOfLines={1}>
|
||||||
|
{item.project.title}
|
||||||
|
</Text>
|
||||||
|
</Pressable>
|
||||||
|
) : (
|
||||||
|
<Pressable
|
||||||
|
accessibilityRole="button"
|
||||||
|
style={({ pressed }) => [styles.row, pressed && styles.rowPressed]}
|
||||||
|
onPress={() =>
|
||||||
|
router.push({
|
||||||
|
pathname: "/media-session",
|
||||||
|
params: { collectionId: item.collection.id },
|
||||||
|
})
|
||||||
|
}
|
||||||
|
>
|
||||||
|
<Text style={styles.rowTitle} numberOfLines={1}>
|
||||||
|
{item.collection.title}
|
||||||
|
</Text>
|
||||||
|
<Text style={styles.rowDetail}>
|
||||||
|
{item.collection.videoCount > 0
|
||||||
|
? `${t("{count} items", { count: item.collection.assetCount })} · ${formatDuration(
|
||||||
|
item.collection.totalDurationMs,
|
||||||
|
)}`
|
||||||
|
: t("{count} items", { count: item.collection.assetCount })}
|
||||||
|
</Text>
|
||||||
|
</Pressable>
|
||||||
|
)
|
||||||
|
}
|
||||||
|
renderSectionFooter={({ section }) =>
|
||||||
|
section.data.length === 0 && !syncing && !needsAccess ? (
|
||||||
|
<Text style={styles.empty}>{t("No sessions yet")}</Text>
|
||||||
|
) : null
|
||||||
|
}
|
||||||
|
/>
|
||||||
|
</View>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function createStyles() {
|
||||||
|
const tokens = mobileTokens();
|
||||||
|
return StyleSheet.create({
|
||||||
|
screen: { flex: 1, backgroundColor: native.page },
|
||||||
|
content: { padding: 16, gap: 8 },
|
||||||
|
sectionTitle: {
|
||||||
|
color: native.secondaryLabel,
|
||||||
|
fontSize: 13,
|
||||||
|
fontWeight: "600",
|
||||||
|
marginTop: 12,
|
||||||
|
marginBottom: 6,
|
||||||
|
},
|
||||||
|
row: {
|
||||||
|
minHeight: 56,
|
||||||
|
paddingHorizontal: 14,
|
||||||
|
paddingVertical: 10,
|
||||||
|
borderRadius: 12,
|
||||||
|
backgroundColor: native.fill,
|
||||||
|
justifyContent: "center",
|
||||||
|
gap: 2,
|
||||||
|
marginBottom: 8,
|
||||||
|
},
|
||||||
|
rowPressed: { backgroundColor: native.fillPressed },
|
||||||
|
rowTitle: { color: native.label, fontSize: 16, fontWeight: "500" },
|
||||||
|
rowDetail: { color: native.secondaryLabel, fontSize: 13 },
|
||||||
|
primaryButton: {
|
||||||
|
minHeight: 48,
|
||||||
|
borderRadius: 12,
|
||||||
|
backgroundColor: tokens.primary,
|
||||||
|
alignItems: "center",
|
||||||
|
justifyContent: "center",
|
||||||
|
marginBottom: 8,
|
||||||
|
},
|
||||||
|
primaryLabel: { color: tokens.primaryForeground, fontSize: 16, fontWeight: "600" },
|
||||||
|
error: { color: tokens.destructive, fontSize: 14, marginBottom: 8 },
|
||||||
|
empty: { color: native.tertiaryLabel, fontSize: 14, paddingVertical: 8 },
|
||||||
|
});
|
||||||
|
}
|
||||||
@ -28,7 +28,11 @@ export default function NewSpace() {
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
router.dismissAll();
|
router.dismissAll();
|
||||||
router.replace("/");
|
// A new project opens in the conversation with its own main agent.
|
||||||
|
const main = space.groups.find((group) => group.kind === "main");
|
||||||
|
router.replace(
|
||||||
|
main ? { pathname: "/group-thread", params: { groupId: main.id, name: main.name } } : "/",
|
||||||
|
);
|
||||||
} catch (reason) {
|
} catch (reason) {
|
||||||
setError(reason instanceof Error ? reason.message : t("Could not create space"));
|
setError(reason instanceof Error ? reason.message : t("Could not create space"));
|
||||||
setPending(false);
|
setPending(false);
|
||||||
|
|||||||
@ -17,10 +17,12 @@ import {
|
|||||||
buildComposerMentionOptions,
|
buildComposerMentionOptions,
|
||||||
type ComposerMention,
|
type ComposerMention,
|
||||||
cloudAgentHttpsUrl,
|
cloudAgentHttpsUrl,
|
||||||
|
deriveWorkspaceFocus,
|
||||||
isApprovalAskBlock,
|
isApprovalAskBlock,
|
||||||
isRunTerminalEvent,
|
isRunTerminalEvent,
|
||||||
isSecretAskBlock,
|
isSecretAskBlock,
|
||||||
latestAnswerableAskMessageId,
|
latestAnswerableAskMessageId,
|
||||||
|
liveScreenRunId,
|
||||||
mentionChipKey,
|
mentionChipKey,
|
||||||
projectMessageReactions,
|
projectMessageReactions,
|
||||||
resolveComposerSendPlan,
|
resolveComposerSendPlan,
|
||||||
@ -288,6 +290,8 @@ function Thread() {
|
|||||||
? { botId }
|
? { botId }
|
||||||
: undefined;
|
: undefined;
|
||||||
const [snap, setSnap] = useState<MobileSnapshot | null>(null);
|
const [snap, setSnap] = useState<MobileSnapshot | null>(null);
|
||||||
|
/** Run that last pointed at the bot's screen, so the live screen is offered only during real screen work. */
|
||||||
|
const [screenRunId, setScreenRunId] = useState<string | null>(null);
|
||||||
const activeThreadId = useRef<string | undefined>(undefined);
|
const activeThreadId = useRef<string | undefined>(undefined);
|
||||||
const [draft, setDraft] = useState("");
|
const [draft, setDraft] = useState("");
|
||||||
const [mentionQuery, setMentionQuery] = useState<string | null>(null);
|
const [mentionQuery, setMentionQuery] = useState<string | null>(null);
|
||||||
@ -860,6 +864,7 @@ function Thread() {
|
|||||||
}
|
}
|
||||||
expandedHistoryThread.current = null;
|
expandedHistoryThread.current = null;
|
||||||
historyEpoch.current += 1;
|
historyEpoch.current += 1;
|
||||||
|
setScreenRunId(null);
|
||||||
const abort = new AbortController();
|
const abort = new AbortController();
|
||||||
void (async () => {
|
void (async () => {
|
||||||
// Pending search jumps load the around-page separately; avoid replacing it with latest.
|
// Pending search jumps load the around-page separately; avoid replacing it with latest.
|
||||||
@ -885,6 +890,8 @@ function Thread() {
|
|||||||
(event) => {
|
(event) => {
|
||||||
cursor = Math.max(cursor, event.seq ?? -1);
|
cursor = Math.max(cursor, event.seq ?? -1);
|
||||||
retryMs = 250;
|
retryMs = 250;
|
||||||
|
const screenRun = groupId ? null : liveScreenRunId(event);
|
||||||
|
if (screenRun) setScreenRunId(screenRun);
|
||||||
if (
|
if (
|
||||||
event.type === "thread.progress" ||
|
event.type === "thread.progress" ||
|
||||||
event.type === "agent.tool.called" ||
|
event.type === "agent.tool.called" ||
|
||||||
@ -1271,6 +1278,23 @@ function Thread() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const answerableAskMessageId = latestAnswerableAskMessageId(snap);
|
const answerableAskMessageId = latestAnswerableAskMessageId(snap);
|
||||||
|
const screenFocus = useMemo(() => {
|
||||||
|
const computer = snap?.computer;
|
||||||
|
if (!snap || !computer || groupId) return null;
|
||||||
|
const focus = deriveWorkspaceFocus(
|
||||||
|
{
|
||||||
|
...snap,
|
||||||
|
computer: {
|
||||||
|
kind: computer.kind ?? "",
|
||||||
|
state: computer.state,
|
||||||
|
screenAvailable: computer.screenAvailable,
|
||||||
|
takeoverRequested: computer.takeoverRequested ?? false,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{ screenRunId },
|
||||||
|
);
|
||||||
|
return focus.pane === "screen" ? focus : null;
|
||||||
|
}, [groupId, screenRunId, snap]);
|
||||||
const runError = snap?.run?.status === "failed" ? (snap.run.error ?? null) : null;
|
const runError = snap?.run?.status === "failed" ? (snap.run.error ?? null) : null;
|
||||||
const liveMessages = useMemo(() => [...visibleMessages].reverse(), [visibleMessages]);
|
const liveMessages = useMemo(() => [...visibleMessages].reverse(), [visibleMessages]);
|
||||||
const messagesById = useMemo(
|
const messagesById = useMemo(
|
||||||
@ -1668,6 +1692,43 @@ function Thread() {
|
|||||||
) : null}
|
) : null}
|
||||||
</View>
|
</View>
|
||||||
<View style={{ paddingBottom: keyboardVisible ? 12 : Math.max(insets.bottom + 12, 24) }}>
|
<View style={{ paddingBottom: keyboardVisible ? 12 : Math.max(insets.bottom + 12, 24) }}>
|
||||||
|
{screenFocus && botId ? (
|
||||||
|
<Pressable
|
||||||
|
accessibilityRole="button"
|
||||||
|
testID="live-screen-pill"
|
||||||
|
onPress={() =>
|
||||||
|
router.push({
|
||||||
|
pathname: "/computer",
|
||||||
|
params: { botId, name: displayName ?? t("Bot") },
|
||||||
|
})
|
||||||
|
}
|
||||||
|
style={({ pressed }) => ({
|
||||||
|
alignSelf: "center",
|
||||||
|
marginTop: 10,
|
||||||
|
flexDirection: "row",
|
||||||
|
alignItems: "center",
|
||||||
|
gap: 8,
|
||||||
|
borderRadius: 999,
|
||||||
|
borderWidth: 1,
|
||||||
|
borderColor: native.fillPressed,
|
||||||
|
backgroundColor: pressed ? native.fillPressed : native.fill,
|
||||||
|
paddingHorizontal: 14,
|
||||||
|
paddingVertical: 8,
|
||||||
|
})}
|
||||||
|
>
|
||||||
|
<View
|
||||||
|
style={{
|
||||||
|
width: 8,
|
||||||
|
height: 8,
|
||||||
|
borderRadius: 4,
|
||||||
|
backgroundColor: screenFocus.takeover ? tokens.warning : tokens.success,
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
<Text style={{ color: tokens.foreground, fontSize: 14, fontWeight: "500" }}>
|
||||||
|
{screenFocus.takeover ? t("Needs takeover") : t("Open computer")}
|
||||||
|
</Text>
|
||||||
|
</Pressable>
|
||||||
|
) : null}
|
||||||
{replyTarget ? (
|
{replyTarget ? (
|
||||||
<View
|
<View
|
||||||
style={{
|
style={{
|
||||||
|
|||||||
@ -728,7 +728,7 @@ export type MobileBot = Pick<
|
|||||||
| "modelId"
|
| "modelId"
|
||||||
| "thinkingLevel"
|
| "thinkingLevel"
|
||||||
> &
|
> &
|
||||||
Partial<Pick<Bot, "parentBotId" | "spaceId">>;
|
Partial<Pick<Bot, "parentBotId" | "spaceId" | "kind">>;
|
||||||
|
|
||||||
export type MobileBotSection = BotSection;
|
export type MobileBotSection = BotSection;
|
||||||
|
|
||||||
@ -790,9 +790,11 @@ export type MobileSnapshot = {
|
|||||||
activeRuns?: Array<{ id: string; botId?: string; status: string }>;
|
activeRuns?: Array<{ id: string; botId?: string; status: string }>;
|
||||||
members?: MobileGroup["members"];
|
members?: MobileGroup["members"];
|
||||||
computer?: {
|
computer?: {
|
||||||
|
kind?: string;
|
||||||
state: string;
|
state: string;
|
||||||
controlHolder: string;
|
controlHolder: string;
|
||||||
screenAvailable: boolean;
|
screenAvailable: boolean;
|
||||||
|
takeoverRequested?: boolean;
|
||||||
mode: ComputerMode;
|
mode: ComputerMode;
|
||||||
busyBotName: string | null;
|
busyBotName: string | null;
|
||||||
};
|
};
|
||||||
@ -896,6 +898,7 @@ type ThreadEvent = {
|
|||||||
type: string;
|
type: string;
|
||||||
seq?: number;
|
seq?: number;
|
||||||
runId?: string;
|
runId?: string;
|
||||||
|
createdAt?: string;
|
||||||
payload?: Record<string, unknown>;
|
payload?: Record<string, unknown>;
|
||||||
};
|
};
|
||||||
|
|
||||||
@ -1074,7 +1077,7 @@ export function applyMobileThreadEvent(
|
|||||||
messages: [...remaining, streaming],
|
messages: [...remaining, streaming],
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
if (event.type === "agent.tool.completed") {
|
if (event.type === "agent.tool.completed" || event.type === "thread.computer") {
|
||||||
return { ...prev, cursor: event.seq ?? prev.cursor };
|
return { ...prev, cursor: event.seq ?? prev.cursor };
|
||||||
}
|
}
|
||||||
if (event.type === "thread.subagent") {
|
if (event.type === "thread.subagent") {
|
||||||
|
|||||||
@ -83,7 +83,7 @@ describe("display and warnings", () => {
|
|||||||
expect(apiBaseWarning("http://127.0.0.1:3100")).toBeNull();
|
expect(apiBaseWarning("http://127.0.0.1:3100")).toBeNull();
|
||||||
expect(apiBaseWarning("http://192.168.1.20:3100")).toBeNull();
|
expect(apiBaseWarning("http://192.168.1.20:3100")).toBeNull();
|
||||||
expect(apiBaseWarning("http://100.64.0.1:3100")).toBeNull();
|
expect(apiBaseWarning("http://100.64.0.1:3100")).toBeNull();
|
||||||
expect(apiBaseWarning("http://100.119.57.55:3100")).toBeNull();
|
expect(apiBaseWarning("http://100.100.100.100:3100")).toBeNull();
|
||||||
expect(apiBaseWarning("http://100.127.255.255:3100")).toBeNull();
|
expect(apiBaseWarning("http://100.127.255.255:3100")).toBeNull();
|
||||||
expect(apiBaseWarning("http://app.example.com")).toMatch(/https/i);
|
expect(apiBaseWarning("http://app.example.com")).toMatch(/https/i);
|
||||||
});
|
});
|
||||||
|
|||||||
@ -111,6 +111,29 @@ describe("spaceInboxItems", () => {
|
|||||||
expect(items.map((item) => item.type)).toEqual(["bot", "group"]);
|
expect(items.map((item) => item.type)).toEqual(["bot", "group"]);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("lists a project's main conversation instead of its main bot", () => {
|
||||||
|
const main = {
|
||||||
|
id: "group-main",
|
||||||
|
name: "Chief",
|
||||||
|
kind: "main" as const,
|
||||||
|
preview: "",
|
||||||
|
pinned: false,
|
||||||
|
sectionId: null,
|
||||||
|
archivedAt: null,
|
||||||
|
unread: false,
|
||||||
|
updatedAt: "2026-01-01T00:00:00.000Z",
|
||||||
|
members: [],
|
||||||
|
};
|
||||||
|
const lead: MobileBot = { ...bot, id: "bot-lead", name: "Chief", kind: "main" };
|
||||||
|
const items = spaceInboxItems([space({ bots: [lead, bot], groups: [main] })]);
|
||||||
|
expect(items.map((item) => (item.type === "heading" ? item.title : item.type))).toEqual([
|
||||||
|
"Work",
|
||||||
|
"bot",
|
||||||
|
"group",
|
||||||
|
]);
|
||||||
|
expect(items.some((item) => item.type === "bot" && item.bot.id === lead.id)).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
it("keeps empty spaces beside a populated space", () => {
|
it("keeps empty spaces beside a populated space", () => {
|
||||||
const personal = space({ id: "personal", isDefault: true, bots: [bot] });
|
const personal = space({ id: "personal", isDefault: true, bots: [bot] });
|
||||||
expect(spaceInboxItems([personal, space()]).map((item) => item.type)).toEqual([
|
expect(spaceInboxItems([personal, space()]).map((item) => item.type)).toEqual([
|
||||||
|
|||||||
@ -32,7 +32,10 @@ export function canDeleteInboxSpace(
|
|||||||
export function spaceInboxItems(spaces: InboxSpace[]): InboxSpaceItem[] {
|
export function spaceInboxItems(spaces: InboxSpace[]): InboxSpaceItem[] {
|
||||||
return spaces.flatMap((space): InboxSpaceItem[] => {
|
return spaces.flatMap((space): InboxSpaceItem[] => {
|
||||||
const chats = [
|
const chats = [
|
||||||
...space.bots.map((chat) => ({ type: "bot" as const, bot: chat, ...chat })),
|
// A main bot is reached through its space's main conversation, not listed again.
|
||||||
|
...space.bots
|
||||||
|
.filter((chat) => chat.kind !== "main")
|
||||||
|
.map((chat) => ({ type: "bot" as const, bot: chat, ...chat })),
|
||||||
...space.groups.map((chat) => ({ type: "group" as const, group: chat, ...chat })),
|
...space.groups.map((chat) => ({ type: "group" as const, group: chat, ...chat })),
|
||||||
];
|
];
|
||||||
const items: InboxSpaceItem[] = [];
|
const items: InboxSpaceItem[] = [];
|
||||||
|
|||||||
@ -600,4 +600,20 @@ export const RU_MESSAGES: Record<string, string> = {
|
|||||||
"{name} (permission denied)": "{name} (в разрешении отказано)",
|
"{name} (permission denied)": "{name} (в разрешении отказано)",
|
||||||
"{name} (unsupported type)": "{name} (неподдерживаемый тип)",
|
"{name} (unsupported type)": "{name} (неподдерживаемый тип)",
|
||||||
"Saved {name} locally": "{name} сохранен локально.",
|
"Saved {name} locally": "{name} сохранен локально.",
|
||||||
|
"Allow photo access": "Разрешить доступ к фото",
|
||||||
|
Approve: "Утвердить",
|
||||||
|
Blocked: "Заблокировано",
|
||||||
|
"Choose photos and videos": "Выбрать фото и видео",
|
||||||
|
"Edit on web or desktop.": "Редактируйте в веб-версии или на компьютере.",
|
||||||
|
Media: "Медиа",
|
||||||
|
"No sessions yet": "Сессий пока нет",
|
||||||
|
"Open Settings": "Открыть настройки",
|
||||||
|
Projects: "Проекты",
|
||||||
|
"Rendering {percent}%": "Рендеринг {percent}%",
|
||||||
|
Sessions: "Сессии",
|
||||||
|
"Upload and create project": "Загрузить и создать проект",
|
||||||
|
Upload: "Загрузить",
|
||||||
|
"Uploading {percent}%": "Загрузка {percent}%",
|
||||||
|
"{count} items could not be uploaded.": "Не удалось загрузить элементов: {count}.",
|
||||||
|
"{count} items": "Элементов: {count}",
|
||||||
};
|
};
|
||||||
|
|||||||
@ -584,4 +584,20 @@ export const ZH_MESSAGES: Record<string, string> = {
|
|||||||
"Extra high": "超高",
|
"Extra high": "超高",
|
||||||
Low: "低",
|
Low: "低",
|
||||||
High: "高",
|
High: "高",
|
||||||
|
"Allow photo access": "允许访问照片",
|
||||||
|
Approve: "批准",
|
||||||
|
Blocked: "已阻止",
|
||||||
|
"Choose photos and videos": "选择照片和视频",
|
||||||
|
"Edit on web or desktop.": "请在网页版或桌面版中编辑。",
|
||||||
|
Media: "媒体",
|
||||||
|
"No sessions yet": "暂无会话",
|
||||||
|
"Open Settings": "打开设置",
|
||||||
|
Projects: "项目",
|
||||||
|
"Rendering {percent}%": "正在渲染 {percent}%",
|
||||||
|
Sessions: "会话",
|
||||||
|
"Upload and create project": "上传并创建项目",
|
||||||
|
Upload: "上传",
|
||||||
|
"Uploading {percent}%": "正在上传 {percent}%",
|
||||||
|
"{count} items could not be uploaded.": "{count} 个项目无法上传。",
|
||||||
|
"{count} items": "{count} 个项目",
|
||||||
};
|
};
|
||||||
|
|||||||
374
apps/mobile/lib/media-api.ts
Normal file
374
apps/mobile/lib/media-api.ts
Normal file
@ -0,0 +1,374 @@
|
|||||||
|
import type {
|
||||||
|
MediaAsset,
|
||||||
|
MediaCapabilities,
|
||||||
|
MediaRender,
|
||||||
|
MediaSource,
|
||||||
|
MediaUpload,
|
||||||
|
} from "@rakazo/contracts";
|
||||||
|
import { fetch as expoFetch } from "expo/fetch";
|
||||||
|
import * as Application from "expo-application";
|
||||||
|
import { File, Paths } from "expo-file-system";
|
||||||
|
import {
|
||||||
|
AssetField,
|
||||||
|
getPermissionsAsync,
|
||||||
|
Asset as LibraryAsset,
|
||||||
|
MediaType,
|
||||||
|
presentPermissionsPicker,
|
||||||
|
Query,
|
||||||
|
requestPermissionsAsync,
|
||||||
|
} from "expo-media-library";
|
||||||
|
import * as Notifications from "expo-notifications";
|
||||||
|
import { useRouter } from "expo-router";
|
||||||
|
import * as SecureStore from "expo-secure-store";
|
||||||
|
import * as Sharing from "expo-sharing";
|
||||||
|
import { useEffect, useState } from "react";
|
||||||
|
import { Platform } from "react-native";
|
||||||
|
import { authHeaders, currentApiBase, rpc, selectedSpaceId } from "./api";
|
||||||
|
import { t } from "./i18n";
|
||||||
|
import type { DeviceAsset, PushInput, PushResult } from "./media-index";
|
||||||
|
import { syncDeviceIndex } from "./media-index";
|
||||||
|
import type { LibraryAccess } from "./media-project";
|
||||||
|
import {
|
||||||
|
libraryAccess,
|
||||||
|
mediaNotificationRoute,
|
||||||
|
renderFileUrl,
|
||||||
|
shareFileName,
|
||||||
|
uploadsFromDevice,
|
||||||
|
videoOutput,
|
||||||
|
} from "./media-project";
|
||||||
|
import type { ChunkBytes, UploadTransport } from "./media-upload";
|
||||||
|
import {
|
||||||
|
combinedProgress,
|
||||||
|
hashOriginal,
|
||||||
|
MediaUploadHttpError,
|
||||||
|
uploadOriginal,
|
||||||
|
} from "./media-upload";
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Native glue for Media Studio on mobile: RPC calls, the media library, the
|
||||||
|
* chunk route, sharing, and push routing. Decisions live in the pure
|
||||||
|
* media-index, media-upload, and media-project modules.
|
||||||
|
*/
|
||||||
|
|
||||||
|
export function mediaRpc<T>(
|
||||||
|
proc: string,
|
||||||
|
body: unknown = {},
|
||||||
|
options?: { timeoutMs?: number | null; signal?: AbortSignal },
|
||||||
|
): Promise<T> {
|
||||||
|
return rpc<T>(`media/${proc}`, body, options);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Media appears only when the server reports Media Studio configured. */
|
||||||
|
export function useMediaAvailable(): boolean {
|
||||||
|
const [available, setAvailable] = useState(false);
|
||||||
|
useEffect(() => {
|
||||||
|
let active = true;
|
||||||
|
void mediaRpc<MediaCapabilities>("capabilities")
|
||||||
|
.then((capabilities) => {
|
||||||
|
if (active) setAvailable(capabilities.configured);
|
||||||
|
})
|
||||||
|
.catch(() => {
|
||||||
|
if (active) setAvailable(false);
|
||||||
|
});
|
||||||
|
return () => {
|
||||||
|
active = false;
|
||||||
|
};
|
||||||
|
}, []);
|
||||||
|
return available;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Library permission
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
const LIBRARY_PERMISSIONS = ["photo", "video"] as const;
|
||||||
|
|
||||||
|
export async function readLibraryAccess(): Promise<LibraryAccess> {
|
||||||
|
return libraryAccess(await getPermissionsAsync(false, [...LIBRARY_PERMISSIONS]));
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function requestLibraryAccess(): Promise<LibraryAccess> {
|
||||||
|
return libraryAccess(await requestPermissionsAsync(false, [...LIBRARY_PERMISSIONS]));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** With limited access, let the user change which items the app may see. */
|
||||||
|
export async function changeLimitedSelection(): Promise<void> {
|
||||||
|
await presentPermissionsPicker(["photo", "video"]);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Device index
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
async function deviceId(): Promise<string> {
|
||||||
|
if (Platform.OS === "ios") {
|
||||||
|
const id = await Application.getIosIdForVendorAsync();
|
||||||
|
if (id) return `ios:${id}`;
|
||||||
|
} else if (Platform.OS === "android") {
|
||||||
|
return `android:${Application.getAndroidId()}`;
|
||||||
|
}
|
||||||
|
throw new Error(t("Failed"));
|
||||||
|
}
|
||||||
|
|
||||||
|
function storeKey(sourceId: string): string {
|
||||||
|
const scope = `${selectedSpaceId() ?? "default"}.${sourceId}`.replace(/[^A-Za-z0-9._-]/g, "_");
|
||||||
|
return `rakazo.media.sync.${scope}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function readSyncToken(sourceId: string): Promise<string | null> {
|
||||||
|
try {
|
||||||
|
return (await SecureStore.getItemAsync(storeKey(sourceId))) || null;
|
||||||
|
} catch {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function writeSyncToken(sourceId: string, token: string | null): Promise<void> {
|
||||||
|
try {
|
||||||
|
if (token) await SecureStore.setItemAsync(storeKey(sourceId), token);
|
||||||
|
} catch {
|
||||||
|
// The next sync then pushes everything again; the server reports it unchanged.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function registerDeviceSource(): Promise<MediaSource> {
|
||||||
|
return mediaRpc<MediaSource>("sources/registerClient", {
|
||||||
|
kind: "ios_library",
|
||||||
|
label: Platform.OS === "ios" ? "iPhone" : "Android",
|
||||||
|
deviceId: await deviceId(),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
async function listDevice(offset: number, limit: number): Promise<DeviceAsset[]> {
|
||||||
|
const rows = await new Query()
|
||||||
|
.within(AssetField.MEDIA_TYPE, [MediaType.IMAGE, MediaType.VIDEO])
|
||||||
|
.orderBy({ key: AssetField.CREATION_TIME, ascending: true })
|
||||||
|
.offset(offset)
|
||||||
|
.limit(limit)
|
||||||
|
.exeForMetadata();
|
||||||
|
return rows.map((row) => ({
|
||||||
|
id: row.id,
|
||||||
|
filename: row.filename,
|
||||||
|
mediaType: row.mediaType,
|
||||||
|
width: row.width,
|
||||||
|
height: row.height,
|
||||||
|
duration: row.duration,
|
||||||
|
creationTime: row.creationTime,
|
||||||
|
modificationTime: row.modificationTime,
|
||||||
|
isFavorite: row.isFavorite,
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
|
||||||
|
async function serverExternalIds(sourceId: string): Promise<string[]> {
|
||||||
|
const ids: string[] = [];
|
||||||
|
let cursor: string | null = null;
|
||||||
|
do {
|
||||||
|
const page: { items: MediaAsset[]; nextCursor: string | null } = await mediaRpc(
|
||||||
|
"assets/search",
|
||||||
|
{ sourceId, cursor, limit: 500 },
|
||||||
|
);
|
||||||
|
for (const asset of page.items) ids.push(asset.externalId);
|
||||||
|
cursor = page.nextCursor;
|
||||||
|
} while (cursor);
|
||||||
|
return ids;
|
||||||
|
}
|
||||||
|
|
||||||
|
let syncing: Promise<MediaSource> | null = null;
|
||||||
|
|
||||||
|
/** Index the device library into its source. One sync runs at a time. */
|
||||||
|
export function syncDeviceLibrary(): Promise<MediaSource> {
|
||||||
|
syncing ??= (async () => {
|
||||||
|
const source = await registerDeviceSource();
|
||||||
|
const result = await syncDeviceIndex({
|
||||||
|
sourceId: source.id,
|
||||||
|
syncToken: await readSyncToken(source.id),
|
||||||
|
io: {
|
||||||
|
listDevice,
|
||||||
|
locate: (id) => new LibraryAsset(id).getLocation(),
|
||||||
|
push: (input: PushInput) =>
|
||||||
|
mediaRpc<PushResult>("sources/pushItems", input, { timeoutMs: 60_000 }),
|
||||||
|
serverAssetCount: async () => {
|
||||||
|
const sources = await mediaRpc<MediaSource[]>("sources/list");
|
||||||
|
return sources.find((entry) => entry.id === source.id)?.assetCount ?? 0;
|
||||||
|
},
|
||||||
|
serverExternalIds: () => serverExternalIds(source.id),
|
||||||
|
},
|
||||||
|
});
|
||||||
|
await writeSyncToken(source.id, result.syncToken);
|
||||||
|
return source;
|
||||||
|
})().finally(() => {
|
||||||
|
syncing = null;
|
||||||
|
});
|
||||||
|
return syncing;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Uploads
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
async function rpcUploadCall<T>(proc: string, body: unknown, timeoutMs?: number): Promise<T> {
|
||||||
|
try {
|
||||||
|
return await mediaRpc<T>(proc, body, timeoutMs ? { timeoutMs } : undefined);
|
||||||
|
} catch (error) {
|
||||||
|
const message = error instanceof Error ? error.message : String(error);
|
||||||
|
if (/expired/i.test(message)) throw new MediaUploadHttpError(409, message);
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function putChunk(
|
||||||
|
uploadId: string,
|
||||||
|
index: number,
|
||||||
|
bytes: ChunkBytes,
|
||||||
|
signal?: AbortSignal,
|
||||||
|
): Promise<MediaUpload> {
|
||||||
|
const url = `${currentApiBase()}/api/media/uploads/${encodeURIComponent(uploadId)}/chunks/${index}`;
|
||||||
|
const response = await expoFetch(url, {
|
||||||
|
method: "PUT",
|
||||||
|
headers: {
|
||||||
|
...(await authHeaders()),
|
||||||
|
origin: "rakazo://",
|
||||||
|
"content-type": "application/octet-stream",
|
||||||
|
},
|
||||||
|
body: bytes,
|
||||||
|
signal,
|
||||||
|
});
|
||||||
|
const body = (await response.json().catch(() => null)) as
|
||||||
|
| (MediaUpload & { error?: undefined })
|
||||||
|
| { error?: string }
|
||||||
|
| null;
|
||||||
|
if (!response.ok) {
|
||||||
|
const message = (body && "error" in body && body.error) || `Upload failed (${response.status})`;
|
||||||
|
throw new MediaUploadHttpError(response.status, message);
|
||||||
|
}
|
||||||
|
return body as MediaUpload;
|
||||||
|
}
|
||||||
|
|
||||||
|
const transport: UploadTransport = {
|
||||||
|
begin: (input) => rpcUploadCall<MediaUpload>("uploads/begin", input),
|
||||||
|
status: (uploadId) => rpcUploadCall<MediaUpload>("uploads/status", { uploadId }),
|
||||||
|
putChunk,
|
||||||
|
complete: (uploadId) => rpcUploadCall<MediaUpload>("uploads/complete", { uploadId }, 10 * 60_000),
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Hashes are slow on large videos; keep them for retries within this session. */
|
||||||
|
const hashes = new Map<string, string>();
|
||||||
|
|
||||||
|
async function uploadOne(
|
||||||
|
asset: MediaAsset,
|
||||||
|
onProgress: (fraction: number) => void,
|
||||||
|
signal?: AbortSignal,
|
||||||
|
): Promise<void> {
|
||||||
|
const original = new LibraryAsset(asset.externalId);
|
||||||
|
const file = new File(await original.getUri());
|
||||||
|
const sizeBytes = file.size;
|
||||||
|
if (!sizeBytes) throw new Error("Original not available");
|
||||||
|
const fileName = (await original.getFilename()) || asset.externalId;
|
||||||
|
const handle = file.open();
|
||||||
|
try {
|
||||||
|
const read = (start: number, length: number) => {
|
||||||
|
handle.offset = start;
|
||||||
|
return handle.readBytes(length);
|
||||||
|
};
|
||||||
|
const key = `${asset.id}:${sizeBytes}`;
|
||||||
|
let sha256 = hashes.get(key);
|
||||||
|
if (!sha256) {
|
||||||
|
sha256 = await hashOriginal(sizeBytes, read, {
|
||||||
|
signal,
|
||||||
|
onProgress: (fraction) => onProgress(fraction * 0.1),
|
||||||
|
});
|
||||||
|
hashes.set(key, sha256);
|
||||||
|
}
|
||||||
|
await uploadOriginal({
|
||||||
|
assetId: asset.id,
|
||||||
|
fileName,
|
||||||
|
sizeBytes,
|
||||||
|
sha256,
|
||||||
|
read,
|
||||||
|
transport,
|
||||||
|
signal,
|
||||||
|
onProgress: (fraction) => onProgress(0.1 + fraction * 0.9),
|
||||||
|
});
|
||||||
|
} finally {
|
||||||
|
handle.close();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Upload every listed item that still waits for its original. Returns the ids that failed. */
|
||||||
|
export async function uploadAssets(
|
||||||
|
assets: MediaAsset[],
|
||||||
|
deviceSourceId: string,
|
||||||
|
onProgress: (fraction: number) => void,
|
||||||
|
signal?: AbortSignal,
|
||||||
|
): Promise<string[]> {
|
||||||
|
const pending = uploadsFromDevice(assets, deviceSourceId);
|
||||||
|
const progress = pending.map(() => ({ sizeBytes: 1, fraction: 0 }));
|
||||||
|
const failed: string[] = [];
|
||||||
|
for (const [index, asset] of pending.entries()) {
|
||||||
|
signal?.throwIfAborted();
|
||||||
|
try {
|
||||||
|
await uploadOne(
|
||||||
|
asset,
|
||||||
|
(fraction) => {
|
||||||
|
const entry = progress[index];
|
||||||
|
if (entry) entry.fraction = fraction;
|
||||||
|
onProgress(combinedProgress(progress));
|
||||||
|
},
|
||||||
|
signal,
|
||||||
|
);
|
||||||
|
} catch (error) {
|
||||||
|
if (signal?.aborted) throw error;
|
||||||
|
failed.push(asset.id);
|
||||||
|
}
|
||||||
|
const entry = progress[index];
|
||||||
|
if (entry) entry.fraction = 1;
|
||||||
|
onProgress(combinedProgress(progress));
|
||||||
|
}
|
||||||
|
return failed;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Playback and sharing
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/** Authenticated source for expo-video. */
|
||||||
|
export async function renderVideoSource(
|
||||||
|
renderId: string,
|
||||||
|
fileName: string,
|
||||||
|
): Promise<{ uri: string; headers: Record<string, string> }> {
|
||||||
|
return {
|
||||||
|
uri: renderFileUrl(currentApiBase(), renderId, fileName),
|
||||||
|
headers: { ...(await authHeaders()), origin: "rakazo://" },
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Download a final render and open the native share sheet. */
|
||||||
|
export async function shareRender(render: MediaRender): Promise<void> {
|
||||||
|
const video = videoOutput(render);
|
||||||
|
if (!video || render.status !== "done") return;
|
||||||
|
const target = new File(Paths.cache, shareFileName(render.id, video.fileName));
|
||||||
|
const file = await File.downloadFileAsync(
|
||||||
|
renderFileUrl(currentApiBase(), render.id, video.fileName, true),
|
||||||
|
target,
|
||||||
|
{ headers: { ...(await authHeaders()), origin: "rakazo://" }, idempotent: true },
|
||||||
|
);
|
||||||
|
await Sharing.shareAsync(file.uri, { mimeType: "video/mp4" });
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Push
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/** Open the project when the user taps a render notification. Mount once in the root layout. */
|
||||||
|
export function useMediaNotificationRouting(): void {
|
||||||
|
const router = useRouter();
|
||||||
|
useEffect(() => {
|
||||||
|
const open = (response: Notifications.NotificationResponse | null) => {
|
||||||
|
const route = mediaNotificationRoute(response?.notification.request.content.data);
|
||||||
|
if (route) router.push(route);
|
||||||
|
};
|
||||||
|
open(Notifications.getLastNotificationResponse());
|
||||||
|
const subscription = Notifications.addNotificationResponseReceivedListener(open);
|
||||||
|
return () => subscription.remove();
|
||||||
|
}, [router]);
|
||||||
|
}
|
||||||
215
apps/mobile/lib/media-index.test.ts
Normal file
215
apps/mobile/lib/media-index.test.ts
Normal file
@ -0,0 +1,215 @@
|
|||||||
|
import { mediaRpcShapes } from "@rakazo/contracts";
|
||||||
|
import { describe, expect, it, vi } from "vitest";
|
||||||
|
import type { DeviceAsset, DeviceIndexIo, PushInput } from "./media-index";
|
||||||
|
import {
|
||||||
|
chunked,
|
||||||
|
parseSyncToken,
|
||||||
|
roundCoordinate,
|
||||||
|
syncDeviceIndex,
|
||||||
|
toClientMediaItem,
|
||||||
|
} from "./media-index";
|
||||||
|
|
||||||
|
function asset(id: string, overrides: Partial<DeviceAsset> = {}): DeviceAsset {
|
||||||
|
return {
|
||||||
|
id,
|
||||||
|
filename: `${id}.mov`,
|
||||||
|
mediaType: "video",
|
||||||
|
width: 1920,
|
||||||
|
height: 1080,
|
||||||
|
duration: 12_345,
|
||||||
|
creationTime: 1_700_000_000_000,
|
||||||
|
modificationTime: 1_700_000_000_000,
|
||||||
|
isFavorite: false,
|
||||||
|
...overrides,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Device library plus a server that keeps what was pushed, like `sources.pushItems`. */
|
||||||
|
function harness(device: DeviceAsset[], serverIds: string[] = []) {
|
||||||
|
const server = new Set(serverIds);
|
||||||
|
const pushes: PushInput[] = [];
|
||||||
|
const io = {
|
||||||
|
listDevice: vi.fn(async (offset: number, limit: number) =>
|
||||||
|
device.slice(offset, offset + limit),
|
||||||
|
),
|
||||||
|
locate: vi.fn(async () => ({ latitude: 52.520_08, longitude: 13.404_95 })),
|
||||||
|
push: vi.fn(async (input: PushInput) => {
|
||||||
|
// Every push must satisfy the RPC contract.
|
||||||
|
mediaRpcShapes.sources.pushItems.input.parse(input);
|
||||||
|
pushes.push(input);
|
||||||
|
for (const item of input.items) server.add(item.externalId);
|
||||||
|
for (const id of input.deletedExternalIds) server.delete(id);
|
||||||
|
return { created: input.items.length, updated: 0, unchanged: 0, deleted: 0 };
|
||||||
|
}),
|
||||||
|
serverAssetCount: vi.fn(async () => server.size),
|
||||||
|
serverExternalIds: vi.fn(async () => [...server]),
|
||||||
|
} satisfies DeviceIndexIo;
|
||||||
|
return { io, server, pushes };
|
||||||
|
}
|
||||||
|
|
||||||
|
describe("toClientMediaItem", () => {
|
||||||
|
it("maps videos with duration and rounded location", () => {
|
||||||
|
const item = toClientMediaItem(asset("v1"), { latitude: 52.520_08, longitude: 13.404_95 });
|
||||||
|
expect(item).toEqual({
|
||||||
|
externalId: "v1",
|
||||||
|
kind: "video",
|
||||||
|
capturedAt: "2023-11-14T22:13:20.000Z",
|
||||||
|
durationMs: 12_345,
|
||||||
|
width: 1920,
|
||||||
|
height: 1080,
|
||||||
|
fps: null,
|
||||||
|
gpsLat: 52.52,
|
||||||
|
gpsLon: 13.4,
|
||||||
|
deviceModel: null,
|
||||||
|
albumNames: [],
|
||||||
|
isFavorite: false,
|
||||||
|
fileName: "v1.mov",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("maps photos without a duration", () => {
|
||||||
|
const item = toClientMediaItem(
|
||||||
|
asset("p1", { mediaType: "image", duration: 0, filename: "p1.heic", isFavorite: true }),
|
||||||
|
null,
|
||||||
|
);
|
||||||
|
expect(item).toMatchObject({ kind: "photo", durationMs: null, gpsLat: null, isFavorite: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("skips audio and unknown entries", () => {
|
||||||
|
expect(toClientMediaItem(asset("a1", { mediaType: "audio" }), null)).toBeNull();
|
||||||
|
expect(toClientMediaItem(asset("u1", { mediaType: "unknown" }), null)).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("drops missing or invalid metadata instead of sending it", () => {
|
||||||
|
const item = toClientMediaItem(
|
||||||
|
asset("v2", { width: null, height: 0, creationTime: null, filename: null }),
|
||||||
|
{ latitude: 0, longitude: 0 },
|
||||||
|
);
|
||||||
|
expect(item).toMatchObject({
|
||||||
|
width: null,
|
||||||
|
height: null,
|
||||||
|
capturedAt: null,
|
||||||
|
fileName: null,
|
||||||
|
gpsLat: null,
|
||||||
|
gpsLon: null,
|
||||||
|
});
|
||||||
|
expect(toClientMediaItem(asset("v3"), { latitude: 120, longitude: 10 })?.gpsLat).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rounds coordinates to about a kilometer", () => {
|
||||||
|
expect(roundCoordinate(48.137_15)).toBe(48.14);
|
||||||
|
expect(roundCoordinate(-33.868_82)).toBe(-33.87);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("helpers", () => {
|
||||||
|
it("parses only numeric sync tokens", () => {
|
||||||
|
expect(parseSyncToken(null)).toBeNull();
|
||||||
|
expect(parseSyncToken("abc")).toBeNull();
|
||||||
|
expect(parseSyncToken("1700000000000")).toBe(1_700_000_000_000);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("chunks lists", () => {
|
||||||
|
expect(chunked([1, 2, 3, 4, 5], 2)).toEqual([[1, 2], [3, 4], [5]]);
|
||||||
|
expect(chunked([], 2)).toEqual([]);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("syncDeviceIndex", () => {
|
||||||
|
it("indexes photos and videos on the first sync and completes", async () => {
|
||||||
|
const device = [
|
||||||
|
asset("v1"),
|
||||||
|
asset("p1", { mediaType: "image", modificationTime: 1_700_000_500_000 }),
|
||||||
|
asset("a1", { mediaType: "audio" }),
|
||||||
|
];
|
||||||
|
const { io, server, pushes } = harness(device);
|
||||||
|
const result = await syncDeviceIndex({ sourceId: "src1", syncToken: null, io });
|
||||||
|
expect([...server].sort()).toEqual(["p1", "v1"]);
|
||||||
|
expect(result).toEqual({ scanned: 2, pushed: 2, deleted: 0, syncToken: "1700000500000" });
|
||||||
|
const last = pushes.at(-1);
|
||||||
|
expect(last).toMatchObject({ complete: true, syncToken: "1700000500000", items: [] });
|
||||||
|
expect(pushes.slice(0, -1).every((push) => !push.complete && push.syncToken === null)).toBe(
|
||||||
|
true,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("pages through large libraries within the push limit", async () => {
|
||||||
|
const device = Array.from({ length: 1_203 }, (_, index) =>
|
||||||
|
asset(`v${index}`, { modificationTime: 1_700_000_000_000 + index }),
|
||||||
|
);
|
||||||
|
const { io, server, pushes } = harness(device);
|
||||||
|
await syncDeviceIndex({ sourceId: "src1", syncToken: null, io });
|
||||||
|
expect(server.size).toBe(1_203);
|
||||||
|
expect(io.listDevice).toHaveBeenCalledTimes(3);
|
||||||
|
expect(pushes.every((push) => push.items.length <= 500)).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("pushes only items changed since the token when nothing else differs", async () => {
|
||||||
|
const device = [
|
||||||
|
asset("old", { modificationTime: 1_000, creationTime: 1_000 }),
|
||||||
|
asset("new", { modificationTime: 3_000, creationTime: 3_000 }),
|
||||||
|
];
|
||||||
|
const { io, pushes } = harness(device, ["old"]);
|
||||||
|
const result = await syncDeviceIndex({ sourceId: "src1", syncToken: "2000", io });
|
||||||
|
expect(pushes.flatMap((push) => push.items.map((item) => item.externalId))).toEqual(["new"]);
|
||||||
|
expect(io.serverExternalIds).not.toHaveBeenCalled();
|
||||||
|
expect(result.syncToken).toBe("3000");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("sends no items when nothing changed and keeps the token", async () => {
|
||||||
|
const device = [asset("v1", { modificationTime: 1_000, creationTime: 1_000 })];
|
||||||
|
const { io, pushes } = harness(device, ["v1"]);
|
||||||
|
const result = await syncDeviceIndex({ sourceId: "src1", syncToken: "1000", io });
|
||||||
|
expect(pushes).toHaveLength(1);
|
||||||
|
expect(pushes[0]).toMatchObject({ items: [], deletedExternalIds: [], complete: true });
|
||||||
|
expect(result.syncToken).toBe("1000");
|
||||||
|
expect(io.locate).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("reports items deleted on the device", async () => {
|
||||||
|
const device = [asset("keep", { modificationTime: 1_000, creationTime: 1_000 })];
|
||||||
|
const { io, server } = harness(device, ["keep", "gone1", "gone2"]);
|
||||||
|
const result = await syncDeviceIndex({ sourceId: "src1", syncToken: "1000", io });
|
||||||
|
expect([...server]).toEqual(["keep"]);
|
||||||
|
expect(result.deleted).toBe(2);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("adds old items newly shared through limited access", async () => {
|
||||||
|
const device = [
|
||||||
|
asset("known", { modificationTime: 1_000, creationTime: 1_000 }),
|
||||||
|
asset("shared", { modificationTime: 500, creationTime: 500 }),
|
||||||
|
];
|
||||||
|
const { io, server } = harness(device, ["known"]);
|
||||||
|
await syncDeviceIndex({ sourceId: "src1", syncToken: "1000", io });
|
||||||
|
expect([...server].sort()).toEqual(["known", "shared"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("splits large deletions and completes only on the last push", async () => {
|
||||||
|
const serverIds = Array.from({ length: 6_000 }, (_, index) => `gone${index}`);
|
||||||
|
const { io, pushes, server } = harness([], serverIds);
|
||||||
|
const result = await syncDeviceIndex({ sourceId: "src1", syncToken: null, io });
|
||||||
|
expect(server.size).toBe(0);
|
||||||
|
expect(result.deleted).toBe(6_000);
|
||||||
|
expect(pushes.map((push) => [push.deletedExternalIds.length, push.complete])).toEqual([
|
||||||
|
[5_000, false],
|
||||||
|
[1_000, true],
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("indexes without location when it is not permitted", async () => {
|
||||||
|
const { io, pushes } = harness([asset("v1")]);
|
||||||
|
io.locate.mockRejectedValue(new Error("ACCESS_MEDIA_LOCATION"));
|
||||||
|
await syncDeviceIndex({ sourceId: "src1", syncToken: null, io });
|
||||||
|
expect(pushes[0]?.items[0]).toMatchObject({ gpsLat: null, gpsLon: null });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("stops when aborted", async () => {
|
||||||
|
const { io } = harness([asset("v1")]);
|
||||||
|
const controller = new AbortController();
|
||||||
|
controller.abort(new Error("stopped"));
|
||||||
|
await expect(
|
||||||
|
syncDeviceIndex({ sourceId: "src1", syncToken: null, io, signal: controller.signal }),
|
||||||
|
).rejects.toThrow("stopped");
|
||||||
|
expect(io.push).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
});
|
||||||
209
apps/mobile/lib/media-index.ts
Normal file
209
apps/mobile/lib/media-index.ts
Normal file
@ -0,0 +1,209 @@
|
|||||||
|
import type { ClientMediaItem } from "@rakazo/contracts";
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Index of the device media library (photos and videos) pushed to the
|
||||||
|
* `ios_library` source through `sources.pushItems`. Only metadata leaves the
|
||||||
|
* device here; originals upload later, per session, when the user asks.
|
||||||
|
*/
|
||||||
|
|
||||||
|
/** Lightweight library metadata as expo-media-library reports it. */
|
||||||
|
export type DeviceAsset = {
|
||||||
|
id: string;
|
||||||
|
filename: string | null;
|
||||||
|
mediaType: string;
|
||||||
|
width: number | null;
|
||||||
|
height: number | null;
|
||||||
|
/** Milliseconds. */
|
||||||
|
duration: number | null;
|
||||||
|
/** Unix milliseconds. */
|
||||||
|
creationTime: number | null;
|
||||||
|
modificationTime: number | null;
|
||||||
|
isFavorite: boolean;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type DeviceLocation = { latitude: number; longitude: number };
|
||||||
|
|
||||||
|
export type PushInput = {
|
||||||
|
sourceId: string;
|
||||||
|
items: ClientMediaItem[];
|
||||||
|
deletedExternalIds: string[];
|
||||||
|
syncToken: string | null;
|
||||||
|
complete: boolean;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type PushResult = { created: number; updated: number; unchanged: number; deleted: number };
|
||||||
|
|
||||||
|
export const PUSH_ITEMS_MAX = 500;
|
||||||
|
export const PUSH_DELETIONS_MAX = 5000;
|
||||||
|
|
||||||
|
/** Coordinates leave the device rounded to two decimals (about 1 km). */
|
||||||
|
export function roundCoordinate(value: number): number {
|
||||||
|
return Math.round(value * 100) / 100;
|
||||||
|
}
|
||||||
|
|
||||||
|
function validCoordinate(value: number, limit: number): boolean {
|
||||||
|
return Number.isFinite(value) && Math.abs(value) <= limit;
|
||||||
|
}
|
||||||
|
|
||||||
|
function positiveInt(value: number | null): number | null {
|
||||||
|
return value !== null && Number.isFinite(value) && value > 0 ? Math.round(value) : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function isoOrNull(ms: number | null): string | null {
|
||||||
|
if (ms === null || !Number.isFinite(ms) || ms <= 0) return null;
|
||||||
|
const date = new Date(ms);
|
||||||
|
return Number.isNaN(date.getTime()) ? null : date.toISOString();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Photos and videos only; audio and unknown entries are not media for this index. */
|
||||||
|
export function isIndexable(asset: Pick<DeviceAsset, "mediaType">): boolean {
|
||||||
|
return asset.mediaType === "image" || asset.mediaType === "video";
|
||||||
|
}
|
||||||
|
|
||||||
|
export function toClientMediaItem(
|
||||||
|
asset: DeviceAsset,
|
||||||
|
location: DeviceLocation | null,
|
||||||
|
): ClientMediaItem | null {
|
||||||
|
if (!isIndexable(asset) || !asset.id || asset.id.length > 200) return null;
|
||||||
|
const hasLocation =
|
||||||
|
location !== null &&
|
||||||
|
validCoordinate(location.latitude, 90) &&
|
||||||
|
validCoordinate(location.longitude, 180) &&
|
||||||
|
!(location.latitude === 0 && location.longitude === 0);
|
||||||
|
const video = asset.mediaType === "video";
|
||||||
|
return {
|
||||||
|
externalId: asset.id,
|
||||||
|
kind: video ? "video" : "photo",
|
||||||
|
capturedAt: isoOrNull(asset.creationTime),
|
||||||
|
durationMs: video ? positiveInt(asset.duration) : null,
|
||||||
|
width: positiveInt(asset.width),
|
||||||
|
height: positiveInt(asset.height),
|
||||||
|
fps: null,
|
||||||
|
gpsLat: hasLocation ? roundCoordinate(location.latitude) : null,
|
||||||
|
gpsLon: hasLocation ? roundCoordinate(location.longitude) : null,
|
||||||
|
deviceModel: null,
|
||||||
|
albumNames: [],
|
||||||
|
isFavorite: asset.isFavorite,
|
||||||
|
fileName: asset.filename ? asset.filename.slice(0, 255) : null,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The sync token is the newest change time seen; later syncs push only newer changes. */
|
||||||
|
export function changeTime(asset: DeviceAsset): number {
|
||||||
|
return Math.max(asset.modificationTime ?? 0, asset.creationTime ?? 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function parseSyncToken(token: string | null): number | null {
|
||||||
|
if (!token || !/^[0-9]{1,16}$/.test(token)) return null;
|
||||||
|
return Number(token);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function chunked<T>(items: T[], size: number): T[][] {
|
||||||
|
const out: T[][] = [];
|
||||||
|
for (let index = 0; index < items.length; index += size)
|
||||||
|
out.push(items.slice(index, index + size));
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
export type DeviceIndexIo = {
|
||||||
|
/** One page of library metadata in a stable order; fewer than `limit` ends the scan. */
|
||||||
|
listDevice(offset: number, limit: number): Promise<DeviceAsset[]>;
|
||||||
|
/** Location of one asset, or null when unknown or not permitted. */
|
||||||
|
locate(id: string): Promise<DeviceLocation | null>;
|
||||||
|
push(input: PushInput): Promise<PushResult>;
|
||||||
|
/** How many assets the server holds for this source. */
|
||||||
|
serverAssetCount(): Promise<number>;
|
||||||
|
/** Every externalId the server holds for this source. */
|
||||||
|
serverExternalIds(): Promise<string[]>;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type DeviceIndexResult = {
|
||||||
|
scanned: number;
|
||||||
|
pushed: number;
|
||||||
|
deleted: number;
|
||||||
|
syncToken: string | null;
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Bring the server index in line with the device library. Pushes items changed
|
||||||
|
* since `syncToken`; when the server count then differs from the device count
|
||||||
|
* (deletions, or items newly shared through limited access), reconciles by id.
|
||||||
|
*/
|
||||||
|
export async function syncDeviceIndex(input: {
|
||||||
|
sourceId: string;
|
||||||
|
syncToken: string | null;
|
||||||
|
io: DeviceIndexIo;
|
||||||
|
pageSize?: number;
|
||||||
|
signal?: AbortSignal;
|
||||||
|
onProgress?: (scanned: number) => void;
|
||||||
|
}): Promise<DeviceIndexResult> {
|
||||||
|
const { io, signal } = input;
|
||||||
|
const pageSize = Math.min(input.pageSize ?? PUSH_ITEMS_MAX, PUSH_ITEMS_MAX);
|
||||||
|
const since = parseSyncToken(input.syncToken);
|
||||||
|
|
||||||
|
const device = new Map<string, DeviceAsset>();
|
||||||
|
for (let offset = 0; ; offset += pageSize) {
|
||||||
|
signal?.throwIfAborted();
|
||||||
|
const page = await io.listDevice(offset, pageSize);
|
||||||
|
for (const asset of page) if (isIndexable(asset)) device.set(asset.id, asset);
|
||||||
|
input.onProgress?.(device.size);
|
||||||
|
if (page.length < pageSize) break;
|
||||||
|
}
|
||||||
|
|
||||||
|
let newest = since ?? 0;
|
||||||
|
for (const asset of device.values()) newest = Math.max(newest, changeTime(asset));
|
||||||
|
const syncToken = newest > 0 ? String(newest) : null;
|
||||||
|
|
||||||
|
let pushed = 0;
|
||||||
|
const sent = new Set<string>();
|
||||||
|
const pushAssets = async (assets: DeviceAsset[]) => {
|
||||||
|
for (const batch of chunked(assets, pageSize)) {
|
||||||
|
signal?.throwIfAborted();
|
||||||
|
const items: ClientMediaItem[] = [];
|
||||||
|
for (const asset of batch) {
|
||||||
|
const location = await io.locate(asset.id).catch(() => null);
|
||||||
|
const item = toClientMediaItem(asset, location);
|
||||||
|
if (item) items.push(item);
|
||||||
|
sent.add(asset.id);
|
||||||
|
}
|
||||||
|
if (items.length === 0) continue;
|
||||||
|
await io.push({
|
||||||
|
sourceId: input.sourceId,
|
||||||
|
items,
|
||||||
|
deletedExternalIds: [],
|
||||||
|
syncToken: null,
|
||||||
|
complete: false,
|
||||||
|
});
|
||||||
|
pushed += items.length;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const changed = [...device.values()].filter(
|
||||||
|
(asset) => since === null || changeTime(asset) > since,
|
||||||
|
);
|
||||||
|
await pushAssets(changed);
|
||||||
|
|
||||||
|
let deletions: string[] = [];
|
||||||
|
if ((await io.serverAssetCount()) !== device.size) {
|
||||||
|
const server = new Set(await io.serverExternalIds());
|
||||||
|
await pushAssets(
|
||||||
|
[...device.values()].filter((asset) => !server.has(asset.id) && !sent.has(asset.id)),
|
||||||
|
);
|
||||||
|
deletions = [...server].filter((id) => !device.has(id));
|
||||||
|
}
|
||||||
|
|
||||||
|
const deletionBatches = chunked(deletions, PUSH_DELETIONS_MAX);
|
||||||
|
if (deletionBatches.length === 0) deletionBatches.push([]);
|
||||||
|
for (const [index, batch] of deletionBatches.entries()) {
|
||||||
|
signal?.throwIfAborted();
|
||||||
|
const last = index === deletionBatches.length - 1;
|
||||||
|
await io.push({
|
||||||
|
sourceId: input.sourceId,
|
||||||
|
items: [],
|
||||||
|
deletedExternalIds: batch,
|
||||||
|
syncToken: last ? syncToken : null,
|
||||||
|
complete: last,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return { scanned: device.size, pushed, deleted: deletions.length, syncToken };
|
||||||
|
}
|
||||||
299
apps/mobile/lib/media-project.test.ts
Normal file
299
apps/mobile/lib/media-project.test.ts
Normal file
@ -0,0 +1,299 @@
|
|||||||
|
import type { MediaEditPlan, MediaRender } from "@rakazo/contracts";
|
||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
import {
|
||||||
|
assetThumbUrl,
|
||||||
|
blockers,
|
||||||
|
canApprove,
|
||||||
|
canShare,
|
||||||
|
formatDuration,
|
||||||
|
isRenderActive,
|
||||||
|
libraryAccess,
|
||||||
|
mediaNotificationRoute,
|
||||||
|
projectReadiness,
|
||||||
|
projectVariants,
|
||||||
|
renderFileUrl,
|
||||||
|
shareFileName,
|
||||||
|
shouldPollProject,
|
||||||
|
uploadsFromDevice,
|
||||||
|
variantState,
|
||||||
|
} from "./media-project";
|
||||||
|
|
||||||
|
function plan(
|
||||||
|
id: string,
|
||||||
|
profileId: string,
|
||||||
|
version: number,
|
||||||
|
createdAt = "2026-01-01T00:00:00.000Z",
|
||||||
|
) {
|
||||||
|
return { id, profileId, version, createdAt } as MediaEditPlan;
|
||||||
|
}
|
||||||
|
|
||||||
|
function render(id: string, planId: string, overrides: Partial<MediaRender> = {}): MediaRender {
|
||||||
|
return {
|
||||||
|
id,
|
||||||
|
planId,
|
||||||
|
projectId: "p1",
|
||||||
|
profileId: "example-short-vertical",
|
||||||
|
quality: "draft",
|
||||||
|
status: "done",
|
||||||
|
progress: 1,
|
||||||
|
outputs: [{ kind: "video", artifactId: null, fileName: "video.mp4", sizeBytes: 10 }],
|
||||||
|
gateReport: null,
|
||||||
|
ffprobe: null,
|
||||||
|
error: null,
|
||||||
|
createdAt: "2026-01-01T00:00:00.000Z",
|
||||||
|
updatedAt: "2026-01-01T00:00:00.000Z",
|
||||||
|
...overrides,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
const profiles = [{ id: "short", name: "Short" }];
|
||||||
|
|
||||||
|
function variantFor(renders: MediaRender[]) {
|
||||||
|
const [variant] = projectVariants({
|
||||||
|
profileIds: ["short"],
|
||||||
|
plans: [plan("plan1", "short", 1)],
|
||||||
|
renders,
|
||||||
|
profiles,
|
||||||
|
});
|
||||||
|
if (!variant) throw new Error("no variant");
|
||||||
|
return variant;
|
||||||
|
}
|
||||||
|
|
||||||
|
describe("projectVariants", () => {
|
||||||
|
it("uses the newest plan version per profile and its renders only", () => {
|
||||||
|
const variants = projectVariants({
|
||||||
|
profileIds: ["short", "wide", "missing"],
|
||||||
|
plans: [plan("a1", "short", 1), plan("a2", "short", 2), plan("b1", "wide", 1)],
|
||||||
|
renders: [
|
||||||
|
render("r-old", "a1", { createdAt: "2026-01-02T00:00:00.000Z" }),
|
||||||
|
render("r-new", "a2"),
|
||||||
|
],
|
||||||
|
profiles,
|
||||||
|
});
|
||||||
|
expect(
|
||||||
|
variants.map((variant) => [variant.profileId, variant.profileName, variant.plan.id]),
|
||||||
|
).toEqual([
|
||||||
|
["short", "Short", "a2"],
|
||||||
|
["wide", "wide", "b1"],
|
||||||
|
]);
|
||||||
|
expect(variants[0]?.latest?.id).toBe("r-new");
|
||||||
|
expect(variants[1]?.latest).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("previews the final render once it is done, else the draft", () => {
|
||||||
|
const draft = render("d", "plan1", { createdAt: "2026-01-01T00:00:00.000Z" });
|
||||||
|
const finalRunning = render("f", "plan1", {
|
||||||
|
quality: "final",
|
||||||
|
status: "rendering",
|
||||||
|
outputs: [],
|
||||||
|
createdAt: "2026-01-02T00:00:00.000Z",
|
||||||
|
});
|
||||||
|
expect(variantFor([draft, finalRunning]).preview?.id).toBe("d");
|
||||||
|
const finalDone = { ...finalRunning, status: "done" as const, outputs: draft.outputs };
|
||||||
|
expect(variantFor([draft, finalDone]).preview?.id).toBe("f");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("variant state and actions", () => {
|
||||||
|
it("is planned without renders and can be approved", () => {
|
||||||
|
const variant = variantFor([]);
|
||||||
|
expect(variantState(variant)).toBe("planned");
|
||||||
|
expect(canApprove(variant)).toBe(true);
|
||||||
|
expect(canShare(variant)).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("is rendering while a draft runs and cannot be approved", () => {
|
||||||
|
const variant = variantFor([render("d", "plan1", { status: "gating", progress: 0.9 })]);
|
||||||
|
expect(variantState(variant)).toBe("rendering");
|
||||||
|
expect(canApprove(variant)).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("approves a finished draft", () => {
|
||||||
|
const variant = variantFor([render("d", "plan1")]);
|
||||||
|
expect(variantState(variant)).toBe("draft_ready");
|
||||||
|
expect(canApprove(variant)).toBe(true);
|
||||||
|
expect(canShare(variant)).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("shares only a finished final render", () => {
|
||||||
|
const running = variantFor([render("f", "plan1", { quality: "final", status: "queued" })]);
|
||||||
|
expect(variantState(running)).toBe("approved_rendering");
|
||||||
|
expect(canShare(running)).toBe(false);
|
||||||
|
const done = variantFor([render("f", "plan1", { quality: "final" })]);
|
||||||
|
expect(variantState(done)).toBe("ready");
|
||||||
|
expect(canShare(done)).toBe(true);
|
||||||
|
expect(canApprove(done)).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("never shares a blocked render and lists its blockers", () => {
|
||||||
|
const blocked = render("f", "plan1", {
|
||||||
|
quality: "final",
|
||||||
|
status: "blocked",
|
||||||
|
gateReport: {
|
||||||
|
profileId: "short",
|
||||||
|
passed: false,
|
||||||
|
checkedAt: "2026-01-01T00:00:00.000Z",
|
||||||
|
sampledFrames: 10,
|
||||||
|
autoFixRound: 3,
|
||||||
|
issues: [
|
||||||
|
{
|
||||||
|
check: "consent",
|
||||||
|
severity: "block",
|
||||||
|
code: "consent_missing",
|
||||||
|
message: "Consent missing",
|
||||||
|
},
|
||||||
|
{ check: "text", severity: "warn", code: "long", message: "Long caption" },
|
||||||
|
],
|
||||||
|
},
|
||||||
|
});
|
||||||
|
const variant = variantFor([blocked]);
|
||||||
|
expect(variantState(variant)).toBe("blocked");
|
||||||
|
expect(canShare(variant)).toBe(false);
|
||||||
|
expect(canApprove(variant)).toBe(false);
|
||||||
|
expect(blockers(variant.latest).map((issue) => issue.code)).toEqual(["consent_missing"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("allows another try after a failure", () => {
|
||||||
|
const variant = variantFor([render("d", "plan1", { status: "failed", outputs: [] })]);
|
||||||
|
expect(variantState(variant)).toBe("failed");
|
||||||
|
expect(canApprove(variant)).toBe(true);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("polling", () => {
|
||||||
|
it("polls only while a render is queued, rendering, or gating", () => {
|
||||||
|
expect(isRenderActive(null)).toBe(false);
|
||||||
|
for (const status of ["queued", "rendering", "gating"] as const) {
|
||||||
|
expect(shouldPollProject([render("r", "p", { status })])).toBe(true);
|
||||||
|
}
|
||||||
|
for (const status of ["done", "blocked", "failed", "canceled"] as const) {
|
||||||
|
expect(shouldPollProject([render("r", "p", { status })])).toBe(false);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("projectReadiness", () => {
|
||||||
|
const device = "src-phone";
|
||||||
|
|
||||||
|
it("waits for this device's uploads first", () => {
|
||||||
|
expect(
|
||||||
|
projectReadiness(
|
||||||
|
[
|
||||||
|
{ sourceId: device, storage: "remote", ingestState: "awaiting_upload" },
|
||||||
|
{ sourceId: device, storage: "uploaded", ingestState: "analyzed" },
|
||||||
|
],
|
||||||
|
true,
|
||||||
|
device,
|
||||||
|
),
|
||||||
|
).toBe("uploading");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("treats items another device must upload as processing", () => {
|
||||||
|
const assets = [
|
||||||
|
{ sourceId: "src-mac", storage: "remote" as const, ingestState: "awaiting_upload" as const },
|
||||||
|
];
|
||||||
|
expect(projectReadiness(assets, true, device)).toBe("processing");
|
||||||
|
expect(uploadsFromDevice(assets, device)).toEqual([]);
|
||||||
|
expect(uploadsFromDevice(assets, null)).toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("waits for analysis when it is available", () => {
|
||||||
|
const assets = [
|
||||||
|
{ sourceId: device, storage: "uploaded" as const, ingestState: "proxied" as const },
|
||||||
|
];
|
||||||
|
expect(projectReadiness(assets, true, device)).toBe("processing");
|
||||||
|
expect(projectReadiness(assets, false, device)).toBe("ready");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("ignores failed items unless all failed", () => {
|
||||||
|
expect(
|
||||||
|
projectReadiness(
|
||||||
|
[
|
||||||
|
{ sourceId: device, storage: "uploaded", ingestState: "failed" },
|
||||||
|
{ sourceId: device, storage: "uploaded", ingestState: "analyzed" },
|
||||||
|
],
|
||||||
|
true,
|
||||||
|
device,
|
||||||
|
),
|
||||||
|
).toBe("ready");
|
||||||
|
expect(
|
||||||
|
projectReadiness(
|
||||||
|
[{ sourceId: device, storage: "uploaded", ingestState: "failed" }],
|
||||||
|
true,
|
||||||
|
device,
|
||||||
|
),
|
||||||
|
).toBe("failed");
|
||||||
|
expect(projectReadiness([], true, device)).toBe("processing");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("urls and names", () => {
|
||||||
|
it("builds render and thumbnail routes", () => {
|
||||||
|
expect(renderFileUrl("https://api.example.test/", "r1", "video.mp4")).toBe(
|
||||||
|
"https://api.example.test/api/media/renders/r1/files/video.mp4",
|
||||||
|
);
|
||||||
|
expect(renderFileUrl("https://api.example.test", "r1", "video.mp4", true)).toBe(
|
||||||
|
"https://api.example.test/api/media/renders/r1/files/video.mp4?download=1",
|
||||||
|
);
|
||||||
|
expect(assetThumbUrl("https://api.example.test", "a 1")).toBe(
|
||||||
|
"https://api.example.test/api/media/assets/a%201/thumb",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("names shared files from the render id", () => {
|
||||||
|
expect(shareFileName("r1", "video.MP4")).toBe("render-r1.mp4");
|
||||||
|
expect(shareFileName("../r1", "video")).toBe("render-r1.mp4");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("formats durations", () => {
|
||||||
|
expect(formatDuration(0)).toBe("0:00");
|
||||||
|
expect(formatDuration(65_400)).toBe("1:05");
|
||||||
|
expect(formatDuration(3_725_000)).toBe("1:02:05");
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("mediaNotificationRoute", () => {
|
||||||
|
it("opens the project of a render push", () => {
|
||||||
|
expect(
|
||||||
|
mediaNotificationRoute({
|
||||||
|
kind: "media.render",
|
||||||
|
projectId: "p1",
|
||||||
|
renderId: "r1",
|
||||||
|
status: "done",
|
||||||
|
}),
|
||||||
|
).toEqual({ pathname: "/media-project", params: { projectId: "p1" } });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("ignores other pushes and malformed ids", () => {
|
||||||
|
expect(mediaNotificationRoute(null)).toBeNull();
|
||||||
|
expect(mediaNotificationRoute({ kind: "completion", threadId: "t1" })).toBeNull();
|
||||||
|
expect(mediaNotificationRoute({ kind: "media.render", projectId: "../x" })).toBeNull();
|
||||||
|
expect(mediaNotificationRoute({ kind: "media.render" })).toBeNull();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("libraryAccess", () => {
|
||||||
|
it("distinguishes full, limited, undecided, denied, and blocked access", () => {
|
||||||
|
expect(
|
||||||
|
libraryAccess({
|
||||||
|
granted: true,
|
||||||
|
canAskAgain: true,
|
||||||
|
status: "granted",
|
||||||
|
accessPrivileges: "all",
|
||||||
|
}),
|
||||||
|
).toBe("all");
|
||||||
|
expect(
|
||||||
|
libraryAccess({
|
||||||
|
granted: true,
|
||||||
|
canAskAgain: true,
|
||||||
|
status: "granted",
|
||||||
|
accessPrivileges: "limited",
|
||||||
|
}),
|
||||||
|
).toBe("limited");
|
||||||
|
expect(libraryAccess({ granted: false, canAskAgain: true, status: "undetermined" })).toBe(
|
||||||
|
"undetermined",
|
||||||
|
);
|
||||||
|
expect(libraryAccess({ granted: false, canAskAgain: true, status: "denied" })).toBe("denied");
|
||||||
|
expect(libraryAccess({ granted: false, canAskAgain: false, status: "denied" })).toBe("blocked");
|
||||||
|
});
|
||||||
|
});
|
||||||
227
apps/mobile/lib/media-project.ts
Normal file
227
apps/mobile/lib/media-project.ts
Normal file
@ -0,0 +1,227 @@
|
|||||||
|
import type {
|
||||||
|
GateIssue,
|
||||||
|
MediaAsset,
|
||||||
|
MediaEditPlan,
|
||||||
|
MediaRender,
|
||||||
|
MediaRenderStatus,
|
||||||
|
PlatformProfileSummary,
|
||||||
|
} from "@rakazo/contracts";
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Pure view logic for the mobile media screens: which plan and render make a
|
||||||
|
* variant, when to poll, what can be approved or shared, and where a media
|
||||||
|
* push notification leads. Screens stay thin; this stays testable.
|
||||||
|
*/
|
||||||
|
|
||||||
|
/** Clients poll while work runs; there is no realtime media subscription. */
|
||||||
|
export const MEDIA_POLL_MS = 2_000;
|
||||||
|
|
||||||
|
const ACTIVE_RENDER: ReadonlySet<MediaRenderStatus> = new Set(["queued", "rendering", "gating"]);
|
||||||
|
|
||||||
|
export function isRenderActive(render: Pick<MediaRender, "status"> | null | undefined): boolean {
|
||||||
|
return Boolean(render && ACTIVE_RENDER.has(render.status));
|
||||||
|
}
|
||||||
|
|
||||||
|
export type MediaVariant = {
|
||||||
|
profileId: string;
|
||||||
|
profileName: string;
|
||||||
|
plan: MediaEditPlan;
|
||||||
|
/** Newest render of the newest plan version, any quality. */
|
||||||
|
latest: MediaRender | null;
|
||||||
|
/** Newest final render of the newest plan version. */
|
||||||
|
final: MediaRender | null;
|
||||||
|
/** Newest render with a playable video: final when done, else the draft. */
|
||||||
|
preview: MediaRender | null;
|
||||||
|
};
|
||||||
|
|
||||||
|
function newer<T extends { createdAt: string; id: string }>(a: T, b: T): boolean {
|
||||||
|
return a.createdAt === b.createdAt ? a.id > b.id : a.createdAt > b.createdAt;
|
||||||
|
}
|
||||||
|
|
||||||
|
function newest<T extends { createdAt: string; id: string }>(items: T[]): T | null {
|
||||||
|
let best: T | null = null;
|
||||||
|
for (const item of items) if (!best || newer(item, best)) best = item;
|
||||||
|
return best;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function videoOutput(render: Pick<MediaRender, "outputs"> | null) {
|
||||||
|
return render?.outputs.find((output) => output.kind === "video") ?? null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** One variant per profile: its newest plan version and that plan's renders. */
|
||||||
|
export function projectVariants(input: {
|
||||||
|
profileIds: string[];
|
||||||
|
plans: MediaEditPlan[];
|
||||||
|
renders: MediaRender[];
|
||||||
|
profiles: Pick<PlatformProfileSummary, "id" | "name">[];
|
||||||
|
}): MediaVariant[] {
|
||||||
|
const names = new Map(input.profiles.map((profile) => [profile.id, profile.name]));
|
||||||
|
const variants: MediaVariant[] = [];
|
||||||
|
for (const profileId of input.profileIds) {
|
||||||
|
const plans = input.plans.filter((plan) => plan.profileId === profileId);
|
||||||
|
let plan: MediaEditPlan | null = null;
|
||||||
|
for (const candidate of plans) {
|
||||||
|
if (
|
||||||
|
!plan ||
|
||||||
|
candidate.version > plan.version ||
|
||||||
|
(candidate.version === plan.version && newer(candidate, plan))
|
||||||
|
) {
|
||||||
|
plan = candidate;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (!plan) continue;
|
||||||
|
const planId = plan.id;
|
||||||
|
const renders = input.renders.filter((render) => render.planId === planId);
|
||||||
|
const final = newest(renders.filter((render) => render.quality === "final"));
|
||||||
|
const playable = renders.filter((render) => render.status === "done" && videoOutput(render));
|
||||||
|
const finalDone = playable.filter((render) => render.quality === "final");
|
||||||
|
variants.push({
|
||||||
|
profileId,
|
||||||
|
profileName: names.get(profileId) ?? profileId,
|
||||||
|
plan,
|
||||||
|
latest: newest(renders),
|
||||||
|
final,
|
||||||
|
preview: newest(finalDone) ?? newest(playable),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return variants;
|
||||||
|
}
|
||||||
|
|
||||||
|
export type VariantState =
|
||||||
|
| "planned"
|
||||||
|
| "rendering"
|
||||||
|
| "draft_ready"
|
||||||
|
| "approved_rendering"
|
||||||
|
| "ready"
|
||||||
|
| "blocked"
|
||||||
|
| "failed";
|
||||||
|
|
||||||
|
export function variantState(variant: MediaVariant): VariantState {
|
||||||
|
const latest = variant.latest;
|
||||||
|
if (!latest) return "planned";
|
||||||
|
if (isRenderActive(latest))
|
||||||
|
return latest.quality === "final" ? "approved_rendering" : "rendering";
|
||||||
|
if (latest.status === "blocked") return "blocked";
|
||||||
|
if (latest.status === "failed" || latest.status === "canceled") return "failed";
|
||||||
|
return latest.quality === "final" ? "ready" : "draft_ready";
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Approval starts the final render; it needs a plan and no render in flight. */
|
||||||
|
export function canApprove(variant: MediaVariant): boolean {
|
||||||
|
const state = variantState(variant);
|
||||||
|
return state === "draft_ready" || state === "planned" || state === "failed";
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Only a final render that passed its gate leaves the app. */
|
||||||
|
export function canShare(variant: MediaVariant): boolean {
|
||||||
|
return Boolean(variant.final && variant.final.status === "done" && videoOutput(variant.final));
|
||||||
|
}
|
||||||
|
|
||||||
|
export function blockers(render: Pick<MediaRender, "gateReport"> | null): GateIssue[] {
|
||||||
|
return render?.gateReport?.issues.filter((issue) => issue.severity === "block") ?? [];
|
||||||
|
}
|
||||||
|
|
||||||
|
export function shouldPollProject(renders: MediaRender[]): boolean {
|
||||||
|
return renders.some(isRenderActive);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Device items that still have to upload before the server can work on them. */
|
||||||
|
export function needsUpload(asset: Pick<MediaAsset, "storage" | "ingestState">): boolean {
|
||||||
|
return asset.storage === "remote" || asset.ingestState === "awaiting_upload";
|
||||||
|
}
|
||||||
|
|
||||||
|
export type ProjectReadiness = "uploading" | "processing" | "ready" | "failed";
|
||||||
|
|
||||||
|
/** Items this device has to upload: only its own library can send them. */
|
||||||
|
export function uploadsFromDevice<
|
||||||
|
T extends Pick<MediaAsset, "storage" | "ingestState" | "sourceId">,
|
||||||
|
>(assets: T[], deviceSourceId: string | null): T[] {
|
||||||
|
return deviceSourceId
|
||||||
|
? assets.filter((asset) => asset.sourceId === deviceSourceId && needsUpload(asset))
|
||||||
|
: [];
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Whether a project's material is ready for automatic plans: every item
|
||||||
|
* uploaded and analyzed (or at least proxied when analysis is unavailable).
|
||||||
|
* "uploading" means this device still holds originals; items waiting for
|
||||||
|
* another device count as processing.
|
||||||
|
*/
|
||||||
|
export function projectReadiness(
|
||||||
|
assets: Pick<MediaAsset, "storage" | "ingestState" | "sourceId">[],
|
||||||
|
analysisAvailable: boolean,
|
||||||
|
deviceSourceId: string | null,
|
||||||
|
): ProjectReadiness {
|
||||||
|
if (assets.length === 0) return "processing";
|
||||||
|
if (uploadsFromDevice(assets, deviceSourceId).length > 0) return "uploading";
|
||||||
|
if (assets.some(needsUpload)) return "processing";
|
||||||
|
const usable = assets.filter((asset) => asset.ingestState !== "failed");
|
||||||
|
if (usable.length === 0) return "failed";
|
||||||
|
const done = analysisAvailable ? ["analyzed"] : ["proxied", "analyzed"];
|
||||||
|
return usable.every((asset) => done.includes(asset.ingestState)) ? "ready" : "processing";
|
||||||
|
}
|
||||||
|
|
||||||
|
/** File name for a render shared from the device. */
|
||||||
|
export function shareFileName(renderId: string, fileName: string): string {
|
||||||
|
const extension = /\.[A-Za-z0-9]{1,5}$/.exec(fileName)?.[0] ?? ".mp4";
|
||||||
|
return `render-${renderId.replace(/[^A-Za-z0-9_-]/g, "").slice(0, 64)}${extension.toLowerCase()}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function renderFileUrl(
|
||||||
|
apiBase: string,
|
||||||
|
renderId: string,
|
||||||
|
fileName: string,
|
||||||
|
download = false,
|
||||||
|
): string {
|
||||||
|
const base = apiBase.replace(/\/+$/, "");
|
||||||
|
const path = `/api/media/renders/${encodeURIComponent(renderId)}/files/${encodeURIComponent(fileName)}`;
|
||||||
|
return `${base}${path}${download ? "?download=1" : ""}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function assetThumbUrl(apiBase: string, assetId: string): string {
|
||||||
|
return `${apiBase.replace(/\/+$/, "")}/api/media/assets/${encodeURIComponent(assetId)}/thumb`;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Push payload the server sends when a render finishes or is blocked. */
|
||||||
|
export type MediaRenderPushData = {
|
||||||
|
kind: "media.render";
|
||||||
|
renderId: string;
|
||||||
|
projectId: string;
|
||||||
|
status: MediaRenderStatus;
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Route a tapped notification to its project, or null for any other push. */
|
||||||
|
export function mediaNotificationRoute(
|
||||||
|
data: unknown,
|
||||||
|
): { pathname: "/media-project"; params: { projectId: string } } | null {
|
||||||
|
if (!data || typeof data !== "object") return null;
|
||||||
|
const record = data as Record<string, unknown>;
|
||||||
|
if (record.kind !== "media.render") return null;
|
||||||
|
const projectId = record.projectId;
|
||||||
|
if (typeof projectId !== "string" || !/^[A-Za-z0-9_-]{1,64}$/.test(projectId)) return null;
|
||||||
|
return { pathname: "/media-project", params: { projectId } };
|
||||||
|
}
|
||||||
|
|
||||||
|
export type LibraryAccess = "undetermined" | "denied" | "blocked" | "limited" | "all";
|
||||||
|
|
||||||
|
/** Collapse a media library permission response into what the screen needs. */
|
||||||
|
export function libraryAccess(permission: {
|
||||||
|
granted: boolean;
|
||||||
|
canAskAgain: boolean;
|
||||||
|
status: string;
|
||||||
|
accessPrivileges?: "all" | "limited" | "none";
|
||||||
|
}): LibraryAccess {
|
||||||
|
if (permission.granted) return permission.accessPrivileges === "limited" ? "limited" : "all";
|
||||||
|
if (permission.status === "undetermined") return "undetermined";
|
||||||
|
return permission.canAskAgain ? "denied" : "blocked";
|
||||||
|
}
|
||||||
|
|
||||||
|
export function formatDuration(ms: number): string {
|
||||||
|
const total = Math.max(0, Math.round(ms / 1000));
|
||||||
|
const hours = Math.floor(total / 3600);
|
||||||
|
const minutes = Math.floor((total % 3600) / 60);
|
||||||
|
const seconds = String(total % 60).padStart(2, "0");
|
||||||
|
return hours > 0
|
||||||
|
? `${hours}:${String(minutes).padStart(2, "0")}:${seconds}`
|
||||||
|
: `${minutes}:${seconds}`;
|
||||||
|
}
|
||||||
44
apps/mobile/lib/media-sha256.test.ts
Normal file
44
apps/mobile/lib/media-sha256.test.ts
Normal file
@ -0,0 +1,44 @@
|
|||||||
|
import { createHash } from "node:crypto";
|
||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
import { Sha256 } from "./media-sha256";
|
||||||
|
|
||||||
|
const node = (data: Uint8Array) => createHash("sha256").update(data).digest("hex");
|
||||||
|
const text = (value: string) => new TextEncoder().encode(value);
|
||||||
|
|
||||||
|
describe("Sha256", () => {
|
||||||
|
it("matches known vectors", () => {
|
||||||
|
expect(new Sha256().hex()).toBe(
|
||||||
|
"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
|
||||||
|
);
|
||||||
|
expect(new Sha256().update(text("abc")).hex()).toBe(
|
||||||
|
"ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("matches node:crypto across block boundaries", () => {
|
||||||
|
for (const size of [1, 55, 56, 63, 64, 65, 119, 120, 128, 1000, 4097]) {
|
||||||
|
const data = new Uint8Array(size);
|
||||||
|
for (let index = 0; index < size; index++) data[index] = (index * 13 + size) & 0xff;
|
||||||
|
expect(new Sha256().update(data).hex()).toBe(node(data));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("gives the same digest however the input is split", () => {
|
||||||
|
const data = new Uint8Array(5000);
|
||||||
|
for (let index = 0; index < data.length; index++) data[index] = (index * 7) & 0xff;
|
||||||
|
for (const step of [1, 3, 63, 64, 100, 4096]) {
|
||||||
|
const hash = new Sha256();
|
||||||
|
for (let offset = 0; offset < data.length; offset += step) {
|
||||||
|
hash.update(data.subarray(offset, offset + step));
|
||||||
|
}
|
||||||
|
expect(hash.hex()).toBe(node(data));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("refuses updates after finishing", () => {
|
||||||
|
const hash = new Sha256();
|
||||||
|
hash.hex();
|
||||||
|
expect(() => hash.update(text("x"))).toThrow();
|
||||||
|
expect(() => hash.hex()).toThrow();
|
||||||
|
});
|
||||||
|
});
|
||||||
123
apps/mobile/lib/media-sha256.ts
Normal file
123
apps/mobile/lib/media-sha256.ts
Normal file
@ -0,0 +1,123 @@
|
|||||||
|
/*
|
||||||
|
* Incremental SHA-256 for media uploads. `uploads.begin` needs the hash of the
|
||||||
|
* whole original before the first chunk, and originals can be gigabytes, so the
|
||||||
|
* file is hashed chunk by chunk instead of being read into memory.
|
||||||
|
*/
|
||||||
|
|
||||||
|
const K = new Uint32Array([
|
||||||
|
0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5, 0x3956c25b, 0x59f111f1, 0x923f82a4, 0xab1c5ed5,
|
||||||
|
0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3, 0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174,
|
||||||
|
0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc, 0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da,
|
||||||
|
0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7, 0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967,
|
||||||
|
0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13, 0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85,
|
||||||
|
0xa2bfe8a1, 0xa81a664b, 0xc24b8b70, 0xc76c51a3, 0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070,
|
||||||
|
0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5, 0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3,
|
||||||
|
0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208, 0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2,
|
||||||
|
]);
|
||||||
|
|
||||||
|
export class Sha256 {
|
||||||
|
private readonly state = new Uint32Array([
|
||||||
|
0x6a09e667, 0xbb67ae85, 0x3c6ef372, 0xa54ff53a, 0x510e527f, 0x9b05688c, 0x1f83d9ab, 0x5be0cd19,
|
||||||
|
]);
|
||||||
|
private readonly block = new Uint8Array(64);
|
||||||
|
private readonly words = new Uint32Array(64);
|
||||||
|
private blockLength = 0;
|
||||||
|
private totalBytes = 0;
|
||||||
|
private finished = false;
|
||||||
|
|
||||||
|
update(data: Uint8Array): this {
|
||||||
|
if (this.finished) throw new Error("Hash already finished");
|
||||||
|
let offset = 0;
|
||||||
|
this.totalBytes += data.byteLength;
|
||||||
|
if (this.blockLength > 0) {
|
||||||
|
const take = Math.min(64 - this.blockLength, data.byteLength);
|
||||||
|
this.block.set(data.subarray(0, take), this.blockLength);
|
||||||
|
this.blockLength += take;
|
||||||
|
offset = take;
|
||||||
|
if (this.blockLength < 64) return this;
|
||||||
|
this.compress(this.block, 0);
|
||||||
|
this.blockLength = 0;
|
||||||
|
}
|
||||||
|
while (offset + 64 <= data.byteLength) {
|
||||||
|
this.compress(data, offset);
|
||||||
|
offset += 64;
|
||||||
|
}
|
||||||
|
if (offset < data.byteLength) {
|
||||||
|
this.block.set(data.subarray(offset), 0);
|
||||||
|
this.blockLength = data.byteLength - offset;
|
||||||
|
}
|
||||||
|
return this;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Lowercase hex digest, the form `uploads.begin` expects. */
|
||||||
|
hex(): string {
|
||||||
|
if (this.finished) throw new Error("Hash already finished");
|
||||||
|
const bitsHigh = Math.floor(this.totalBytes / 0x20000000);
|
||||||
|
const bitsLow = (this.totalBytes * 8) >>> 0;
|
||||||
|
const padLength = this.blockLength < 56 ? 56 - this.blockLength : 120 - this.blockLength;
|
||||||
|
const tail = new Uint8Array(padLength + 8);
|
||||||
|
tail[0] = 0x80;
|
||||||
|
const view = new DataView(tail.buffer);
|
||||||
|
view.setUint32(padLength, bitsHigh);
|
||||||
|
view.setUint32(padLength + 4, bitsLow);
|
||||||
|
const total = this.totalBytes;
|
||||||
|
this.update(tail);
|
||||||
|
this.totalBytes = total;
|
||||||
|
this.finished = true;
|
||||||
|
let out = "";
|
||||||
|
for (const word of this.state) out += word.toString(16).padStart(8, "0");
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
private compress(data: Uint8Array, offset: number): void {
|
||||||
|
const w = this.words;
|
||||||
|
for (let i = 0; i < 16; i++) {
|
||||||
|
const j = offset + i * 4;
|
||||||
|
w[i] =
|
||||||
|
((data[j] ?? 0) << 24) |
|
||||||
|
((data[j + 1] ?? 0) << 16) |
|
||||||
|
((data[j + 2] ?? 0) << 8) |
|
||||||
|
(data[j + 3] ?? 0);
|
||||||
|
}
|
||||||
|
for (let i = 16; i < 64; i++) {
|
||||||
|
const a = w[i - 15] ?? 0;
|
||||||
|
const b = w[i - 2] ?? 0;
|
||||||
|
const s0 = ((a >>> 7) | (a << 25)) ^ ((a >>> 18) | (a << 14)) ^ (a >>> 3);
|
||||||
|
const s1 = ((b >>> 17) | (b << 15)) ^ ((b >>> 19) | (b << 13)) ^ (b >>> 10);
|
||||||
|
w[i] = ((w[i - 16] ?? 0) + s0 + (w[i - 7] ?? 0) + s1) | 0;
|
||||||
|
}
|
||||||
|
const s = this.state;
|
||||||
|
let a = s[0] ?? 0;
|
||||||
|
let b = s[1] ?? 0;
|
||||||
|
let c = s[2] ?? 0;
|
||||||
|
let d = s[3] ?? 0;
|
||||||
|
let e = s[4] ?? 0;
|
||||||
|
let f = s[5] ?? 0;
|
||||||
|
let g = s[6] ?? 0;
|
||||||
|
let h = s[7] ?? 0;
|
||||||
|
for (let i = 0; i < 64; i++) {
|
||||||
|
const s1 = ((e >>> 6) | (e << 26)) ^ ((e >>> 11) | (e << 21)) ^ ((e >>> 25) | (e << 7));
|
||||||
|
const ch = (e & f) ^ (~e & g);
|
||||||
|
const t1 = (h + s1 + ch + (K[i] ?? 0) + (w[i] ?? 0)) | 0;
|
||||||
|
const s0 = ((a >>> 2) | (a << 30)) ^ ((a >>> 13) | (a << 19)) ^ ((a >>> 22) | (a << 10));
|
||||||
|
const maj = (a & b) ^ (a & c) ^ (b & c);
|
||||||
|
const t2 = (s0 + maj) | 0;
|
||||||
|
h = g;
|
||||||
|
g = f;
|
||||||
|
f = e;
|
||||||
|
e = (d + t1) | 0;
|
||||||
|
d = c;
|
||||||
|
c = b;
|
||||||
|
b = a;
|
||||||
|
a = (t1 + t2) | 0;
|
||||||
|
}
|
||||||
|
s[0] = (s[0] ?? 0) + a;
|
||||||
|
s[1] = (s[1] ?? 0) + b;
|
||||||
|
s[2] = (s[2] ?? 0) + c;
|
||||||
|
s[3] = (s[3] ?? 0) + d;
|
||||||
|
s[4] = (s[4] ?? 0) + e;
|
||||||
|
s[5] = (s[5] ?? 0) + f;
|
||||||
|
s[6] = (s[6] ?? 0) + g;
|
||||||
|
s[7] = (s[7] ?? 0) + h;
|
||||||
|
}
|
||||||
|
}
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
x
Reference in New Issue
Block a user