cynarAI/evOo-local
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

ops: build the web app in the Evo deploy and add a preview-capable stack unit (#110)

Browse Source 
* ops: build the web app in the Evo deploy and add a preview-capable stack unit

The Evo deploy restarted the dev stack without ever building the web app, so a
broken production build went unnoticed and the server could not serve the
build at all. Service workers and web push also need the production web app
behind HTTPS, which the deploy notes did not cover.

deploy.sh now builds the web app after install into a staging directory and
swaps it into apps/web/dist right before the restart, so a failing build
stops the deploy before migrations and the restart and never touches the
build a running vite preview serves. Any failure after the checkout rolls
back the code, reinstalls, regenerates the client, rebuilds the web app when
this deploy had already replaced it, restarts and health-checks; the outcome
is recorded as rolled-back or rollback-failed. The health check now fails
when the API check fails even if the web check passes. Build output goes to a
log that is printed only on failure.

start-stack.sh runs api, worker and sandbox supervisor as pnpm dev does and
serves the web app either from the Vite dev server (default) or from the
build with vite preview on the same port (EVO_WEB_MODE=preview). Preview mode
first checks that a build exists and that RAKAZO_HOST and SCREEN_PROXY_SECRET
resolve the way vite.config.ts resolves them, using the same secret guard.
evoo-local.service is a user unit template for it; the README documents
installing it, switching modes, the catalog caveat for production builds,
and HTTPS through Caddy with EXTRA_TRUSTED_ORIGINS.

A testkit test runs the real scripts against fixture checkouts with fake
pnpm, curl and restart hooks. It replaces the unwired shell regression test,
which exercised a copy of the SHA capture rather than deploy.sh itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u

* ops: show the build error instead of stack frames when a deploy build fails

A failing web build printed the last 40 lines of its log, which for a syntax
error are mostly bundler stack frames below the message that names the file
and line. Drop the frames before printing the tail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u

* test: look up the home main conversation in the scheduling journey

Since every project has its own main conversation, the journey that
schedules for a specialist found the user's main conversation with
findFirst over all of them. With the Lab project's conversation also
matching, it picked either one, so the check failed about one run in
three. The lookup now names the home space, where the specialist joined
and the routine is created.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MRtGG9ARfqqhma3t85Ro2u

---------

Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
cynarAI 2026-10-04 06:35:38 +02:00 committed by GitHub
parent c4c5ce3c2d
commit 735322b016
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
6 changed files with 626 additions and 86 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

128
ops/evo-deploy/README.md
View File

@ -1,16 +1,16 @@
# Evo auto-deploy (evOo-local) # Evo auto-deploy (evOo-local)
Idempotent deploy helper for the **source checkout** on the Evo host (`~/Projects/evOo-local`, user `cynar`). Pulls `main` (or a pinned SHA), runs migrations, restarts the existing stack, health-checks API `:3100` and web `:5173`, and rolls back on failure. Idempotent deploy helper for the **source checkout** on the Evo host, run as `<deploy-user>`. Pulls `main` (or a pinned SHA), installs dependencies, builds the web app, runs migrations, restarts the stack, health-checks API `:3100` and web `:5173`, and rolls back on failure.
This is separate from production Docker deploys under `/srv/rakazo` (see `docs/self-host.md` and `.github/workflows/ci.yml` `deploy-production`). This is separate from production Docker deploys (see `docs/self-host.md` and `.github/workflows/ci.yml` `deploy-production`).
## Prerequisites on Evo ## Prerequisites on Evo
1. Git checkout at `~/Projects/evOo-local` (or set `EVO_DEPLOY_DIR`). 1. Git checkout at `$HOME/Projects/evOo-local` (or set `EVO_DEPLOY_DIR`).
2. `.env` with a working `DATABASE_URL` (Postgres reachable for `pnpm db:migrate`). 2. `.env` with a working `DATABASE_URL` (Postgres reachable for `pnpm db:migrate`).
3. A restart path the script can use (pick one): 3. A restart path the script can use (pick one):
- **systemd (recommended for unattended deploy):** user unit, e.g. `evoo-local.service`, that starts `pnpm dev` or your stack wrapper. Set repo variable `EVO_SYSTEMD_UNIT`. - **systemd (recommended for unattended deploy):** the user unit from [Stack unit](#stack-unit). Set repo variable `EVO_SYSTEMD_UNIT`.
- **tmux:** session `evoo` running `pnpm dev` (matches `scripts/start-tmux.sh` on Mac; on Evo set `EVO_TMUX_SESSION` if different). - **tmux:** session `evoo` running `pnpm dev` (set `EVO_TMUX_SESSION` if different).
- **compose:** `infra/compose/docker-compose.yml` with `api` / `web` / `worker` (dev stack). - **compose:** `infra/compose/docker-compose.yml` with `api` / `web` / `worker` (dev stack).
- **script:** executable hook at `EVO_RESTART_SCRIPT`. - **script:** executable hook at `EVO_RESTART_SCRIPT`.
4. Self-hosted GitHub Actions runner on Evo with labels `[self-hosted, evoo-ci, Linux]` (already used by CI). 4. Self-hosted GitHub Actions runner on Evo with labels `[self-hosted, evoo-ci, Linux]` (already used by CI).
@ -20,23 +20,88 @@ No SSH keys or deploy secrets are stored in this repo. The workflow runs **on**
## Enable the workflow ## Enable the workflow
1. Merge this PR (or copy `ops/evo-deploy/` and `.github/workflows/evo-deploy.yml` to `main`). 1. Merge this PR (or copy `ops/evo-deploy/` and `.github/workflows/evo-deploy.yml` to `main`).
2. In GitHub → **Settings → Secrets and variables → Actions → Variables** for `cynarAI/evOo-local`: 2. In GitHub → **Settings → Secrets and variables → Actions → Variables** for this repository:
- `EVO_DEPLOY_ENABLED` = `true` - `EVO_DEPLOY_ENABLED` = `true`
- Optional: `EVO_DEPLOY_DIR` (default `~/Projects/evOo-local` on the runner user) - Optional: `EVO_DEPLOY_DIR` (default `$HOME/Projects/evOo-local` of the runner user)
- Optional: `EVO_RESTART_MODE` (`auto`, `systemd`, `tmux`, `compose`, `script`, `none`) - Optional: `EVO_RESTART_MODE` (`auto`, `systemd`, `tmux`, `compose`, `script`, `none`)
- Optional: `EVO_SYSTEMD_UNIT`, `EVO_TMUX_SESSION`, health URLs, timeouts - Optional: `EVO_SYSTEMD_UNIT`, `EVO_TMUX_SESSION`, health URLs, timeouts
3. Confirm the runner user can read/write `EVO_DEPLOY_DIR` and restart services (`systemctl --user` or tmux). 3. Confirm the runner user can read/write `EVO_DEPLOY_DIR` and restart services (`systemctl --user` or tmux).
Until `EVO_DEPLOY_ENABLED` is `true`, pushes to `main` do not deploy (workflow job is skipped). Until `EVO_DEPLOY_ENABLED` is `true`, pushes to `main` do not deploy (workflow job is skipped).
## Stack unit
`start-stack.sh` starts api, worker and sandbox supervisor exactly as `pnpm dev` does. The web app runs in one of two modes, chosen by `EVO_WEB_MODE`:
- `dev` (default): the Vite dev server, same as `pnpm dev`.
- `preview`: `vite preview` serves the production build from `apps/web/dist` on the same port. `deploy.sh` rebuilds it on every deploy.
`evoo-local.service` is a systemd user unit template that runs the script. Install it once, as `<deploy-user>` in the checkout:
```bash
systemctl --user cat evoo-local.service # note what the current unit runs; keep a copy if needed
install -D -m 0644 ops/evo-deploy/evoo-local.service ~/.config/systemd/user/evoo-local.service
# Edit WorkingDirectory in the installed copy if the checkout is not at ~/Projects/evOo-local.
systemctl --user daemon-reload
systemctl --user enable evoo-local.service
systemctl --user restart evoo-local.service
loginctl enable-linger "$USER" # keep user services running without a login session
```
The unit starts the script through a login shell, so node and pnpm come from the same `PATH` that `deploy.sh` uses. It stays in dev mode, so nothing visible changes. Keep the repo variables `EVO_RESTART_MODE=systemd` and `EVO_SYSTEMD_UNIT=evoo-local.service`. Logs: `journalctl --user -u evoo-local.service -f`.
### Switch to preview
The production build drops message source text, so a message missing from the Lingui catalogs shows as a hash id instead of English. Switch only once the catalogs are complete (`pnpm --filter @rakazo/web intl:extract` adds no new entries).
1. In the root `.env`, set `RAKAZO_HOST` to the hostname browsers use (see [HTTPS with Caddy](#https-with-caddy)) and `SCREEN_PROXY_SECRET` to a dedicated random value of at least 32 characters (`openssl rand -hex 32`), different from `BETTER_AUTH_SECRET` and `SANDBOX_SUPERVISOR_TOKEN`. `vite preview` runs with `NODE_ENV=production` and refuses development placeholders; the API reads the same secret.
2. Check the configuration without restarting anything:
```bash
EVO_WEB_MODE=preview bash ops/evo-deploy/start-stack.sh --check
```
3. Switch the unit and restart:
```bash
systemctl --user edit evoo-local.service
# add:
# [Service]
# Environment=EVO_WEB_MODE=preview
systemctl --user restart evoo-local.service
```
4. Verify: `curl -fsS -o /dev/null -w '%{http_code}\n' http://127.0.0.1:5173/`.
To switch back, remove the override (`systemctl --user revert evoo-local.service`) and restart.
## HTTPS with Caddy
Service workers and web push need a secure context, so phones and other devices need HTTPS; `http://<host-ip>:5173` is not one. Run Caddy on the host and proxy everything to the web port. Both the dev server and `vite preview` forward `/api`, `/rpc` and the screen proxy to the API, so `:3100` stays private:
```Caddyfile
app.example.com {
reverse_proxy 127.0.0.1:5173
}
```
This is the host proxy from `docs/self-host.md`; `infra/compose/Caddyfile.prod` routes the same paths to Compose containers and is not used here. Caddy obtains the certificate itself when the hostname resolves to the host and ports 80 and 443 are reachable; otherwise configure a DNS challenge.
Root `.env`, then restart the unit:
```dotenv
RAKAZO_HOST=app.example.com
EXTRA_TRUSTED_ORIGINS=https://app.example.com
```
- `RAKAZO_HOST` is the hostname `vite preview` accepts. Caddy forwards the original `Host` header; the dev server accepts any host.
- `EXTRA_TRUSTED_ORIGINS` (comma-separated) lets the HTTPS origin sign in. Keep `BETTER_AUTH_URL` on the plain-http origin and set it explicitly, because it falls back to `WEB_ORIGIN`: an `https://` value makes Better Auth use `__Secure-` cookies, which browsers drop over plain http, so signing in via `http://…:5173` would stop working.
- `WEB_ORIGIN` is the origin of links the API builds, such as the agent computer screen. Set it to the HTTPS origin if that is the address you use; an `http://` screen link is blocked as mixed content on the HTTPS page.
## Manual / dry-run ## Manual / dry-run
On Evo (or any machine with the checkout and stack): On Evo (or any machine with the checkout and stack):
```bash ```bash
cd ~/Projects/evOo-local cd <checkout>
# Show what would happen (no git reset, migrate, restart, or health wait) # Show what would happen (no git reset, install, build, migrate, restart, or health wait)
bash ops/evo-deploy/deploy.sh --dry-run bash ops/evo-deploy/deploy.sh --dry-run
# Deploy main tip # Deploy main tip
@ -51,30 +116,28 @@ EVO_RESTART_MODE=systemd EVO_SYSTEMD_UNIT=evoo-local.service bash ops/evo-deploy
GitHub: **Actions → evo-deploy → Run workflow** (optional `ref`, `dry_run`, `skip_migrate`). GitHub: **Actions → evo-deploy → Run workflow** (optional `ref`, `dry_run`, `skip_migrate`).
## Rollback behavior ## Deploy and rollback behavior
### Regression: SHA capture vs. git stdout 1. Before `git fetch`, the script records `previousSha` (current `HEAD`), then resets the checkout to the target.
2. `pnpm install --frozen-lockfile`, then the web build into `.evo-deploy/web-dist`. Its output goes to `.evo-deploy/web-build.log` and is printed only when the build fails.
`deploy.sh` captures `previousSha` and `targetSha` via process substitution (`read … < <(git_fetch_and_checkout)`). Git commands such as `reset --hard` print progress to **stdout** (`HEAD is now at …`), which previously polluted that capture and produced invalid `.evo-deploy/state.json` (e.g. `deployedSha: "is now at 84bc45e …"`, `previousSha: "HEAD"`). Rollback then became a no-op. 3. `pnpm db:generate && pnpm db:migrate`.
4. The new build replaces `apps/web/dist`, the stack restarts, and the script waits up to `EVO_HEALTH_TIMEOUT_SEC` (default 120s) for:
**Fix:** all git invocations inside `git_fetch_and_checkout` (and rollback) redirect stdout to stderr; only the final `printf '%s %s\n'` emits SHAs on stdout. `write_state` and rollback reject values that are not 7–40 hex characters.
1. Before `git fetch`, the script records `previousSha` (current `HEAD`).
2. After pull, migrate, and restart, it waits up to `EVO_HEALTH_TIMEOUT_SEC` (default 120s) for:
- `EVO_API_HEALTH_URL` (default `http://127.0.0.1:3100/health`) - `EVO_API_HEALTH_URL` (default `http://127.0.0.1:3100/health`)
- `EVO_WEB_HEALTH_URL` (default `http://127.0.0.1:5173/`) - `EVO_WEB_HEALTH_URL` (default `http://127.0.0.1:5173/`)
3. On failure: `git reset --hard` to `previousSha`, restart services again, exit non-zero. 5. Any failure rolls back: `git reset --hard` to `previousSha`, reinstall, regenerate the client, rebuild the web app if this deploy had already replaced it, restart, health-check, exit non-zero. A failing install, build or migration therefore never restarts the stack on the new commit, and `apps/web/dist` keeps the build that was being served. Migrations only move forward and stay applied.
4. Last outcome is written to `.evo-deploy/state.json` under the checkout. 6. The outcome (`success`, `rolled-back` or `rollback-failed`) is written to `.evo-deploy/state.json` under the checkout.
`deploy.sh` captures `previousSha` and `targetSha` via process substitution, so every git command inside `git_fetch_and_checkout` and rollback redirects stdout to stderr (`reset --hard` prints `HEAD is now at …`). `write_state` and rollback reject values that are not 7–40 hex characters. `packages/testkit/src/evo-deploy.test.ts` runs the script against a fixture checkout and covers this, the build gate and the rollback.
Manual rollback to a known good SHA: Manual rollback to a known good SHA:
```bash ```bash
cd ~/Projects/evOo-local cd <checkout>
git reset --hard <good-sha> EVO_RESTART_MODE=systemd EVO_SYSTEMD_UNIT=evoo-local.service EVO_DEPLOY_REF=<good-sha> \
EVO_RESTART_MODE=systemd EVO_SYSTEMD_UNIT=evoo-local.service bash ops/evo-deploy/deploy.sh --skip-migrate --ref HEAD bash ops/evo-deploy/deploy.sh --skip-migrate
``` ```
(or restart your unit/tmux session by hand, then verify health URLs). (or `git reset --hard <good-sha>`, `pnpm install --frozen-lockfile`, `pnpm --filter @rakazo/web build`, restart the unit, then verify the health URLs).
## Autopilot / Grok Bot after merge ## Autopilot / Grok Bot after merge
@ -83,7 +146,7 @@ After a PR merges to `main`:
1. **Preferred:** rely on the `evo-deploy` workflow (once `EVO_DEPLOY_ENABLED=true`). No SSH step required. 1. **Preferred:** rely on the `evo-deploy` workflow (once `EVO_DEPLOY_ENABLED=true`). No SSH step required.
2. **Fallback (manual):** SSH to Evo and run: 2. **Fallback (manual):** SSH to Evo and run:
```bash ```bash
cd ~/Projects/evOo-local && bash ops/evo-deploy/deploy.sh cd <checkout> && bash ops/evo-deploy/deploy.sh
``` ```
3. **Verify:** API `curl -fsS http://127.0.0.1:3100/health`, web `curl -fsS -o /dev/null -w '%{http_code}\n' http://127.0.0.1:5173/`. 3. **Verify:** API `curl -fsS http://127.0.0.1:3100/health`, web `curl -fsS -o /dev/null -w '%{http_code}\n' http://127.0.0.1:5173/`.
@ -91,6 +154,8 @@ Do not commit host passwords, tokens, or SSH private keys. Configure restart tar
## Environment reference ## Environment reference
`deploy.sh`:
| Variable | Default | Purpose | | Variable | Default | Purpose |
|----------|---------|---------| |----------|---------|---------|
| `EVO_DEPLOY_DIR` | `$HOME/Projects/evOo-local` | Git checkout root | | `EVO_DEPLOY_DIR` | `$HOME/Projects/evOo-local` | Git checkout root |
@ -100,10 +165,17 @@ Do not commit host passwords, tokens, or SSH private keys. Configure restart tar
| `EVO_SYSTEMD_SCOPE` | `user` | `user` or `system` | | `EVO_SYSTEMD_SCOPE` | `user` | `user` or `system` |
| `EVO_TMUX_SESSION` | `evoo` | tmux session name | | `EVO_TMUX_SESSION` | `evoo` | tmux session name |
| `EVO_TMUX_COMMAND` | `pnpm dev` | Command started in new tmux session | | `EVO_TMUX_COMMAND` | `pnpm dev` | Command started in new tmux session |
| `EVO_MIGRATE` | `true` | Run `pnpm db:generate` + `pnpm db:migrate` | | `EVO_MIGRATE` | `true` | Run `pnpm db:generate` + `pnpm db:migrate` (`--skip-migrate`) |
| `EVO_INSTALL` | `true` | Run `pnpm install --frozen-lockfile` after checkout | | `EVO_INSTALL` | `true` | Run `pnpm install --frozen-lockfile` after checkout (`--skip-install`) |
| `EVO_BUILD` | `true` | Build the web app before the restart (`--skip-build`) |
| `EVO_API_HEALTH_URL` | `http://127.0.0.1:3100/health` | API health probe | | `EVO_API_HEALTH_URL` | `http://127.0.0.1:3100/health` | API health probe |
| `EVO_WEB_HEALTH_URL` | `http://127.0.0.1:5173/` | Web health probe | | `EVO_WEB_HEALTH_URL` | `http://127.0.0.1:5173/` | Web health probe |
| `EVO_HEALTH_TIMEOUT_SEC` | `120` | Per-check timeout (seconds) | | `EVO_HEALTH_TIMEOUT_SEC` | `120` | Per-check timeout (seconds) |
If `EVO_RESTART_MODE=auto` cannot detect systemd, tmux, compose, or a restart script, deploy fails with a clear error — set `EVO_RESTART_MODE` and the matching unit/session explicitly. `start-stack.sh`:
| Variable | Default | Purpose |
|----------|---------|---------|
| `EVO_WEB_MODE` | `dev` | `dev` (Vite dev server) or `preview` (production build) |
If `EVO_RESTART_MODE=auto` cannot detect systemd, tmux, compose, or a restart script, it falls back to `none` and only health-checks — set `EVO_RESTART_MODE` and the matching unit/session explicitly.

40
ops/evo-deploy/deploy-sha-capture.test.sh
View File

@ -1,40 +0,0 @@
#!/usr/bin/env bash
# Regression: git_fetch_and_checkout must emit only two SHAs on stdout.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
TMP="$(mktemp -d)"
trap 'rm -rf "$TMP"' EXIT
git init -q "$TMP/repo"
git -C "$TMP/repo" config user.email "test@example.com"
git -C "$TMP/repo" config user.name "Test"
echo first >"$TMP/repo/file.txt"
git -C "$TMP/repo" add file.txt
git -C "$TMP/repo" commit -q -m "first"
FIRST_SHA="$(git -C "$TMP/repo" rev-parse HEAD)"
echo second >"$TMP/repo/file.txt"
git -C "$TMP/repo" commit -q -am "second"
SECOND_SHA="$(git -C "$TMP/repo" rev-parse HEAD)"
# Minimal reimplementation of the capture contract under test.
git_fetch_and_checkout() {
local previous_sha target_sha deploy_dir="$1" ref="$2"
previous_sha="$(git -C "$deploy_dir" rev-parse HEAD)"
git -C "$deploy_dir" fetch origin --prune >&2 2>/dev/null || true
target_sha="$(git -C "$deploy_dir" rev-parse "${ref}^{commit}")"
git -C "$deploy_dir" reset --hard "$target_sha" >&2
printf '%s %s\n' "$previous_sha" "$target_sha"
}
git -C "$TMP/repo" remote add origin "$TMP/repo"
git -C "$TMP/repo" checkout -q "$FIRST_SHA"
capture="$(git_fetch_and_checkout "$TMP/repo" "$SECOND_SHA")"
read -r got_previous got_target <<<"$capture"
[[ "$got_previous" == "$FIRST_SHA" ]] || { echo "expected previous $FIRST_SHA, got $got_previous"; exit 1; }
[[ "$got_target" == "$SECOND_SHA" ]] || { echo "expected target $SECOND_SHA, got $got_target"; exit 1; }
[[ "$capture" != *"HEAD is now at"* ]] || { echo "git reset output leaked into capture: $capture"; exit 1; }
echo "ok: deploy-sha-capture regression passed"

94
ops/evo-deploy/deploy.sh
View File

@ -1,10 +1,9 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# evOo-local — idempotent deploy helper for the Evo host checkout. # evOo-local — idempotent deploy helper for the Evo host checkout.
# Pull (or pin) a revision, migrate, restart services, health-check, rollback on failure. # Pull (or pin) a revision, install, build the web app, migrate, restart services, health-check,
# roll back on failure.
set -euo pipefail set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# --- defaults (override via env; see ops/evo-deploy/README.md) --- # --- defaults (override via env; see ops/evo-deploy/README.md) ---
EVO_DEPLOY_DIR="${EVO_DEPLOY_DIR:-${HOME}/Projects/evOo-local}" EVO_DEPLOY_DIR="${EVO_DEPLOY_DIR:-${HOME}/Projects/evOo-local}"
EVO_DEPLOY_REF="${EVO_DEPLOY_REF:-main}" EVO_DEPLOY_REF="${EVO_DEPLOY_REF:-main}"
@ -23,6 +22,7 @@ EVO_WEB_HEALTH_URL="${EVO_WEB_HEALTH_URL:-http://127.0.0.1:5173/}"
EVO_HEALTH_TIMEOUT_SEC="${EVO_HEALTH_TIMEOUT_SEC:-120}" EVO_HEALTH_TIMEOUT_SEC="${EVO_HEALTH_TIMEOUT_SEC:-120}"
EVO_MIGRATE="${EVO_MIGRATE:-true}" EVO_MIGRATE="${EVO_MIGRATE:-true}"
EVO_INSTALL="${EVO_INSTALL:-true}" EVO_INSTALL="${EVO_INSTALL:-true}"
EVO_BUILD="${EVO_BUILD:-true}"
EVO_DRY_RUN="${EVO_DRY_RUN:-false}" EVO_DRY_RUN="${EVO_DRY_RUN:-false}"
EVO_STATE_FILE="${EVO_STATE_FILE:-${EVO_DEPLOY_DIR}/.evo-deploy/state.json}" EVO_STATE_FILE="${EVO_STATE_FILE:-${EVO_DEPLOY_DIR}/.evo-deploy/state.json}"
@ -37,7 +37,7 @@ is_valid_git_sha() {
usage() { usage() {
cat <<'EOF' cat <<'EOF'
Usage: deploy.sh [--dry-run] [--ref REF] [--skip-migrate] [--skip-install] Usage: deploy.sh [--dry-run] [--ref REF] [--skip-migrate] [--skip-install] [--skip-build]
Environment: EVO_DEPLOY_DIR, EVO_DEPLOY_REF, EVO_RESTART_MODE, EVO_SYSTEMD_UNIT, Environment: EVO_DEPLOY_DIR, EVO_DEPLOY_REF, EVO_RESTART_MODE, EVO_SYSTEMD_UNIT,
EVO_TMUX_SESSION, EVO_API_HEALTH_URL, EVO_WEB_HEALTH_URL, EVO_HEALTH_TIMEOUT_SEC, … EVO_TMUX_SESSION, EVO_API_HEALTH_URL, EVO_WEB_HEALTH_URL, EVO_HEALTH_TIMEOUT_SEC, …
@ -66,6 +66,10 @@ parse_args() {
EVO_INSTALL=false EVO_INSTALL=false
shift shift
;; ;;
--skip-build)
EVO_BUILD=false
shift
;;
-h | --help) -h | --help)
usage usage
exit 0 exit 0
@ -212,8 +216,7 @@ wait_for_url() {
} }
health_check() { health_check() {
wait_for_url "$EVO_API_HEALTH_URL" "API" wait_for_url "$EVO_API_HEALTH_URL" "API" && wait_for_url "$EVO_WEB_HEALTH_URL" "web"
wait_for_url "$EVO_WEB_HEALTH_URL" "web"
} }
write_state() { write_state() {
@ -267,19 +270,78 @@ run_migrate() {
run_shell "cd $(printf '%q' "$EVO_DEPLOY_DIR") && pnpm db:generate && pnpm db:migrate" run_shell "cd $(printf '%q' "$EVO_DEPLOY_DIR") && pnpm db:generate && pnpm db:migrate"
} }
# Rollback regenerates the client for the previous schema; applied migrations stay applied.
run_generate() {
[[ "$EVO_MIGRATE" == "true" ]] || return 0
log "running pnpm db:generate"
run_shell "cd $(printf '%q' "$EVO_DEPLOY_DIR") && pnpm db:generate"
}
run_install() { run_install() {
[[ "$EVO_INSTALL" == "true" ]] || { log "skipping install (EVO_INSTALL=false)"; return 0; } [[ "$EVO_INSTALL" == "true" ]] || { log "skipping install (EVO_INSTALL=false)"; return 0; }
log "running pnpm install --frozen-lockfile" log "running pnpm install --frozen-lockfile"
run_shell "cd $(printf '%q' "$EVO_DEPLOY_DIR") && pnpm install --frozen-lockfile" run_shell "cd $(printf '%q' "$EVO_DEPLOY_DIR") && pnpm install --frozen-lockfile"
} }
# The build goes to a staging directory, so a failing build never touches the one a running
# `vite preview` serves; activate_web_build swaps it in right before the restart.
WEB_STAGING_DIR="$EVO_DEPLOY_DIR/.evo-deploy/web-dist"
WEB_BUILD_LOG="$EVO_DEPLOY_DIR/.evo-deploy/web-build.log"
WEB_DIST_DIR="$EVO_DEPLOY_DIR/apps/web/dist"
WEB_BUILD_REPLACED=false
build_web() {
[[ "$EVO_BUILD" == "true" ]] || { log "skipping web build (EVO_BUILD=false)"; return 0; }
log "building the web app"
if [[ "$DRY_RUN" == true ]]; then
log "[dry-run] pnpm --filter @rakazo/web build --outDir $WEB_STAGING_DIR"
return 0
fi
rm -rf "$WEB_STAGING_DIR" && mkdir -p "$(dirname "$WEB_BUILD_LOG")" || return 1
if run_shell "cd $(printf '%q' "$EVO_DEPLOY_DIR") && pnpm --filter @rakazo/web build --outDir $(printf '%q' "$WEB_STAGING_DIR") --emptyOutDir" >"$WEB_BUILD_LOG" 2>&1 &&
[[ -f "$WEB_STAGING_DIR/index.html" ]]; then
return 0
fi
# Stack frames bury the message that names the file and line.
grep -v -E '^[[:space:]]+at ' "$WEB_BUILD_LOG" | tail -n 60 >&2 || true
log "ERROR: web build failed (full log: $WEB_BUILD_LOG)"
return 1
}
activate_web_build() {
[[ "$EVO_BUILD" == "true" ]] || return 0
local outgoing="$WEB_STAGING_DIR.outgoing"
log "activating the web build"
WEB_BUILD_REPLACED=true
run rm -rf "$outgoing" || return 1
if [[ -d "$WEB_DIST_DIR" ]]; then
run mv "$WEB_DIST_DIR" "$outgoing" || return 1
fi
run mv "$WEB_STAGING_DIR" "$WEB_DIST_DIR" || return 1
run rm -rf "$outgoing"
}
# Restores the previous commit with its dependencies, client and web build, restarts and
# health-checks it. Every step is attempted so the service comes back even if one of them fails.
rollback() { rollback() {
local previous_sha="$1" local previous_sha="$1" status="rolled-back"
is_valid_git_sha "$previous_sha" || die "rollback aborted: invalid previous_sha: $previous_sha" is_valid_git_sha "$previous_sha" || die "rollback aborted: invalid previous_sha: $previous_sha"
log "ROLLBACK: resetting to $previous_sha and restarting services" log "ROLLBACK: resetting to $previous_sha"
run git -C "$EVO_DEPLOY_DIR" reset --hard "$previous_sha" >&2 if ! run git -C "$EVO_DEPLOY_DIR" reset --hard "$previous_sha" >&2; then
restart_services write_state "$previous_sha" "$previous_sha" "rollback-failed"
write_state "$previous_sha" "$previous_sha" "rolled-back" die "deploy failed and the checkout could not be reset to $previous_sha"
fi
run_install || status="rollback-failed"
run_generate || status="rollback-failed"
# Until activation the served build is still the previous one; rebuild only once it was replaced.
if [[ "$WEB_BUILD_REPLACED" == true ]] && ! { build_web && activate_web_build; }; then
status="rollback-failed"
fi
if ! { restart_services && health_check; }; then
status="rollback-failed"
fi
write_state "$previous_sha" "$previous_sha" "$status"
[[ "$status" == "rolled-back" ]] || die "deploy failed; rollback to $previous_sha did not complete"
die "deploy failed; rolled back to $previous_sha" die "deploy failed; rolled back to $previous_sha"
} }
@ -300,16 +362,12 @@ main() {
is_valid_git_sha "$target_sha" || die "invalid target_sha captured from checkout: $target_sha" is_valid_git_sha "$target_sha" || die "invalid target_sha captured from checkout: $target_sha"
if [[ "$previous_sha" == "$target_sha" && "$DRY_RUN" != true ]]; then if [[ "$previous_sha" == "$target_sha" && "$DRY_RUN" != true ]]; then
log "already at $target_sha — continuing with migrate/restart/health" log "already at $target_sha — continuing with install/build/migrate/restart/health"
fi fi
run_install # Any failure rolls back; a failing install, build or migration stops before the restart.
run_migrate if ! { run_install && build_web && run_migrate && activate_web_build && restart_services && health_check; }; then
restart_services
if ! health_check; then
rollback "$previous_sha" rollback "$previous_sha"
exit 1
fi fi
write_state "$previous_sha" "$target_sha" "success" write_state "$previous_sha" "$target_sha" "success"

19
ops/evo-deploy/evoo-local.service Normal file
View File

@ -0,0 +1,19 @@
# systemd user unit for the evOo stack in this checkout. Template: install and switch modes as
# described in ops/evo-deploy/README.md.
[Unit]
Description=evOo stack (api, worker, sandbox supervisor, web)
[Service]
Type=simple
# Same default checkout as deploy.sh (EVO_DEPLOY_DIR); change it if the checkout lives elsewhere.
WorkingDirectory=%h/Projects/evOo-local
# dev (Vite dev server) or preview (production build); see the README before switching.
Environment=EVO_WEB_MODE=dev
# A login shell puts node and pnpm on PATH the same way deploy.sh finds them.
ExecStart=/bin/bash -lc 'exec ./ops/evo-deploy/start-stack.sh'
Restart=on-failure
RestartSec=5
TimeoutStopSec=30
[Install]
WantedBy=default.target

79
ops/evo-deploy/start-stack.sh Executable file
View File

@ -0,0 +1,79 @@
#!/usr/bin/env bash
# evOo-local — start the stack from this checkout (used by evoo-local.service).
# api, worker and sandbox supervisor run as `pnpm dev` runs them. The web app runs as the Vite dev
# server (EVO_WEB_MODE=dev, the default) or serves the production build with `vite preview` on the
# same port (EVO_WEB_MODE=preview). `--check` validates the configuration and exits.
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
EVO_WEB_MODE="${EVO_WEB_MODE:-dev}"
log() { printf '[evo-stack] %s\n' "$*" >&2; }
die() { log "ERROR: $*"; exit 1; }
# `vite preview` runs with NODE_ENV=production, so apps/web/vite.config.ts refuses development
# secrets. Resolve the env the way that config does (process env first, then the root .env files)
# and run the same guard, so a missing value fails here instead of inside a restart loop.
check_preview() {
[[ -f "$ROOT/apps/web/dist/index.html" ]] ||
die "apps/web/dist has no build; run: pnpm --filter @rakazo/web build"
(cd "$ROOT/apps/web" && NODE_ENV=production "$ROOT/node_modules/.bin/tsx" --input-type=module -e '
import { loadEnv } from "vite";
import { resolveScreenProxySecret } from "../../packages/core/src/secrets-guard.ts";
const files = loadEnv("production", "../..", "");
const env = { ...process.env };
for (const key of ["RAKAZO_HOST", "SCREEN_PROXY_SECRET", "SANDBOX_SUPERVISOR_TOKEN", "BETTER_AUTH_SECRET"]) {
env[key] = files[key];
}
const problems = [];
if (!env.RAKAZO_HOST?.trim()) problems.push("Set RAKAZO_HOST to the hostname the web app is served on.");
try {
resolveScreenProxySecret(env);
} catch (error) {
problems.push(error.message);
}
for (const problem of problems) console.error("[evo-stack] ERROR: " + problem);
process.exitCode = problems.length > 0 ? 1 : 0;
')
}
start_preview() {
pnpm dev '--filter=!@rakazo/web' &
SERVICES_PID=$!
NODE_ENV=production pnpm --filter @rakazo/web preview --strictPort &
WEB_PID=$!
trap 'kill "$SERVICES_PID" "$WEB_PID" 2>/dev/null || true' EXIT
trap 'exit 143' TERM
trap 'exit 130' INT
# Polling instead of `wait -n` keeps this working on bash 3.2.
while kill -0 "$SERVICES_PID" 2>/dev/null && kill -0 "$WEB_PID" 2>/dev/null; do
sleep 1
done
die "a service exited; stopping the stack"
}
main() {
case "${1:-}" in
"" | --check) ;;
*) die "usage: start-stack.sh [--check]" ;;
esac
case "$EVO_WEB_MODE" in
dev | preview) ;;
*) die "EVO_WEB_MODE must be dev or preview (got: $EVO_WEB_MODE)" ;;
esac
cd "$ROOT"
if [[ "$EVO_WEB_MODE" == preview ]]; then
check_preview
fi
if [[ "${1:-}" == --check ]]; then
log "ok ($EVO_WEB_MODE)"
return 0
fi
if [[ "$EVO_WEB_MODE" == dev ]]; then
exec pnpm dev
fi
start_preview
}
main "$@"

352
packages/testkit/src/evo-deploy.test.ts Normal file
View File

@ -0,0 +1,352 @@
import { execFileSync, spawnSync } from "node:child_process";
import {
chmodSync,
copyFileSync,
existsSync,
mkdirSync,
mkdtempSync,
readFileSync,
realpathSync,
rmSync,
symlinkSync,
writeFileSync,
} from "node:fs";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it } from "vitest";
const repoRoot = path.resolve(import.meta.dirname, "../../..");
const deployScript = path.join(repoRoot, "ops/evo-deploy/deploy.sh");
const temporaryDirectories: string[] = [];
afterEach(() => {
for (const directory of temporaryDirectories.splice(0)) {
rmSync(directory, { recursive: true, force: true });
}
});
function write(file: string, content: string, mode?: number) {
mkdirSync(path.dirname(file), { recursive: true });
writeFileSync(file, content);
if (mode !== undefined) chmodSync(file, mode);
}
function temporaryRoot(prefix: string) {
const root = realpathSync(mkdtempSync(path.join(os.tmpdir(), prefix)));
temporaryDirectories.push(root);
return root;
}
function searchPath(bin: string) {
return [bin, path.dirname(process.execPath), "/usr/bin", "/bin"].join(path.delimiter);
}
function readLines(file: string) {
return existsSync(file) ? readFileSync(file, "utf8").trim().split("\n") : [];
}
interface Revision {
build?: "ok" | "fail";
health?: "ok" | "down";
}
/**
* A checkout at revision A whose served build came from A, with revision B to deploy. Fake pnpm,
* curl and restart hook record what ran against which checked-out revision.
*/
function deployFixture(next: Revision, previous: Revision = {}) {
const root = temporaryRoot("evo-deploy-");
const checkout = path.join(root, "checkout");
const bin = path.join(root, "bin");
const home = path.join(root, "home");
const commandLog = path.join(root, "commands.log");
const env = {
HOME: home,
PATH: searchPath(bin),
COMMAND_LOG: commandLog,
GIT_AUTHOR_NAME: "Test",
GIT_AUTHOR_EMAIL: "test@example.com",
GIT_COMMITTER_NAME: "Test",
GIT_COMMITTER_EMAIL: "test@example.com",
GIT_CONFIG_NOSYSTEM: "1",
};
const git = (...args: string[]) =>
execFileSync("git", args, {
cwd: checkout,
env,
encoding: "utf8",
stdio: ["ignore", "pipe", "pipe"],
}).trim();
// deploy.sh runs pnpm through a login shell, which may reset PATH from the system profile.
write(path.join(home, ".bash_profile"), `export PATH="${bin}:$PATH"\n`);
write(
path.join(bin, "pnpm"),
`#!/bin/bash
printf '%s %s\\n' "$(cat version.txt)" "$*" >> "$COMMAND_LOG"
if [[ "$1 $2 $3 $4" == "--filter @rakazo/web build --outDir" ]]; then
echo "vite building for production"
mkdir -p "$5"
if [[ "$(cat build.txt)" != ok ]]; then echo "simulated build error" >&2; exit 1; fi
echo "built $(cat version.txt)" > "$5/index.html"
fi
`,
0o755,
);
write(
path.join(bin, "curl"),
`#!/bin/bash\n[[ "$(cat "$EVO_DEPLOY_DIR/health.txt")" == ok ]]\n`,
0o755,
);
write(
path.join(bin, "restart"),
`#!/bin/bash
served="$(cat "$EVO_DEPLOY_DIR/apps/web/dist/index.html" 2>/dev/null || echo none)"
printf 'restart %s %s\\n' "$(cat "$EVO_DEPLOY_DIR/version.txt")" "$served" >> "$COMMAND_LOG"
`,
0o755,
);
mkdirSync(checkout, { recursive: true });
git("init", "-q", "-b", "main");
const commit = (version: string, revision: Revision) => {
write(
path.join(checkout, "package.json"),
'{"scripts":{"db:migrate":"prisma migrate deploy"}}',
);
write(path.join(checkout, "version.txt"), version);
write(path.join(checkout, "build.txt"), revision.build ?? "ok");
write(path.join(checkout, "health.txt"), revision.health ?? "ok");
git("add", "-A");
git("commit", "-q", "-m", version);
return git("rev-parse", "HEAD");
};
const a = commit("A", previous);
const b = commit("B", next);
git("reset", "-q", "--hard", a);
git("remote", "add", "origin", checkout);
write(path.join(checkout, "apps/web/dist/index.html"), "built A");
return {
a,
b,
checkout,
run(args: string[] = []) {
const result = spawnSync("/bin/bash", [deployScript, ...args], {
env: {
...env,
EVO_DEPLOY_DIR: checkout,
EVO_DEPLOY_REF: b,
EVO_RESTART_MODE: "script",
EVO_RESTART_SCRIPT: path.join(bin, "restart"),
EVO_HEALTH_TIMEOUT_SEC: "1",
},
encoding: "utf8",
timeout: 30_000,
});
expect(result.error).toBeUndefined();
return result;
},
commands: () => readLines(commandLog),
head: () => git("rev-parse", "HEAD"),
served: () => readFileSync(path.join(checkout, "apps/web/dist/index.html"), "utf8").trim(),
state: () => JSON.parse(readFileSync(path.join(checkout, ".evo-deploy/state.json"), "utf8")),
build: (version: string) =>
`${version} --filter @rakazo/web build --outDir ${path.join(checkout, ".evo-deploy/web-dist")} --emptyOutDir`,
};
}
describe("evo deploy", () => {
it("builds before migrating and restarts on the new build", () => {
const f = deployFixture({});
const result = f.run();
expect(result.status, result.stderr).toBe(0);
expect(f.commands()).toEqual([
"B install --frozen-lockfile",
f.build("B"),
"B db:generate",
"B db:migrate",
"restart B built B",
]);
expect(result.stderr).not.toContain("vite building for production");
expect(f.served()).toBe("built B");
expect(existsSync(path.join(f.checkout, ".evo-deploy/web-dist"))).toBe(false);
// Regression: git output on stdout once polluted the captured SHAs.
expect(f.state()).toMatchObject({ previousSha: f.a, deployedSha: f.b, status: "success" });
});
it("stops a failing build before the restart and keeps the served build", () => {
const f = deployFixture({ build: "fail" });
const result = f.run();
expect(result.status).not.toBe(0);
expect(result.stderr).toContain("simulated build error");
expect(result.stderr).toContain(`deploy failed; rolled back to ${f.a}`);
expect(f.commands()).toEqual([
"B install --frozen-lockfile",
f.build("B"),
"A install --frozen-lockfile",
"A db:generate",
"restart A built A",
]);
expect(f.head()).toBe(f.a);
expect(f.served()).toBe("built A");
expect(f.state()).toMatchObject({ previousSha: f.a, deployedSha: f.a, status: "rolled-back" });
});
it("rolls back code, dependencies and build when the new revision is unhealthy", () => {
const f = deployFixture({ health: "down" });
const result = f.run();
expect(result.status).not.toBe(0);
expect(f.commands()).toEqual([
"B install --frozen-lockfile",
f.build("B"),
"B db:generate",
"B db:migrate",
"restart B built B",
"A install --frozen-lockfile",
"A db:generate",
f.build("A"),
"restart A built A",
]);
expect(f.head()).toBe(f.a);
expect(f.served()).toBe("built A");
expect(f.state()).toMatchObject({ deployedSha: f.a, status: "rolled-back" });
});
it("reports a rollback that does not come back healthy", () => {
const f = deployFixture({ health: "down" }, { health: "down" });
const result = f.run();
expect(result.status).not.toBe(0);
expect(result.stderr).toContain(`rollback to ${f.a} did not complete`);
expect(f.state()).toMatchObject({ deployedSha: f.a, status: "rollback-failed" });
});
it("changes nothing in a dry run", () => {
const f = deployFixture({});
const result = f.run(["--dry-run"]);
expect(result.status, result.stderr).toBe(0);
expect(f.commands()).toEqual([]);
expect(f.head()).toBe(f.a);
expect(f.served()).toBe("built A");
expect(existsSync(path.join(f.checkout, ".evo-deploy/state.json"))).toBe(false);
});
});
/** A checkout layout around a copy of start-stack.sh, with the real tsx and vite installs. */
function stackFixture(options: { build?: boolean; env?: string } = {}) {
const root = temporaryRoot("evo-stack-");
const bin = path.join(root, "bin");
const commandLog = path.join(root, "commands.log");
const script = path.join(root, "ops/evo-deploy/start-stack.sh");
mkdirSync(path.dirname(script), { recursive: true });
copyFileSync(path.join(repoRoot, "ops/evo-deploy/start-stack.sh"), script);
for (const linked of ["node_modules", "packages/core", "apps/web/node_modules"]) {
mkdirSync(path.dirname(path.join(root, linked)), { recursive: true });
symlinkSync(path.join(repoRoot, linked), path.join(root, linked));
}
if (options.build !== false) write(path.join(root, "apps/web/dist/index.html"), "<html></html>");
if (options.env !== undefined) write(path.join(root, ".env"), options.env);
write(
path.join(bin, "pnpm"),
`#!/bin/bash
printf '%s|%s|%s\\n' "$PWD" "\${NODE_ENV:-}" "$*" >> "$COMMAND_LOG"
if [[ "$*" == "dev --filter=!@rakazo/web" ]]; then exec sleep 30; fi
if [[ "$1" == --filter ]]; then
# Exit only after the services were started, so both launches are on record.
for _ in $(seq 50); do
grep -q -e "--filter=!@rakazo/web" "$COMMAND_LOG" && exit 3
sleep 0.1
done
exit 4
fi
`,
0o755,
);
return {
root,
run(args: string[] = [], mode?: string) {
const result = spawnSync("/bin/bash", [script, ...args], {
env: {
HOME: path.join(root, "home"),
PATH: searchPath(bin),
COMMAND_LOG: commandLog,
...(mode ? { EVO_WEB_MODE: mode } : {}),
},
encoding: "utf8",
timeout: 30_000,
});
expect(result.error).toBeUndefined();
return result;
},
commands: () => readLines(commandLog),
};
}
const previewEnv =
"RAKAZO_HOST=app.example.com\nSCREEN_PROXY_SECRET=fake-screen-proxy-secret-for-tests-0123456789\n";
describe("evo stack start script", () => {
it("runs pnpm dev by default", () => {
const f = stackFixture();
const result = f.run();
expect(result.status, result.stderr).toBe(0);
expect(f.commands()).toEqual([`${f.root}||dev`]);
});
it("rejects an unknown web mode", () => {
const result = stackFixture().run(["--check"], "production");
expect(result.status).toBe(1);
expect(result.stderr).toContain("EVO_WEB_MODE must be dev or preview (got: production)");
});
it.each([
["no build", { build: false, env: previewEnv }, ["apps/web/dist has no build"]],
[
"no host or secret",
{ env: "" },
["Set RAKAZO_HOST", "Set SCREEN_PROXY_SECRET to a dedicated random screen proxy credential."],
],
[
"the example secret",
{
env: "RAKAZO_HOST=app.example.com\nSCREEN_PROXY_SECRET=replace-with-32-plus-character-screen-proxy-secret\n",
},
["Replace the SCREEN_PROXY_SECRET placeholder"],
],
])("refuses preview with %s before starting anything", (_name, options, messages) => {
const f = stackFixture(options);
const result = f.run([], "preview");
expect(result.status).toBe(1);
for (const message of messages) expect(result.stderr).toContain(message);
expect(f.commands()).toEqual([]);
});
it("accepts a preview configuration from the root .env", () => {
const f = stackFixture({ env: previewEnv });
const result = f.run(["--check"], "preview");
expect(result.status, result.stderr).toBe(0);
expect(result.stderr).toContain("ok (preview)");
expect(f.commands()).toEqual([]);
});
it("serves the build next to the other services and stops when one exits", () => {
const f = stackFixture({ env: previewEnv });
const result = f.run([], "preview");
expect(result.status).toBe(1);
expect(result.stderr).toContain("a service exited");
expect(f.commands().sort()).toEqual([
`${f.root}|production|--filter @rakazo/web preview --strictPort`,
`${f.root}||dev --filter=!@rakazo/web`,
]);
});
});