evOo-local/ops/evo-deploy/README.md

# Evo auto-deploy (evOo-local)

Idempotent deploy helper for the **source checkout** on the Evo host, run as `<deploy-user>`. Pulls `main` (or a pinned SHA), installs dependencies, builds the web app, runs migrations, restarts the stack, health-checks API `:3100` and web `:5173`, and rolls back on failure.

This is separate from production Docker deploys (see `docs/self-host.md` and `.github/workflows/ci.yml` `deploy-production`).

## Prerequisites on Evo

1. Git checkout at `$HOME/Projects/evOo-local` (or set `EVO_DEPLOY_DIR`).
2. `.env` with a working `DATABASE_URL` (Postgres reachable for `pnpm db:migrate`).
3. A restart path the script can use (pick one):
   - **systemd (recommended for unattended deploy):** the user unit from [Stack unit](#stack-unit). Set repo variable `EVO_SYSTEMD_UNIT`.
   - **tmux:** session `evoo` running `pnpm dev` (set `EVO_TMUX_SESSION` if different).
   - **compose:** `infra/compose/docker-compose.yml` with `api` / `web` / `worker` (dev stack).
   - **script:** executable hook at `EVO_RESTART_SCRIPT`.
4. Self-hosted GitHub Actions runner on Evo with labels `[self-hosted, evoo-ci, Linux]` (already used by CI).

No SSH keys or deploy secrets are stored in this repo. The workflow runs **on** the Evo runner against the local checkout.

## Enable the workflow

1. Merge this PR (or copy `ops/evo-deploy/` and `.github/workflows/evo-deploy.yml` to `main`).
2. In GitHub → **Settings → Secrets and variables → Actions → Variables** for this repository:
   - `EVO_DEPLOY_ENABLED` = `true`
   - Optional: `EVO_DEPLOY_DIR` (default `$HOME/Projects/evOo-local` of the runner user)
   - Optional: `EVO_RESTART_MODE` (`auto`, `systemd`, `tmux`, `compose`, `script`, `none`)
   - **Evo (recommended):** pin `EVO_RESTART_MODE=systemd`, `EVO_SYSTEMD_UNIT=evoo-local.service`, and `EVO_SYSTEMD_SCOPE=user` as repo variables (or the same values in the runner host env). Auto-detect prefers systemd when that unit is active or enabled, but pinning avoids ambiguity if `infra/compose/docker-compose.yml` is present and Compose env vars are not configured for production-like deploys.
   - Optional: `EVO_TMUX_SESSION`, health URLs, timeouts
3. Confirm the runner user can read/write `EVO_DEPLOY_DIR` and restart services (`systemctl --user` or tmux).

Until `EVO_DEPLOY_ENABLED` is `true`, pushes to `main` do not deploy (workflow job is skipped).

## Stack unit

`start-stack.sh` starts api, worker and sandbox supervisor exactly as `pnpm dev` does. The web app runs in one of two modes, chosen by `EVO_WEB_MODE`:

- `dev` (default): the Vite dev server, same as `pnpm dev`.
- `preview`: `vite preview` serves the production build from `apps/web/dist` on the same port. `deploy.sh` rebuilds it on every deploy.

`evoo-local.service` is a systemd user unit template that runs the script. Install it once, as `<deploy-user>` in the checkout:

```bash
systemctl --user cat evoo-local.service   # note what the current unit runs; keep a copy if needed
install -D -m 0644 ops/evo-deploy/evoo-local.service ~/.config/systemd/user/evoo-local.service
# Edit WorkingDirectory in the installed copy if the checkout is not at ~/Projects/evOo-local.
systemctl --user daemon-reload
systemctl --user enable evoo-local.service
systemctl --user restart evoo-local.service
loginctl enable-linger "$USER"            # keep user services running without a login session
```

The unit starts the script through a login shell, so node and pnpm come from the same `PATH` that `deploy.sh` uses. It stays in dev mode, so nothing visible changes. Keep the repo variables `EVO_RESTART_MODE=systemd` and `EVO_SYSTEMD_UNIT=evoo-local.service`. Logs: `journalctl --user -u evoo-local.service -f`.

### Switch to preview

The production build drops message source text, so a message missing from the Lingui catalogs shows as a hash id instead of English. Switch only once the catalogs are complete (`pnpm --filter @rakazo/web intl:extract` adds no new entries).

1. In the root `.env`, set `RAKAZO_HOST` to the hostname browsers use (see [HTTPS with Caddy](#https-with-caddy)) and `SCREEN_PROXY_SECRET` to a dedicated random value of at least 32 characters (`openssl rand -hex 32`), different from `BETTER_AUTH_SECRET` and `SANDBOX_SUPERVISOR_TOKEN`. `vite preview` runs with `NODE_ENV=production` and refuses development placeholders; the API reads the same secret.
2. Check the configuration without restarting anything:
   ```bash
   EVO_WEB_MODE=preview bash ops/evo-deploy/start-stack.sh --check
   ```
3. Switch the unit and restart:
   ```bash
   systemctl --user edit evoo-local.service
   # add:
   # [Service]
   # Environment=EVO_WEB_MODE=preview
   systemctl --user restart evoo-local.service
   ```
4. Verify: `curl -fsS -o /dev/null -w '%{http_code}\n' http://127.0.0.1:5173/`.

To switch back, remove the override (`systemctl --user revert evoo-local.service`) and restart.

## HTTPS with Caddy

Service workers and web push need a secure context, so phones and other devices need HTTPS; `http://<host-ip>:5173` is not one. Run Caddy on the host and proxy everything to the web port. Both the dev server and `vite preview` forward `/api`, `/rpc` and the screen proxy to the API, so `:3100` stays private:

```Caddyfile
app.example.com {
	reverse_proxy 127.0.0.1:5173
}
```

This is the host proxy from `docs/self-host.md`; `infra/compose/Caddyfile.prod` routes the same paths to Compose containers and is not used here. Caddy obtains the certificate itself when the hostname resolves to the host and ports 80 and 443 are reachable; otherwise configure a DNS challenge.

Root `.env`, then restart the unit:

```dotenv
RAKAZO_HOST=app.example.com
EXTRA_TRUSTED_ORIGINS=https://app.example.com
```

- `RAKAZO_HOST` is the hostname `vite preview` accepts. Caddy forwards the original `Host` header; the dev server accepts any host.
- `EXTRA_TRUSTED_ORIGINS` (comma-separated) lets the HTTPS origin sign in. Keep `BETTER_AUTH_URL` on the plain-http origin and set it explicitly, because it falls back to `WEB_ORIGIN`: an `https://` value makes Better Auth use `__Secure-` cookies, which browsers drop over plain http, so signing in via `http://…:5173` would stop working.
- `WEB_ORIGIN` is the origin of links the API builds, such as the agent computer screen. Set it to the HTTPS origin if that is the address you use; an `http://` screen link is blocked as mixed content on the HTTPS page.

## Manual / dry-run

On Evo (or any machine with the checkout and stack):

```bash
cd <checkout>

# Show what would happen (no git reset, install, build, migrate, restart, or health wait)
bash ops/evo-deploy/deploy.sh --dry-run

# Deploy main tip
bash ops/evo-deploy/deploy.sh

# Pin a SHA
EVO_DEPLOY_REF=abc1234 bash ops/evo-deploy/deploy.sh

# Force systemd restart (unit optional — defaults to evoo-local.service)
EVO_RESTART_MODE=systemd bash ops/evo-deploy/deploy.sh
EVO_RESTART_MODE=systemd EVO_SYSTEMD_UNIT=evoo-local.service bash ops/evo-deploy/deploy.sh
```

GitHub: **Actions → evo-deploy → Run workflow** (optional `ref`, `dry_run`, `skip_migrate`).

## Deploy and rollback behavior

Restart mode and its prerequisites (systemd unit name, tmux, compose file, restart script, valid `EVO_SYSTEMD_SCOPE`) are validated **before** `git fetch`, install, build, migrate, or web activation. A bad configuration exits immediately without changing the checkout or database.

1. Before `git fetch`, the script records `previousSha` (current `HEAD`), then resets the checkout to the target.
2. `pnpm install --frozen-lockfile`, then the web build into `.evo-deploy/web-dist`. Its output goes to `.evo-deploy/web-build.log` and is printed only when the build fails.
3. `pnpm db:generate && pnpm db:migrate`.
4. The new build replaces `apps/web/dist`, the stack restarts, and the script waits up to `EVO_HEALTH_TIMEOUT_SEC` (default 120s) for:
   - `EVO_API_HEALTH_URL` (default `http://127.0.0.1:3100/health`)
   - `EVO_WEB_HEALTH_URL` (default `http://127.0.0.1:5173/`)
5. Any failure rolls back: `git reset --hard` to `previousSha`, reinstall, regenerate the client, rebuild the web app if this deploy had already replaced it, restart, health-check, exit non-zero. A failing install, build or migration therefore never restarts the stack on the new commit, and `apps/web/dist` keeps the build that was being served. Migrations only move forward and stay applied.
6. The outcome (`success`, `rolled-back` or `rollback-failed`) is written to `.evo-deploy/state.json` under the checkout.

`deploy.sh` captures `previousSha` and `targetSha` via process substitution, so every git command inside `git_fetch_and_checkout` and rollback redirects stdout to stderr (`reset --hard` prints `HEAD is now at …`). `write_state` and rollback reject values that are not 7–40 hex characters. `packages/testkit/src/evo-deploy.test.ts` runs the script against a fixture checkout and covers this, the build gate and the rollback.

Manual rollback to a known good SHA:

```bash
cd <checkout>
EVO_RESTART_MODE=systemd EVO_SYSTEMD_UNIT=evoo-local.service EVO_DEPLOY_REF=<good-sha> \
  bash ops/evo-deploy/deploy.sh --skip-migrate
```

(or `git reset --hard <good-sha>`, `pnpm install --frozen-lockfile`, `pnpm --filter @rakazo/web build`, restart the unit, then verify the health URLs).

## Autopilot / Grok Bot after merge

After a PR merges to `main`:

1. **Preferred:** rely on the `evo-deploy` workflow (once `EVO_DEPLOY_ENABLED=true`). No SSH step required.
2. **Fallback (manual):** SSH to Evo and run:
   ```bash
   cd <checkout> && bash ops/evo-deploy/deploy.sh
   ```
3. **Verify:** API `curl -fsS http://127.0.0.1:3100/health`, web `curl -fsS -o /dev/null -w '%{http_code}\n' http://127.0.0.1:5173/`.

Do not commit host passwords, tokens, or SSH private keys. Configure restart targets via repo variables or env on the runner host only.

## Environment reference

`deploy.sh`:

| Variable | Default | Purpose |
|----------|---------|---------|
| `EVO_DEPLOY_DIR` | `$HOME/Projects/evOo-local` | Git checkout root |
| `EVO_DEPLOY_REF` | `main` | Branch name or commit SHA |
| `EVO_RESTART_MODE` | `auto` | `systemd` / `tmux` / `compose` / `script` / `none` |
| `EVO_SYSTEMD_UNIT` | _(empty)_ | Systemd unit to restart; when `EVO_RESTART_MODE=systemd` and unset, defaults to `EVO_SYSTEMD_AUTO_UNIT` (same as auto mode). Auto mode sets this when a unit is found |
| `EVO_SYSTEMD_SCOPE` | `user` | `user` or `system` (auto-detect picks the scope where the unit is active/enabled) |
| `EVO_SYSTEMD_AUTO_UNIT` | `evoo-local.service` | Unit name probed when `EVO_RESTART_MODE=auto` and `EVO_SYSTEMD_UNIT` is unset |
| `EVO_TMUX_SESSION` | `evoo` | tmux session name |
| `EVO_TMUX_COMMAND` | `pnpm dev` | Command started in new tmux session |
| `EVO_MIGRATE` | `true` | Run `pnpm db:generate` + `pnpm db:migrate` (`--skip-migrate`) |
| `EVO_INSTALL` | `true` | Run `pnpm install --frozen-lockfile` after checkout (`--skip-install`) |
| `EVO_BUILD` | `true` | Build the web app before the restart (`--skip-build`) |
| `EVO_API_HEALTH_URL` | `http://127.0.0.1:3100/health` | API health probe |
| `EVO_WEB_HEALTH_URL` | `http://127.0.0.1:5173/` | Web health probe |
| `EVO_HEALTH_TIMEOUT_SEC` | `120` | Per-check timeout (seconds) |

`start-stack.sh`:

| Variable | Default | Purpose |
|----------|---------|---------|
| `EVO_WEB_MODE` | `dev` | `dev` (Vite dev server) or `preview` (production build) |

When `EVO_RESTART_MODE=auto`, detection order is:

1. **systemd** — `EVO_SYSTEMD_UNIT` is set, or `EVO_SYSTEMD_AUTO_UNIT` (default `evoo-local.service`) is active, enabled, or loaded under `systemctl --user` then `systemctl` (system scope).
2. **script** — `EVO_RESTART_SCRIPT` exists and is executable.
3. **tmux** — session `EVO_TMUX_SESSION` exists.
4. **compose** — `EVO_COMPOSE_FILE` exists under the checkout.
5. **none** — skip restart; health-check only.

If auto mode picks the wrong path (for example Compose on Evo because the compose file exists but the stack runs under systemd), pin `EVO_RESTART_MODE=systemd` and `EVO_SYSTEMD_UNIT=evoo-local.service` in repo variables or host env.