diff --git a/.claude-code-supervisor.yml b/.claude-code-supervisor.yml index 0328145..d6bf909 100644 --- a/.claude-code-supervisor.yml +++ b/.claude-code-supervisor.yml @@ -6,7 +6,7 @@ triage: # Command that accepts a prompt on stdin and returns text on stdout. # Default: claude -p (uses Claude Code's own auth) command: "claude -p --no-session-persistence" - model: "claude-haiku-4-20250414" + model: "claude-haiku-4-5-20251001" max_tokens: 150 notify: diff --git a/.claude/hooks/supervisor/lib.sh b/.claude/hooks/supervisor/lib.sh index 68d3892..b28cdf7 100755 --- a/.claude/hooks/supervisor/lib.sh +++ b/.claude/hooks/supervisor/lib.sh @@ -65,7 +65,7 @@ ccs_triage() { local triage_cmd model max_tokens triage_cmd=$(ccs_config_get "$config_file" "triage.command" "claude -p --no-session-persistence") - model=$(ccs_config_get "$config_file" "triage.model" "claude-haiku-4-20250414") + model=$(ccs_config_get "$config_file" "triage.model" "claude-haiku-4-5-20251001") max_tokens=$(ccs_config_get "$config_file" "triage.max_tokens" "150") echo "$prompt" | $triage_cmd --model "$model" --max-tokens "$max_tokens" 2>/dev/null diff --git a/AGENTS.md b/AGENTS.md index 5bb13e8..a043e4b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,185 +1,68 @@ # Agent Brief: Harb Stack -## Core Concepts -- KRAIKEN couples Harberger-tax staking with a dominant Uniswap V3 liquidity manager to create asymmetric slippage, sentiment-driven pricing, and VWAP "price memory" safeguards. -- Liquidity dominance is mission-critical; treat any regression that weakens the LiquidityManager's control as a priority incident. -- Harberger staking supplies the sentiment oracle that drives Optimizer parameters, which in turn tune liquidity placement and supply expansion. +## What is KRAIKEN? +KRAIKEN couples Harberger-tax staking with a dominant Uniswap V3 liquidity manager to create asymmetric slippage, sentiment-driven pricing, and VWAP "price memory" safeguards. Liquidity dominance is mission-critical; treat any regression that weakens the LiquidityManager's control as a priority incident. ## User Journey -1. **Buy** - Acquire KRAIKEN on Uniswap. -2. **Stake** - Declare a tax rate on kraiken.org to earn from protocol growth. -3. **Compete** - Snatch undervalued positions to optimise returns. +1. **Buy** — Acquire KRAIKEN on Uniswap. +2. **Stake** — Declare a tax rate on kraiken.org to earn from protocol growth. +3. **Compete** — Snatch undervalued positions to optimise returns. -## Operating the Stack +## Directory Map +| Path | What | Guide | +|------|------|-------| +| `onchain/` | Solidity + Foundry contracts, deploy scripts, fuzzing | [onchain/AGENTS.md](onchain/AGENTS.md) | +| `services/ponder/` | Ponder indexer powering the GraphQL API | [services/ponder/AGENTS.md](services/ponder/AGENTS.md) | +| `landing/` | Vue 3 marketing + staking interface | [landing/AGENTS.md](landing/AGENTS.md) | +| `web-app/` | Staking UI | [web-app/AGENTS.md](web-app/AGENTS.md) | +| `kraiken-lib/` | Shared TypeScript helpers for clients and bots | [kraiken-lib/AGENTS.md](kraiken-lib/AGENTS.md) | +| `services/txnBot/` | Automation bot for `recenter()` and `payTax()` upkeep | [services/txnBot/AGENTS.md](services/txnBot/AGENTS.md) | +| `scripts/` | `dev.sh`, bootstrap, build helpers | — | +| `tests/e2e/` | Playwright end-to-end tests | — | +| `docs/` | Architecture, product truth, environment, ops guides | — | -### Quick Start +## Quick Start ```bash -nohup ./scripts/dev.sh start & # start (takes ~3-6 min first time) -tail -f nohup.out # watch progress -./scripts/dev.sh health # verify all services healthy -./scripts/dev.sh stop # stop and clean up +./scripts/dev.sh start # boots full stack (~3-6 min first time) +./scripts/dev.sh health # verify all services healthy +./scripts/dev.sh stop # stop and clean up ``` -Do not launch services individually — `dev.sh` enforces phased startup with health gates. +See [docs/dev-environment.md](docs/dev-environment.md) for restart modes, ports, Docker topology, and common pitfalls. -### Restart Modes -- `./scripts/dev.sh restart --light` — Fast (~10-20s): only webapp + txnbot, preserves Anvil/Ponder state. Use for frontend changes. -- `./scripts/dev.sh restart --full` — Full (~3-6min): redeploys contracts, fresh state. Use for contract changes. +## Key Patterns +- **ES Modules everywhere**: The entire stack uses `"type": "module"` and `import` syntax. +- **`token0isWeth`**: Flips amount semantics; confirm ordering before seeding or interpreting liquidity. +- **Price^2 (X96)**: VWAP, `ethScarcity`, and Optimizer outputs operate on price^2. Avoid "normalising" to sqrt inadvertently. +- **LiquidityManager funding**: Fund with Base WETH (`0x4200...0006`) before expecting `recenter()` to succeed. +- **Ponder state**: Stored in `.ponder/`; drop the directory if schema changes break migrations. +- **Harberger staking** supplies the sentiment oracle that drives Optimizer parameters, which in turn tune liquidity placement and supply expansion. -### Common Pitfalls -- **Docker disk full**: `dev.sh start` refuses to run if Docker disk usage exceeds 20GB. Fix: `./scripts/dev.sh stop` (auto-prunes) or `docker system prune -af --volumes`. -- **Stale Ponder state**: If Ponder fails with schema errors after contract changes, delete its state: `rm -rf services/ponder/.ponder/` then `./scripts/dev.sh restart --full`. -- **kraiken-lib out of date**: If services fail with import errors or missing exports, rebuild: `./scripts/build-kraiken-lib.sh`. The dev script does this automatically on `start`, but manual rebuilds are needed if you change kraiken-lib while the stack is already running. -- **Container not found errors**: `dev.sh` expects Docker Compose v2 container names (`harb-anvil-1`, hyphens not underscores). Verify with `docker compose version`. -- **Port conflicts**: The stack uses ports 8545 (Anvil), 5173 (webapp), 5174 (landing), 42069 (Ponder), 43069 (txnBot), 5100 (Otterscan), 8081 (Caddy). Check with `lsof -i :` if startup fails. -- **npm ci failures in containers**: Named Docker volumes cache `node_modules/`. If dependencies change and installs fail, remove the volume: `docker volume rm harb_webapp_node_modules` (or similar), then restart. - -### Environments -Supported: `BASE_SEPOLIA_LOCAL_FORK` (default Anvil fork), `BASE_SEPOLIA`, and `BASE`. Match contract addresses and RPCs accordingly. - -### Prerequisites -Docker Engine (Linux) or Colima (Mac). See `docs/docker.md` for installation. - -## Component Guides -- `onchain/` - Solidity + Foundry contracts, deploy scripts, and fuzzing helpers ([details](onchain/AGENTS.md)). -- `services/ponder/` - Ponder indexer powering the GraphQL API ([details](services/ponder/AGENTS.md)). -- `landing/` - Vue 3 marketing + staking interface ([details](landing/AGENTS.md)). -- `kraiken-lib/` - Shared TypeScript helpers for clients and bots ([details](kraiken-lib/AGENTS.md)). -- `services/txnBot/` - Automation bot for `recenter()` and `payTax()` upkeep ([details](services/txnBot/AGENTS.md)). - -## Testing & Tooling -- Contracts: run `forge build`, `forge test`, and `forge snapshot` inside `onchain/`. -- Fuzzing: scripts under `onchain/analysis/` (e.g., `./analysis/run-fuzzing.sh [optimizer] debugCSV`) generate replayable scenarios. -- Integration: after the stack boots, inspect Anvil logs, hit `http://localhost:8081/api/graphql` for Ponder, and poll `http://localhost:8081/api/txn/status` for txnBot health. -- **E2E Tests**: Playwright-based full-stack tests in `tests/e2e/` verify complete user journeys (mint ETH → swap KRK → stake). Run with `npm run test:e2e` from repo root. Tests use mocked wallet provider with Anvil accounts. In CI, the Woodpecker e2e pipeline runs these against pre-built service images. - -## Version Validation System -- **Contract VERSION**: `Kraiken.sol` exposes a `VERSION` constant (currently v2) that must be incremented for breaking changes to TAX_RATES, events, or core data structures. -- **Ponder Validation**: On startup, Ponder reads the contract VERSION and validates against `COMPATIBLE_CONTRACT_VERSIONS` in `kraiken-lib/src/version.ts`. Fails hard (exit 1) on mismatch to prevent indexing wrong data. -- **Frontend Check**: Web-app validates `KRAIKEN_LIB_VERSION` at runtime (currently placeholder; future: query Ponder GraphQL for full 3-way validation). -- **CI Enforcement**: Woodpecker `release.yml` pipeline validates that contract VERSION matches `COMPATIBLE_CONTRACT_VERSIONS` before release. -- See `VERSION_VALIDATION.md` for complete architecture, workflows, and troubleshooting. - -## Docker -See `docs/docker.md` for Docker/Colima installation and the full service topology. Key facts: -- `docker-compose.yml` has NO `depends_on`. Service ordering is handled by `scripts/dev.sh`. -- Startup phases: anvil+postgres → bootstrap → ponder → webapp/landing/txn-bot → caddy → smoke test. -- Shared bootstrap: `scripts/bootstrap-common.sh` (sourced by both local dev and CI). -- 20GB disk limit enforced. `dev.sh stop` auto-prunes. Named volumes for `node_modules`. -- All services have log rotation (30MB max per container) and PostgreSQL WAL limits configured. - -## Guardrails & Tips -- `token0isWeth` flips amount semantics; confirm ordering before seeding or interpreting liquidity. -- VWAP, `ethScarcity`, and Optimizer outputs operate on price^2 (X96). Avoid "normalising" to sqrt inadvertently. -- Fund the LiquidityManager with Base WETH (`0x4200...0006`) before expecting `recenter()` to succeed. -- Ponder stores data in `.ponder/`; drop the directory if schema changes break migrations. -- Keep git clean before committing; never leave commented-out code or untested changes. -- **ES Modules**: The entire stack uses ES modules. kraiken-lib, txnBot, Ponder, and web-app all require `"type": "module"` in package.json and use `import` syntax. -- **kraiken-lib Build**: Run `./scripts/build-kraiken-lib.sh` before `docker-compose up` so containers mount a fresh `kraiken-lib/dist` from the host. -- **Live Reload**: `scripts/watch-kraiken-lib.sh` rebuilds on file changes (requires inotify-tools) and restarts dependent containers automatically. +## Before Opening a PR +1. `forge build && forge test` in `onchain/` — contracts must compile and pass. +2. Run `npm run test:e2e` from repo root if you touched frontend or services. +3. `git diff --check` — no trailing whitespace or merge markers. +4. Keep commits clean; never leave commented-out code or untested changes. +5. If you changed `kraiken-lib`, rebuild: `./scripts/build-kraiken-lib.sh`. +6. If you changed contract VERSION or events, update `COMPATIBLE_CONTRACT_VERSIONS` in `kraiken-lib/src/version.ts`. ## Code Quality & Git Hooks -- **Pre-commit Hooks**: Husky runs lint-staged on all staged files before commits. Each component (onchain, kraiken-lib, ponder, txnBot, web-app, landing) has `.lintstagedrc.json` configured for ESLint + Prettier. -- **Version Validation (Future)**: Pre-commit hook includes validation logic that will enforce version sync between `onchain/src/Kraiken.sol` (contract VERSION constant) and `kraiken-lib/src/version.ts` (COMPATIBLE_CONTRACT_VERSIONS array). This validation only runs if both files exist and contain version information. -- **Husky Setup**: `.husky/pre-commit` orchestrates all pre-commit checks. Modify this file to add new validation steps. -- To test hooks manually: `git add && .husky/pre-commit` +Pre-commit hooks (Husky + lint-staged) run ESLint + Prettier on staged files. Each component has its own `.lintstagedrc.json`. To test manually: `git add && .husky/pre-commit`. -## Handy Commands -- `foundryup` - update Foundry toolchain. -- `anvil --fork-url https://sepolia.base.org` - manual fork when diagnosing outside the helper script. -- `cast call "slot0()"` - inspect pool state. -- `PONDER_NETWORK=BASE_SEPOLIA_LOCAL_FORK npm run dev` (inside `services/ponder/`) - focused indexer debugging when the full stack is already running. -- `curl -X POST http://localhost:8081/api/graphql -d '{"query":"{ stats(id:\"0x01\"){kraikenTotalSupply}}"}'` -- `curl http://localhost:8081/api/txn/status` - -## Woodpecker CI - -### Infrastructure -- **Server**: Woodpecker 3.10.0 runs as a **systemd service** (`woodpecker-server.service`), NOT a Docker container. Binary at `/usr/local/bin/woodpecker-server`. -- **Host**: `https://ci.sovraigns.network` (port 8000 locally at `http://127.0.0.1:8000`) -- **Forge**: Codeberg (Gitea-compatible) — repo `johba/harb`, forge remote ID `800173` -- **Database**: PostgreSQL at `127.0.0.1:5432`, database `woodpecker`, user `woodpecker` -- **Config**: `/etc/woodpecker/server.env` (contains secrets — agent secret, Gitea OAuth secret, DB credentials) -- **CLI**: Downloaded to `/tmp/woodpecker-cli` (v3.10.0). Requires `WOODPECKER_SERVER` and `WOODPECKER_TOKEN` env vars. -- **Logs**: `journalctl -u woodpecker-server -f` (NOT `docker logs`) - -### Pipeline Configs -- `.woodpecker/build-ci-images.yml` — Builds Docker CI images using unified `docker/Dockerfile.service-ci`. Triggers on **push** to `master` or `feature/ci` when files in `docker/`, `.woodpecker/`, `containers/`, `kraiken-lib/`, `onchain/`, `services/`, `web-app/`, or `landing/` change. -- `.woodpecker/e2e.yml` — Runs Playwright E2E tests. Bootstrap step sources `scripts/bootstrap-common.sh` for shared deploy/seed logic. Health checks use `scripts/wait-for-service.sh`. Triggers on **pull_request** to `master`. -- Pipeline numbering: even = build-ci-images (push events), odd = E2E (pull_request events). This is not guaranteed but was the observed pattern. - -### Monitoring Pipelines via DB -Since the Woodpecker API requires authentication (tokens are cached in server memory; DB-only token changes don't work without a server restart), monitor pipelines directly via PostgreSQL: -```bash -# Latest pipelines -PGPASSWORD='' psql -h 127.0.0.1 -U woodpecker -d woodpecker -c \ - "SELECT number, status, branch, event, commit FROM pipelines - WHERE repo_id = (SELECT id FROM repos WHERE full_name = 'johba/harb') - ORDER BY number DESC LIMIT 5;" - -# Step details for a specific pipeline -PGPASSWORD='' psql -h 127.0.0.1 -U woodpecker -d woodpecker -c \ - "SELECT s.name, s.state, - CASE WHEN s.finished > 0 AND s.started > 0 THEN (s.finished - s.started)::int::text || 's' - ELSE '-' END as duration, s.exit_code - FROM steps s WHERE s.pipeline_id = ( - SELECT id FROM pipelines WHERE number = - AND repo_id = (SELECT id FROM repos WHERE full_name = 'johba/harb')) - ORDER BY s.started NULLS LAST;" -``` - -### Triggering Pipelines -- **Normal flow**: Push to Codeberg → Codeberg fires webhook to `https://ci.sovraigns.network/api/hook` → Woodpecker creates pipeline. -- **Known issue**: Codeberg webhooks can stop firing if `ci.sovraigns.network` becomes unreachable (DNS/connectivity). Check Codeberg repo settings → Webhooks to verify delivery history and re-trigger. -- **Manual trigger via API** (requires valid token — see known issues): - ```bash - WOODPECKER_SERVER=http://127.0.0.1:8000 WOODPECKER_TOKEN= \ - /tmp/woodpecker-cli pipeline create --branch feature/ci johba/harb - ``` -- **API auth limitation**: The server caches user token hashes in memory. Inserting a token directly into the DB does not work without restarting the server (`sudo systemctl restart woodpecker-server`). - -### CI Docker Images -- `docker/Dockerfile.service-ci` — Unified parameterized Dockerfile for all service CI images (ponder, webapp, landing, txnBot). Uses `--build-arg` for service-specific configuration (SERVICE_DIR, SERVICE_PORT, ENTRYPOINT_SCRIPT, NEEDS_SYMLINKS, etc.). - - **sync-tax-rates**: Builder stage runs `scripts/sync-tax-rates.mjs` to sync tax rates from `Stake.sol` into kraiken-lib before TypeScript compilation. - - **Symlinks fix** (webapp only, `NEEDS_SYMLINKS=true`): Creates `/web-app`, `/kraiken-lib`, `/onchain` symlinks to work around Vite's `removeBase()` stripping `/app/` prefix from filesystem paths. - - **CI env detection** (`CI=true`): Disables Vue DevTools plugin in `vite.config.ts` to prevent 500 errors caused by path resolution issues with `/app/` base path. - - **HEALTHCHECK**: Configurable via build args; webapp uses `--retries=84 --interval=5s` = 420s (7 min), aligned with `wait-for-stack` step timeout. -- **Shared entrypoints**: Each service uses a unified entrypoint script (`containers/-entrypoint.sh`) that branches on `CI=true` env var for CI vs local dev paths. Common helpers in `containers/entrypoint-common.sh`. -- **Shared bootstrap**: `scripts/bootstrap-common.sh` contains shared contract deployment, seeding, and funding functions used by both `containers/bootstrap.sh` (local dev) and `.woodpecker/e2e.yml` (CI). -- CI images are tagged with git SHA and `latest`, pushed to a local registry. - -### CI Agent & Registry Auth -- **Agent**: Runs as user `ci` (uid 1001) on `harb-staging`, same host as the dev environment. Binary at `/usr/local/bin/woodpecker-agent`. -- **Registry credentials**: The `ci` user must have Docker auth configured at `/home/ci/.docker/config.json` to pull private images from `registry.niovi.voyage`. If images fail to pull with "no basic auth credentials", fix with: - ```bash - sudo mkdir -p /home/ci/.docker - sudo cp /home/debian/.docker/config.json /home/ci/.docker/config.json - sudo chown -R ci:ci /home/ci/.docker - sudo chmod 600 /home/ci/.docker/config.json - ``` -- **Shared Docker daemon**: The `ci` and `debian` users share the same Docker daemon. Running `docker system prune` as `debian` removes images cached for CI pipelines. If CI image pulls fail after a prune, either fix registry auth (above) or pre-pull images as `debian`: `docker pull registry.niovi.voyage/harb/ponder-ci:latest` etc. - -### CI Debugging Tips -- If pipelines aren't being created after a push, check Codeberg webhook delivery logs first. -- The Woodpecker server needs `sudo` to restart. Without it, you cannot: refresh API tokens, clear cached state, or recover from webhook auth issues. -- E2E pipeline failures often come from `wait-for-stack` timing out. Check the webapp HEALTHCHECK alignment and Ponder indexing time. -- The `web-app/vite.config.ts` `allowedHosts` array must include container hostnames (`webapp`, `caddy`) for health checks to succeed inside Docker networks. -- **Never use `bash -lc`** in Woodpecker pipeline commands — login shell resets PATH via `/etc/profile`, losing Foundry and other tools set by Docker ENV. Use `bash -c` instead. - -## Codeberg API Access -- **Auth**: Codeberg API tokens are stored in `~/.netrc` (standard `curl --netrc` format, `chmod 600`): - ``` - machine codeberg.org - login johba - password - ``` - The `password` field holds the API token — this is standard `.netrc` convention, not an actual password. -- **Generate tokens** at `https://codeberg.org/user/settings/applications`. -- **Usage**: Pass `--netrc` to curl for authenticated Codeberg API calls: - ```bash - curl --netrc -s https://codeberg.org/api/v1/repos/johba/harb/issues | jq '.[0].title' - ``` -- **Note**: The repo uses SSH for git push/pull (`ssh://git@codeberg.org`), so `.netrc` is only used for REST API interactions (issues, PRs, releases). +## Deeper Docs +| Topic | File | +|-------|------| +| Dev environment, Docker, ports, pitfalls | [docs/dev-environment.md](docs/dev-environment.md) | +| Woodpecker CI setup and debugging | [docs/ci-pipeline.md](docs/ci-pipeline.md) | +| Testing: Foundry, E2E, version validation | [docs/testing.md](docs/testing.md) | +| Codeberg API access and webhooks | [docs/codeberg-api.md](docs/codeberg-api.md) | +| Product truth and positioning | [docs/PRODUCT-TRUTH.md](docs/PRODUCT-TRUTH.md) | +| Architecture overview | [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) | +| UX decisions | [docs/UX-DECISIONS.md](docs/UX-DECISIONS.md) | +| Environment configuration | [docs/ENVIRONMENT.md](docs/ENVIRONMENT.md) | +| Version validation architecture | [VERSION_VALIDATION.md](VERSION_VALIDATION.md) | +| Uniswap V3 math deep dive | [onchain/UNISWAP_V3_MATH.md](onchain/UNISWAP_V3_MATH.md) | +| Technical appendix | [TECHNICAL_APPENDIX.md](TECHNICAL_APPENDIX.md) | +| Harberger tax mechanics | [HARBERG.md](HARBERG.md) | ## References - Deployment history: `onchain/deployments-local.json`, `onchain/broadcast/`. -- Deep dives: `TECHNICAL_APPENDIX.md`, `HARBERG.md`, and `onchain/UNISWAP_V3_MATH.md`. diff --git a/docs/ci-pipeline.md b/docs/ci-pipeline.md new file mode 100644 index 0000000..2355f72 --- /dev/null +++ b/docs/ci-pipeline.md @@ -0,0 +1,73 @@ +# Woodpecker CI + +## Infrastructure +- **Server**: Woodpecker 3.10.0 runs as a **systemd service** (`woodpecker-server.service`), NOT a Docker container. Binary at `/usr/local/bin/woodpecker-server`. +- **Host**: `https://ci.sovraigns.network` (port 8000 locally at `http://127.0.0.1:8000`) +- **Forge**: Codeberg (Gitea-compatible) — repo `johba/harb`, forge remote ID `800173` +- **Database**: PostgreSQL at `127.0.0.1:5432`, database `woodpecker`, user `woodpecker` +- **Config**: `/etc/woodpecker/server.env` (contains secrets — agent secret, Gitea OAuth secret, DB credentials) +- **CLI**: Downloaded to `/tmp/woodpecker-cli` (v3.10.0). Requires `WOODPECKER_SERVER` and `WOODPECKER_TOKEN` env vars. +- **Logs**: `journalctl -u woodpecker-server -f` (NOT `docker logs`) + +## Pipeline Configs +- `.woodpecker/build-ci-images.yml` — Builds Docker CI images using unified `docker/Dockerfile.service-ci`. Triggers on **push** to `master` or `feature/ci` when files in `docker/`, `.woodpecker/`, `containers/`, `kraiken-lib/`, `onchain/`, `services/`, `web-app/`, or `landing/` change. +- `.woodpecker/e2e.yml` — Runs Playwright E2E tests. Bootstrap step sources `scripts/bootstrap-common.sh` for shared deploy/seed logic. Health checks use `scripts/wait-for-service.sh`. Triggers on **pull_request** to `master`. +- Pipeline numbering: even = build-ci-images (push events), odd = E2E (pull_request events). This is not guaranteed but was the observed pattern. + +## Monitoring Pipelines via DB +Since the Woodpecker API requires authentication (tokens are cached in server memory; DB-only token changes don't work without a server restart), monitor pipelines directly via PostgreSQL: +```bash +# Latest pipelines +PGPASSWORD='' psql -h 127.0.0.1 -U woodpecker -d woodpecker -c \ + "SELECT number, status, branch, event, commit FROM pipelines + WHERE repo_id = (SELECT id FROM repos WHERE full_name = 'johba/harb') + ORDER BY number DESC LIMIT 5;" + +# Step details for a specific pipeline +PGPASSWORD='' psql -h 127.0.0.1 -U woodpecker -d woodpecker -c \ + "SELECT s.name, s.state, + CASE WHEN s.finished > 0 AND s.started > 0 THEN (s.finished - s.started)::int::text || 's' + ELSE '-' END as duration, s.exit_code + FROM steps s WHERE s.pipeline_id = ( + SELECT id FROM pipelines WHERE number = + AND repo_id = (SELECT id FROM repos WHERE full_name = 'johba/harb')) + ORDER BY s.started NULLS LAST;" +``` + +## Triggering Pipelines +- **Normal flow**: Push to Codeberg → Codeberg fires webhook to `https://ci.sovraigns.network/api/hook` → Woodpecker creates pipeline. +- **Known issue**: Codeberg webhooks can stop firing if `ci.sovraigns.network` becomes unreachable (DNS/connectivity). Check Codeberg repo settings → Webhooks to verify delivery history and re-trigger. +- **Manual trigger via API** (requires valid token — see known issues): + ```bash + WOODPECKER_SERVER=http://127.0.0.1:8000 WOODPECKER_TOKEN= \ + /tmp/woodpecker-cli pipeline create --branch feature/ci johba/harb + ``` +- **API auth limitation**: The server caches user token hashes in memory. Inserting a token directly into the DB does not work without restarting the server (`sudo systemctl restart woodpecker-server`). + +## CI Docker Images +- `docker/Dockerfile.service-ci` — Unified parameterized Dockerfile for all service CI images (ponder, webapp, landing, txnBot). Uses `--build-arg` for service-specific configuration (SERVICE_DIR, SERVICE_PORT, ENTRYPOINT_SCRIPT, NEEDS_SYMLINKS, etc.). + - **sync-tax-rates**: Builder stage runs `scripts/sync-tax-rates.mjs` to sync tax rates from `Stake.sol` into kraiken-lib before TypeScript compilation. + - **Symlinks fix** (webapp only, `NEEDS_SYMLINKS=true`): Creates `/web-app`, `/kraiken-lib`, `/onchain` symlinks to work around Vite's `removeBase()` stripping `/app/` prefix from filesystem paths. + - **CI env detection** (`CI=true`): Disables Vue DevTools plugin in `vite.config.ts` to prevent 500 errors caused by path resolution issues with `/app/` base path. + - **HEALTHCHECK**: Configurable via build args; webapp uses `--retries=84 --interval=5s` = 420s (7 min), aligned with `wait-for-stack` step timeout. +- **Shared entrypoints**: Each service uses a unified entrypoint script (`containers/-entrypoint.sh`) that branches on `CI=true` env var for CI vs local dev paths. Common helpers in `containers/entrypoint-common.sh`. +- **Shared bootstrap**: `scripts/bootstrap-common.sh` contains shared contract deployment, seeding, and funding functions used by both `containers/bootstrap.sh` (local dev) and `.woodpecker/e2e.yml` (CI). +- CI images are tagged with git SHA and `latest`, pushed to a local registry. + +## CI Agent & Registry Auth +- **Agent**: Runs as user `ci` (uid 1001) on `harb-staging`, same host as the dev environment. Binary at `/usr/local/bin/woodpecker-agent`. +- **Registry credentials**: The `ci` user must have Docker auth configured at `/home/ci/.docker/config.json` to pull private images from `registry.niovi.voyage`. If images fail to pull with "no basic auth credentials", fix with: + ```bash + sudo mkdir -p /home/ci/.docker + sudo cp /home/debian/.docker/config.json /home/ci/.docker/config.json + sudo chown -R ci:ci /home/ci/.docker + sudo chmod 600 /home/ci/.docker/config.json + ``` +- **Shared Docker daemon**: The `ci` and `debian` users share the same Docker daemon. Running `docker system prune` as `debian` removes images cached for CI pipelines. If CI image pulls fail after a prune, either fix registry auth (above) or pre-pull images as `debian`: `docker pull registry.niovi.voyage/harb/ponder-ci:latest` etc. + +## Debugging Tips +- If pipelines aren't being created after a push, check Codeberg webhook delivery logs first. +- The Woodpecker server needs `sudo` to restart. Without it, you cannot: refresh API tokens, clear cached state, or recover from webhook auth issues. +- E2E pipeline failures often come from `wait-for-stack` timing out. Check the webapp HEALTHCHECK alignment and Ponder indexing time. +- The `web-app/vite.config.ts` `allowedHosts` array must include container hostnames (`webapp`, `caddy`) for health checks to succeed inside Docker networks. +- **Never use `bash -lc`** in Woodpecker pipeline commands — login shell resets PATH via `/etc/profile`, losing Foundry and other tools set by Docker ENV. Use `bash -c` instead. diff --git a/docs/codeberg-api.md b/docs/codeberg-api.md new file mode 100644 index 0000000..e4a0829 --- /dev/null +++ b/docs/codeberg-api.md @@ -0,0 +1,32 @@ +# Codeberg API Access + +## Authentication +Codeberg API tokens are stored in `~/.netrc` (standard `curl --netrc` format, `chmod 600`): +``` +machine codeberg.org +login johba +password +``` +The `password` field holds the API token — this is standard `.netrc` convention, not an actual password. + +## Generating Tokens +Generate tokens at `https://codeberg.org/user/settings/applications`. + +## Usage +Pass `--netrc` to curl for authenticated Codeberg API calls: +```bash +# List issues +curl --netrc -s https://codeberg.org/api/v1/repos/johba/harb/issues | jq '.[0].title' + +# Get a specific issue +curl --netrc -s https://codeberg.org/api/v1/repos/johba/harb/issues/42 | jq '.title, .body' + +# List pull requests +curl --netrc -s https://codeberg.org/api/v1/repos/johba/harb/pulls | jq '.[].title' +``` + +## Git vs API +The repo uses SSH for git push/pull (`ssh://git@codeberg.org`), so `.netrc` is only used for REST API interactions (issues, PRs, releases). + +## Webhooks +Codeberg sends webhooks to `https://ci.sovraigns.network/api/hook` to trigger Woodpecker CI pipelines. If webhooks stop firing (e.g. DNS issues), check Codeberg repo settings → Webhooks to verify delivery history and re-trigger. diff --git a/docs/dev-environment.md b/docs/dev-environment.md new file mode 100644 index 0000000..db2c001 --- /dev/null +++ b/docs/dev-environment.md @@ -0,0 +1,67 @@ +# Dev Environment + +## Prerequisites +Docker Engine (Linux) or Colima (Mac). See `docs/docker.md` for installation. + +## Quick Start +```bash +nohup ./scripts/dev.sh start & # start (takes ~3-6 min first time) +tail -f nohup.out # watch progress +./scripts/dev.sh health # verify all services healthy +./scripts/dev.sh stop # stop and clean up +``` +Do not launch services individually — `dev.sh` enforces phased startup with health gates. + +## Restart Modes +- `./scripts/dev.sh restart --light` — Fast (~10-20s): only webapp + txnbot, preserves Anvil/Ponder state. Use for frontend changes. +- `./scripts/dev.sh restart --full` — Full (~3-6min): redeploys contracts, fresh state. Use for contract changes. + +## Environments +Supported: `BASE_SEPOLIA_LOCAL_FORK` (default Anvil fork), `BASE_SEPOLIA`, and `BASE`. Match contract addresses and RPCs accordingly. + +## Ports +The stack uses these ports: +| Port | Service | +|-------|------------| +| 8545 | Anvil | +| 5173 | webapp | +| 5174 | landing | +| 42069 | Ponder | +| 43069 | txnBot | +| 5100 | Otterscan | +| 8081 | Caddy | + +Check with `lsof -i :` if startup fails. + +## Docker Topology +- `docker-compose.yml` has NO `depends_on`. Service ordering is handled by `scripts/dev.sh`. +- Startup phases: anvil+postgres → bootstrap → ponder → webapp/landing/txn-bot → caddy → smoke test. +- Shared bootstrap: `scripts/bootstrap-common.sh` (sourced by both local dev and CI). +- 20GB disk limit enforced. `dev.sh stop` auto-prunes. Named volumes for `node_modules`. +- All services have log rotation (30MB max per container) and PostgreSQL WAL limits configured. + +## kraiken-lib Build +- Run `./scripts/build-kraiken-lib.sh` before `docker-compose up` so containers mount a fresh `kraiken-lib/dist` from the host. +- `scripts/watch-kraiken-lib.sh` rebuilds on file changes (requires inotify-tools) and restarts dependent containers automatically. +- The dev script runs the build automatically on `start`, but manual rebuilds are needed if you change kraiken-lib while the stack is already running. + +## Common Pitfalls +- **Docker disk full**: `dev.sh start` refuses to run if Docker disk usage exceeds 20GB. Fix: `./scripts/dev.sh stop` (auto-prunes) or `docker system prune -af --volumes`. +- **Stale Ponder state**: If Ponder fails with schema errors after contract changes, delete its state: `rm -rf services/ponder/.ponder/` then `./scripts/dev.sh restart --full`. +- **kraiken-lib out of date**: If services fail with import errors or missing exports, rebuild: `./scripts/build-kraiken-lib.sh`. +- **Container not found errors**: `dev.sh` expects Docker Compose v2 container names (`harb-anvil-1`, hyphens not underscores). Verify with `docker compose version`. +- **Port conflicts**: See Ports table above. Check with `lsof -i :` if startup fails. +- **npm ci failures in containers**: Named Docker volumes cache `node_modules/`. If dependencies change and installs fail, remove the volume: `docker volume rm harb_webapp_node_modules` (or similar), then restart. + +## Handy Commands +```bash +foundryup # update Foundry toolchain +anvil --fork-url https://sepolia.base.org # manual fork for diagnosing outside dev.sh + +# inspect services while stack is running +curl http://localhost:8081/api/txn/status +curl -X POST http://localhost:8081/api/graphql \ + -d '{"query":"{ stats(id:\"0x01\"){kraikenTotalSupply}}"}' +cast call "slot0()" # inspect pool state +PONDER_NETWORK=BASE_SEPOLIA_LOCAL_FORK npm run dev # focused Ponder debugging (inside services/ponder/) +``` diff --git a/docs/testing.md b/docs/testing.md new file mode 100644 index 0000000..b0c464c --- /dev/null +++ b/docs/testing.md @@ -0,0 +1,41 @@ +# Testing + +## Contract Tests (Foundry) +Run inside `onchain/`: +```bash +forge build # compile contracts +forge test # run unit + fork tests +forge snapshot # gas snapshot +``` + +## Fuzzing +Scripts under `onchain/analysis/` generate replayable scenarios: +```bash +./analysis/run-fuzzing.sh [optimizer] debugCSV +``` + +## Integration Testing +After the stack boots via `dev.sh`: +- Anvil logs: check for revert errors +- Ponder GraphQL: `http://localhost:8081/api/graphql` +- txnBot health: `http://localhost:8081/api/txn/status` + +## E2E Tests (Playwright) +Full-stack tests in `tests/e2e/` verify complete user journeys (mint ETH → swap KRK → stake). + +```bash +npm run test:e2e # from repo root +``` + +- Tests use a mocked wallet provider with Anvil accounts. +- In CI, the Woodpecker `e2e.yml` pipeline runs these against pre-built service images. +- See [docs/ci-pipeline.md](ci-pipeline.md) for CI-specific E2E details. + +## Version Validation System +The stack enforces version compatibility across contracts, indexer, and frontend: + +- **Contract VERSION**: `Kraiken.sol` exposes a `VERSION` constant (currently v2) that must be incremented for breaking changes to TAX_RATES, events, or core data structures. +- **Ponder Validation**: On startup, Ponder reads the contract VERSION and validates against `COMPATIBLE_CONTRACT_VERSIONS` in `kraiken-lib/src/version.ts`. Fails hard (exit 1) on mismatch to prevent indexing wrong data. +- **Frontend Check**: Web-app validates `KRAIKEN_LIB_VERSION` at runtime (currently placeholder; future: query Ponder GraphQL for full 3-way validation). +- **CI Enforcement**: Woodpecker `release.yml` pipeline validates that contract VERSION matches `COMPATIBLE_CONTRACT_VERSIONS` before release. +- See `VERSION_VALIDATION.md` (repo root) for complete architecture, workflows, and troubleshooting. diff --git a/onchain/.claude-code-supervisor.yml b/onchain/.claude-code-supervisor.yml index a4f7ee1..d1e9922 100644 --- a/onchain/.claude-code-supervisor.yml +++ b/onchain/.claude-code-supervisor.yml @@ -1,6 +1,6 @@ triage: command: "claude -p --no-session-persistence" - model: "claude-haiku-4-20250414" + model: "claude-haiku-4-5-20251001" notify: command: "/tmp/supervisor-notify.sh"