harb/AGENTS.md

151 lines
13 KiB
Markdown
Raw Normal View History

2025-09-23 14:18:04 +02:00
# Agent Brief: Harb Stack
2025-09-24 09:57:20 +02:00
## Core Concepts
- KRAIKEN couples Harberger-tax staking with a dominant Uniswap V3 liquidity manager to create asymmetric slippage, sentiment-driven pricing, and VWAP "price memory" safeguards.
- Liquidity dominance is mission-critical; treat any regression that weakens the LiquidityManager's control as a priority incident.
- Harberger staking supplies the sentiment oracle that drives Optimizer parameters, which in turn tune liquidity placement and supply expansion.
## User Journey
1. **Buy** - Acquire KRAIKEN on Uniswap.
2. **Stake** - Declare a tax rate on kraiken.org to earn from protocol growth.
3. **Compete** - Snatch undervalued positions to optimise returns.
## Operating the Stack
- Start everything with `nohup ./scripts/dev.sh start &` and stop via `./scripts/dev.sh stop`. Do not launch services individually.
- **Restart modes** for faster iteration:
- `./scripts/dev.sh restart --light` - Fast restart (~10-20s): only webapp + txnbot, preserves Anvil/Ponder state. Use for frontend changes.
- `./scripts/dev.sh restart --full` - Full restart (~3-4min): redeploys contracts, fresh state. Use for contract changes.
2025-09-24 09:57:20 +02:00
- Supported environments: `BASE_SEPOLIA_LOCAL_FORK` (default Anvil fork), `BASE_SEPOLIA`, and `BASE`. Match contract addresses and RPCs accordingly.
- The stack uses Docker containers orchestrated via docker-compose. The script boots Anvil, deploys contracts, seeds liquidity, starts Ponder, launches the landing site, and runs the txnBot. Wait for logs to settle before manual testing.
- **Prerequisites**: Docker Engine (Linux) or Colima (Mac). See installation instructions below.
2025-09-24 09:57:20 +02:00
## Component Guides
- `onchain/` - Solidity + Foundry contracts, deploy scripts, and fuzzing helpers ([details](onchain/AGENTS.md)).
- `services/ponder/` - Ponder indexer powering the GraphQL API ([details](services/ponder/AGENTS.md)).
- `landing/` - Vue 3 marketing + staking interface ([details](landing/AGENTS.md)).
- `kraiken-lib/` - Shared TypeScript helpers for clients and bots ([details](kraiken-lib/AGENTS.md)).
- `services/txnBot/` - Automation bot for `recenter()` and `payTax()` upkeep ([details](services/txnBot/AGENTS.md)).
## Testing & Tooling
- Contracts: run `forge build`, `forge test`, and `forge snapshot` inside `onchain/`.
- Fuzzing: scripts under `onchain/analysis/` (e.g., `./analysis/run-fuzzing.sh [optimizer] debugCSV`) generate replayable scenarios.
2025-10-11 10:55:49 +00:00
- Integration: after the stack boots, inspect Anvil logs, hit `http://localhost:8081/api/graphql` for Ponder, and poll `http://localhost:8081/api/txn/status` for txnBot health.
- **E2E Tests**: Playwright-based full-stack tests in `tests/e2e/` verify complete user journeys (mint ETH → swap KRK → stake). Run with `npm run test:e2e` from repo root. Tests use mocked wallet provider with Anvil accounts and automatically start/stop the stack. See `INTEGRATION_TEST_STATUS.md` and `SWAP_VERIFICATION.md` for details.
2025-09-24 09:57:20 +02:00
## Version Validation System
- **Contract VERSION**: `Kraiken.sol` exposes a `VERSION` constant (currently v1) that must be incremented for breaking changes to TAX_RATES, events, or core data structures.
- **Ponder Validation**: On startup, Ponder reads the contract VERSION and validates against `COMPATIBLE_CONTRACT_VERSIONS` in `kraiken-lib/src/version.ts`. Fails hard (exit 1) on mismatch to prevent indexing wrong data.
- **Frontend Check**: Web-app validates `KRAIKEN_LIB_VERSION` at runtime (currently placeholder; future: query Ponder GraphQL for full 3-way validation).
- **CI Enforcement**: GitHub workflow validates that contract VERSION is in `COMPATIBLE_CONTRACT_VERSIONS` before merging PRs.
- See `VERSION_VALIDATION.md` for complete architecture, workflows, and troubleshooting.
## Docker Installation & Setup
- **Linux**: Install Docker Engine via package manager or `curl -fsSL https://get.docker.com | sh`, then add user to docker group: `sudo usermod -aG docker $USER` (logout/login required)
- **Mac**: Use Colima (open-source Docker Desktop alternative):
```bash
brew install colima docker docker-compose
colima start --cpu 4 --memory 8 --disk 100
docker ps # verify installation
```
- **Container Orchestration**: `docker-compose.yml` has NO `depends_on` declarations. All service ordering is handled in `scripts/dev.sh` via phased startup with explicit health checks.
- **Startup Phases**: (1) Create all containers, (2) Start anvil+postgres and wait for healthy, (3) Start bootstrap and wait for completion, (4) Start ponder and wait for healthy, (5) Start webapp/landing/txn-bot, (6) Start caddy.
- **Logging Configuration**: All services have log rotation configured (max 10MB per file, 3 files max = 30MB per container) to prevent disk bloat. Logs are automatically rotated by Docker.
- **Disk Management** (Portable, No Per-Machine Setup Required):
- **20GB Hard Limit**: The stack enforces a 20GB total Docker disk usage limit (images + containers + volumes + build cache).
- **Pre-flight Checks**: `./scripts/dev.sh start` checks Docker disk usage before starting and refuses to start if over 20GB.
- **Aggressive Auto-Cleanup on Stop**: `./scripts/dev.sh stop` automatically prunes ALL unused Docker resources including build cache (the primary cause of bloat).
- **Named Volumes for node_modules**: All Node.js services (ponder, webapp, landing, txnBot) use named Docker volumes for `node_modules/` instead of writing to the host filesystem. This prevents host pollution (20-30GB savings) and ensures `docker system prune --volumes` cleans them up.
- **npm Best Practices**: All entrypoints use `npm ci` (not `npm install`) for reproducible builds and `npm cache clean --force` to remove ~50-100MB of cache per service.
- **PostgreSQL WAL Limits**: Postgres configured with `wal_level=minimal`, `max_wal_size=128MB`, and `archive_mode=off` to prevent unbounded WAL file growth in the postgres volume.
- **Log Rotation**: All containers limited to 30MB logs (10MB × 3 files) via docker-compose logging configuration.
- **.dockerignore**: Excludes `node_modules/`, caches, and build outputs from Docker build context to speed up builds and reduce image size.
- **Monitoring**: The stack displays current Docker disk usage on startup and warns at 80% (16GB).
- **Note**: Docker has no built-in portable disk quotas. All limits are enforced via aggressive pruning, bounded configurations, and isolation of dependencies to Docker volumes.
2025-09-24 09:57:20 +02:00
## Guardrails & Tips
- `token0isWeth` flips amount semantics; confirm ordering before seeding or interpreting liquidity.
- VWAP, `ethScarcity`, and Optimizer outputs operate on price^2 (X96). Avoid "normalising" to sqrt inadvertently.
- Fund the LiquidityManager with Base WETH (`0x4200...0006`) before expecting `recenter()` to succeed.
- Ponder stores data in `.ponder/`; drop the directory if schema changes break migrations.
- Keep git clean before committing; never leave commented-out code or untested changes.
- **ES Modules**: The entire stack uses ES modules. kraiken-lib, txnBot, Ponder, and web-app all require `"type": "module"` in package.json and use `import` syntax.
- **kraiken-lib Build**: Run `./scripts/build-kraiken-lib.sh` before `docker-compose up` so containers mount a fresh `kraiken-lib/dist` from the host.
- **Live Reload**: `scripts/watch-kraiken-lib.sh` rebuilds on file changes (requires inotify-tools) and restarts dependent containers automatically.
2025-09-24 09:57:20 +02:00
## Code Quality & Git Hooks
- **Pre-commit Hooks**: Husky runs lint-staged on all staged files before commits. Each component (onchain, kraiken-lib, ponder, txnBot, web-app, landing) has `.lintstagedrc.json` configured for ESLint + Prettier.
- **Version Validation (Future)**: Pre-commit hook includes validation logic that will enforce version sync between `onchain/src/Kraiken.sol` (contract VERSION constant) and `kraiken-lib/src/version.ts` (COMPATIBLE_CONTRACT_VERSIONS array). This validation only runs if both files exist and contain version information.
- **Husky Setup**: `.husky/pre-commit` orchestrates all pre-commit checks. Modify this file to add new validation steps.
- To test hooks manually: `git add <files> && .husky/pre-commit`
2025-09-24 09:57:20 +02:00
## Handy Commands
- `foundryup` - update Foundry toolchain.
- `anvil --fork-url https://sepolia.base.org` - manual fork when diagnosing outside the helper script.
- `cast call <POOL> "slot0()"` - inspect pool state.
- `PONDER_NETWORK=BASE_SEPOLIA_LOCAL_FORK npm run dev` (inside `services/ponder/`) - focused indexer debugging when the full stack is already running.
2025-10-11 10:55:49 +00:00
- `curl -X POST http://localhost:8081/api/graphql -d '{"query":"{ stats(id:\"0x01\"){kraikenTotalSupply}}"}'`
- `curl http://localhost:8081/api/txn/status`
2025-09-23 14:18:04 +02:00
## Woodpecker CI
### Infrastructure
- **Server**: Woodpecker 3.10.0 runs as a **systemd service** (`woodpecker-server.service`), NOT a Docker container. Binary at `/usr/local/bin/woodpecker-server`.
- **Host**: `https://ci.sovraigns.network` (port 8000 locally at `http://127.0.0.1:8000`)
- **Forge**: Codeberg (Gitea-compatible) — repo `johba/harb`, forge remote ID `800173`
- **Database**: PostgreSQL at `127.0.0.1:5432`, database `woodpecker`, user `woodpecker`
- **Config**: `/etc/woodpecker/server.env` (contains secrets — agent secret, Gitea OAuth secret, DB credentials)
- **CLI**: Downloaded to `/tmp/woodpecker-cli` (v3.10.0). Requires `WOODPECKER_SERVER` and `WOODPECKER_TOKEN` env vars.
- **Logs**: `journalctl -u woodpecker-server -f` (NOT `docker logs`)
### Pipeline Configs
- `.woodpecker/build-ci-images.yml` — Builds Docker CI images. Triggers on **push** to `master` or `feature/ci` when files in `docker/`, `.woodpecker/`, `kraiken-lib/`, `onchain/out/`, or `web-app/` change.
- `.woodpecker/e2e.yml` — Runs Playwright E2E tests. Triggers on **pull_request** to `master`.
- Pipeline numbering: even = build-ci-images (push events), odd = E2E (pull_request events). This is not guaranteed but was the observed pattern.
### Monitoring Pipelines via DB
Since the Woodpecker API requires authentication (tokens are cached in server memory; DB-only token changes don't work without a server restart), monitor pipelines directly via PostgreSQL:
```bash
# Latest pipelines
PGPASSWORD='<db_password>' psql -h 127.0.0.1 -U woodpecker -d woodpecker -c \
"SELECT number, status, branch, event, commit FROM pipelines
WHERE repo_id = (SELECT id FROM repos WHERE full_name = 'johba/harb')
ORDER BY number DESC LIMIT 5;"
# Step details for a specific pipeline
PGPASSWORD='<db_password>' psql -h 127.0.0.1 -U woodpecker -d woodpecker -c \
"SELECT s.name, s.state,
CASE WHEN s.finished > 0 AND s.started > 0 THEN (s.finished - s.started)::int::text || 's'
ELSE '-' END as duration, s.exit_code
FROM steps s WHERE s.pipeline_id = (
SELECT id FROM pipelines WHERE number = <N>
AND repo_id = (SELECT id FROM repos WHERE full_name = 'johba/harb'))
ORDER BY s.started NULLS LAST;"
```
### Triggering Pipelines
- **Normal flow**: Push to Codeberg → Codeberg fires webhook to `https://ci.sovraigns.network/api/hook` → Woodpecker creates pipeline.
- **Known issue**: Codeberg webhooks can stop firing if `ci.sovraigns.network` becomes unreachable (DNS/connectivity). Check Codeberg repo settings → Webhooks to verify delivery history and re-trigger.
- **Manual trigger via API** (requires valid token — see known issues):
```bash
WOODPECKER_SERVER=http://127.0.0.1:8000 WOODPECKER_TOKEN=<token> \
/tmp/woodpecker-cli pipeline create --branch feature/ci johba/harb
```
- **API auth limitation**: The server caches user token hashes in memory. Inserting a token directly into the DB does not work without restarting the server (`sudo systemctl restart woodpecker-server`).
### CI Docker Images
- `docker/Dockerfile.webapp-ci` — Webapp CI image with Vite dev server.
- **Symlinks fix** (lines 57-59): Creates `/web-app`, `/kraiken-lib`, `/onchain` symlinks to work around Vite's `removeBase()` stripping `/app/` prefix from filesystem paths.
- **CI env detection** (`CI=true`): Disables Vue DevTools plugin in `vite.config.ts` to prevent 500 errors caused by path resolution issues with `/app/` base path.
- **HEALTHCHECK**: `--retries=84 --interval=5s` = 420s (7 min) total wait, aligned with `wait-for-stack` step timeout.
- CI images are tagged with git SHA and `latest`, pushed to a local registry.
### CI Debugging Tips
- If pipelines aren't being created after a push, check Codeberg webhook delivery logs first.
- The Woodpecker server needs `sudo` to restart. Without it, you cannot: refresh API tokens, clear cached state, or recover from webhook auth issues.
- E2E pipeline failures often come from `wait-for-stack` timing out. Check the webapp HEALTHCHECK alignment and Ponder indexing time.
- The `web-app/vite.config.ts` `allowedHosts` array must include container hostnames (`webapp`, `caddy`) for health checks to succeed inside Docker networks.
2025-09-24 09:57:20 +02:00
## References
- Deployment history: `onchain/deployments-local.json`, `onchain/broadcast/`.
- Deep dives: `TECHNICAL_APPENDIX.md`, `HARBERG.md`, and `onchain/UNISWAP_V3_MATH.md`.