Add formulas/AGENTS.md documenting sense vs act type distinction,
cron conventions, step ID naming rules, TOML structure skeleton,
and a how-to-add-a-new-formula walkthrough.
Add scripts/harb-evaluator/AGENTS.md covering the evaluator runtime:
directory layout, exit code convention, stack lifecycle, evidence
output, and how to add a new evaluator script.
Update root AGENTS.md directory map to link both new files.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- #864: Add comment documenting that MEMORY_FILE and REPORT_DIR both
resolve to $REPO_ROOT/tmp (intentional coupling, previously undocumented)
- #579: POOL die guard already present (added in a2f8996, issue #854)
- #775: feeDest address already corrected (fixed in 0e33d6c, issue #760)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implement the attack catalogue loop (step 5a) in red-team.sh that was
previously a forward spec in the formula. The loop replays every *.jsonl
attack file through AttackRunner.s.sol with snapshot revert between files,
records LM total ETH before/after each attack, and injects results into
the adversarial agent prompt so it knows which strategies are already
catalogued.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add CANDIDATE env var support to bootstrap-light.sh. When set to a
.push3 file path, the script:
1. Invokes push3-transpiler to regenerate OptimizerV3Push3.sol
2. Extracts the function body into OptimizerV3Push3Lib.sol
3. Deploys contracts normally via DeployLocal.sol
4. Deploys OptimizerV3 and upgrades the UUPS proxy via upgradeTo()
Also updates formulas/run-red-team.toml to reflect the implementation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add infrastructure.weth to deployments-local.json output in both
bootstrap-common.sh (write_deployments_json) and bootstrap-light.sh,
so non-Base local forks get the correct WETH address from the
deployment file instead of silently falling back to the Base hardcode.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add scripts/harb-evaluator/run-resources.sh: collects disk, RAM,
Anthropic API usage, and Woodpecker CI queue metrics
- Add scripts/harb-evaluator/run-protocol.sh: collects TVL, fees,
position data, and rebalance events from LiquidityManager
- Fix run-protocol.toml: positions accessed via positions(uint8) not
named getters (floorPosition/anchorPosition/discoveryPosition)
- Fix event signature: Recentered(int24,bool) not Recenter(int24,int24,int24)
Addresses review findings: missing implementation files and contract
interface mismatch.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fixes#1066
## Changes
Done. Here's what was changed:
**`evidence/README.md`**
- Added `"candidate_commit": "abc1234"` to the red-team schema JSON example
- Added `candidate_commit | string | Git commit SHA of the optimizer under test` row to the field table
**`scripts/harb-evaluator/red-team.sh`**
- Captures `CANDIDATE_COMMIT` from `git rev-parse HEAD` at startup (alongside existing `CANDIDATE_NAME`/`OPTIMIZER_PROFILE`)
- Added a new step (9a-pre) that writes `evidence/red-team/YYYY-MM-DD.json` at the end of each run, including `candidate_commit` plus all other schema fields (`candidate`, `optimizer_profile`, `lm_eth_before`, `lm_eth_after`, `eth_extracted`, `floor_held`, `verdict`, `attacks`)
Co-authored-by: openhands <openhands@all-hands.dev>
Reviewed-on: https://codeberg.org/johba/harb/pulls/1075
Reviewed-by: Disinto_bot <disinto_bot@noreply.codeberg.org>
- cleanup_worktree: add `git branch -D $BRANCH` to prevent stale local
branch refs accumulating on push failure (bug fix)
- .netrc parser: replace fragile line-count awk with field-iteration
approach that handles both multi-line and single-line .netrc formats
- ETH formatting: pass values as argv to python3 instead of interpolating
into the code string, removing the injection surface
- mktemp -u: generate path without pre-creating directory; git worktree
add creates it, avoiding the "already exists" error on some git versions
- mkdir -p guard before cp to attacks destination directory
- sed portability: `s/-\+/-/g` → `s/--*/-/g` (POSIX-compliant)
- red-team.sh: capture PIPESTATUS[0] from promote-attacks pipe and emit
a distinct warning log line when promotion fails
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add scripts/harb-evaluator/promote-attacks.sh which:
- Reads tmp/red-team-attacks.jsonl after a successful red-team run
- Deduplicates by op-type fingerprint against all existing attack files
- Classifies attack type (staking, il-crystallization, fee-drain-oscillation,
floor-ratchet, lp-manipulation, floor-attack) from the op sequence
- Creates an isolated git worktree branch from origin/master
- Commits the attack file to onchain/script/backtesting/attacks/<type>-<candidate>.jsonl
- Opens a Codeberg PR with attack type, ETH extracted, and optimizer profile
Integrate into red-team.sh: when the floor breaks (ETH extracted) and an
attack export exists, promote-attacks.sh is called automatically (non-fatal).
Gracefully no-ops when CODEBERG_TOKEN / ~/.netrc are absent.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
red-team.sh produces the stream JSONL that export-attacks.py parses, so
they must agree on addresses. Update SWAP_ROUTER and NPM in red-team.sh
to Base Sepolia and fix the invariant comment in export-attacks.py.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Update SWAP_ROUTER_ADDR and NPM_ADDR in export-attacks.py from Base
mainnet addresses to the correct Base Sepolia addresses, matching
helpers/market.ts and helpers/swap.ts.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Address AI reviewer feedback on d1f75a7:
- Wrap cross_file append in try/except so a write failure never prevents
the memory trim-write from running (bug fix)
- Stamp sweep_id on pre-trim exported entries using the SWEEP_ID env var;
pass SWEEP_ID from red-team-sweep.sh so entries are attributable to a
sweep run (data-consistency fix)
- Add inline comment explaining the 3-tuple dedup key (run, ts, strategy)
and its relationship to step-4c's identity check (clarity nit)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Before trimming MEMORY_FILE to 50 entries, export any entries that would
be dropped (non-DECREASED entries outside the last 10) directly to
CROSS_PATTERNS_FILE. This ensures no entries are permanently lost before
red-team-sweep.sh step 4c reads the memory file.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace fixed \`sleep 1\` in the container teardown poll loop with exponential
backoff (100ms → 200ms → … → 2000ms cap). The 30s hard timeout is preserved.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The extract_memory regex previously matched any "lm.?eth" mention,
including mid-execution "Total LM ETH: X wei" output lines produced by
the agent's cast check commands. During a staking step these lines
reflect an intermediate chain state (ETH temporarily locked/moved)
rather than the final reverted state, causing strategies to be recorded
as DECREASED even when the runner confirmed ETH_SAFE.
Fix: narrow the capture to the structured `lm_eth_after: <value>`
label that the agent writes in its final RED-TEAM REPORT block.
Mid-execution total-ETH lines no longer match and cannot corrupt the
per-strategy result in memory or the cross-patterns file.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Move CROSS_PATTERNS_FILE from /tmp/red-team-cross-patterns.jsonl to
tools/red-team/cross-patterns.jsonl (repo-tracked path)
- Remove the reset (> file) at sweep start so patterns accumulate across runs
- Generate a SWEEP_ID (sweep-YYYYMMDD-HHMMSS) at sweep start and stamp
each new entry with sweep_id for traceability
- Deduplicate on (pattern, candidate, result): entries already present in
the file are skipped; intra-batch duplicates are also suppressed
- Create tools/red-team/ directory with .gitkeep
- Add mkdir -p guards in both scripts so the directory is created on first run
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
bootstrap-light.sh now extracts the Uniswap V3 pool address from
DeployLocal.sol deploy output and writes both Pool and V3Factory
(Base Sepolia: 0x4752ba5DBc23f44D87826276BF6Fd6b1C372aD24) into
deployments-local.json alongside the existing contract addresses.
red-team.sh now reads V3_FACTORY and POOL from deployments-local.json
instead of hardcoding the Base mainnet factory address
(0x33128a8fC17869897dcE68Ed026d694621f6FDfD), and removes the getPool()
RPC call that always failed with "contract does not have any code" on
the Sepolia fork.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Document MIN_RECENTER_INTERVAL (60 s, LiquidityManager.sol:61) and
PRICE_STABILITY_INTERVAL (300 s, PriceOracle.sol:14) in
docs/ARCHITECTURE.md and docs/PRODUCT-TRUTH.md so that agent-facing
and product-facing copy stays traceable to source constants.
Add an inline HTML comment in red-team-program.md next to the
hardcoded 60s/300s sentence pointing to the two source constants,
making drift detectable during code review.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Inject Kraiken.sol (outstandingSupply, mint/burn mechanics) and Stake.sol
(snatch, withdrawal, KRK exclusion from floor denominator) into the red-team
agent prompt so agents can reason from actual source rather than guesses.
- red-team.sh: read SOL_KRAIKEN and SOL_STAKE from onchain/src/ alongside
the other six contracts already injected
- red-team-program.md: add ### Kraiken.sol and ### Stake.sol sections in the
Source Code reference block (after PriceOracle.sol)
- AGENTS.md: document the full list of injected contracts in a new
"Red-team Agent Context" section; both files are now listed as in-scope
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- red-team-sweep.sh: reset CROSS_PATTERNS_FILE at sweep start to prevent
stale patterns from prior invocations contaminating a fresh run
- red-team-sweep.sh: wrap pattern-extraction Python in set +e/set -e and
capture output so log() prefix is applied; move memory truncation outside
the if-block so it runs unconditionally even if Python fails
- red-team.sh: filter entries where candidate == current_candidate before
grouping, removing self-referential cross-candidate evidence
- red-team.sh: skip entries with empty pattern key (both pattern and
strategy fields empty) to prevent spurious bucket merging
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- red-team-sweep.sh: after each candidate completes, extract all memory
entries into /tmp/red-team-cross-patterns.jsonl (append), then clear
the raw memory file so the next candidate starts with a fresh state
- red-team.sh: define CROSS_PATTERNS_FILE; before building the prompt,
read the cross-patterns file and generate a "Cross-Candidate
Intelligence" section grouped by abstract op pattern — universal
patterns (broke 2+ candidates), candidate-specific wins, and patterns
that held everywhere — each annotated with optimizer profiles
- The new section is injected into the Claude prompt above the existing
Previous Findings block, satisfying all acceptance criteria
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- make_pattern: replace text.find('stake')/find('unstake') with
re.search(r'\bstake\b')/re.search(r'\bunstake\b') so 'stake' is never
found as a substring of 'unstake' (bug #1)
- make_pattern: track first-occurrence position of each op and sort by
position before building the sequence string, preserving actual
execution order instead of a hardcoded canonical order (bug #2)
- insight capture: track insight_pri on the current dict; only overwrite
stored insight when new match has strictly higher priority (lower index),
preventing a late 'because...' clause from silently replacing an earlier
'Key Insight:' capture (warning #3)
- run_num: compute max(run)+1 from JSON entries instead of wc -l so run
numbers stay monotonically increasing after memory trim (info #4)
- red-team-sweep.sh: also set adaptive flag when any r37-r40 register has
a variable-form assignment (r40 = uint256(someVar)), catching candidates
where only one branch uses constants (warning #5)
- red-team-sweep.sh: remove unnecessary 'import sys as _sys' in except
block; sys is already in scope (nit #6)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add CANDIDATE_NAME and OPTIMIZER_PROFILE env vars to red-team.sh
(defaults to "unknown" for standalone runs)
- Update extract_memory Python: new fields candidate, optimizer_profile,
pattern (abstract op sequence via make_pattern()), and improved insight
extraction that also captures WHY explanations (because/since/due to)
- Update MEMORY_SECTION Python: entries now grouped by candidate;
universal patterns (DECREASED across multiple candidates) surfaced first
- Update prompt: add "Current Attack Target" table with candidate/profile,
optimizer parameter explanations (CI/AW/AS/DD behavioral impact),
Rule 9 requiring pattern+insight per strategy, updated report format
with Pattern/Insight fields and universal-pattern conclusion field
- Update red-team-sweep.sh: after inject, parse OptimizerV3Push3.sol for
r40/r39/r38/r37 constants to build OPTIMIZER_PROFILE string; pass
CANDIDATE_NAME and OPTIMIZER_PROFILE as env vars to red-team.sh
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
## What
- `tools/push3-transpiler/inject.sh` — shared transpile+inject logic used by both batch-eval and red-team-sweep
- `batch-eval.sh` — replaced inline 60-line Python block with `inject.sh` call
- `scripts/harb-evaluator/red-team-sweep.sh` — red-teams each kindergarten seed using existing `red-team.sh`, with random smoke test gate
## Why
Sweep script kept breaking because I rewrote the injection logic instead of reusing batch-eval's proven Python. Now there's one copy.
## Testing
- inject.sh tested manually on DO box with optimizer_v3 seed
- Smoke test picks random seed, injects + compiles before starting sweep
Co-authored-by: openhands <openhands@all-hands.dev>
Reviewed-on: https://codeberg.org/johba/harb/pulls/806
Reviewed-by: review_bot <review_bot@noreply.codeberg.org>
red-team.sh called bare `sudo docker compose up/down` which applies
env_reset and drops FORK_URL before anvil-entrypoint.sh can read it.
Change both calls to `sudo -E` so the caller's FORK_URL override is
propagated to docker-compose and into the anvil container.
Update ENVIRONMENT.md to reflect that a plain `FORK_URL=... bash
red-team.sh` invocation now works correctly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
