- Add fitness_flags="token_value_inflation" to evo_run004_champion in
manifest.jsonl so callers can detect the inflated value without
discarding the entry entirely.
- Add effective_fitness() helper in evolve.sh pool admission (step 5)
that returns 0 for any entry with a token_value_inflation flag,
preventing inflated scores from biasing the top-100 evolved pool
ranking or eviction decisions.
- Document in evolve.sh that raw fitness values are only comparable
within the same evaluation run.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fixes#667
## Changes
## Summary
Implemented persistent top-100 candidate pool in `tools/push3-evolution/evolve.sh`:
### Changes
**`--run-id <N>` flag** (line 96)
- Optional integer; auto-increments from highest `run` field in `manifest.jsonl` when omitted
- Zero-padded to 3 digits (`001`, `002`, …)
**Seeds pool constants** (after path canonicalization)
- `SEEDS_DIR` → `$SCRIPT_DIR/seeds/`
- `POOL_MANIFEST` → `seeds/manifest.jsonl`
- `ADMISSION_THRESHOLD` → `6000000000000000000000` (6e21 wei)
**`--diverse-seeds` mode** now has two paths:
1. **Pool mode** (pool non-empty): random-shuffles the pool and takes up to `POPULATION` candidates — real evolved diversity, not parametric clones
2. **Fallback** (pool empty): original `seed-gen-cli` parametric variant behavior
- Both paths fall back to mutating `--seed` to fill any shortfall
**Step 5 — End-of-run admission** (after the diff step):
1. Scans all `generation_*.jsonl` in `OUTPUT_DIR` for candidates with `fitness ≥ 6e21`
2. Maps `candidate_id` (e.g. `gen2_c005`) back to `.push3` files in `WORK_DIR` (still exists since cleanup fires on EXIT)
3. Deduplicates by SHA-256 content hash against existing pool
4. Names new files `run{RUN_ID}_gen{N}_c{MMM}.push3`
5. Merges with existing pool, sorts by fitness descending, keeps top 100
6. Copies admitted files to `seeds/`, removes evicted evolved files (never hand-written), rewrites `manifest.jsonl`
Co-authored-by: openhands <openhands@all-hands.dev>
Reviewed-on: https://codeberg.org/johba/harb/pulls/683
Reviewed-by: review_bot <review_bot@noreply.codeberg.org>
## Three bugs in evolve.sh
1. **Heredoc stdin conflict** — `py_stats()` used `<<PYEOF` heredoc which stole stdin from the pipe, so python never received score values → stats always `min=0 max=0 mean=0`
2. **Bash integer overflow** — global best comparison used `[ $MAX -gt $GLOBAL_BEST_FITNESS ]` which overflows on uint256 wei values (>9.2e18) → best always tracked as 0
3. **candidate_id mismatch** — evolve.sh looked up `gen0_c000` but batch-eval produces `candidate_000` (derived from filename) → score lookup always returned default 0
All 3 previous evolution runs (150+ candidates) reported all zeros despite batch-eval correctly scoring them at ~8.26e21 wei.
## Fix
- `py_stats`: heredoc → `python3 -c` inline
- Global best: bash `[ -gt ]` → `python3` big number comparison
- Score lookup: use `basename $CAND_FILE` instead of synthetic CID
Co-authored-by: root <root@debian-g-2vcpu-8gb-ams3-01>
Reviewed-on: https://codeberg.org/johba/harb/pulls/665
Reviewed-by: review_bot <review_bot@noreply.codeberg.org>
- evolve.sh: fix fail-in-subshell bug — run seed-gen-cli as a direct
command so its exit code is checked by the parent shell and fail()
aborts the script correctly; redirect stderr to log file instead of
discarding it with 2>/dev/null
- seed-generator.ts: reorder enumerateVariants() to put
STAKED_THRESHOLDS outermost (192 entries/block) so that
selectVariants(6) with stride=192 covers all 6 staked% thresholds;
remove false doc claim about "first variant is current seed config";
add comments explaining CI=0n is intentional in all presets
- seed-gen-cli.ts: emit a stderr diagnostic when count exceeds the
1152-variant cap so the cap is visible rather than silently producing
fewer files than requested
- test: strengthen n=6 test to assert all STAKED_THRESHOLDS values are
represented in the selected variants
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add seed-generator.ts module and seed-gen-cli.ts CLI that produce
parametric Push3 variants for initial population seeding.
Variants systematically cover:
- Staked% thresholds: 80, 85, 88, 91, 94, 97
- Penalty thresholds: 30, 50, 70, 100
- Bull params: 4 presets (aggressive → mild)
- Bear params: 4 presets (standard → very mild)
- Tax distributions: exponential (seed), linear, sqrt
Total combination space: 6×4×4×4×3 = 1152 variants.
selectVariants(n) samples evenly so every axis is represented.
evolve.sh gains --diverse-seeds flag: when set, gen_0 is seeded with
parametric variants instead of N copies of the same mutated seed.
Remaining slots (if population > generated variants) fall back to
mutations of the base seed.
All generated programs pass transpiler stack validation (33 new tests).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three defensive layers so every Push3 program runs without reverting:
Layer A (transpiler/index.ts): assign bear defaults (CI=0, AS=0.3e18,
AW=100, DD=0.3e18) to all four outputs at the top of calculateParams.
Any output the evolved program does not overwrite keeps the safe default.
Layer B (transpiler/transpiler.ts): graceful stack underflow — dpop/bpop
return '0'/'false' instead of throwing, and the final output-pop falls
back to bear-default literals when fewer than 4 values remain on the
stack. Wrong output count no longer aborts transpilation.
Layer C (transpiler/transpiler.ts + index.ts): wrap the entire function
body in `unchecked {}` so integer overflow wraps (matching Push3), and
emit `(b == 0 ? 0 : a / b)` for every DYADIC./ (div-by-zero → 0,
matching Push3 no-op semantics).
Layer 2 (Optimizer.sol getLiquidityParams): clamp the three fraction
outputs (capitalInefficiency, anchorShare, discoveryDepth) to [0, 1e18]
after abi.decode so a buggy evolved program cannot produce out-of-range
values even if it runs without reverting.
Regenerated OptimizerV3Push3.sol with the updated transpiler; all 193
tests pass (34 Optimizer/OptimizerV3Push3 tests explicitly).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Optimizer.sol: move CALCULATE_PARAMS_GAS_LIMIT constant to top of
contract (after error declaration) to avoid mid-contract placement.
Expand natspec with EIP-150 63/64 note: callers need ~203 175 gas to
deliver the full 200 000 budget to the inner staticcall.
- Optimizer.sol: add ret.length < 128 guard before abi.decode in
getLiquidityParams(). Malformed return data (truncated / wrong ABI)
from an evolved program now falls back to _bearDefaults() instead of
propagating an unhandled revert. The 128-byte minimum is the ABI
encoding of (uint256, uint256, uint24, uint256) — four 32-byte slots.
- Optimizer.sol: add cross-reference comment to _bearDefaults() noting
that its values must stay in sync with LiquidityManager.recenter()'s
catch block to prevent silent divergence.
- FitnessEvaluator.t.sol: add CALCULATE_PARAMS_GAS_LIMIT mirror constant
(must match Optimizer.sol). Disqualify candidates whose measured gas
exceeds the production cap with fitness=0 and error="gas_over_limit"
— prevents the pipeline from selecting programs that are functionally
dead on-chain (would always produce bear defaults in production).
- batch-eval.sh: update output format comment to document the gas_used
field and over-gas-limit error object added by this feature.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adding SeedSwapper alongside DeployLocal in the same .sol file caused
forge to error "Multiple contracts in the target path" when no --tc flag
was specified, silently failing the CI bootstrap step.
Add --tc DeployLocal to all forge script invocations of DeployLocal.sol:
- scripts/bootstrap-common.sh (CI / local bootstrap)
- tools/deploy-optimizer.sh (manual deploy tool)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Deploy scripts (DeployLocal.sol and DeployBase.sol) now execute a
seed buy + double-recenter sequence before handing control to users:
1. Temporarily grant deployer recenterAccess (via self as feeDestination)
2. Fund LM with a small amount and call recenter() -> places thin positions
3. SeedSwapper executes a small buy, generating a non-zero WETH fee
4. Second recenter() hits the cumulativeVolume==0 bootstrap path with
ethFee>0 -> _recordVolumeAndPrice fires -> cumulativeVolume>0
5. Revoke recenterAccess and restore the real feeDestination
After deployment, cumulativeVolume>0, so the bootstrap path is
unreachable by external users and cannot be front-run by an attacker
inflating the initial VWAP anchor with a whale buy.
Also adds:
- tools/deploy-optimizer.sh: verification step checks cumulativeVolume>0
after a fresh local deployment
- test_vwapBootstrappedBySeedTrade() in VWAPFloorProtection.t.sol:
confirms the deploy sequence (recenter + buy + recenter) leaves
cumulativeVolume>0 and getVWAP()>0
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Skip UUPS upgradeTo: etch + vm.store ERC1967 implementation slot directly
(OptimizerV3Push3 is standalone, no UUPS inheritance needed for evolution)
- Use deployedBytecode (runtime) instead of bytecode (creation) for vm.etch
- Inject transpiled body into OptimizerV3.sol (has getLiquidityParams via Optimizer)
instead of using standalone OptimizerV3Push3.sol
- Wrap buy/sell/stake/unstake in try/catch — attack ops should not abort the batch
- Add /tmp read to fs_permissions for batch-eval manifest files
- Bootstrap recenter returns bool instead of reverting (soft-fail per candidate)
Reviewer noted that `< 4` only catches underflow; programs leaving 5+
values on the DYADIC stack silently passed isValid(). Change the guard
to `!== 4` so both under- and overflow are rejected, matching the
documented 'exactly 4 outputs' contract.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace silent ?? '0' fallbacks with an explicit length check that
throws when the DYADIC stack holds fewer than 4 values at program
termination. isValid() in the evolution pipeline now correctly
rejects underflow programs instead of silently scoring them as valid
with zeroed outputs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace per-candidate Anvil+forge-script pipeline with in-process EVM
execution using Foundry's native revm backend, achieving 10-100× speedup
for evolutionary search at scale.
New files:
- onchain/test/FitnessEvaluator.t.sol — Forge test that forks Base once,
deploys the full KRAIKEN stack, then for each candidate uses vm.etch to
inject the compiled optimizer bytecode, UUPS-upgrades the proxy, runs all
attack sequences with in-memory vm.snapshot/revertTo (no RPC overhead),
and emits one {"candidate_id","fitness"} JSON line per candidate.
Skips gracefully when BASE_RPC_URL is unset (CI-safe).
- tools/push3-evolution/revm-evaluator/batch-eval.sh — Wrapper that
transpiles+compiles each candidate sequentially, writes a two-file
manifest (ids.txt + bytecodes.txt), then invokes FitnessEvaluator.t.sol
in a single forge test run and parses the score JSON from stdout.
Modified:
- tools/push3-evolution/evolve.sh — Adds EVAL_MODE env var (anvil|revm).
When EVAL_MODE=revm, batch-scores every candidate in a generation with
one batch-eval.sh call instead of N sequential fitness.sh processes;
scores are looked up from the JSONL output in the per-candidate loop.
Default remains EVAL_MODE=anvil for backward compatibility.
Key design decisions:
- Per-candidate Solidity compilation is unavoidable (each Push3 candidate
produces different Solidity); the speedup is in the evaluation phase.
- vm.snapshot/revertTo in forge test are O(1) memory operations (true
revm), not RPC calls — this is the core speedup vs Anvil.
- recenterAccess is set in bootstrap so TWAP stability checks are bypassed
during attack sequences (mirrors the existing fitness.sh bootstrap).
- Test skips cleanly when BASE_RPC_URL is absent, keeping CI green.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add virtual to Optimizer.calculateParams() for UUPS override
- Create OptimizerV3.sol: UUPS-upgradeable optimizer with transpiled Push3 logic
- Update deploy-optimizer.sh to deploy OptimizerV3 instead of Optimizer
- Add ~/.foundry/bin to PATH in evolve.sh, fitness.sh, deploy-optimizer.sh
Address round-2 review findings:
- Move BASELINE_SNAP before deploy-optimizer.sh so cleanup fully reverts the
deploy on a shared Anvil; fixes nonce/address collision when a second
sequential evaluation reuses the same chain
- Revert deploy output to capture-and-suppress on success / surface on failure;
removes per-candidate stderr noise in evolution loop batch runs
- Fix cast rpc anvil_mine arg order to match all other cast rpc calls in script
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Address review findings:
- Bug: add BASELINE_SNAP before bootstrap; cleanup reverts it on shared Anvil
to undo setRecenterAccess/WETH-funding/recenter mutations (was dead code before)
- Bug: require ANVIL_FORK_URL when cold-starting Anvil — DeployLocal.sol needs
live Base contracts (Uniswap V3 Factory, WETH) that don't exist on a plain fork
- Warning: flag DIRTY and emit warning when anvil_revert fails instead of || true
- Warning: tee deploy-optimizer.sh output to both log file and stderr so progress
is visible and preserved for post-failure diagnosis
- Nit: replace 50×evm_mine loop with single anvil_mine 0x32 (49 fewer RTTs)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements the five Push3 mutation operators and the meta-operator for
the optimizer evolution pipeline:
- mutateConstant: shifts a random integer literal by ±δ (clamped to 0)
- swapOperator: swaps ADD↔SUB, MUL↔DIV, GT↔LT, GTE↔LTE
- deleteInstruction: removes a random non-EXEC.IF instr; validates result
- insertInstruction: inserts stack-neutral pair (push 0 + DYADIC.POP)
- crossover: single-point crossover of two programs at instruction boundaries
- mutate: applies N random mutations from the four single-program operators
All mutations validate output via transpile() symbolic stack simulation.
Invalid mutations silently return the original program.
35 unit tests cover all operators, edge cases (empty program, single
instruction, deep stack), and the acceptance criterion that
mutate(optimizer_v3, 3) produces ≥10 distinct valid variants in 20 trials.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
