Merge pull request 'fix: Red-team schema should document snapshot-isolation methodology for lm_eth fields (#1083)' (#1145) from fix/issue-1083 into master
This commit is contained in:
commit
e1cd283f6a
1 changed files with 30 additions and 0 deletions
|
|
@ -265,6 +265,36 @@ Records one adversarial red-team run against a candidate optimizer.
|
|||
| `attacks[].delta_bps` | integer | LM ETH change in basis points |
|
||||
| `attacks[].insight` | string | Key finding from this strategy |
|
||||
|
||||
### Snapshot-Isolation Methodology
|
||||
|
||||
All red-team runs use **snapshot isolation** as the standard methodology. This
|
||||
ensures that each attack is evaluated independently against the same initial
|
||||
state, rather than against a cumulative balance modified by prior attacks.
|
||||
|
||||
**How it works:**
|
||||
|
||||
1. Before the first attack, the test runner records the initial `lm_eth_before`
|
||||
value and takes an Anvil snapshot via the `anvil_snapshot` RPC method.
|
||||
2. Each attack executes against this snapshot: run the attack, measure
|
||||
`lm_eth_after`, compute `delta_bps`, then revert to the snapshot via
|
||||
the `anvil_revert` RPC method.
|
||||
3. The next attack begins from the exact same chain state as the previous one.
|
||||
|
||||
**Field semantics under snapshot isolation:**
|
||||
|
||||
| Field | Semantics |
|
||||
|-------|-----------|
|
||||
| `lm_eth_before` | LM total ETH at the shared initial snapshot — identical for every attack in the run |
|
||||
| `lm_eth_after` | LM total ETH measured after this specific attack, before reverting |
|
||||
| `attacks[].delta_bps` | Change relative to the shared `lm_eth_before`, not relative to any prior attack |
|
||||
|
||||
**Key implications:**
|
||||
|
||||
- `lm_eth_before` and `lm_eth_after` reflect **per-attack state**, not
|
||||
cumulative historical balance. Each attack sees the same starting ETH.
|
||||
- Attack results are independent and order-insensitive — reordering attacks does
|
||||
not change any individual `delta_bps` value.
|
||||
|
||||
---
|
||||
|
||||
## Schema: `holdout/YYYY-MM-DD-prNNN.json`
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue