johba/harb
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix: fix: red-team cross-pattern export records intermediate states as DECREASED (#852)

Browse source 
The extract_memory regex previously matched any "lm.?eth" mention,
including mid-execution "Total LM ETH: X wei" output lines produced by
the agent's cast check commands.  During a staking step these lines
reflect an intermediate chain state (ETH temporarily locked/moved)
rather than the final reverted state, causing strategies to be recorded
as DECREASED even when the runner confirmed ETH_SAFE.

Fix: narrow the capture to the structured `lm_eth_after: <value>`
label that the agent writes in its final RED-TEAM REPORT block.
Mid-execution total-ETH lines no longer match and cannot corrupt the
per-strategy result in memory or the cross-patterns file.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
openhands 2026-03-16 13:19:03 +00:00
parent da230b6c14
commit f3fb1c3db0
1 changed files with 6 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

8
scripts/harb-evaluator/red-team.sh
View file

@ -298,8 +298,12 @@ for text in texts:
} }
if current: if current:
# Capture floor readings — take the last match in the block (most recent value) # Capture lm_eth_after only from the structured final-report label
floor_matches = list(re.finditer(r"(?:floor|ethPerToken|lm.?eth)[^\d]*?(\d{4,})\s*(?:wei)?", text, re.IGNORECASE)) # ("lm_eth_after: <value> wei"). Mid-execution "Total LM ETH: X wei"
# lines are deliberately excluded: they reflect intermediate chain state
# (e.g. after staking before revert) and must not be recorded as the
# confirmed post-strategy ETH balance.
floor_matches = list(re.finditer(r"lm_eth_after\s*:\s*(\d+)", text, re.IGNORECASE))
if floor_matches: if floor_matches:
current["lm_eth_after"] = int(floor_matches[-1].group(1)) current["lm_eth_after"] = int(floor_matches[-1].group(1))