johba/harb
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix: Red-team schema should add candidate_commit field (#1066) (#1075)

Browse source 
Fixes #1066

## Changes
Done. Here's what was changed:

**`evidence/README.md`**
- Added `"candidate_commit": "abc1234"` to the red-team schema JSON example
- Added `candidate_commit | string | Git commit SHA of the optimizer under test` row to the field table

**`scripts/harb-evaluator/red-team.sh`**
- Captures `CANDIDATE_COMMIT` from `git rev-parse HEAD` at startup (alongside existing `CANDIDATE_NAME`/`OPTIMIZER_PROFILE`)
- Added a new step (9a-pre) that writes `evidence/red-team/YYYY-MM-DD.json` at the end of each run, including `candidate_commit` plus all other schema fields (`candidate`, `optimizer_profile`, `lm_eth_before`, `lm_eth_after`, `eth_extracted`, `floor_held`, `verdict`, `attacks`)

Co-authored-by: openhands <openhands@all-hands.dev>
Reviewed-on: https://codeberg.org/johba/harb/pulls/1075
Reviewed-by: Disinto_bot <disinto_bot@noreply.codeberg.org>
This commit is contained in:
johba 2026-03-21 13:47:13 +01:00
parent 636ba989ee
commit 46e928ea97
3 changed files with 78 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
evidence/README.md
View file

@ -82,6 +82,7 @@ Records one adversarial red-team run against a candidate optimizer.
{
"date": "YYYY-MM-DD",
"candidate": "OptimizerV3",
"candidate_commit": "abc1234",
"optimizer_profile": "push3-default",
"lm_eth_before": 1000000000000000000000,
"lm_eth_after": 998500000000000000000,
@ -104,6 +105,7 @@ Records one adversarial red-team run against a candidate optimizer.
|-------|------|-------------|
| `date` | string (ISO) | Date of the run |
| `candidate` | string | Optimizer under test |
| `candidate_commit` | string | Git commit SHA of the optimizer under test |
| `optimizer_profile` | string | Named profile / push3 variant |
| `lm_eth_before` | integer (wei) | LM total ETH at start |
| `lm_eth_after` | integer (wei) | LM total ETH at end |