| dev/cruxeval | dev/humaneval_plus | dev/thealgorithms | holdout/mbpp_plus | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| started | commit | corpus | workflow | pass | fail | error | total | pass | fail | error | total | pass | fail | error | total | pass | fail | error | total | wall seconds |
| 2026-09-22T11:12:34Z | cdc91528f5c8 | dev corpus-dev-2026-09-20-c037740453c2, holdout corpus-holdout-2026-09-20-c037740453c2 | run 35718600858 (attempt 1) | 716 (+35) | 0 | 72 (-35) | 788 | 160 (+6) | 0 | 8 (-6) | 168 | 1753 (+79) | 0 (-1) | 554 (-78) | 2307 | 164 (+3) | 1 | 38 (-3) | 203 | 2633 |
| 2026-09-20T13:56:32Z | 576a336fc4e6 | dev corpus-dev-2026-09-20-c037740453c2, holdout corpus-holdout-2026-09-20-c037740453c2 | - | 681 | 0 | 107 | 788 | 154 | 0 | 14 | 168 | 1674 | 1 | 632 | 2307 | 161 | 1 | 41 | 203 | 1650 |