| status | function | dataset |
|---|---|---|
| pass -> error | data_structures/binary_tree/avl_tree.py:del_node | dev/thealgorithms |
| pass -> error | data_structures/binary_tree/basic_binary_tree.py:__iter__ | dev/thealgorithms |
| pass -> error | data_structures/binary_tree/is_sum_tree.py:__iter__ | dev/thealgorithms |
| pass -> error | data_structures/hashing/hash_map.py:__init__ | dev/thealgorithms |
| pass -> error | geometry/geometry.py:area | dev/thealgorithms |
| pass -> error | geometry/geometry.py:perimeter | dev/thealgorithms |
| pass -> error | graphs/bidirectional_a_star.py:__init__ | dev/thealgorithms |
| pass -> error | graphs/bidirectional_breadth_first_search.py:__init__ | dev/thealgorithms |
| error -> pass | data_structures/binary_tree/binary_search_tree.py:inorder | dev/thealgorithms |
| error -> pass | data_structures/binary_tree/segment_tree_other.py:__init__ | dev/thealgorithms |
| error -> pass | graphs/greedy_best_first.py:__init__ | dev/thealgorithms |
A broken line with a dotted vertical line marks where the measured functions changed; values on either side are not comparable.
One row per measurement, newest first. Timeouts and crashed workers can differ between measurements of the same code.
| dev/cruxeval | dev/humaneval_plus | dev/thealgorithms | holdout/mbpp_plus | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| started | commit | corpus | CI job | pass | fail | error | total | pass | fail | error | total | pass | fail | error | total | pass | fail | error | total |
| 2026-10-17T18:06:01Z | 443af03fcb1b | corpus-dev-2026-10-12-5ca4be0093b1 corpus-holdout-2026-10-12-5ca4be0093b1 | 39056274864 | 725 | 0 | 63 | 788 | 158 | 0 | 8 | 166 | 1697 | 0 | 398 | 2095 | 166 | 0 | 37 | 203 |
| 2026-10-17T04:29:45Z | bb9b041e58bd | corpus-dev-2026-10-12-5ca4be0093b1 corpus-holdout-2026-10-12-5ca4be0093b1 | 38944196856 | 725 | 0 | 63 | 788 | 158 | 0 | 8 | 166 | 1697 | 0 | 398 | 2095 | 166 | 0 | 37 | 203 |
| 2026-10-12T12:39:04Z | 13ea545ff509 | corpus-dev-2026-10-12-5ca4be0093b1 corpus-holdout-2026-10-12-5ca4be0093b1 | 38249365453 | 725 | 0 | 63 | 788 | 158 (-2) | 0 | 8 | 166 (-2) | 1697 (-160) | 0 | 398 (-52) | 2095 (-212) | 166 | 0 | 37 | 203 |
| 2026-10-12T09:34:31Z | dcfbaecb4bda | corpus-dev-2026-10-12-c34a75dd6a1d corpus-holdout-2026-10-12-c34a75dd6a1d | 38229977058 | 725 (+8) | 0 | 63 (-8) | 788 | 160 | 0 | 8 | 168 | 1857 (+25) | 0 | 450 (-25) | 2307 | 166 | 0 | 37 | 203 |
| 2026-09-28T18:07:13Z | 82f7c976e525 | corpus-dev-2026-09-20-c037740453c2 corpus-holdout-2026-09-20-c037740453c2 | 36462291622 | 717 | 0 | 71 | 788 | 160 | 0 | 8 | 168 | 1832 (+1) | 0 | 475 (-1) | 2307 | 166 | 0 | 37 | 203 |
| 2026-09-24T18:04:31Z | 84b5e3ca6c25 | corpus-dev-2026-09-20-c037740453c2 corpus-holdout-2026-09-20-c037740453c2 | 36038309389 | 717 | 0 | 71 | 788 | 160 | 0 | 8 | 168 | 1831 (+20) | 0 | 476 (-20) | 2307 | 166 | 0 | 37 | 203 |
| 2026-09-23T18:28:57Z | 0a9ad5100ebc | corpus-dev-2026-09-20-c037740453c2 corpus-holdout-2026-09-20-c037740453c2 | 35902215662 | 717 (+1) | 0 | 71 (-1) | 788 | 160 | 0 | 8 | 168 | 1811 (+58) | 0 | 496 (-58) | 2307 | 166 (+2) | 0 (-1) | 37 (-1) | 203 |
| 2026-09-22T11:12:34Z | cdc91528f5c8 | corpus-dev-2026-09-20-c037740453c2 corpus-holdout-2026-09-20-c037740453c2 | 35718600858 | 716 (+35) | 0 | 72 (-35) | 788 | 160 (+6) | 0 | 8 (-6) | 168 | 1753 (+79) | 0 (-1) | 554 (-78) | 2307 | 164 (+3) | 1 | 38 (-3) | 203 |
| 2026-09-20T13:56:32Z | 576a336fc4e6 | corpus-dev-2026-09-20-c037740453c2 corpus-holdout-2026-09-20-c037740453c2 | - | 681 | 0 | 107 | 788 | 154 | 0 | 14 | 168 | 1674 | 1 | 632 | 2307 | 161 | 1 | 41 | 203 |