OpenBench

OpenBench Testing Framework

Finished
World	Reckless	ext-trigger	diff	40.0+0.40	LLR: -2.28 (-2.25, 2.89) [0.00, 3.00] Games: 41368 W: 10111 L: 10152 D: 21105 Ptnml(0-2): 17, 4730, 11231, 4689, 17
World	Reckless	larger-batch-size	diff	8.0+0.08	LLR: -2.35 (-2.25, 2.89) [0.00, 3.00] Games: 5150 W: 1249 L: 1377 D: 2524 Ptnml(0-2): 18, 676, 1309, 560, 12
World	Reckless	larger-batch-size	diff	N=25000	Elo: -4.98 +- 2.01 (95%) [N=40000] Games: 40334 W: 11157 L: 11735 D: 17442 Ptnml(0-2): 581, 5126, 9267, 4676, 517
World	Reckless	ext-trigger	diff	8.0+0.08	LLR: 2.95 (-2.25, 2.89) [0.00, 3.00] Games: 44144 W: 11194 L: 10922 D: 22028 Ptnml(0-2): 139, 5089, 11319, 5411, 114
Cj5716	Reckless	eval-correction-qsfut	diff	8.0+0.08	LLR: 2.93 (-2.25, 2.89) [0.00, 3.00] Games: 65648 W: 16683 L: 16362 D: 32603 Ptnml(0-2): 216, 7689, 16691, 8014, 214
World	Reckless	mp-more-weights-2	diff	40.0+0.40	LLR: -0.85 (-2.25, 2.89) [0.00, 3.00] Games: 6424 W: 1567 L: 1601 D: 3256 Ptnml(0-2): 7, 747, 1734, 721, 3
Cj5716	Reckless	eval-correction-fut	diff	8.0+0.08	LLR: -2.28 (-2.25, 2.89) [0.00, 3.00] Games: 17962 W: 4467 L: 4561 D: 8934 Ptnml(0-2): 42, 2173, 4638, 2093, 35
World	Reckless	probcut-in-check-sanity-2	diff	8.0+0.08	LLR: -1.34 (-2.25, 2.89) [-2.75, 0.25] Games: 45930 W: 11485 L: 11650 D: 22795 Ptnml(0-2): 192, 5420, 11888, 5291, 174
World	Reckless	low-depth-guard	diff	40.0+0.40	LLR: -0.24 (-2.25, 2.89) [0.00, 3.00] Games: 12768 W: 3145 L: 3132 D: 6491 Ptnml(0-2): 7, 1456, 3448, 1463, 10
World	Reckless	mp-more-weights-2	diff	8.0+0.08	LLR: 2.90 (-2.25, 2.89) [0.00, 3.00] Games: 16380 W: 4248 L: 4038 D: 8094 Ptnml(0-2): 54, 1845, 4200, 2019, 72
World	Reckless	low-depth-guard	diff	8.0+0.08	LLR: 3.03 (-2.25, 2.89) [0.00, 3.00] Games: 26234 W: 6733 L: 6492 D: 13009 Ptnml(0-2): 73, 3098, 6556, 3295, 95
World	Reckless	probcut-in-check-safety	diff	8.0+0.08	LLR: -2.06 (-2.25, 2.89) [-2.75, 0.25] Games: 11068 W: 2673 L: 2815 D: 5580 Ptnml(0-2): 29, 1396, 2822, 1262, 25	a bit more safety for in check probcut since Score::NONE is always > beta
World	Reckless	mp-more-weights	diff	40.0+0.40	LLR: 2.92 (-2.25, 2.89) [0.00, 3.00] Games: 21192 W: 5388 L: 5183 D: 10621 Ptnml(0-2): 7, 2288, 5801, 2493, 7
World	Reckless	mp-more-weights	diff	8.0+0.08	LLR: 2.93 (-2.25, 2.89) [0.00, 3.00] Games: 16328 W: 4173 L: 3966 D: 8189 Ptnml(0-2): 51, 1791, 4260, 2024, 38
World	Reckless	mp-noisy-gives-check	diff	8.0+0.08	LLR: -1.99 (-2.25, 2.89) [0.00, 3.00] Games: 25340 W: 6342 L: 6403 D: 12595 Ptnml(0-2): 67, 3019, 6545, 2986, 53
World	Reckless	qs-quiet-tt-move-stage	diff	8.0+0.08	LLR: -2.29 (-2.25, 2.89) [-2.75, 0.25] Games: 35510 W: 8777 L: 8977 D: 17756 Ptnml(0-2): 114, 4262, 9181, 4106, 92	search only the quiet TT move, `bench 16 1 20` finishes in under 2 mins
World	Reckless	neg-ext-less	diff	8.0+0.08	LLR: -2.51 (-2.25, 2.89) [0.00, 3.00] Games: 16648 W: 4137 L: 4248 D: 8263 Ptnml(0-2): 53, 1992, 4333, 1905, 41
World	Reckless	qs-check-count	diff	8.0+0.08	LLR: 0.98 (-2.25, 2.89) [-2.75, 0.25] Games: 15892 W: 4001 L: 3973 D: 7918 Ptnml(0-2): 40, 1829, 4178, 1861, 38
87	Reckless	tt-cutoff-penalty	diff	8.0+0.08	LLR: -2.27 (-2.25, 2.89) [0.00, 3.00] Games: 36452 W: 9145 L: 9199 D: 18108 Ptnml(0-2): 99, 4414, 9260, 4348, 105
Joshka	Reckless	joshka/optimize-quiet-move-scoring	diff	4.0+0.04	LLR: -2.33 (-2.25, 2.89) [0.00, 3.00] Games: 16322 W: 4144 L: 4245 D: 7933 Ptnml(0-2): 80, 1836, 4423, 1749, 73	This one should have the correct bench values (I think)
87	Reckless	mp-checking-non-threatened	diff	8.0+0.08	LLR: -2.28 (-2.25, 2.89) [0.00, 3.00] Games: 24942 W: 6259 L: 6340 D: 12343 Ptnml(0-2): 98, 3009, 6340, 2924, 100
Proton	Reckless	ps_qsearch107	diff	40.0+0.40	LLR: 0.21 (-2.25, 2.89) [-2.75, 0.25] Games: 17084 W: 4195 L: 4212 D: 8677 Ptnml(0-2): 7, 1717, 5112, 1698, 8	Try to limit search explosions in qsearch.
Proton	Reckless	ps_qsearch106	diff	40.0+0.40	LLR: 0.05 (-2.25, 2.89) [-2.75, 0.25] Games: 24674 W: 5992 L: 6032 D: 12650 Ptnml(0-2): 8, 2839, 6685, 2795, 10	fix.
Proton	Reckless	ps_qsearch105	diff	40.0+0.40	LLR: -1.83 (-2.25, 2.89) [-2.75, 0.25] Games: 34464 W: 8377 L: 8537 D: 17550 Ptnml(0-2): 19, 3913, 9519, 3771, 10	Simplify skip-quiets. If this sucks, it's Peregrine's fault. :)
World	Reckless	s2-less-selfplay	diff	8.0+0.08	LLR: -2.25 (-2.25, 2.89) [0.00, 3.00] Games: 6534 W: 1619 L: 1742 D: 3173 Ptnml(0-2): 34, 850, 1610, 751, 22

1 2 3 76 77 78 79 80 463 464 465