mirror of
https://github.com/sockspls/badfish
synced 2025-04-30 16:53:09 +00:00
194 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Linmiao Xu
|
5d81071953 |
Update default main net to nn-1111cefa1111.nnue
Created from 2 distinct spsa tunes of the latest main net (nn-31337bea577c.nnue) and applying the params to the prior main net (nn-e8bac1c07a5a.nnue). This effectively reverts the modifications to output weights and biases in https://github.com/official-stockfish/Stockfish/pull/5509 SPSA: A: 6000, alpha: 0.602, gamma: 0.101 1st - 437 feature transformer biases where values are < 25 54k / 120k games at 180+1.8 https://tests.stockfishchess.org/tests/view/66af98ac4ff211be9d4edad0 nn-808259761cca.nnue 2nd - 208 L2 weights where values are zero 112k / 120k games at 180+1.8 https://tests.stockfishchess.org/tests/view/66b0c3074ff211be9d4edbe5 nn-a56cb8c3d477.nnue When creating the above 2 nets (nn-808259761cca.nnue, nn-a56cb8c3d477.nnue), spsa params were unintentionally applied to nn-e8bac1c07a5a.nnue rather than nn-31337bea577c.nnue due to an issue in a script that creates nets by applying spsa results to base nets. Since they both passed STC and were neutral or slightly positive at LTC, they were combined to see if the elo from each set of params was additive. The 2 nets can be merged on top of nn-e8bac1c07a5a.nnue with: https://github.com/linrock/nnue-tools/blob/90942d3/spsa/combine_nnue.py ``` python3 combine_nnue.py \ nn-e8bac1c07a5a.nnue \ nn-808259761cca.nnue \ nn-a56cb8c3d477.nnue ``` Merging yields nn-87caa003fc6a.nnue which was renamed to nn-1111cefa1111.nnue with an updated nnue-namer around 10x faster than before by: - using a prefix trie for efficient prefix matches - modifying 4 non-functional bytes near the end of the file instead of 2 https://github.com/linrock/nnue-namer Thanks to @MinetaS for pointing out in #nnue-dev what the non-functional bytes are: L3 is 32, 4 bytes for biases, 32 bytes for weights. (fc_2) So -38 and -37 are technically -2 and -1 of fc_1 (type AffineTransform<30, 32>) And since InputDimension is padded to 32 there are total 32 of 2 adjacent bytes padding. So yes, it's non-functional whatever values are there. It's possible to tweak bytes at -38 - 32 * N and -37 - 32 * N given N = 0 ... 31 The net renamed with the new method passed non-regression STC vs. the original net: https://tests.stockfishchess.org/tests/view/66c0f0a821503a509c13b332 To print the spsa params with nnue-pytorch: ``` import features from serialize import NNUEReader feature_set = features.get_feature_set_from_name("HalfKAv2_hm") with open("nn-31337bea577c.nnue", "rb") as f: model = NNUEReader(f, feature_set).model c_end = 16 for i,ft_bias in enumerate(model.input.bias.data[:3072]): value = int(ft_bias * 254) if abs(value) < 25: print(f"ftB[{i}],{value},-1024,1024,{c_end},0.0020") c_end = 6 for i in range(8): for j in range(32): for k in range(30): value = int(model.layer_stacks.l2.weight.data[32 * i + j, k] * 64) if value == 0: print(f"twoW[{i}][{j}][{k}],{value},-127,127,{c_end},0.0020") ``` New params found with the same method as: https://github.com/official-stockfish/Stockfish/pull/5459 Passed STC: https://tests.stockfishchess.org/tests/view/66b4d4464ff211be9d4edf6e LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 136416 W: 35753 L: 35283 D: 65380 Ptnml(0-2): 510, 16159, 34416, 16597, 526 Passed LTC: https://tests.stockfishchess.org/tests/view/66b76e814ff211be9d4ee1cc LLR: 2.95 (-2.94,2.94) <0.50,2.50> Total: 159336 W: 40753 L: 40178 D: 78405 Ptnml(0-2): 126, 17497, 43864, 18038, 143 closes https://github.com/official-stockfish/Stockfish/pull/5534 bench 1613043 |
||
|
Linmiao Xu
|
b55217fd02 |
Update default main net to nn-31337bea577c.nnue
Created by updating output weights (256) and biases (8) of the previous main net with values found with spsa around 101k / 120k games at 140+1.4. 264 spsa params: output weights and biases in nn-e8bac1c07a5a.nnue A: 6000, alpha: 0.602, gamma: 0.101 weights: [-127, 127], c_end = 6 biases: [-8192, 8192], c_end = 64 Among the 264 params, 189 weights and all 8 biases were changed. Changes in the weights: - mean: -0.111 +/- 3.57 - range: [-8, 8] Found with the same method as: https://github.com/official-stockfish/Stockfish/pull/5459 Due to the original name (nn-ea8c9128c325.nnue) being too similar to the previous main net (nn-e8bac1c07a5a.nnue) and creating confusion, it was renamed by making non-functional changes to the .nnue file the same way as past nets with: https://github.com/linrock/nnue-namer To verify that bench is the same and view the modified non-functional bytes: ``` echo -e "setoption name EvalFile value nn-ea8c9128c325.nnue\nbench" | ./stockfish echo -e "setoption name EvalFile value nn-31337bea577c.nnue\nbench" | ./stockfish cmp -l nn-ea8c9128c325.nnue nn-31337bea577c.nnue diff <(xxd nn-ea8c9128c325.nnue) <(xxd nn-31337bea577c.nnue) ``` Passed STC: https://tests.stockfishchess.org/tests/view/669564154ff211be9d4ec080 LLR: 2.93 (-2.94,2.94) <0.00,2.00> Total: 57280 W: 15139 L: 14789 D: 27352 Ptnml(0-2): 209, 6685, 14522, 6995, 229 Passed LTC: https://tests.stockfishchess.org/tests/view/669694204ff211be9d4ec1b4 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 63030 W: 16093 L: 15720 D: 31217 Ptnml(0-2): 47, 6766, 17516, 7139, 47 closes https://github.com/official-stockfish/Stockfish/pull/5509 bench 1371485 |
||
|
Linmiao Xu
|
b209f14b1e |
Update default main net to nn-e8bac1c07a5a.nnue
Created by modifying L2 weights from the previous main net (nn-74f1d263ae9a.nnue)
with params found by spsa around 9k / 120k games at 120+1.2.
370 spsa params - L2 weights in nn-74f1d263ae9a.nnue where |val| >= 50
A: 6000, alpha: 0.602, gamma: 0.101
weights: [-127, 127], c_end = 6
To print the spsa params with nnue-pytorch:
```
import features
from serialize import NNUEReader
feature_set = features.get_feature_set_from_name("HalfKAv2_hm")
with open("nn-74f1d263ae9a.nnue", "rb") as f:
model = NNUEReader(f, feature_set).model
c_end = 6
for i in range(8):
for j in range(32):
for k in range(30):
value = int(model.layer_stacks.l2.weight[32 * i + j, k] * 64)
if abs(value) >= 50:
print(f"twoW[{i}][{j}][{k}],{value},-127,127,{c_end},0.0020")
```
Among the 370 params, 229 weights were changed.
avg change: 0.0961 ± 1.67
range: [-4, 3]
The number of weights changed, grouped by layer stack index,
shows more weights were modified in the lower piece count buckets:
[54, 52, 29, 23, 22, 18, 14, 17]
Found with the same method described in:
https://github.com/official-stockfish/Stockfish/pull/5459
Passed STC:
https://tests.stockfishchess.org/tests/view/668aec9a58083e5fd88239e7
LLR: 3.00 (-2.94,2.94) <0.00,2.00>
Total: 52384 W: 13569 L: 13226 D: 25589
Ptnml(0-2): 127, 6141, 13335, 6440, 149
Passed LTC:
https://tests.stockfishchess.org/tests/view/668af50658083e5fd8823a0b
LLR: 2.94 (-2.94,2.94) <0.50,2.50>
Total: 46974 W: 12006 L: 11668 D: 23300
Ptnml(0-2): 25, 4992, 13121, 5318, 31
closes https://github.com/official-stockfish/Stockfish/pull/5466
bench 1300471
|
||
|
Linmiao Xu
|
5752529cab |
Update default main net to nn-74f1d263ae9a.nnue
Created by setting output weights (256) and biases (8) of the previous main net nn-ddcfb9224cdb.nnue to values found around 12k / 120k spsa games at 120+1.2 This used modified fishtest dev workers to construct .nnue files from spsa params, then load them with EvalFile when running tests: https://github.com/linrock/fishtest/tree/spsa-file-modified-nnue/worker Inspired by researching loading spsa params from files: https://github.com/official-stockfish/fishtest/pull/1926 Scripts for modifying nnue files and preparing params: https://github.com/linrock/nnue-pytorch/tree/no-gpu-modify-nnue spsa params: weights: [-127, 127], c_end = 6 biases: [-8192, 8192], c_end = 64 Example of reading output weights and biases from the previous main net using nnue-pytorch and printing spsa params in a format compatible with fishtest: ``` import features from serialize import NNUEReader feature_set = features.get_feature_set_from_name("HalfKAv2_hm") with open("nn-ddcfb9224cdb.nnue", "rb") as f: model = NNUEReader(f, feature_set).model c_end_weights = 6 c_end_biases = 64 for i in range(8): for j in range(32): value = round(int(model.layer_stacks.output.weight[i, j] * 600 * 16) / 127) print(f"oW[{i}][{j}],{value},-127,127,{c_end_weights},0.0020") for i in range(8): value = int(model.layer_stacks.output.bias[i] * 600 * 16) print(f"oB[{i}],{value},-8192,8192,{c_end_biases},0.0020") ``` For more info on spsa tuning params in nets: https://github.com/official-stockfish/Stockfish/pull/5149 https://github.com/official-stockfish/Stockfish/pull/5254 Passed STC: https://tests.stockfishchess.org/tests/view/66894d64e59d990b103f8a37 LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 32000 W: 8443 L: 8137 D: 15420 Ptnml(0-2): 80, 3627, 8309, 3875, 109 Passed LTC: https://tests.stockfishchess.org/tests/view/6689668ce59d990b103f8b8b LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 172176 W: 43822 L: 43225 D: 85129 Ptnml(0-2): 97, 18821, 47633, 19462, 75 closes https://github.com/official-stockfish/Stockfish/pull/5459 bench 1120091 |
||
|
Linmiao Xu
|
cb4a623119 |
Update default smallnet to nn-37f18f62d772.nnue
Created by training L1-128 from scratch with:
- skipping based on simple eval in the trainer, for compatibility with
regular binpacks without requiring pre-filtering all binpacks
- minimum simple eval of 950, lower than 1000 previously
- usage of some hse-v1 binpacks with minimum simple eval 1000
- addition of hse-v6 binpacks with minimum simple eval 500
- permuting the FT with 10k positions from fishpack32.binpack
- torch.compile to speed up smallnet training
Training is significantly slower when using non-pre-filtered binpacks due to
the increased skipping required.
This net was reached at epoch 339.
```
experiment-name: 128--S1-hse-1k-T80-v6-unfilt-less-sf--se-gt950-no-wld-skip
training-dataset:
/data/:
- dfrc99-16tb7p.v2.min.binpack
/data/hse-v1/:
- leela96-filt-v2.min.high-simple-eval-1k.min-v2.binpack
- test60-novdec2021-12tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.min-v2.binpack
- test77-nov2021-2tb7p.no-db.min.high-simple-eval-1k.min-v2.binpack
- test77-dec2021-16tb7p.no-db.min.high-simple-eval-1k.min-v2.binpack
- test77-jan2022-2tb7p.high-simple-eval-1k.min-v2.binpack
- test78-jantomay2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.min-v2.binpack
- test78-juntosep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.min-v2.binpack
- test79-apr2022-16tb7p.min.high-simple-eval-1k.min-v2.binpack
- test79-may2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.min-v2.binpack
- test80-apr2022-16tb7p.min.high-simple-eval-1k.min-v2.binpack
- test80-may2022-16tb7p.high-simple-eval-1k.min-v2.binpack
- test80-jun2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.min-v2.binpack
- test80-jul2022-16tb7p.v6-dd.min.high-simple-eval-1k.min-v2.binpack
- test80-sep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.min-v2.binpack
- test80-nov2022-16tb7p-v6-dd.min.high-simple-eval-1k.min-v2.binpack
/data/S11-mar2024/:
- test80-2022-08-aug-16tb7p.v6-dd.min.binpack
- test80-2022-10-oct-16tb7p.v6-dd.binpack
- test80-2022-12-dec-16tb7p.min.binpack
- test80-2023-01-jan-16tb7p.v6-sk20.min.binpack
- test80-2023-02-feb-16tb7p.v6-sk20.min.binpack
- test80-2023-03-mar-2tb7p.v6-sk16.min.binpack
- test80-2023-04-apr-2tb7p.v6-sk16.min.binpack
- test80-2023-05-may-2tb7p.v6.min.binpack
- test80-2023-06-jun-2tb7p.binpack.min-v2.binpack
- test80-2023-07-jul-2tb7p.binpack.min-v2.binpack
- test80-2023-08-aug-2tb7p.v6.min.binpack
- test80-2023-09-sep-2tb7p.binpack.hse-v6.binpack
- test80-2023-10-oct-2tb7p.binpack.hse-v6.binpack
- test80-2023-11-nov-2tb7p.binpack.hse-v6.binpack
- test80-2023-12-dec-2tb7p.binpack.hse-v6.binpack
- test80-2024-01-jan-2tb7p.binpack.hse-v6.binpack
- test80-2024-02-feb-2tb7p.binpack.hse-v6.binpack
- test80-2024-03-mar-2tb7p.binpack
wld-fen-skipping: False
nnue-pytorch-branch: linrock/nnue-pytorch/128-skipSimpleEval-lt950-torch-compile
engine-test-branch: linrock/Stockfish/L1-128-nolazy
engine-base-branch: linrock/Stockfish/L1-128
start-from-engine-test-net: False
num-epochs: 500
start-lambda: 1.0
end-lambda: 1.0
```
Training data can be found at:
https://robotmoon.com/nnue-training-data/
Passed STC:
https://tests.stockfishchess.org/tests/view/66549c16a86388d5e27daff5
LLR: 2.93 (-2.94,2.94) <0.00,2.00>
Total: 196608 W: 51254 L: 50697 D: 94657
Ptnml(0-2): 722, 23244, 49796, 23839, 703
Passed LTC:
https://tests.stockfishchess.org/tests/view/6658d1aa6b0e318cefa90122
LLR: 2.96 (-2.94,2.94) <0.50,2.50>
Total: 122538 W: 31332 L: 30835 D: 60371
Ptnml(0-2): 69, 13407, 33811, 13922, 60
closes https://github.com/official-stockfish/Stockfish/pull/5333
bench
|
||
|
Linmiao Xu
|
35aff79843 |
Update default main net to nn-ddcfb9224cdb.nnue
Created by further tuning the spsa-tuned main net `nn-c721dfca8cd3.nnue` with the same methods described in https://github.com/official-stockfish/Stockfish/pull/5254 This net was reached at 61k / 120k spsa games at 70+0.7 th 7: https://tests.stockfishchess.org/tests/view/665639d0a86388d5e27dd259 Passed STC: https://tests.stockfishchess.org/tests/view/6657d44e6b0e318cefa8d771 LLR: 2.95 (-2.94,2.94) <0.00,2.00> Total: 114688 W: 29775 L: 29344 D: 55569 Ptnml(0-2): 274, 13633, 29149, 13964, 324 Passed LTC: https://tests.stockfishchess.org/tests/view/6657e1e46b0e318cefa8d7a6 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 88152 W: 22412 L: 21988 D: 43752 Ptnml(0-2): 56, 9560, 24409, 10006, 45 closes https://github.com/official-stockfish/Stockfish/pull/5308 Bench: 1434678 |
||
|
Linmiao Xu
|
d92d1f3180 |
Move smallnet threshold logic into a function
Now that the smallnet threshold is no longer a constant, use a function to organize it with other eval code. Passed non-regression STC: https://tests.stockfishchess.org/tests/view/66459fa093ce6da3e93b5ba2 LLR: 2.95 (-2.94,2.94) <-1.75,0.25> Total: 217600 W: 56281 L: 56260 D: 105059 Ptnml(0-2): 756, 23787, 59729, 23736, 792 closes https://github.com/official-stockfish/Stockfish/pull/5255 No functional change |
||
|
Linmiao Xu
|
1b7dea3f85 |
Update default main net to nn-c721dfca8cd3.nnue
Created by first retraining the spsa-tuned main net `nn-ae6a388e4a1a.nnue` with: - using v6-dd data without bestmove captures removed - addition of T80 mar2024 data - increasing loss by 20% when Q is too high - torch.compile changes for marginal training speed gains And then SPSA tuning weights of epoch 899 following methods described in: https://github.com/official-stockfish/Stockfish/pull/5149 This net was reached at 92k out of 120k steps in this 70+0.7 th 7 SPSA tuning run: https://tests.stockfishchess.org/tests/view/66413b7df9f4e8fc783c9bbb Thanks to @Viren6 for suggesting usage of: - c value 4 for the weights - c value 128 for the biases Scripts for automating applying fishtest spsa params to exporting tuned .nnue are in: https://github.com/linrock/nnue-tools/tree/master/spsa Before spsa tuning, epoch 899 was nn-f85738aefa84.nnue https://tests.stockfishchess.org/tests/view/663e5c893a2f9702074bc167 After initially training with max-epoch 800, training was resumed with max-epoch 1000. ``` experiment-name: 3072--S11--more-data-v6-dd-t80-mar2024--see-ge0-20p-more-loss-high-q-sk28-l8 nnue-pytorch-branch: linrock/nnue-pytorch/3072-r21-skip-more-wdl-see-ge0-20p-more-loss-high-q-torch-compile-more start-from-engine-test-net: False start-from-model: /data/config/apr2024-3072/nn-ae6a388e4a1a.nnue early-fen-skipping: 28 training-dataset: /data/S11-mar2024/: - leela96.v2.min.binpack - test60-2021-11-12-novdec-12tb7p.v6-dd.min.binpack - test78-2022-01-to-05-jantomay-16tb7p.v6-dd.min.binpack - test80-2022-06-jun-16tb7p.v6-dd.min.binpack - test80-2022-08-aug-16tb7p.v6-dd.min.binpack - test80-2022-09-sep-16tb7p.v6-dd.min.binpack - test80-2023-01-jan-16tb7p.v6-sk20.min.binpack - test80-2023-02-feb-16tb7p.v6-sk20.min.binpack - test80-2023-03-mar-2tb7p.v6-sk16.min.binpack - test80-2023-04-apr-2tb7p.v6-sk16.min.binpack - test80-2023-05-may-2tb7p.v6.min.binpack # https://github.com/official-stockfish/Stockfish/pull/4782 - test80-2023-06-jun-2tb7p.binpack - test80-2023-07-jul-2tb7p.binpack # https://github.com/official-stockfish/Stockfish/pull/4972 - test80-2023-08-aug-2tb7p.v6.min.binpack - test80-2023-09-sep-2tb7p.binpack - test80-2023-10-oct-2tb7p.binpack # S9 new data: https://github.com/official-stockfish/Stockfish/pull/5056 - test80-2023-11-nov-2tb7p.binpack - test80-2023-12-dec-2tb7p.binpack # S10 new data: https://github.com/official-stockfish/Stockfish/pull/5149 - test80-2024-01-jan-2tb7p.binpack - test80-2024-02-feb-2tb7p.binpack # S11 new data - test80-2024-03-mar-2tb7p.binpack /data/filt-v6-dd/: - test77-dec2021-16tb7p-filter-v6-dd.binpack - test78-juntosep2022-16tb7p-filter-v6-dd.binpack - test79-apr2022-16tb7p-filter-v6-dd.binpack - test79-may2022-16tb7p-filter-v6-dd.binpack - test80-jul2022-16tb7p-filter-v6-dd.binpack - test80-oct2022-16tb7p-filter-v6-dd.binpack - test80-nov2022-16tb7p-filter-v6-dd.binpack num-epochs: 1000 lr: 4.375e-4 gamma: 0.995 start-lambda: 0.8 end-lambda: 0.7 ``` Training data can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move: nn-epoch899.nnue : 4.6 +/- 1.4 Passed STC: https://tests.stockfishchess.org/tests/view/6645454893ce6da3e93b31ae LLR: 2.95 (-2.94,2.94) <0.00,2.00> Total: 95232 W: 24598 L: 24194 D: 46440 Ptnml(0-2): 294, 11215, 24180, 11647, 280 Passed LTC: https://tests.stockfishchess.org/tests/view/6645522d93ce6da3e93b31df LLR: 2.95 (-2.94,2.94) <0.50,2.50> Total: 320544 W: 81432 L: 80524 D: 158588 Ptnml(0-2): 164, 35659, 87696, 36611, 142 closes https://github.com/official-stockfish/Stockfish/pull/5254 bench 1995552 |
||
|
Linmiao Xu
|
47597641dc |
Lower smallnet threshold linearly as pawn count decreases
Passed STC: https://tests.stockfishchess.org/tests/view/6644f677324e96f42f89d894 LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 377920 W: 97135 L: 96322 D: 184463 Ptnml(0-2): 1044, 44259, 97588, 44978, 1091 Passed LTC: https://tests.stockfishchess.org/tests/view/664548af93ce6da3e93b31b3 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 169056 W: 42901 L: 42312 D: 83843 Ptnml(0-2): 58, 18538, 46753, 19115, 64 closes https://github.com/official-stockfish/Stockfish/pull/5252 Bench: 1991750 |
||
|
Linmiao Xu
|
0b08953174 |
Re-evaluate some small net positions for more accurate evals
Use main net evals when small net evals hint that higher eval accuracy may be worth the slower eval speeds. With Finny caches, re-evals with the main net are less expensive than before. Original idea by mstembera who I've added as co-author to this PR. Based on reEval tests by mstembera: https://tests.stockfishchess.org/tests/view/65e69187b6345c1b934866e5 https://tests.stockfishchess.org/tests/view/65e863aa0ec64f0526c3e991 A few variants of this patch also passed LTC: https://tests.stockfishchess.org/tests/view/663d2108507ebe1c0e91f407 https://tests.stockfishchess.org/tests/view/663e388c3a2f9702074bc152 Passed STC: https://tests.stockfishchess.org/tests/view/663dadbd1a61d6377f190e2c LLR: 2.93 (-2.94,2.94) <0.00,2.00> Total: 92320 W: 23941 L: 23531 D: 44848 Ptnml(0-2): 430, 10993, 22931, 11349, 457 Passed LTC: https://tests.stockfishchess.org/tests/view/663ef48b2948bf9aa698690c LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 98934 W: 24907 L: 24457 D: 49570 Ptnml(0-2): 48, 10952, 27027, 11382, 58 closes https://github.com/official-stockfish/Stockfish/pull/5238 bench 1876282 Co-Authored-By: mstembera <5421953+mstembera@users.noreply.github.com> |
||
|
cj5716
|
8ee9905d8b |
Remove PSQT-only mode
Passed STC: LLR: 2.94 (-2.94,2.94) <-1.75,0.25> Total: 94208 W: 24270 L: 24112 D: 45826 Ptnml(0-2): 286, 11186, 24009, 11330, 293 https://tests.stockfishchess.org/tests/view/6635ddd773559a8aa8582826 Passed LTC: LLR: 2.95 (-2.94,2.94) <-1.75,0.25> Total: 114960 W: 29107 L: 28982 D: 56871 Ptnml(0-2): 37, 12683, 31924, 12790, 46 https://tests.stockfishchess.org/tests/view/663604a973559a8aa85881ed closes #5214 Bench 1653939 |
||
|
gab8192
|
49ef4c935a |
Implement accumulator refresh table
For each thread persist an accumulator cache for the network, where each cache contains multiple entries for each of the possible king squares. When the accumulator needs to be refreshed, the cached entry is used to more efficiently update the accumulator, instead of rebuilding it from scratch. This idea, was first described by Luecx (author of Koivisto) and is commonly referred to as "Finny Tables". When the accumulator needs to be refreshed, instead of filling it with biases and adding every piece from scratch, we... 1. Take the `AccumulatorRefreshEntry` associated with the new king bucket 2. Calculate the features to activate and deactivate (from differences between bitboards in the entry and bitboards of the actual position) 3. Apply the updates on the refresh entry 4. Copy the content of the refresh entry accumulator to the accumulator we were refreshing 5. Copy the bitboards from the position to the refresh entry, to match the newly updated accumulator Results at STC: https://tests.stockfishchess.org/tests/view/662301573fe04ce4cefc1386 (first version) https://tests.stockfishchess.org/tests/view/6627fa063fe04ce4cefc6560 (final) Non-Regression between first and final: https://tests.stockfishchess.org/tests/view/662801e33fe04ce4cefc660a STC SMP: https://tests.stockfishchess.org/tests/view/662808133fe04ce4cefc667c closes https://github.com/official-stockfish/Stockfish/pull/5183 No functional change |
||
|
Muzhen Gaming
|
1adf8e1ae6 |
VVLTC search tune
Parameters were tuned in 3 stages: * Using an earlier L1-3072 net, and with triple extension margin manually set to 0: https://tests.stockfishchess.org/tests/view/65ffdf5d0ec64f0526c544f2 (~30k games) * Continue tuning, but with the previous master net (L1-2560). https://tests.stockfishchess.org/tests/view/660663f00ec64f0526c59c41 (~27k games) * Starting with the parameters from step 2, use the current L1-3072 net, and allow the triple extension margin to be tuned starting from 0: https://tests.stockfishchess.org/tests/view/660c16b8216a13d9498e7536 (40k games) Passed VVLTC 1st sprt: https://tests.stockfishchess.org/tests/view/66115eacbfeb43334bf7eddd LLR: 2.95 (-2.94,2.94) <0.00,2.00> Total: 27138 W: 7045 L: 6789 D: 13304 Ptnml(0-2): 1, 2421, 8471, 2673, 3 Passed VVLTC 2nd sprt: https://tests.stockfishchess.org/tests/view/661483623eb00c8ccc0049c1 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 26242 W: 6807 L: 6535 D: 12900 Ptnml(0-2): 0, 2353, 8143, 2625, 0 STC Elo estimate: https://tests.stockfishchess.org/tests/view/66175ca55a4693796d96608c Elo: -10.53 ± 2.4 (95%) LOS: 0.0% Total: 21584 W: 5294 L: 5948 D: 10342 Ptnml(0-2): 102, 2937, 5363, 2293, 97 nElo: -19.99 ± 4.7 (95%) PairsRatio: 0.79 closes https://github.com/official-stockfish/Stockfish/pull/5162 Bench: 1381387 |
||
|
Viren6
|
0716b845fd |
Update NNUE architecture to SFNNv9 and net nn-ae6a388e4a1a.nnue
Part 1: PyTorch Training, linrock Trained with a 10-stage sequence from scratch, starting in May 2023: https://github.com/linrock/nnue-tools/blob/master/exp-sequences/3072-10stage-SFNNv9.yml While the training methods were similar to the L1-2560 training sequence, the last two stages introduced min-v2 binpacks, where bestmove capture and in-check position scores were not zeroed during minimization, for compatibility with skipping SEE >= 0 positions and future research. Training data can be found at: https://robotmoon.com/nnue-training-data This net was tested at epoch 679 of the 10th training stage: https://tests.stockfishchess.org/tests/view/65f32e460ec64f0526c48dbc Part 2: SPSA Training, Viren6 The net was then SPSA tuned. This consisted of the output weights (32 * 8) and biases (8) as well as the L3 biases (32 * 8) and L2 biases (16 * 8), totalling 648 params in total. The SPSA tune can be found here: https://tests.stockfishchess.org/tests/view/65fc33ba0ec64f0526c512e3 With the help of Disservin , the initial weights were extracted with: https://github.com/Viren6/Stockfish/tree/new228 The net was saved with the tuned weights using: https://github.com/Viren6/Stockfish/tree/new241 Earlier nets of the SPSA failed STC compared to the base 3072 net of part 1: https://tests.stockfishchess.org/tests/view/65ff356e0ec64f0526c53c98 Therefore it is suspected that the SPSA at VVLTC has added extra scaling on top of the scaling of increasing the L1 size. Passed VVLTC 1: https://tests.stockfishchess.org/tests/view/6604a9020ec64f0526c583da LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 53042 W: 13554 L: 13256 D: 26232 Ptnml(0-2): 12, 5147, 15903, 5449, 10 Passed VVLTC 2: https://tests.stockfishchess.org/tests/view/660ad1b60ec64f0526c5dd23 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 17506 W: 4574 L: 4315 D: 8617 Ptnml(0-2): 1, 1567, 5362, 1818, 5 STC Elo estimate: https://tests.stockfishchess.org/tests/view/660b834d01aaec5069f87cb0 Elo: -7.66 ± 3.8 (95%) LOS: 0.0% Total: 9618 W: 2440 L: 2652 D: 4526 Ptnml(0-2): 80, 1281, 2261, 1145, 42 nElo: -13.94 ± 6.9 (95%) PairsRatio: 0.87 closes https://tests.stockfishchess.org/tests/view/660b834d01aaec5069f87cb0 bench 1823302 Co-Authored-By: Linmiao Xu <lin@robotmoon.com> |
||
|
Muzhen Gaming
|
d99f89506b |
VVLTC search tune
This set of parameters was derived from 3 tuning attempts:
https://tests.stockfishchess.org/tests/view/65d19ab61d8e83c78bfd8436 (80+0.8 x8, ~40k games)
Then tuned with one of linrock's early L1-3072 nets:
https://tests.stockfishchess.org/tests/view/65def7b04b19edc854ebdec8 (VVLTC, ~36k games)
Starting from the result of this tuning, the parameters were then tuned with the current master net:
https://tests.stockfishchess.org/tests/view/65f11c420ec64f0526c46fc4 (VVLTC, ~45k games)
Additionally, at the start of the third tuning phase, 2 parameters were manually changed:
Notably, the triple extension margin was decreased from 78 to 22. This idea was given by Vizvezdenec:
https://tests.stockfishchess.org/tests/view/65f0a2360ec64f0526c46752.
The PvNode extension margin was also adjusted from 50 to 40.
This tune also differs from previous tuning attempts by tuning the evaluation thresholds for smallnet and psqt-only.
The former was increased through the tuning, and this is hypothesized to scale better at VVLTC,
although there is not much evidence of it.
Passed VVLTC 1st sprt: https://tests.stockfishchess.org/tests/view/65f6761d0ec64f0526c4be88
LLR: 2.94 (-2.94,2.94) <0.00,2.00>
Total: 44688 W: 11421 L: 11140 D: 22127
Ptnml(0-2): 1, 4170, 13722, 4449, 2
Passed VVLTC 2nd sprt: https://tests.stockfishchess.org/tests/view/65fa31a30ec64f0526c4f611
LLR: 2.94 (-2.94,2.94) <0.50,2.50>
Total: 27450 W: 7057 L: 6778 D: 13615
Ptnml(0-2): 4, 2545, 8346, 2828, 2
STC Elo estimate: https://tests.stockfishchess.org/tests/view/65fd3e540ec64f0526c521ae
Elo: -7.84 ± 1.8 (95%) LOS: 0.0%
Total: 40000 W: 9899 L: 10802 D: 19299
Ptnml(0-2): 203, 5221, 10025, 4378, 173
nElo: -14.91 ± 3.4 (95%) PairsRatio: 0.84
closes https://github.com/official-stockfish/Stockfish/pull/5130
Bench: 1876107
|
||
|
Gahtan Nahdi
|
1a6c22c511 |
Evaluation adjustment for different eval types
Gives different eval scaling parameters for the three different types of evaluation (bignet, smallnet, psqtOnly). Passed STC: https://tests.stockfishchess.org/tests/view/65f4b0020ec64f0526c4a3bd LLR: 2.96 (-2.94,2.94) <0.00,2.00> Total: 168064 W: 43507 L: 42987 D: 81570 Ptnml(0-2): 662, 19871, 42445, 20393, 661 Passed LTC: https://tests.stockfishchess.org/tests/view/65f6be1a0ec64f0526c4c361 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 162564 W: 41188 L: 40604 D: 80772 Ptnml(0-2): 120, 18112, 44216, 18732, 102 closes https://github.com/official-stockfish/Stockfish/pull/5122 Bench: 2113576 |
||
|
FauziAkram
|
627974c99f |
Search + Eval + Movepick Tune
Passed STC: https://tests.stockfishchess.org/tests/view/65ef15220ec64f0526c44b04 LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 24480 W: 6459 L: 6153 D: 11868 Ptnml(0-2): 101, 2798, 6184, 3008, 149 Passed LTC: https://tests.stockfishchess.org/tests/view/65ef4bac0ec64f0526c44f50 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 53316 W: 13561 L: 13203 D: 26552 Ptnml(0-2): 27, 5925, 14408, 6259, 39 closes https://github.com/official-stockfish/Stockfish/pull/5104 Bench: 1715522 |
||
|
Disservin
|
1a26d698de |
Refactor Network Usage
Continuing from PR #4968, this update improves how Stockfish handles network usage, making it easier to manage and modify networks in the future. With the introduction of a dedicated Network class, creating networks has become straightforward. See uci.cpp: ```cpp NN::NetworkBig({EvalFileDefaultNameBig, "None", ""}, NN::embeddedNNUEBig) ``` The new `Network` encapsulates all network-related logic, significantly reducing the complexity previously required to support multiple network types, such as the distinction between small and big networks #4915. Non-Regression STC: https://tests.stockfishchess.org/tests/view/65edd26c0ec64f0526c43584 LLR: 2.94 (-2.94,2.94) <-1.75,0.25> Total: 33760 W: 8887 L: 8661 D: 16212 Ptnml(0-2): 143, 3795, 8808, 3961, 173 Non-Regression SMP STC: https://tests.stockfishchess.org/tests/view/65ed71970ec64f0526c42fdd LLR: 2.96 (-2.94,2.94) <-1.75,0.25> Total: 59088 W: 15121 L: 14931 D: 29036 Ptnml(0-2): 110, 6640, 15829, 6880, 85 Compiled with `make -j profile-build` ``` bash ./bench_parallel.sh ./stockfish ./stockfish-nnue 13 50 sf_base = 1568540 +/- 7637 (95%) sf_test = 1573129 +/- 7301 (95%) diff = 4589 +/- 8720 (95%) speedup = 0.29260% +/- 0.556% (95%) ``` Compiled with `make -j build` ``` bash ./bench_parallel.sh ./stockfish ./stockfish-nnue 13 50 sf_base = 1472653 +/- 7293 (95%) sf_test = 1491928 +/- 7661 (95%) diff = 19275 +/- 7154 (95%) speedup = 1.30886% +/- 0.486% (95%) ``` closes https://github.com/official-stockfish/Stockfish/pull/5100 No functional change |
||
|
Muzhen Gaming
|
10e2732978 |
VVLTC search tune
Result of 32k games of tuning at 60+0.6 8-thread. Link to the tuning attempt: https://tests.stockfishchess.org/tests/view/65def7b04b19edc854ebdec8 Passed VVLTC first SPRT: https://tests.stockfishchess.org/tests/view/65e51b53416ecd92c162ab7f LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 37570 W: 9613 L: 9342 D: 18615 Ptnml(0-2): 2, 3454, 11601, 3727, 1 Passed VVLTC second SPRT: https://tests.stockfishchess.org/tests/view/65e87d1c0ec64f0526c3eb39 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 123158 W: 31463 L: 31006 D: 60689 Ptnml(0-2): 5, 11589, 37935, 12044, 6 Note: The small net and psqt-only thresholds have been moved to evaluate.h. The reasoning is that these values are used in both `evaluate.cpp` and `evaluate_nnue.cpp`, and thus unifying their usage avoids inconsistencies during testing, where one occurrence is changed without the other (this happened during the search tune SPRT). closes https://github.com/official-stockfish/Stockfish/pull/5101 Bench: 1741218 |
||
|
Linmiao Xu
|
bd579ab5d1 |
Update default main net to nn-1ceb1ade0001.nnue
Created by retraining the previous main net `nn-b1a57edbea57.nnue` with: - some of the same options as before: - ranger21, more WDL skipping, 15% more loss when Q is too high - removal of the huge 514G pre-interleaved binpack - removal of SF-generated dfrc data (dfrc99-16tb7p-filt-v2.min.binpack) - interleaving many binpacks at training time - training with some bestmove capture positions where SEE < 0 - increased usage of torch.compile to speed up training by up to 40% ```yaml experiment-name: 2560--S10-dfrc0-to-dec2023-skip-more-wdl-15p-more-loss-high-q-see-ge0-sk28 nnue-pytorch-branch: linrock/nnue-pytorch/r21-more-wdl-skip-15p-more-loss-high-q-skip-see-ge0-torch-compile-more start-from-engine-test-net: True early-fen-skipping: 28 training-dataset: # similar, not the exact same as: # https://github.com/official-stockfish/Stockfish/pull/4635 - /data/S5-5af/leela96.v2.min.binpack - /data/S5-5af/test60-2021-11-12-novdec-12tb7p.v6-dd.min.binpack - /data/S5-5af/test77-2021-12-dec-16tb7p.v6-dd.min.binpack - /data/S5-5af/test78-2022-01-to-05-jantomay-16tb7p.v6-dd.min.binpack - /data/S5-5af/test78-2022-06-to-09-juntosep-16tb7p.v6-dd.min.binpack - /data/S5-5af/test79-2022-04-apr-16tb7p.v6-dd.min.binpack - /data/S5-5af/test79-2022-05-may-16tb7p.v6-dd.min.binpack - /data/S5-5af/test80-2022-06-jun-16tb7p.v6-dd.min.unmin.binpack - /data/S5-5af/test80-2022-07-jul-16tb7p.v6-dd.min.binpack - /data/S5-5af/test80-2022-08-aug-16tb7p.v6-dd.min.binpack - /data/S5-5af/test80-2022-09-sep-16tb7p.v6-dd.min.unmin.binpack - /data/S5-5af/test80-2022-10-oct-16tb7p.v6-dd.min.binpack - /data/S5-5af/test80-2022-11-nov-16tb7p.v6-dd.min.binpack - /data/S5-5af/test80-2023-01-jan-16tb7p.v6-sk20.min.binpack - /data/S5-5af/test80-2023-02-feb-16tb7p.v6-dd.min.binpack - /data/S5-5af/test80-2023-03-mar-2tb7p.min.unmin.binpack - /data/S5-5af/test80-2023-04-apr-2tb7p.binpack - /data/S5-5af/test80-2023-05-may-2tb7p.min.dd.binpack # https://github.com/official-stockfish/Stockfish/pull/4782 - /data/S6-1ee1aba5ed/test80-2023-06-jun-2tb7p.binpack - /data/S6-1ee1aba5ed/test80-2023-07-jul-2tb7p.min.binpack # https://github.com/official-stockfish/Stockfish/pull/4972 - /data/S8-baff1edbea57/test80-2023-08-aug-2tb7p.v6.min.binpack - /data/S8-baff1edbea57/test80-2023-09-sep-2tb7p.binpack - /data/S8-baff1edbea57/test80-2023-10-oct-2tb7p.binpack # https://github.com/official-stockfish/Stockfish/pull/5056 - /data/S9-b1a57edbea57/test80-2023-11-nov-2tb7p.binpack - /data/S9-b1a57edbea57/test80-2023-12-dec-2tb7p.binpack num-epochs: 800 lr: 4.375e-4 gamma: 0.995 start-lambda: 1.0 end-lambda: 0.7 ``` This particular net was reached at epoch 759. Use of more torch.compile decorators in nnue-pytorch model.py than in the previous main net training run sped up training by up to 40% on Tesla gpus when using recent pytorch compiled with cuda 12: https://github.com/linrock/nnue-tools/blob/7fb9831/Dockerfile Skipping positions with bestmove captures where static exchange evaluation is >= 0 is based on the implementation from Sopel's NNUE training & experimentation log: https://docs.google.com/document/d/1gTlrr02qSNKiXNZ_SuO4-RjK4MXBiFlLE6jvNqqMkAY Experiment 293 - only skip captures with see>=0 Positions with bestmove captures where score == 0 are always skipped for compatibility with minimized binpacks, since the original minimizer sets scores to 0 for slight improvements in compression. The trainer branch used was: https://github.com/linrock/nnue-pytorch/tree/r21-more-wdl-skip-15p-more-loss-high-q-skip-see-ge0-torch-compile-more Binpacks were renamed to be sorted chronologically by default when sorted by name. The binpack data are otherwise the same as binpacks with similar names in the prior naming convention. Training data can be found at: https://robotmoon.com/nnue-training-data/ Passed STC: https://tests.stockfishchess.org/tests/view/65e3ddd1f2ef6c733362ae5c LLR: 2.92 (-2.94,2.94) <0.00,2.00> Total: 149792 W: 39153 L: 38661 D: 71978 Ptnml(0-2): 675, 17586, 37905, 18032, 698 Passed LTC: https://tests.stockfishchess.org/tests/view/65e4d91c416ecd92c162a69b LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 64416 W: 16517 L: 16135 D: 31764 Ptnml(0-2): 38, 7218, 17313, 7602, 37 closes https://github.com/official-stockfish/Stockfish/pull/5090 Bench: 1373183 |
||
|
Linmiao Xu
|
8e75548f2a |
Update default main net to nn-b1a57edbea57.nnue
Created by retraining the previous main net `nn-baff1edbea57.nnue` with: - some of the same options as before: ranger21, more WDL skipping - the addition of T80 nov+dec 2023 data - increasing loss by 15% when prediction is too high, up from 10% - use of torch.compile to speed up training by over 25% ```yaml experiment-name: 2560--S9-514G-T80-augtodec2023-more-wdl-skip-15p-more-loss-high-q-sk28 training-dataset: # https://github.com/official-stockfish/Stockfish/pull/4782 - /data/S6-514G-1ee1aba5ed.binpack - /data/test80-aug2023-2tb7p.v6.min.binpack - /data/test80-sep2023-2tb7p.binpack - /data/test80-oct2023-2tb7p.binpack - /data/test80-nov2023-2tb7p.binpack - /data/test80-dec2023-2tb7p.binpack early-fen-skipping: 28 start-from-engine-test-net: True nnue-pytorch-branch: linrock/nnue-pytorch/r21-more-wdl-skip-15p-more-loss-high-q-torch-compile num-epochs: 1000 lr: 4.375e-4 gamma: 0.995 start-lambda: 1.0 end-lambda: 0.7 ``` Epoch 819 trained with the above config led to this PR. Use of torch.compile decorators in nnue-pytorch model.py was found to speed up training by at least 25% on Ampere gpus when using recent pytorch compiled with cuda 12: https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch See recent main net PRs for more info on - ranger21 and more WDL skipping: https://github.com/official-stockfish/Stockfish/pull/4942 - increasing loss when Q is too high: https://github.com/official-stockfish/Stockfish/pull/4972 Training data can be found at: https://robotmoon.com/nnue-training-data/ Passed STC: https://tests.stockfishchess.org/tests/view/65cd76151d8e83c78bfd2f52 LLR: 2.98 (-2.94,2.94) <0.00,2.00> Total: 78336 W: 20504 L: 20115 D: 37717 Ptnml(0-2): 317, 9225, 19721, 9562, 343 Passed LTC: https://tests.stockfishchess.org/tests/view/65ce5be61d8e83c78bfd43e9 LLR: 2.95 (-2.94,2.94) <0.50,2.50> Total: 41016 W: 10492 L: 10159 D: 20365 Ptnml(0-2): 22, 4533, 11071, 4854, 28 closes https://github.com/official-stockfish/Stockfish/pull/5056 Bench: 1351997 |
||
|
Disservin
|
88331add0d |
Remove the dependency on a Worker from evaluate
Also remove dead code, `rootSimpleEval` is no longer used since the introduction of dual net. `iterBestValue` is also no longer used in evaluate and can be reduced to a local variable. closes https://github.com/official-stockfish/Stockfish/pull/4979 No functional change |
||
|
Disservin
|
a107910951 |
Refactor global variables
This aims to remove some of the annoying global structure which Stockfish has. Overall there is no major elo regression to be expected. Non regression SMP STC (paused, early version): https://tests.stockfishchess.org/tests/view/65983d7979aa8af82b9608f1 LLR: 0.23 (-2.94,2.94) <-1.75,0.25> Total: 76232 W: 19035 L: 19096 D: 38101 Ptnml(0-2): 92, 8735, 20515, 8690, 84 Non regression STC (early version): https://tests.stockfishchess.org/tests/view/6595b3a479aa8af82b95da7f LLR: 2.93 (-2.94,2.94) <-1.75,0.25> Total: 185344 W: 47027 L: 46972 D: 91345 Ptnml(0-2): 571, 21285, 48943, 21264, 609 Non regression SMP STC: https://tests.stockfishchess.org/tests/view/65a0715c79aa8af82b96b7e4 LLR: 2.94 (-2.94,2.94) <-1.75,0.25> Total: 142936 W: 35761 L: 35662 D: 71513 Ptnml(0-2): 209, 16400, 38135, 16531, 193 These global structures/variables add hidden dependencies and allow data to be mutable from where it shouldn't it be (i.e. options). They also prevent Stockfish from internal selfplay, which would be a nice thing to be able to do, i.e. instantiate two Stockfish instances and let them play against each other. It will also allow us to make Stockfish a library, which can be easier used on other platforms. For consistency with the old search code, `thisThread` has been kept, even though it is not strictly necessary anymore. This the first major refactor of this kind (in recent time), and future changes are required, to achieve the previously described goals. This includes cleaning up the dependencies, transforming the network to be self contained and coming up with a plan to deal with proper tablebase memory management (see comments for more information on this). The removal of these global structures has been discussed in parts with Vondele and Sopel. closes https://github.com/official-stockfish/Stockfish/pull/4968 No functional change |
||
|
Linmiao Xu
|
6deb88728f |
Update default main net to nn-baff1edbea57.nnue
Created by retraining the previous main net nn-b1e55edbea57.nnue with: - some of the same options as before: ranger21 optimizer, more WDL skipping - adding T80 aug filter-v6, sep, and oct 2023 data to the previous best dataset - increasing training loss for positions where predicted win rates were higher than estimated match results from training data position scores ```yaml experiment-name: 2560--S8-r21-more-wdl-skip-10p-more-loss-high-q-sk28 training-dataset: # https://github.com/official-stockfish/Stockfish/pull/4782 - /data/S6-1ee1aba5ed.binpack - /data/test80-aug2023-2tb7p.v6.min.binpack - /data/test80-sep2023-2tb7p.binpack - /data/test80-oct2023-2tb7p.binpack early-fen-skipping: 28 start-from-engine-test-net: True nnue-pytorch-branch: linrock/nnue-pytorch/r21-more-wdl-skip-10p-more-loss-high-q num-epochs: 1000 lr: 4.375e-4 gamma: 0.995 start-lambda: 1.0 end-lambda: 0.7 ``` Training data can be found at: https://robotmoon.com/nnue-training-data/ Training loss was increased by 10% for positions where predicted win rates were higher than suggested by the win rate model based on the training data, by multiplying with: ((qf > pt) * 0.1 + 1). This was a variant of experiments from Sopel's NNUE training & experimentation log: https://docs.google.com/document/d/1gTlrr02qSNKiXNZ_SuO4-RjK4MXBiFlLE6jvNqqMkAY Experiment 302 - increase loss when prediction too high, vondele’s idea Experiment 309 - increase loss when prediction too high, normalize in a batch Passed STC: https://tests.stockfishchess.org/tests/view/6597a21c79aa8af82b95fd5c LLR: 2.93 (-2.94,2.94) <0.00,2.00> Total: 148320 W: 37960 L: 37475 D: 72885 Ptnml(0-2): 542, 17565, 37383, 18206, 464 Passed LTC: https://tests.stockfishchess.org/tests/view/659834a679aa8af82b960845 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 55188 W: 13955 L: 13592 D: 27641 Ptnml(0-2): 34, 6162, 14834, 6535, 29 closes https://github.com/official-stockfish/Stockfish/pull/4972 Bench: 1219824 |
||
|
Disservin
|
99cdb920fc |
Cleanup Evalfile handling
This cleans up the EvalFile handling after the merge of #4915, which has become a bit confusing on what it is actually doing. closes https://github.com/official-stockfish/Stockfish/pull/4971 No functional change |
||
|
Linmiao Xu
|
f09adaa4a4 |
Update smallnet to nn-baff1ede1f90.nnue with wider eval range
Created by training an L1-128 net from scratch with a wider range of
evals in the training data and wld-fen-skipping disabled during
training. The differences in this training data compared to the first
dual nnue PR are:
- removal of all positions with 3 pieces
- when piece count >= 16, keep positions with simple eval above 750
- when piece count < 16, remove positions with simple eval above 3000
The asymmetric data filtering was meant to flatten the training data
piece count distribution, which was previously heavily skewed towards
positions with low piece counts.
Additionally, the simple eval range where the smallnet is used was
widened to cover more positions previously evaluated by the big net and
simple eval.
```yaml
experiment-name: 128--S1-hse-S7-v4-S3-v1-no-wld-skip
training-dataset:
- /data/hse/S3/leela96-filt-v2.min.high-simple-eval-1k.binpack
- /data/hse/S3/dfrc99-16tb7p-eval-filt-v2.min.high-simple-eval-1k.binpack
- /data/hse/S3/test80-apr2022-16tb7p.min.high-simple-eval-1k.binpack
- /data/hse/S7/test60-2020-2tb7p.v6-3072.high-simple-eval-v4.binpack
- /data/hse/S7/test60-novdec2021-12tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack
- /data/hse/S7/test77-nov2021-2tb7p.v6-3072.min.high-simple-eval-v4.binpack
- /data/hse/S7/test77-dec2021-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack
- /data/hse/S7/test77-jan2022-2tb7p.high-simple-eval-v4.binpack
- /data/hse/S7/test78-jantomay2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack
- /data/hse/S7/test78-juntosep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack
- /data/hse/S7/test79-apr2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack
- /data/hse/S7/test79-may2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack
- /data/hse/S7/test80-may2022-16tb7p.high-simple-eval-v4.binpack
- /data/hse/S7/test80-jun2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack
- /data/hse/S7/test80-jul2022-16tb7p.v6-dd.min.high-simple-eval-v4.binpack
- /data/hse/S7/test80-aug2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack
- /data/hse/S7/test80-sep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack
- /data/hse/S7/test80-oct2022-16tb7p.v6-dd.high-simple-eval-v4.binpack
- /data/hse/S7/test80-nov2022-16tb7p-v6-dd.min.high-simple-eval-v4.binpack
- /data/hse/S7/test80-jan2023-3of3-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack
- /data/hse/S7/test80-feb2023-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-v4.binpack
- /data/hse/S7/test80-mar2023-2tb7p.v6-sk16.min.high-simple-eval-v4.binpack
- /data/hse/S7/test80-apr2023-2tb7p-filter-v6-sk16.min.high-simple-eval-v4.binpack
- /data/hse/S7/test80-may2023-2tb7p.v6.min.high-simple-eval-v4.binpack
- /data/hse/S7/test80-jun2023-2tb7p.v6-3072.min.high-simple-eval-v4.binpack
- /data/hse/S7/test80-jul2023-2tb7p.v6-3072.min.high-simple-eval-v4.binpack
- /data/hse/S7/test80-aug2023-2tb7p.v6.min.high-simple-eval-v4.binpack
- /data/hse/S7/test80-sep2023-2tb7p.high-simple-eval-v4.binpack
- /data/hse/S7/test80-oct2023-2tb7p.high-simple-eval-v4.binpack
wld-fen-skipping: False
start-from-engine-test-net: False
nnue-pytorch-branch: linrock/nnue-pytorch/L1-128
engine-test-branch: linrock/Stockfish/L1-128-nolazy
engine-base-branch: linrock/Stockfish/L1-128
num-epochs: 500
start-lambda: 1.0
end-lambda: 1.0
```
Experiment yaml configs converted to easy_train.sh commands with:
https://github.com/linrock/nnue-tools/blob/4339954/yaml_easy_train.py
Binpacks interleaved at training time with:
https://github.com/official-stockfish/nnue-pytorch/pull/259
FT weights permuted with 10k positions from fishpack32.binpack with:
https://github.com/official-stockfish/nnue-pytorch/pull/254
Data filtered for high simple eval positions (v4) with:
https://github.com/linrock/Stockfish/blob/b9c8440/src/tools/transform.cpp#L640-L675
Training data can be found at:
https://robotmoon.com/nnue-training-data/
Local elo at 25k nodes per move of
L1-128 smallnet (nnue-only eval) vs. L1-128 trained on standard S1 data:
nn-epoch319.nnue : -241.7 +/- 3.2
Passed STC vs.
|
||
|
Linmiao Xu
|
584d9efedc |
Dual NNUE with L1-128 smallnet
Credit goes to @mstembera for: - writing the code enabling dual NNUE: https://github.com/official-stockfish/Stockfish/pull/4898 - the idea of trying L1-128 trained exclusively on high simple eval positions The L1-128 smallnet is: - epoch 399 of a single-stage training from scratch - trained only on positions from filtered data with high material difference - defined by abs(simple_eval) > 1000 ```yaml experiment-name: 128--S1-only-hse-v2 training-dataset: - /data/hse/S3/dfrc99-16tb7p-eval-filt-v2.min.high-simple-eval-1k.binpack - /data/hse/S3/leela96-filt-v2.min.high-simple-eval-1k.binpack - /data/hse/S3/test80-apr2022-16tb7p.min.high-simple-eval-1k.binpack - /data/hse/S7/test60-2020-2tb7p.v6-3072.high-simple-eval-1k.binpack - /data/hse/S7/test60-novdec2021-12tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test77-nov2021-2tb7p.v6-3072.min.high-simple-eval-1k.binpack - /data/hse/S7/test77-dec2021-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test77-jan2022-2tb7p.high-simple-eval-1k.binpack - /data/hse/S7/test78-jantomay2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test78-juntosep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test79-apr2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test79-may2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack # T80 2022 - /data/hse/S7/test80-may2022-16tb7p.high-simple-eval-1k.binpack - /data/hse/S7/test80-jun2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test80-jul2022-16tb7p.v6-dd.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-aug2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test80-sep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test80-oct2022-16tb7p.v6-dd.high-simple-eval-1k.binpack - /data/hse/S7/test80-nov2022-16tb7p-v6-dd.min.high-simple-eval-1k.binpack # T80 2023 - /data/hse/S7/test80-jan2023-3of3-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test80-feb2023-16tb7p-filter-v6-dd.min-mar2023.unmin.high-simple-eval-1k.binpack - /data/hse/S7/test80-mar2023-2tb7p.v6-sk16.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-apr2023-2tb7p-filter-v6-sk16.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-may2023-2tb7p.v6.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-jun2023-2tb7p.v6-3072.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-jul2023-2tb7p.v6-3072.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-aug2023-2tb7p.v6.min.high-simple-eval-1k.binpack - /data/hse/S7/test80-sep2023-2tb7p.high-simple-eval-1k.binpack - /data/hse/S7/test80-oct2023-2tb7p.high-simple-eval-1k.binpack start-from-engine-test-net: False nnue-pytorch-branch: linrock/nnue-pytorch/L1-128 engine-test-branch: linrock/Stockfish/L1-128-nolazy engine-base-branch: linrock/Stockfish/L1-128 num-epochs: 500 lambda: 1.0 ``` Experiment yaml configs converted to easy_train.sh commands with: https://github.com/linrock/nnue-tools/blob/4339954/yaml_easy_train.py Binpacks interleaved at training time with: https://github.com/official-stockfish/nnue-pytorch/pull/259 Data filtered for high simple eval positions with: https://github.com/linrock/nnue-data/blob/32d6a68/filter_high_simple_eval_plain.py https://github.com/linrock/Stockfish/blob/61dbfe/src/tools/transform.cpp#L626-L655 Training data can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move of L1-128 smallnet (nnue-only eval) vs. L1-128 trained on standard S1 data: nn-epoch399.nnue : -318.1 +/- 2.1 Passed STC: https://tests.stockfishchess.org/tests/view/6574cb9d95ea6ba1fcd49e3b LLR: 2.93 (-2.94,2.94) <0.00,2.00> Total: 62432 W: 15875 L: 15521 D: 31036 Ptnml(0-2): 177, 7331, 15872, 7633, 203 Passed LTC: https://tests.stockfishchess.org/tests/view/6575da2d4d789acf40aaac6e LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 64830 W: 16118 L: 15738 D: 32974 Ptnml(0-2): 43, 7129, 17697, 7497, 49 closes https://github.com/official-stockfish/Stockfish/pulls Bench: 1330050 Co-Authored-By: mstembera <5421953+mstembera@users.noreply.github.com> |
||
|
Disservin
|
b987d4f033 |
Use type aliases instead of enums for Value types
The primary rationale behind this lies in the fact that enums were not originally designed to be employed in the manner we currently utilize them. The Value enum was used like a type alias throughout the code and was often misused. Furthermore, changing the underlying size of the enum to int16_t broke everything, mostly because of the operator overloads for the Value enum, were causing data to be truncated. Since Value is now a type alias, the operator overloads are no longer required. Passed Non-Regression STC: https://tests.stockfishchess.org/tests/view/6593b8bb79aa8af82b95b401 LLR: 2.95 (-2.94,2.94) <-1.75,0.25> Total: 235296 W: 59919 L: 59917 D: 115460 Ptnml(0-2): 743, 27085, 62054, 26959, 807 closes https://github.com/official-stockfish/Stockfish/pull/4960 No functional change |
||
|
Disservin
|
444f03ee95 |
Update copyright year
closes https://github.com/official-stockfish/Stockfish/pull/4954 No functional change |
||
|
Linmiao Xu
|
f12035c88c |
Update default net to nn-b1e55edbea57.nnue
Created by retraining the master big net `nn-0000000000a0.nnue` on the same dataset with the ranger21 optimizer and more WDL skipping at training time. More WDL skipping is meant to increase lambda accuracy and train on fewer misevaluated positions where position scores are unlikely to correlate with game outcomes. Inspired by: - repeated reports in discord #events-discuss about SF misplaying due to wrong endgame evals, possibly due to Leela's endgame weaknesses reflected in training data - an attempt to reduce the skewed dataset piece count distribution where there are much more positions with less than 16 pieces, since the target piece count distribution in the trainer is symmetric around 16 The faster convergence seen with ranger21 is meant to: - prune experiment ideas more quickly since fewer epochs are needed to reach elo maxima - research faster potential trainings by shortening each run ```yaml experiment-name: 2560-S7-Re-514G-ranger21-more-wdl-skip training-dataset: /data/S6-514G.binpack early-fen-skipping: 28 start-from-engine-test-net: True nnue-pytorch-branch: linrock/nnue-pytorch/r21-more-wdl-skip num-epochs: 1200 lr: 4.375e-4 gamma: 0.995 start-lambda: 1.0 end-lambda: 0.7 ``` Experiment yaml configs converted to easy_train.sh commands with: https://github.com/linrock/nnue-tools/blob/4339954/yaml_easy_train.py Implementations based off of Sopel's NNUE training & experimentation log: https://docs.google.com/document/d/1gTlrr02qSNKiXNZ_SuO4-RjK4MXBiFlLE6jvNqqMkAY - Experiment 336 - ranger21 https://github.com/Sopel97/nnue-pytorch/tree/experiment_336 - Experiment 351 - more WDL skipping The version of the ranger21 optimizer used is: https://github.com/lessw2020/Ranger21/blob/b507df6/ranger21/ranger21.py The dataset is the exact same as in: https://github.com/official-stockfish/Stockfish/pull/4782 Local elo at 25k nodes per move: nn-epoch619.nnue : 6.2 +/- 4.2 Passed STC: https://tests.stockfishchess.org/tests/view/658a029779aa8af82b94fbe6 LLR: 2.93 (-2.94,2.94) <0.00,2.00> Total: 46528 W: 11985 L: 11650 D: 22893 Ptnml(0-2): 154, 5489, 11688, 5734, 199 Passed LTC: https://tests.stockfishchess.org/tests/view/658a448979aa8af82b95010f LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 265326 W: 66378 L: 65574 D: 133374 Ptnml(0-2): 153, 30175, 71254, 30877, 204 This was additionally tested with the latest DualNNUE and passed SPRTs: Passed STC vs. https://github.com/official-stockfish/Stockfish/pull/4919 https://tests.stockfishchess.org/tests/view/658bcd5c79aa8af82b951846 LLR: 2.93 (-2.94,2.94) <0.00,2.00> Total: 296128 W: 76273 L: 75554 D: 144301 Ptnml(0-2): 1223, 35768, 73617, 35979, 1477 Passed LTC vs. https://github.com/official-stockfish/Stockfish/pull/4919 https://tests.stockfishchess.org/tests/view/658c988d79aa8af82b95240f LLR: 2.95 (-2.94,2.94) <0.50,2.50> Total: 75618 W: 19085 L: 18680 D: 37853 Ptnml(0-2): 45, 8420, 20497, 8779, 68 closes https://github.com/official-stockfish/Stockfish/pull/4942 Bench: 1304666 |
||
|
Disservin
|
2d0237db3f |
add clang-format
This introduces clang-format to enforce a consistent code style for Stockfish. Having a documented and consistent style across the code will make contributing easier for new developers, and will make larger changes to the codebase easier to make. To facilitate formatting, this PR includes a Makefile target (`make format`) to format the code, this requires clang-format (version 17 currently) to be installed locally. Installing clang-format is straightforward on most OS and distros (e.g. with https://apt.llvm.org/, brew install clang-format, etc), as this is part of quite commonly used suite of tools and compilers (llvm / clang). Additionally, a CI action is present that will verify if the code requires formatting, and comment on the PR as needed. Initially, correct formatting is not required, it will be done by maintainers as part of the merge or in later commits, but obviously this is encouraged. fixes https://github.com/official-stockfish/Stockfish/issues/3608 closes https://github.com/official-stockfish/Stockfish/pull/4790 Co-Authored-By: Joost VandeVondele <Joost.VandeVondele@gmail.com> |
||
|
Linmiao Xu
|
afe7f4d9b0 |
Update default net to nn-0000000000a0.nnue
This is a later epoch from the same experiment that led to the previous master net. In training stage 6, max-epoch was raised to 1,200 near the end of the first 1,000 epochs. For more details, see https://github.com/official-stockfish/Stockfish/pull/4795 Local elo at 25k nodes per move (vs. L1-2048 nn-1ee1aba5ed4c.nnue) ep1079 : 15.6 +/- 1.2 Passed STC: https://tests.stockfishchess.org/tests/view/651503b3b3e74811c8af1e2a LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 29408 W: 7607 L: 7304 D: 14497 Ptnml(0-2): 97, 3277, 7650, 3586, 94 Passed LTC: https://tests.stockfishchess.org/tests/view/651585ceb3e74811c8af2a5f LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 73164 W: 18828 L: 18440 D: 35896 Ptnml(0-2): 30, 7749, 20644, 8121, 38 closes https://github.com/official-stockfish/Stockfish/pull/4810 Bench: 1453057 |
||
|
Linmiao Xu
|
70ba9de85c |
Update NNUE architecture to SFNNv8: L1-2560 nn-ac1dbea57aa3.nnue
Creating this net involved: - a 6-stage training process from scratch. The datasets used in stages 1-5 were fully minimized. - permuting L1 weights with https://github.com/official-stockfish/nnue-pytorch/pull/254 A strong epoch after each training stage was chosen for the next. The 6 stages were: ``` 1. 400 epochs, lambda 1.0, default LR and gamma UHOx2-wIsRight-multinet-dfrc-n5000 (135G) nodes5000pv2_UHO.binpack data_pv-2_diff-100_nodes-5000.binpack wrongIsRight_nodes5000pv2.binpack multinet_pv-2_diff-100_nodes-5000.binpack dfrc_n5000.binpack 2. 800 epochs, end-lambda 0.75, LR 4.375e-4, gamma 0.995, skip 12 LeelaFarseer-T78juntoaugT79marT80dec.binpack (141G) T60T70wIsRightFarseerT60T74T75T76.binpack test78-junjulaug2022-16tb7p.no-db.min.binpack test79-mar2022-16tb7p.no-db.min.binpack test80-dec2022-16tb7p.no-db.min.binpack 3. 800 epochs, end-lambda 0.725, LR 4.375e-4, gamma 0.995, skip 20 leela93-v1-dfrc99-v2-T78juntosepT80jan-v6dd-T78janfebT79aprT80aprmay.min.binpack leela93-filt-v1.min.binpack dfrc99-16tb7p-filt-v2.min.binpack test78-juntosep2022-16tb7p-filter-v6-dd.min-mar2023.binpack test80-jan2023-3of3-16tb7p-filter-v6-dd.min-mar2023.binpack test78-janfeb2022-16tb7p.min.binpack test79-apr2022-16tb7p.min.binpack test80-apr2022-16tb7p.min.binpack test80-may2022-16tb7p.min.binpack 4. 800 epochs, end-lambda 0.7, LR 4.375e-4, gamma 0.995, skip 24 leela96-dfrc99-v2-T78juntosepT79mayT80junsepnovjan-v6dd-T80mar23-v6-T60novdecT77decT78aprmayT79aprT80may23.min.binpack leela96-filt-v2.min.binpack dfrc99-16tb7p-filt-v2.min.binpack test78-juntosep2022-16tb7p-filter-v6-dd.min-mar2023.binpack test79-may2022-16tb7p.filter-v6-dd.min.binpack test80-jun2022-16tb7p.filter-v6-dd.min.binpack test80-sep2022-16tb7p.filter-v6-dd.min.binpack test80-nov2022-16tb7p.filter-v6-dd.min.binpack test80-jan2023-3of3-16tb7p-filter-v6-dd.min-mar2023.binpack test80-mar2023-2tb7p.v6-sk16.min.binpack test60-novdec2021-16tb7p.min.binpack test77-dec2021-16tb7p.min.binpack test78-aprmay2022-16tb7p.min.binpack test79-apr2022-16tb7p.min.binpack test80-may2023-2tb7p.min.binpack 5. 960 epochs, end-lambda 0.7, LR 4.375e-4, gamma 0.995, skip 28 Increased max-epoch to 960 near the end of the first 800 epochs 5af11540bbfe dataset: https://github.com/official-stockfish/Stockfish/pull/4635 6. 1000 epochs, end-lambda 0.7, LR 4.375e-4, gamma 0.995, skip 28 Increased max-epoch to 1000 near the end of the first 800 epochs 1ee1aba5ed dataset: https://github.com/official-stockfish/Stockfish/pull/4782 ``` L1 weights permuted with: ```bash python3 serialize.py $nnue $nnue_permuted \ --features=HalfKAv2_hm \ --ft_optimize \ --ft_optimize_data=/data/fishpack32.binpack \ --ft_optimize_count=10000 ``` Speed measurements from 100 bench runs at depth 13 with profile-build x86-64-avx2: ``` sf_base = 1329051 +/- 2224 (95%) sf_test = 1163344 +/- 2992 (95%) diff = -165706 +/- 4913 (95%) speedup = -12.46807% +/- 0.370% (95%) ``` Training data can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move (vs. L1-2048 nn-1ee1aba5ed4c.nnue) ep959 : 16.2 +/- 2.3 Failed 10+0.1 STC: https://tests.stockfishchess.org/tests/view/6501beee2cd016da89abab21 LLR: -2.92 (-2.94,2.94) <0.00,2.00> Total: 13184 W: 3285 L: 3535 D: 6364 Ptnml(0-2): 85, 1662, 3334, 1440, 71 Failed 180+1.8 VLTC: https://tests.stockfishchess.org/tests/view/6505cf9a72620bc881ea908e LLR: -2.94 (-2.94,2.94) <0.00,2.00> Total: 64248 W: 16224 L: 16374 D: 31650 Ptnml(0-2): 26, 6788, 18640, 6650, 20 Passed 60+0.6 th 8 VLTC SMP (STC bounds): https://tests.stockfishchess.org/tests/view/65084a4618698b74c2e541dc LLR: 2.95 (-2.94,2.94) <0.00,2.00> Total: 90630 W: 23372 L: 23033 D: 44225 Ptnml(0-2): 13, 8490, 27968, 8833, 11 Passed 60+0.6 th 8 VLTC SMP: https://tests.stockfishchess.org/tests/view/6501d45d2cd016da89abacdb LLR: 2.95 (-2.94,2.94) <0.50,2.50> Total: 137804 W: 35764 L: 35276 D: 66764 Ptnml(0-2): 31, 13006, 42326, 13522, 17 closes https://github.com/official-stockfish/Stockfish/pull/4795 bench 1246812 |
||
|
Sebastian Buchwald
|
952740b36c |
Let CI check C++ includes
The commit adds a CI workflow that uses the included-what-you-use (IWYU) tool to check for missing or superfluous includes in .cpp files and their corresponding .h files. This means that some .h files (especially in the nnue folder) are not checked yet. The CI setup looks like this: - We build IWYU from source to include some yet unreleased fixes. This IWYU version targets LLVM 17. Thus, we get the latest release candidate of LLVM 17 from LLVM's nightly packages. - The Makefile now has an analyze target that just build the object files (without linking) - The CI uses the analyze target with the IWYU tool as compiler to analyze the compiled .cpp file and its corresponding .h file. - If IWYU suggests a change the build fails (-Xiwyu --error). - To avoid false positives we use LLVM's libc++ as standard library - We have a custom mappings file that adds some mappings that are missing in IWYU's default mappings We also had to add one IWYU pragma to prevent a false positive in movegen.h. https://github.com/official-stockfish/Stockfish/pull/4783 No functional change |
||
|
Linmiao Xu
|
3d1b067d85 |
Update default net to nn-1ee1aba5ed4c.nnue
Created by retraining the master net on a dataset composed by: - adding Leela data from T60 jul-dec 2020, T77 nov 2021, T80 jun-jul 2023 - deduplicating and unminimizing parts of the dataset before interleaving Trained initially with max epoch 800, then increased near the end of training twice. First to 960, then 1200. After training, post-processing involved: - greedy permuting L1 weights with https://github.com/official-stockfish/Stockfish/pull/4620 - greedy 2- and 3- cycle permuting with https://github.com/official-stockfish/Stockfish/pull/4640 python3 easy_train.py \ --experiment-name 2048-retrain-S6-sk28 \ --training-dataset /data/S6.binpack \ --early-fen-skipping 28 \ --start-from-engine-test-net True \ --max_epoch 1200 \ --lr 4.375e-4 \ --gamma 0.995 \ --start-lambda 1.0 \ --end-lambda 0.7 \ --tui False \ --seed $RANDOM \ --gpus 0 In the list of datasets below, periods in the filename represent the sequence of steps applied to arrive at the particular binpack. For example: test77-dec2021-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack 1. test77 dec2021 data rescored with 16 TB of syzygy tablebases during data conversion 2. filtered with csv_filter_v6_dd.py - v6 filtering and deduplication in one step 3. minimized with the original mar2023 implementation of `minimize_binpack` in the tools branch 4. unminimized by removing all positions with score == 32002 (`VALUE_NONE`) Binpacks were: - filtered with: https://github.com/linrock/nnue-data - unminimized with: https://github.com/linrock/Stockfish/tree/tools-unminify - deduplicated with: https://github.com/linrock/Stockfish/tree/tools-dd DATASETS=( leela96-filt-v2.min.unminimized.binpack dfrc99-16tb7p-eval-filt-v2.min.unminimized.binpack # most of the 0dd1cebea57 v6-dd dataset (without test80-jul2022) # https://github.com/official-stockfish/Stockfish/pull/4606 test60-novdec2021-12tb7p.filter-v6-dd.min-mar2023.unminimized.binpack test77-dec2021-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack test78-jantomay2022-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack test78-juntosep2022-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack test79-apr2022-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack test79-may2022-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack test80-jun2022-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack test80-aug2022-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack test80-sep2022-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack test80-oct2022-16tb7p.filter-v6-dd.min.binpack test80-nov2022-16tb7p.filter-v6-dd.min.binpack test80-jan2023-3of3-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack test80-feb2023-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack # older Leela data, recently converted test60-octnovdec2020-2tb7p.min.unminimized.binpack test60-julaugsep2020-2tb7p.min.binpack test77-nov2021-2tb7p.min.dd.binpack # newer Leela data test80-mar2023-2tb7p.min.unminimized.binpack test80-apr2023-2tb7p.filter-v6-sk16.min.unminimized.binpack test80-may2023-2tb7p.min.dd.binpack test80-jun2023-2tb7p.min.binpack test80-jul2023-2tb7p.binpack ) python3 interleave_binpacks.py ${DATASETS[@]} /data/S6.binpack Training data can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move: nn-epoch1059 : 2.7 +/- 1.6 Passed STC: https://tests.stockfishchess.org/tests/view/64fc8d705dab775b5359db42 LLR: 2.93 (-2.94,2.94) <0.00,2.00> Total: 168352 W: 43216 L: 42704 D: 82432 Ptnml(0-2): 599, 19672, 43134, 20160, 611 Passed LTC: https://tests.stockfishchess.org/tests/view/64fd44a75dab775b5359f065 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 154194 W: 39436 L: 38881 D: 75877 Ptnml(0-2): 78, 16577, 43238, 17120, 84 closes https://github.com/official-stockfish/Stockfish/pull/4782 Bench: 1603079 |
||
|
Stéphane Nicolet
|
b25d68f6ee |
Introduce simple_eval() for lazy evaluations
This patch implements the pure materialistic evaluation called simple_eval() to gain a speed-up during Stockfish search. We use the so-called lazy evaluation trick: replace the accurate but slow NNUE network evaluation by the super-fast simple_eval() if the position seems to be already won (high material advantage). To guard against some of the most obvious blunders introduced by this idea, this patch uses the following features which will raise the lazy evaluation threshold in some situations: - avoid lazy evals on shuffling branches in the search tree - avoid lazy evals if the position at root already has a material imbalance - avoid lazy evals if the search value at root is already winning/losing. Moreover, we add a small random noise to the simple_eval() term. This idea (stochastic mobility in the minimax tree) was worth about 200 Elo in the pure simple_eval() player on Lichess. Overall, the current implementation in this patch evaluates about 2% of the leaves in the search tree lazily. -------------------------------------------- STC: LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 60352 W: 15585 L: 15234 D: 29533 Ptnml(0-2): 216, 6906, 15578, 7263, 213 https://tests.stockfishchess.org/tests/view/64f1d9bcbd9967ffae366209 LTC: LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 35106 W: 8990 L: 8678 D: 17438 Ptnml(0-2): 14, 3668, 9887, 3960, 24 https://tests.stockfishchess.org/tests/view/64f25204f5b0c54e3f04c0e7 verification run at VLTC: LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 74362 W: 19088 L: 18716 D: 36558 Ptnml(0-2): 6, 7226, 22348, 7592, 9 https://tests.stockfishchess.org/tests/view/64f2ecdbf5b0c54e3f04d3ae All three tests above were run with adjudication off, we also verified that there was no regression on matetracker (thanks Disservin!). ---------------------------------------------- closes https://github.com/official-stockfish/Stockfish/pull/4771 Bench: 1393714 |
||
|
Disservin
|
3c0e86a91e |
Cleanup includes
Reorder a few includes, include "position.h" where it was previously missing and apply include-what-you-use suggestions. Also make the order of the includes consistent, in the following way: 1. Related header (for .cpp files) 2. A blank line 3. C/C++ headers 4. A blank line 5. All other header files closes https://github.com/official-stockfish/Stockfish/pull/4763 fixes https://github.com/official-stockfish/Stockfish/issues/4707 No functional change |
||
|
Sebastian Buchwald
|
f972947492 |
Cleanup code after removal of classical evaluation
This includes the following changes: - Remove declaration of removed global variable - Adapt string that mentions removed UCI option closes https://github.com/official-stockfish/Stockfish/pull/4675 No functional change |
||
|
Linmiao Xu
|
e699fee513 |
Update default net to nn-c38c3d8d3920.nnue
This was a later epoch from the same experiment that led to the previous master net. After training, it was prepared the same way: 1. greedy permuting L1 weights with https://github.com/official-stockfish/Stockfish/pull/4620 2. leb128 compression with https://github.com/glinscott/nnue-pytorch/pull/251 3. greedy 2- and 3- cycle permuting with https://github.com/official-stockfish/Stockfish/pull/4640 Local elo at 25k nodes per move (vs. L1-1536 nn-fdc1d0fe6455.nnue): nn-epoch739.nnue : 20.2 +/- 1.7 Passed STC: https://tests.stockfishchess.org/tests/view/64a050b33ee09aa549c4e4c8 LLR: 2.95 (-2.94,2.94) <0.00,2.00> Total: 195552 W: 49977 L: 49430 D: 96145 Ptnml(0-2): 556, 22775, 50607, 23242, 596 Passed LTC: https://tests.stockfishchess.org/tests/view/64a127bd3ee09aa549c4f60c LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 235452 W: 60327 L: 59609 D: 115516 Ptnml(0-2): 119, 25173, 66426, 25887, 121 closes https://github.com/official-stockfish/Stockfish/pull/4666 bench 2427629 |
||
|
Linmiao Xu
|
915532181f |
Update NNUE architecture to SFNNv7 with larger L1 size of 2048
Creating this net involved: - a 5-step training process from scratch - greedy permuting L1 weights with https://github.com/official-stockfish/Stockfish/pull/4620 - leb128 compression with https://github.com/glinscott/nnue-pytorch/pull/251 - greedy 2- and 3- cycle permuting with https://github.com/official-stockfish/Stockfish/pull/4640 The 5 training steps were: 1. 400 epochs, lambda 1.0, lr 9.75e-4 UHOx2-wIsRight-multinet-dfrc-n5000-largeGensfen-d9.binpack (178G) nodes5000pv2_UHO.binpack data_pv-2_diff-100_nodes-5000.binpack wrongIsRight_nodes5000pv2.binpack multinet_pv-2_diff-100_nodes-5000.binpack dfrc_n5000.binpack large_gensfen_multipvdiff_100_d9.binpack ep399 chosen as start model for step2 2. 800 epochs, end-lambda 0.75, skip 16 LeelaFarseer-T78juntoaugT79marT80dec.binpack (141G) T60T70wIsRightFarseerT60T74T75T76.binpack test78-junjulaug2022-16tb7p.no-db.min.binpack test79-mar2022-16tb7p.no-db.min.binpack test80-dec2022-16tb7p.no-db.min.binpack ep559 chosen as start model for step3 3. 800 epochs, end-lambda 0.725, skip 20 leela96-dfrc99-v2-T80dectofeb-sk20-mar-v6-T77decT78janfebT79apr.binpack (223G) leela96-filt-v2.min.binpack dfrc99-16tb7p-eval-filt-v2.min.binpack test80-dec2022-16tb7p-filter-v6-sk20.min-mar2023.binpack test80-jan2023-16tb7p-filter-v6-sk20.min-mar2023.binpack test80-feb2023-16tb7p-filter-v6-sk20.min-mar2023.binpack test80-mar2023-2tb7p-filter-v6.min.binpack test77-dec2021-16tb7p.no-db.min.binpack test78-janfeb2022-16tb7p.no-db.min.binpack test79-apr2022-16tb7p.no-db.min.binpack ep499 chosen as start model for step4 4. 800 epochs, end-lambda 0.7, skip 24 0dd1cebea57 dataset https://github.com/official-stockfish/Stockfish/pull/4606 ep599 chosen as start model for step5 5. 800 epochs, end-lambda 0.7, skip 28 same dataset as step4 ep619 became nn-1b951f8b449d.nnue For the final step5 training: python3 easy_train.py \ --experiment-name L1-2048-S5-sameData-sk28-S4-0dd1cebea57-shuffled-S3-leela96-dfrc99-v2-T80dectofeb-sk20-mar-v6-T77decT78janfebT79apr-sk20-S2-LeelaFarseerT78T79T80-ep399-S1-UHOx2-wIsRight-multinet-dfrc-n5000-largeGensfen-d9 \ --training-dataset /data/leela96-dfrc99-T60novdec-v2-T80juntonovjanfebT79aprmayT78jantosepT77dec-v6dd-T80apr.binpack \ --early-fen-skipping 28 \ --nnue-pytorch-branch linrock/nnue-pytorch/misc-fixes-L1-2048 \ --engine-test-branch linrock/Stockfish/L1-2048 \ --start-from-engine-test-net False \ --start-from-model /data/experiments/experiment_L1-2048-S4-0dd1cebea57-shuffled-S3-leela96-dfrc99-v2-T80dectofeb-sk20-mar-v6-T77decT78janfebT79apr-sk20-S2-LeelaFarseerT78T79T80-ep399-S1-UHOx2-wIsRight-multinet-dfrc-n5000-largeGensfen-d9/training/run_0/nn-epoch599.nnue --max_epoch 800 \ --lr 4.375e-4 \ --gamma 0.995 \ --start-lambda 1.0 \ --end-lambda 0.7 \ --tui False \ --seed $RANDOM \ --gpus 0 SF training data components for the step1 dataset: https://drive.google.com/drive/folders/1yLCEmioC3Xx9KQr4T7uB6GnLm5icAYGU Leela training data for steps 2-5 can be found at: https://robotmoon.com/nnue-training-data/ Due to larger L1 size and slower inference, the speed penalty loses elo at STC. Measurements from 100 bench runs at depth 13 with x86-64-modern on Intel Core i5-1038NG7 2.00GHz: sf_base = 1240730 +/- 3443 (95%) sf_test = 1153341 +/- 2832 (95%) diff = -87388 +/- 1616 (95%) speedup = -7.04330% +/- 0.130% (95%) Local elo at 25k nodes per move (vs. L1-1536 nn-fdc1d0fe6455.nnue): nn-epoch619.nnue : 21.1 +/- 3.2 Failed STC: https://tests.stockfishchess.org/tests/view/6498ee93dc7002ce609cf979 LLR: -2.95 (-2.94,2.94) <0.00,2.00> Total: 11680 W: 3058 L: 3299 D: 5323 Ptnml(0-2): 44, 1422, 3149, 1181, 44 LTC: https://tests.stockfishchess.org/tests/view/649b32f5dc7002ce609d20cf Elo: 0.68 ± 1.5 (95%) LOS: 80.5% Total: 40000 W: 10887 L: 10809 D: 18304 Ptnml(0-2): 36, 3938, 11958, 4048, 20 nElo: 1.50 ± 3.4 (95%) PairsRatio: 1.02 Passed VLTC 180+1.8: https://tests.stockfishchess.org/tests/view/64992b43dc7002ce609cfd20 LLR: 3.06 (-2.94,2.94) <0.00,2.00> Total: 38086 W: 10612 L: 10338 D: 17136 Ptnml(0-2): 9, 3316, 12115, 3598, 5 Passed VLTC SMP 60+0.6 th 8: https://tests.stockfishchess.org/tests/view/649a21fedc7002ce609d0c7d LLR: 2.95 (-2.94,2.94) <0.50,2.50> Total: 38936 W: 11091 L: 10820 D: 17025 Ptnml(0-2): 1, 2948, 13305, 3207, 7 closes https://github.com/official-stockfish/Stockfish/pull/4646 Bench: 2505168 |
||
|
Daniel Monroe
|
ef94f77f8c |
Update default net to nn-a3d1bfca1672.nnue
faster permutation of master net weights
Activation data taken from https://drive.google.com/drive/folders/1Ec9YuuRx4N03GPnVPoQOW70eucOKngQe?usp=sharing
Permutation found using
|
||
|
Linmiao Xu
|
a49b3ba7ed |
Update default net to nn-5af11540bbfe.nnue
Created by retraining the sparsified master net (nn-cd2ff4716c34.nnue) on a 100% minified dataset including Leela transformers data from T80 may2023. Weights permuted with the exact methods and code in: https://github.com/official-stockfish/Stockfish/pull/4620 LEB128 compression done with the new serialize.py param in: https://github.com/glinscott/nnue-pytorch/pull/251 Initially trained with max epoch 800. Around epoch 780, training was paused and max epoch raised to 960. python3 easy_train.py \ --experiment-name L1-1536-sparse-master-retrain \ --training-dataset /data/leela96-dfrc99-v2-T60novdecT77decT78jantosepT79aprmayT80juntonovjan-v6dd-T80febtomay2023.min.binpack \ --early-fen-skipping 27 \ --start-from-engine-test-net True \ --max_epoch 960 \ --lr 4.375e-4 \ --gamma 0.995 \ --start-lambda 1.0 \ --end-lambda 0.7 \ --tui False \ --seed $RANDOM \ --gpus 0 For preparing the training dataset (interleaved size 328G): python3 interleave_binpacks.py \ leela96-filt-v2.min.binpack \ dfrc99-16tb7p-eval-filt-v2.min.binpack \ filt-v6-dd-min/test60-novdec2021-12tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test77-dec2021-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test78-jantomay2022-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test78-juntosep2022-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test79-apr2022-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test79-may2022-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test80-jun2022-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test80-jul2022-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test80-aug2022-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test80-sep2022-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test80-oct2022-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test80-nov2022-16tb7p-filter-v6-dd.min.binpack \ filt-v6-dd-min/test80-jan2023-16tb7p-filter-v6-dd.min.binpack \ test80-2023/test80-feb2023-16tb7p-no-db.min.binpack \ test80-2023/test80-mar2023-2tb7p-no-db.min.binpack \ test80-2023/test80-apr2023-2tb7p-no-db.min.binpack \ test80-2023/test80-may2023-2tb7p-no-db.min.binpack \ /data/leela96-dfrc99-v2-T60novdecT77decT78jantosepT79aprmayT80juntonovjan-v6dd-T80febtomay2023.min.binpack Minified binpacks and Leela T80 training data from 2023 available at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move: nn-epoch879.nnue : 3.9 +/- 5.7 Passed STC: https://tests.stockfishchess.org/tests/view/64928c1bdc7002ce609c7690 LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 72000 W: 19242 L: 18889 D: 33869 Ptnml(0-2): 182, 7787, 19716, 8126, 189 Passed LTC: https://tests.stockfishchess.org/tests/view/64930a37dc7002ce609c82e3 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 54552 W: 14978 L: 14647 D: 24927 Ptnml(0-2): 23, 5123, 16650, 5460, 20 closes https://github.com/official-stockfish/Stockfish/pull/4635 bench 2593605 |
||
|
maxim
|
a46087ee30 |
Compressed network parameters
Implemented LEB128 (de)compression for the feature transformer. Reduces embedded network size from 70 MiB to 39 Mib. The new nn-78bacfcee510.nnue corresponds to the master net compressed. closes https://github.com/official-stockfish/Stockfish/pull/4617 No functional change |
||
|
AndrovT
|
32d3284df5 |
Permute master net weights to increase sparsity
Activation data collection using
|
||
|
AndrovT
|
38e61663d8 |
Use block sparse input for the first layer.
Use block sparse input for the first fully connected layer on architectures with at least SSSE3. Depending on the CPU architecture, this yields a speedup of up to 10%, e.g. ``` Result of 100 runs of 'bench 16 1 13 default depth NNUE' base (...ockfish-base) = 959345 +/- 7477 test (...ckfish-patch) = 1054340 +/- 9640 diff = +94995 +/- 3999 speedup = +0.0990 P(speedup > 0) = 1.0000 CPU: 8 x AMD Ryzen 7 5700U with Radeon Graphics Hyperthreading: on ``` Passed STC: https://tests.stockfishchess.org/tests/view/6485aa0965ffe077ca12409c LLR: 2.93 (-2.94,2.94) <0.00,2.00> Total: 8864 W: 2479 L: 2223 D: 4162 Ptnml(0-2): 13, 829, 2504, 1061, 25 This commit includes a net with reordered weights, to increase the likelihood of block sparse inputs, but otherwise equivalent to the previous master net (nn-ea57bea57e32.nnue). Activation data collected with https://github.com/AndrovT/Stockfish/tree/log-activations, running bench 16 1 13 varied_1000.epd depth NNUE on this data. Net parameters permuted with https://gist.github.com/AndrovT/9e3fbaebb7082734dc84d27e02094cb3. closes https://github.com/official-stockfish/Stockfish/pull/4612 No functional change |
||
|
Linmiao Xu
|
932f5a2d65 |
Update default net to nn-ea57bea57e32.nnue
Created by retraining an earlier epoch (ep659) of the experiment that led to the first SFNNv6 net: - First retrained on the nn-0dd1cebea573 dataset - Then retrained with skip 20 on a smaller dataset containing unfiltered Leela data - And then retrained again with skip 27 on the nn-0dd1cebea573 dataset The equivalent 7-step training sequence from scratch that led here was: 1. max-epoch 400, lambda 1.0, constant LR 9.75e-4, T79T77-filter-v6-dd.min.binpack ep379 chosen for retraining in step2 2. max-epoch 800, end-lambda 0.75, T60T70wIsRightFarseerT60T74T75T76.binpack ep679 chosen for retraining in step3 3. max-epoch 800, end-lambda 0.75, skip 28, nn-e1fb1ade4432 dataset ep799 chosen for retraining in step4 4. max-epoch 800, end-lambda 0.7, skip 28, nn-e1fb1ade4432 dataset ep759 became nn-8d69132723e2.nnue (first SFNNv6 net) ep659 chosen for retraining in step5 5. max-epoch 800, end-lambda 0.7, skip 28, nn-0dd1cebea573 dataset ep759 chosen for retraining in step6 6. max-epoch 800, end-lambda 0.7, skip 20, leela-dfrc-v2-T77decT78janfebT79aprT80apr.binpack ep639 chosen for retraining in step7 7. max-epoch 800, end-lambda 0.7, skip 27, nn-0dd1cebea573 dataset ep619 became nn-ea57bea57e32.nnue For the last retraining (step7): python3 easy_train.py --experiment-name L1-1536-Re6-masterShuffled-ep639-sk27-Re5-leela-dfrc-v2-T77toT80small-Re4-masterShuffled-ep659-Re3-sameAs-Re2-leela96-dfrc99-16t-v2-T60novdecT80juntonovjanfebT79aprmayT78jantosepT77dec-v6dd-Re1-LeelaFarseer-new-T77T79 \ --training-dataset /data/leela96-dfrc99-T60novdec-v2-T80juntonovjanfebT79aprmayT78jantosepT77dec-v6dd-T80apr.binpack \ --nnue-pytorch-branch linrock/nnue-pytorch/misc-fixes-L1-1536 \ --early-fen-skipping 27 \ --start-lambda 1.0 \ --end-lambda 0.7 \ --max_epoch 800 \ --start-from-engine-test-net False \ --start-from-model /data/L1-1536-Re5-leela-dfrc-v2-T77toT80small-epoch639.nnue \ --lr 4.375e-4 \ --gamma 0.995 \ --tui False \ --seed $RANDOM \ --gpus "0," For preparing the step6 leela-dfrc-v2-T77decT78janfebT79aprT80apr.binpack dataset: python3 interleave_binpacks.py \ leela96-filt-v2.binpack \ dfrc99-16tb7p-eval-filt-v2.binpack \ test77-dec2021-16tb7p.no-db.min-mar2023.binpack \ test78-janfeb2022-16tb7p.no-db.min-mar2023.binpack \ test79-apr2022-16tb7p-filter-v6-dd.binpack \ test80-apr2022-16tb7p.no-db.min-mar2023.binpack \ /data/leela-dfrc-v2-T77decT78janfebT79aprT80apr.binpack The unfiltered Leela data used for the step6 dataset can be found at: https://robotmoon.com/nnue-training-data Local elo at 25k nodes per move: nn-epoch619.nnue : 2.3 +/- 1.9 Passed STC: https://tests.stockfishchess.org/tests/view/6480d43c6e6ce8d9fc6d7cc8 LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 40992 W: 11017 L: 10706 D: 19269 Ptnml(0-2): 113, 4400, 11170, 4689, 124 Passed LTC: https://tests.stockfishchess.org/tests/view/648119ac6e6ce8d9fc6d8208 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 129174 W: 35059 L: 34579 D: 59536 Ptnml(0-2): 66, 12548, 38868, 13050, 55 closes https://github.com/official-stockfish/Stockfish/pull/4611 bench: 2370027 |
||
|
Linmiao Xu
|
373359b44d |
Update default net to nn-0dd1cebea573.nnue
Created by retraining an earlier epoch of the experiment leading to the first SFNNv6 net on a more-randomized version of the nn-e1fb1ade4432.nnue dataset mixed with unfiltered T80 apr2023 data. Trained using early-fen-skipping 28 and max-epoch 960. The trainer settings and epochs used in the 5-step training sequence leading here were: 1. train from scratch for 400 epochs, lambda 1.0, constant LR 9.75e-4, T79T77-filter-v6-dd.min.binpack 2. retrain ep379, max-epoch 800, end-lambda 0.75, T60T70wIsRightFarseerT60T74T75T76.binpack 3. retrain ep679, max-epoch 800, end-lambda 0.75, skip 28, nn-e1fb1ade4432 dataset 4. retrain ep799, max-epoch 800, end-lambda 0.7, skip 28, nn-e1fb1ade4432 dataset 5. retrain ep439, max-epoch 960, end-lambda 0.7, skip 28, shuffled nn-e1fb1ade4432 + T80 apr2023 This net was epoch 559 of the final (step 5) retraining: ```bash python3 easy_train.py \ --experiment-name L1-1536-Re4-leela96-dfrc99-T60novdec-v2-T80juntonovjanfebT79aprmayT78jantosepT77dec-v6dd-T80apr-shuffled-sk28 \ --training-dataset /data/leela96-dfrc99-T60novdec-v2-T80juntonovjanfebT79aprmayT78jantosepT77dec-v6dd-T80apr.binpack \ --nnue-pytorch-branch linrock/nnue-pytorch/misc-fixes-L1-1536 \ --early-fen-skipping 28 \ --start-lambda 1.0 \ --end-lambda 0.7 \ --max_epoch 960 \ --start-from-engine-test-net False \ --start-from-model /data/L1-1536-Re3-nn-epoch439.nnue \ --engine-test-branch linrock/Stockfish/L1-1536 \ --lr 4.375e-4 \ --gamma 0.995 \ --tui False \ --seed $RANDOM \ --gpus "0," ``` During data preparation, most binpacks were unminimized by removing positions with score 32002 (`VALUE_NONE`). This makes the tradeoff of increasing dataset filesize on disk to increase the randomness of positions in interleaved datasets. The code used for unminimizing is at: https://github.com/linrock/Stockfish/tree/tools-unminify For preparing the dataset used in this experiment: ```bash python3 interleave_binpacks.py \ leela96-filt-v2.binpack \ dfrc99-16tb7p-eval-filt-v2.binpack \ filt-v6-dd-min/test60-novdec2021-12tb7p-filter-v6-dd.min-mar2023.unmin.binpack \ filt-v6-dd-min/test80-aug2022-16tb7p-filter-v6-dd.min-mar2023.unmin.binpack \ filt-v6-dd-min/test80-sep2022-16tb7p-filter-v6-dd.min-mar2023.unmin.binpack \ filt-v6-dd-min/test80-jun2022-16tb7p-filter-v6-dd.min-mar2023.unmin.binpack \ filt-v6-dd/test80-jul2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test80-oct2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test80-nov2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd-min/test80-jan2023-3of3-16tb7p-filter-v6-dd.min-mar2023.unmin.binpack \ filt-v6-dd-min/test80-feb2023-16tb7p-filter-v6-dd.min-mar2023.unmin.binpack \ filt-v6-dd/test79-apr2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test79-may2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd-min/test78-jantomay2022-16tb7p-filter-v6-dd.min-mar2023.unmin.binpack \ filt-v6-dd/test78-juntosep2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test77-dec2021-16tb7p-filter-v6-dd.binpack \ test80-apr2023-2tb7p.binpack \ /data/leela96-dfrc99-T60novdec-v2-T80juntonovjanfebT79aprmayT78jantosepT77dec-v6dd-T80apr.binpack ``` T80 apr2023 data was converted using lc0-rescorer with ~2tb of tablebases and can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move vs. nn-e1fb1ade4432.nnue (L1 size 1024): nn-epoch559.nnue : 25.7 +/- 1.6 Passed STC: https://tests.stockfishchess.org/tests/view/647cd3b87cf638f0f53f9cbb LLR: 2.95 (-2.94,2.94) <0.00,2.00> Total: 59200 W: 16000 L: 15660 D: 27540 Ptnml(0-2): 159, 6488, 15996, 6768, 189 Passed LTC: https://tests.stockfishchess.org/tests/view/647d58de726f6b400e4085d8 LLR: 2.95 (-2.94,2.94) <0.50,2.50> Total: 58800 W: 16002 L: 15657 D: 27141 Ptnml(0-2): 44, 5607, 17748, 5962, 39 closes https://github.com/official-stockfish/Stockfish/pull/4606 bench 2141197 |
||
|
Linmiao Xu
|
c1fff71650 |
Update NNUE architecture to SFNNv6 with larger L1 size of 1536
Created by training a new net from scratch with L1 size increased from 1024 to 1536. Thanks to Vizvezdenec for the idea of exploring larger net sizes after recent training data improvements. A new net was first trained with lambda 1.0 and constant LR 8.75e-4. Then a strong net from a later epoch in the training run was chosen for retraining with start-lambda 1.0 and initial LR 4.375e-4 decaying with gamma 0.995. Retraining was performed a total of 3 times, for this 4-step process: 1. 400 epochs, lambda 1.0 on filtered T77+T79 v6 deduplicated data 2. 800 epochs, end-lambda 0.75 on T60T70wIsRightFarseerT60T74T75T76.binpack 3. 800 epochs, end-lambda 0.75 and early-fen-skipping 28 on the master dataset 4. 800 epochs, end-lambda 0.7 and early-fen-skipping 28 on the master dataset In the training sequence that reached the new nn-8d69132723e2.nnue net, the epochs used for the 3x retraining runs were: 1. epoch 379 trained on T77T79-filter-v6-dd.min.binpack 2. epoch 679 trained on T60T70wIsRightFarseerT60T74T75T76.binpack 3. epoch 799 trained on the master dataset For training from scratch: python3 easy_train.py \ --experiment-name new-L1-1536-T77T79-filter-v6dd \ --training-dataset /data/T77T79-filter-v6-dd.min.binpack \ --max_epoch 400 \ --lambda 1.0 \ --start-from-engine-test-net False \ --engine-test-branch linrock/Stockfish/L1-1536 \ --nnue-pytorch-branch linrock/Stockfish/misc-fixes-L1-1536 \ --tui False \ --gpus "0," \ --seed $RANDOM Retraining commands were similar to each other. For the 3rd retraining run: python3 easy_train.py \ --experiment-name L1-1536-T77T79-v6dd-Re1-LeelaFarseer-Re2-masterDataset-Re3-sameData \ --training-dataset /data/leela96-dfrc99-v2-T60novdecT80juntonovjanfebT79aprmayT78jantosepT77dec-v6dd.binpack \ --early-fen-skipping 28 \ --max_epoch 800 \ --start-lambda 1.0 \ --end-lambda 0.7 \ --lr 4.375e-4 \ --gamma 0.995 \ --start-from-engine-test-net False \ --start-from-model /data/L1-1536-T77T79-v6dd-Re1-LeelaFarseer-Re2-masterDataset-nn-epoch799.nnue \ --engine-test-branch linrock/Stockfish/L1-1536 \ --nnue-pytorch-branch linrock/nnue-pytorch/misc-fixes-L1-1536 \ --tui False \ --gpus "0," \ --seed $RANDOM The T77+T79 data used is a subset of the master dataset available at: https://robotmoon.com/nnue-training-data/ T60T70wIsRightFarseerT60T74T75T76.binpack is available at: https://drive.google.com/drive/folders/1S9-ZiQa_3ApmjBtl2e8SyHxj4zG4V8gG Local elo at 25k nodes per move vs. nn-e1fb1ade4432.nnue (L1 size 1024): nn-epoch759.nnue : 26.9 +/- 1.6 Failed STC https://tests.stockfishchess.org/tests/view/64742485d29264e4cfa75f97 LLR: -2.94 (-2.94,2.94) <0.00,2.00> Total: 13728 W: 3588 L: 3829 D: 6311 Ptnml(0-2): 71, 1661, 3610, 1482, 40 Failing LTC https://tests.stockfishchess.org/tests/view/64752d7c4a36543c4c9f3618 LLR: -1.91 (-2.94,2.94) <0.50,2.50> Total: 35424 W: 9522 L: 9603 D: 16299 Ptnml(0-2): 24, 3579, 10585, 3502, 22 Passed VLTC 180+1.8 https://tests.stockfishchess.org/tests/view/64752df04a36543c4c9f3638 LLR: 2.95 (-2.94,2.94) <0.50,2.50> Total: 47616 W: 13174 L: 12863 D: 21579 Ptnml(0-2): 13, 4261, 14952, 4566, 16 Passed VLTC SMP 60+0.6 th 8 https://tests.stockfishchess.org/tests/view/647446ced29264e4cfa761e5 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 19942 W: 5694 L: 5451 D: 8797 Ptnml(0-2): 6, 1504, 6707, 1749, 5 closes https://github.com/official-stockfish/Stockfish/pull/4593 bench 2222567 |
||
|
Linmiao Xu
|
41f50b2c83 |
Update default net to nn-e1fb1ade4432.nnue
Created by retraining nn-dabb1ed23026.nnue with a dataset composed of: * The previous best dataset (nn-1ceb1a57d117.nnue dataset) * Adding de-duplicated T80 data from feb2023 and the last 10 days of jan2023, filtered with v6-dd Initially trained with the same options as the recent master net (nn-1ceb1a57d117.nnue). Around epoch 890, training was manually stopped and max epoch increased to 1000. ``` python3 easy_train.py \ --experiment-name leela96-dfrc99-T60novdec-v2-T80augsep-v6-T80junjuloctnovjanfebT79aprmayT78jantosepT77dec-v6dd \ --training-dataset /data/leela96-dfrc99-T60novdec-v2-T80augsep-v6-T80junjuloctnovjanfebT79aprmayT78jantosepT77dec-v6dd.binpack \ --nnue-pytorch-branch linrock/nnue-pytorch/misc-fixes \ --start-from-engine-test-net True \ --early-fen-skipping 30 \ --start-lambda 1.0 \ --end-lambda 0.7 \ --max_epoch 900 \ --lr 4.375e-4 \ --gamma 0.995 \ --tui False \ --gpus "0," \ --seed $RANDOM ``` The same v6-dd filtering and binpack minimizer was used for preparing the recent nn-1ceb1a57d117.nnue dataset. ``` python3 interleave_binpacks.py \ leela96-filt-v2.binpack \ dfrc99-filt-v2.binpack \ T60-nov2021-12tb7p-eval-filt-v2.binpack \ T60-dec2021-12tb7p-eval-filt-v2.binpack \ filt-v6/test80-aug2022-16tb7p-filter-v6.min-mar2023.binpack \ filt-v6/test80-sep2022-16tb7p-filter-v6.min-mar2023.binpack \ filt-v6-dd/test80-jun2022-16tb7p-filter-v6-dd.min-mar2023.binpack \ filt-v6-dd/test80-jul2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test80-oct2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test80-nov2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test80-jan2022-3of3-16tb7p-filter-v6-dd.min-mar2023.binpack \ filt-v6-dd/test80-feb2023-16tb7p-filter-v6-dd.min-mar2023.binpack \ filt-v6-dd/test79-apr2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test79-may2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test78-jantomay2022-16tb7p-filter-v6-dd.min-mar2023.binpack \ filt-v6-dd/test78-juntosep2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test77-dec2021-16tb7p-filter-v6-dd.binpack \ /data/leela96-dfrc99-T60novdec-v2-T80augsep-v6-T80junjuloctnovjanfebT79aprmayT78jantosepT77dec-v6dd.binpack ``` Links for downloading the training data components can be found at: https://robotmoon.com/nnue-training-data/ Local elo at 25k nodes per move: nn-epoch919.nnue : 2.6 +/- 2.8 Passed STC vs. nn-dabb1ed23026.nnue https://tests.stockfishchess.org/tests/view/644420df94ff3db5625f2af5 LLR: 2.94 (-2.94,2.94) <0.00,2.00> Total: 125960 W: 33898 L: 33464 D: 58598 Ptnml(0-2): 351, 13920, 34021, 14320, 368 Passed LTC vs. nn-1ceb1a57d117.nnue https://tests.stockfishchess.org/tests/view/64469f128d30316529b3dc46 LLR: 2.95 (-2.94,2.94) <0.50,2.50> Total: 24544 W: 6817 L: 6542 D: 11185 Ptnml(0-2): 8, 2252, 7488, 2505, 19 closes https://github.com/official-stockfish/Stockfish/pull/4546 bench 3714847 |
||
|
Linmiao Xu
|
c3ce220408 |
Created by retraining the master net with these changes to the dataset:
* Extending v6 filtering to data from T77 dec2021, T79 may2022, and T80 nov2022 * Reducing the number of duplicate positions, prioritizing position scores seen later in time * Using a binpack minimizer to reduce the overall data size Trained the same way as the previous master net, aside from the dataset changes: ``` python3 easy_train.py \ --experiment-name leela96-dfrc99-T60novdec-v2-T80augsep-v6-T80junjuloctnovT79aprmayT78jantosepT77dec-v6dd \ --training-dataset /data/leela96-dfrc99-T60novdec-v2-T80augsep-v6-T80junjuloctnovT79aprmayT78jantosepT77dec-v6dd.binpack \ --nnue-pytorch-branch linrock/nnue-pytorch/misc-fixes \ --start-from-engine-test-net True \ --early-fen-skipping 30 \ --start-lambda 1.0 \ --end-lambda 0.7 \ --max_epoch 900 \ --lr 4.375e-4 \ --gamma 0.995 \ --tui False \ --gpus "0," \ --seed $RANDOM ``` The new v6-dd filtering reduces duplicate positions by iterating over hourly data files within leela test runs, starting with the most recent, then keeping positions the first time they're seen and ignoring positions that are seen again. This ordering was done with the assumption that position scores seen later in time are generally more accurate than scores seen earlier in the test run. Positions are de-duplicated based on piece orientations, the first token in fen strings. The binpack minimizer was run with default settings after first merging monthly data into single binpacks. ``` python3 interleave_binpacks.py \ leela96-filt-v2.binpack \ dfrc99-filt-v2.binpack \ T60-nov2021-12tb7p-eval-filt-v2.binpack \ T60-dec2021-12tb7p-eval-filt-v2.binpack \ filt-v6/test80-aug2022-16tb7p-filter-v6.min-mar2023.binpack \ filt-v6/test80-sep2022-16tb7p-filter-v6.min-mar2023.binpack \ filt-v6-dd/test80-jun2022-16tb7p-filter-v6-dd.min-mar2023.binpack \ filt-v6-dd/test80-jul2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test80-oct2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test80-nov2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test79-apr2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test79-may2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test78-jantomay2022-16tb7p-filter-v6-dd.min-mar2023.binpack \ filt-v6-dd/test78-juntosep2022-16tb7p-filter-v6-dd.binpack \ filt-v6-dd/test77-dec2021-16tb7p-filter-v6-dd.binpack \ /data/leela96-dfrc99-T60novdec-v2-T80augsep-v6-T80junjuloctnovT79aprmayT78jantosepT77dec-v6dd.binpack ``` The code for v6-dd filtering is available along with training data preparation scripts at: https://github.com/linrock/nnue-data Links for downloading the training data components: https://robotmoon.com/nnue-training-data/ The binpack minimizer is from: #4447 Local elo at 25k nodes per move: nn-epoch859.nnue : 1.2 +/- 2.6 Passed STC: https://tests.stockfishchess.org/tests/view/643aad7db08900ff1bc5a832 LLR: 2.93 (-2.94,2.94) <0.00,2.00> Total: 565040 W: 150225 L: 149162 D: 265653 Ptnml(0-2): 1875, 62137, 153229, 63608, 1671 Passed LTC: https://tests.stockfishchess.org/tests/view/643ecf2fa43cf30e719d2042 LLR: 2.94 (-2.94,2.94) <0.50,2.50> Total: 1014840 W: 274645 L: 272456 D: 467739 Ptnml(0-2): 515, 98565, 306970, 100956, 414 closes https://github.com/official-stockfish/Stockfish/pull/4545 bench 3476305 |