`speedtest [threads] [hash_MiB] [time_s]`. `threads` default to system concurrency. `hash_MiB` defaults to `threads*128`. `time_s` defaults to 150.
Intended to be used with default parameters, as a stable hardware benchmark.
Example:
```
C:\dev\stockfish-master\src>stockfish.exe speedtest
Stockfish dev-20240928-nogit by the Stockfish developers (see AUTHORS file)
info string Using 16 threads
Warmup position 3/3
Position 258/258
===========================
Version : Stockfish dev-20240928-nogit
Compiled by : g++ (GNUC) 13.2.0 on MinGW64
Compilation architecture : x86-64-vnni256
Compilation settings : 64bit VNNI BMI2 AVX2 SSE41 SSSE3 SSE2 POPCNT
Compiler __VERSION__ macro : 13.2.0
Large pages : yes
User invocation : speedtest
Filled invocation : speedtest 16 2048 150
Available processors : 0-15
Thread count : 16
Thread binding : none
TT size [MiB] : 2048
Hash max, avg [per mille] :
single search : 40, 21
single game : 631, 428
Total nodes searched : 2099917842
Total search time [s] : 153.937
Nodes/second : 13641410
```
-------------------------------
Small unrelated tweaks:
- Network verification output is now handled as a callback.
- TT hashfull queries allow specifying maximum entry age.
closes https://github.com/official-stockfish/Stockfish/pull/5354
No functional change
This patch introduces three additional correction histories, namely,
Major Piece Correction History, Minor Piece Correction History, and
Non-Pawn Correction History.
Introduced by @mcthouacbb in Sirius
(https://github.com/mcthouacbb/Sirius) chess engine. The Major Piece
Correction History is indexed by side-to-move and the Zobrist key
representing the position of the King, Rook, and Queen of both sides.
Likewise, the Minor Piece Correction History is indexed by side-to-move
and the Zobrist key representing the position of the King, Knight, and
Bishop of both sides.
Also See:
97b85bbaac3099cdef2f
Introduced by @zzzzz151 in Starzix (https://github.com/zzzzz151/Starzix)
chess engine. Non-Pawn correction history consists of side-to-move, side
of Zobrist key, and a Zobrist key representing of the position of all
non-pawn pieces of **one side**. The non-pawn correction values for both
key sides are then summed.
Also See:
34911772f133e0df8dd2
The weights on the final correction value of the above correction
histories, as well as existing correction histories, are then tuned in
two separate SPSA sessions, totaling 75k games.
SPSA1:
https://tests.stockfishchess.org/tests/view/66e5243886d5ee47d953a86b
(Stopped early due to some weights reaching the maximum value)
SPSA2:
https://tests.stockfishchess.org/tests/view/66e6a26f86d5ee47d953a965
Also thanks to @martinnovaak, (Motor
https://github.com/martinnovaak/motor author) for insights and
suggestions.
Passed STC:
LLR: 2.94 (-2.94,2.94) <0.00,2.00>
Total: 23328 W: 6197 L: 5901 D: 11230
Ptnml(0-2): 82, 2582, 6041, 2876, 83
https://tests.stockfishchess.org/tests/view/66e8787b86d5ee47d953ab6f
Passed LTC:
LLR: 2.94 (-2.94,2.94) <0.50,2.50>
Total: 10626 W: 2826 L: 2560 D: 5240
Ptnml(0-2): 4, 1054, 2941, 1300, 14
https://tests.stockfishchess.org/tests/view/66e8ab2386d5ee47d953aba8
closes https://github.com/official-stockfish/Stockfish/pull/5598
Bench: 1011161
Idea of this patch comes from the fact that current history heuristics
are mostly populated by low depth entries since our stat bonus reaches
maximum value at depth 5-6 and number of low depth nodes is much bigger
than number of high depth nodes. But it doesn't make a whole lost of
sense to use this low-depth centered histories to sort moves at root.
Current patch introduces special history table that is used exclusively
at root, it remembers which quiet moves were good and which quiet moves
were not good there and uses this information for move ordering.
Passed STC:
https://tests.stockfishchess.org/tests/view/66dda74adc53972b68218cc9
LLR: 2.93 (-2.94,2.94) <0.00,2.00>
Total: 127680 W: 33579 L: 33126 D: 60975
Ptnml(0-2): 422, 15098, 32391, 15463, 466
Passed LTC:
https://tests.stockfishchess.org/tests/view/66dead2adc53972b68218d34
LLR: 2.95 (-2.94,2.94) <0.50,2.50>
Total: 381978 W: 96958 L: 95923 D: 189097
Ptnml(0-2): 277, 42165, 105089, 43162, 296
closes https://github.com/official-stockfish/Stockfish/pull/5595
Bench: 1611283
Since an unknown amount of time the instrumented CI has been a bit
flawed, explained here
https://github.com/official-stockfish/Stockfish/issues/5185. It also
experiences random timeout issues where restarting the workflow fixes it
or very long run times (more than other workflows) and is not very
portable.
The intention of this commit is to port the instrumented.sh to python
which also works on other operating systems. It should also be
relatively easy for beginners to add new tests to assert stockfish's
output and to run it.
From the source directory the following command can be run.
`python3 ../tests/instrumented.py --none ./stockfish`
A test runner will go over the test suites and run the test cases.
All instrumented tests should have been ported over.
The required python version for this is should be 3.7 (untested) + the
requests package, testing.py includes some infrastructure code which
setups the testing.
fixes https://github.com/official-stockfish/Stockfish/issues/5185
closes https://github.com/official-stockfish/Stockfish/pull/5583
No functional change
The recent commit introduced a bug in the net downloading script that
the file is not downloaded correctly and the content is redirected to
stdout.
closes https://github.com/official-stockfish/Stockfish/pull/5585
No functional change
- A small quality of file change is to change the type of engine path
from a string to an optional string, skips the binary directory
lookup, which is commonly disabled by people who create wasm builds or
include stockfish as a library.
closes https://github.com/official-stockfish/Stockfish/pull/5575
No functional change
Fixes https://github.com/official-stockfish/Stockfish/issues/5564
This patch extracts the net downloading script in Makefile into an
external script file. Also the script is moderately rewritten for
improved readability and speed.
* Use wget preferentially over curl, as curl is known to have slight
overhead.
* Use command instead of hash to check if command exists. Reportedly,
hash always returns zero in some POSIX shells even when the command
fails.
* Command existence checks (wget/curl, sha256sum) are performed only
once at the beginning.
* Each of common patterns is encapsulated in a function
(get_nnue_filename, validate_network).
* Print out error/warning messages to stderr.
closes https://github.com/official-stockfish/Stockfish/pull/5563
No functional change
Co-authored-by: Disservin <disservin.social@gmail.com>
Official release version of Stockfish 17
Bench: 1484730
---
Stockfish 17
Today we have the pleasure to announce a new major release of Stockfish. As
always, you can freely download it at https://stockfishchess.org/download and
use it in the GUI of your choice.
Don’t forget to join our Discord server[1] to get in touch with the community
of developers and users of the project!
*Quality of chess play*
In tests against Stockfish 16, this release brings an Elo gain of up to 46
points[2] and wins up to 4.5 times more game pairs[3] than it loses. In
practice, high-quality moves are now found in less time, with a user upgrading
from Stockfish 14 being able to analyze games at least 6 times[4] faster with
Stockfish 17 while maintaining roughly the same quality.
During this development period, Stockfish won its 9th consecutive first place
in the main league of the Top Chess Engine Championship (TCEC)[5], and the 24th
consecutive first place in the main events (bullet, blitz, and rapid) of the
Computer Chess Championship (CCC)[6].
*Update highlights*
*Improved engine lines*
This release introduces principal variations (PVs) that are more informative
for mate and decisive table base (TB) scores. In both cases, the PV will
contain all moves up to checkmate. For mate scores, the PV shown is the best
variation known to the engine at that point, while for table base wins, it
follows, based on the TB, a sequence of moves that preserves the game outcome
to checkmate.
*NUMA performance optimization*
For high-end computers with multiple CPUs (typically a dual-socket architecture
with 100+ cores), this release automatically improves performance with a
`NumaPolicy` setting that optimizes non-uniform memory access (NUMA). Although
typical consumer hardware will not benefit, speedups of up to 2.8x[7] have been
measured.
*Shoutouts*
*ChessDB*
During the past 1.5 years, hundreds of cores have been continuously running
Stockfish to grow a database of analyzed positions. This chess cloud
database[8] now contains well over 45 billion positions, providing excellent
coverage of all openings and commonly played lines. This database is already
integrated into GUIs such as En Croissant[9] and Nibbler[10], which access it
through the public API.
*Leela Chess Zero*
Generally considered to be the strongest GPU engine, it continues to provide
open data which is essential for training our NNUE networks. They released
version 0.31.1[11] of their engine a few weeks ago, check it out!
*Website redesign*
Our website has undergone a redesign in recent months, most notably in our home
page[12], now featuring a darker color scheme and a more modern aesthetic,
while still maintaining its core identity. We hope you'll like it as much as we
do!
*Thank you*
The Stockfish project builds on a thriving community of enthusiasts (thanks
everybody!) who contribute their expertise, time, and resources to build a free
and open-source chess engine that is robust, widely available, and very strong.
We would like to express our gratitude for the 11k stars[13] that light up our
GitHub project! Thank you for your support and encouragement – your recognition
means a lot to us.
We invite our chess fans to join the Fishtest testing framework[14] to
contribute compute resources needed for development. Programmers can contribute
to the project either directly to Stockfish[15] (C++), to Fishtest[16] (HTML,
CSS, JavaScript, and Python), to our trainer nnue-pytorch[17] (C++ and Python),
or to our website[18] (HTML, CSS/SCSS, and JavaScript).
The Stockfish team
[1] https://discord.gg/GWDRS3kU6R
[2] https://tests.stockfishchess.org/tests/view/66d738ba9de3e7f9b33d159a
[3] https://tests.stockfishchess.org/tests/view/66d738f39de3e7f9b33d15a0
[4] https://github.com/official-stockfish/Stockfish/wiki/Useful-data#equivalent-time-odds-and-normalized-game-pair-elo
[5] https://en.wikipedia.org/wiki/Stockfish_(chess)#Top_Chess_Engine_Championship
[6] https://en.wikipedia.org/wiki/Stockfish_(chess)#Chess.com_Computer_Chess_Championship
[7] https://github.com/official-stockfish/Stockfish/pull/5285
[8] https://chessdb.cn/queryc_en/
[9] https://encroissant.org/
[10] https://github.com/rooklift/nibbler
[11] https://github.com/LeelaChessZero/lc0/releases/tag/v0.31.1
[12] https://stockfishchess.org/
[13] https://github.com/official-stockfish/Stockfish/stargazers
[14] https://github.com/official-stockfish/fishtest/wiki/Running-the-worker
[15] https://github.com/official-stockfish/Stockfish
[16] https://github.com/official-stockfish/fishtest
[17] https://github.com/official-stockfish/nnue-pytorch
[18] https://github.com/official-stockfish/stockfish-web
Created from 2 distinct spsa tunes of the latest main net (nn-31337bea577c.nnue)
and applying the params to the prior main net (nn-e8bac1c07a5a.nnue). This
effectively reverts the modifications to output weights and biases in
https://github.com/official-stockfish/Stockfish/pull/5509
SPSA:
A: 6000, alpha: 0.602, gamma: 0.101
1st - 437 feature transformer biases where values are < 25
54k / 120k games at 180+1.8
https://tests.stockfishchess.org/tests/view/66af98ac4ff211be9d4edad0
nn-808259761cca.nnue
2nd - 208 L2 weights where values are zero
112k / 120k games at 180+1.8
https://tests.stockfishchess.org/tests/view/66b0c3074ff211be9d4edbe5
nn-a56cb8c3d477.nnue
When creating the above 2 nets (nn-808259761cca.nnue, nn-a56cb8c3d477.nnue),
spsa params were unintentionally applied to nn-e8bac1c07a5a.nnue rather
than nn-31337bea577c.nnue due to an issue in a script that creates nets
by applying spsa results to base nets.
Since they both passed STC and were neutral or slightly positive at LTC,
they were combined to see if the elo from each set of params was additive.
The 2 nets can be merged on top of nn-e8bac1c07a5a.nnue with:
https://github.com/linrock/nnue-tools/blob/90942d3/spsa/combine_nnue.py
```
python3 combine_nnue.py \
nn-e8bac1c07a5a.nnue \
nn-808259761cca.nnue \
nn-a56cb8c3d477.nnue
```
Merging yields nn-87caa003fc6a.nnue which was renamed to nn-1111cefa1111.nnue
with an updated nnue-namer around 10x faster than before by:
- using a prefix trie for efficient prefix matches
- modifying 4 non-functional bytes near the end of the file instead of 2
https://github.com/linrock/nnue-namer
Thanks to @MinetaS for pointing out in #nnue-dev what the non-functional bytes are:
L3 is 32, 4 bytes for biases, 32 bytes for weights. (fc_2)
So -38 and -37 are technically -2 and -1 of fc_1 (type AffineTransform<30, 32>)
And since InputDimension is padded to 32 there are total 32 of 2 adjacent bytes padding.
So yes, it's non-functional whatever values are there.
It's possible to tweak bytes at -38 - 32 * N and -37 - 32 * N given N = 0 ... 31
The net renamed with the new method passed non-regression STC vs. the original net:
https://tests.stockfishchess.org/tests/view/66c0f0a821503a509c13b332
To print the spsa params with nnue-pytorch:
```
import features
from serialize import NNUEReader
feature_set = features.get_feature_set_from_name("HalfKAv2_hm")
with open("nn-31337bea577c.nnue", "rb") as f:
model = NNUEReader(f, feature_set).model
c_end = 16
for i,ft_bias in enumerate(model.input.bias.data[:3072]):
value = int(ft_bias * 254)
if abs(value) < 25:
print(f"ftB[{i}],{value},-1024,1024,{c_end},0.0020")
c_end = 6
for i in range(8):
for j in range(32):
for k in range(30):
value = int(model.layer_stacks.l2.weight.data[32 * i + j, k] * 64)
if value == 0:
print(f"twoW[{i}][{j}][{k}],{value},-127,127,{c_end},0.0020")
```
New params found with the same method as:
https://github.com/official-stockfish/Stockfish/pull/5459
Passed STC:
https://tests.stockfishchess.org/tests/view/66b4d4464ff211be9d4edf6e
LLR: 2.94 (-2.94,2.94) <0.00,2.00>
Total: 136416 W: 35753 L: 35283 D: 65380
Ptnml(0-2): 510, 16159, 34416, 16597, 526
Passed LTC:
https://tests.stockfishchess.org/tests/view/66b76e814ff211be9d4ee1cc
LLR: 2.95 (-2.94,2.94) <0.50,2.50>
Total: 159336 W: 40753 L: 40178 D: 78405
Ptnml(0-2): 126, 17497, 43864, 18038, 143
closes https://github.com/official-stockfish/Stockfish/pull/5534
bench 1613043
This patch moves the DotProd code into the propagation function which
has sequential access optimization. To prove the speedup, the comparison
is done without the sparse layer. With the sparse layer the effect is
marginal (GCC 0.3%, LLVM/Clang 0.1%).
For both tests, binary is compiled with GCC 14.1. Each test had 50 runs.
Sparse layer included:
```
speedup = +0.0030
P(speedup > 0) = 1.0000
```
Sparse layer excluded:
```
speedup = +0.0561
P(speedup > 0) = 1.0000
```
closes https://github.com/official-stockfish/Stockfish/pull/5520
No functional change
Since simplification of quiet checks in qsearch this depth isn't used by
any function at all apart movepicker, which also doesn't use passed
qsearch depth in any way, so can be removed. No functional change.
closes https://github.com/official-stockfish/Stockfish/pull/5514
No functional change
Created by updating output weights (256) and biases (8)
of the previous main net with values found with spsa around
101k / 120k games at 140+1.4.
264 spsa params: output weights and biases in nn-e8bac1c07a5a.nnue
A: 6000, alpha: 0.602, gamma: 0.101
weights: [-127, 127], c_end = 6
biases: [-8192, 8192], c_end = 64
Among the 264 params, 189 weights and all 8 biases were changed.
Changes in the weights:
- mean: -0.111 +/- 3.57
- range: [-8, 8]
Found with the same method as:
https://github.com/official-stockfish/Stockfish/pull/5459
Due to the original name (nn-ea8c9128c325.nnue) being too similar
to the previous main net (nn-e8bac1c07a5a.nnue) and creating confusion,
it was renamed by making non-functional changes to the .nnue file
the same way as past nets with:
https://github.com/linrock/nnue-namer
To verify that bench is the same and view the modified non-functional bytes:
```
echo -e "setoption name EvalFile value nn-ea8c9128c325.nnue\nbench" | ./stockfish
echo -e "setoption name EvalFile value nn-31337bea577c.nnue\nbench" | ./stockfish
cmp -l nn-ea8c9128c325.nnue nn-31337bea577c.nnue
diff <(xxd nn-ea8c9128c325.nnue) <(xxd nn-31337bea577c.nnue)
```
Passed STC:
https://tests.stockfishchess.org/tests/view/669564154ff211be9d4ec080
LLR: 2.93 (-2.94,2.94) <0.00,2.00>
Total: 57280 W: 15139 L: 14789 D: 27352
Ptnml(0-2): 209, 6685, 14522, 6995, 229
Passed LTC:
https://tests.stockfishchess.org/tests/view/669694204ff211be9d4ec1b4
LLR: 2.94 (-2.94,2.94) <0.50,2.50>
Total: 63030 W: 16093 L: 15720 D: 31217
Ptnml(0-2): 47, 6766, 17516, 7139, 47
closes https://github.com/official-stockfish/Stockfish/pull/5509
bench 1371485
