Currently, we first calculate some bitboards at the top of Evaluation::space()
and then check whether we actually need them. Invert the ordering. Of course this
does not make a difference in current master because the constexpr bitboard
calculations are in fact done at compile time by any decent compiler, but I find
my version a bit healthier since it will always meet or exceed current implementation
even if we eventually change the spaceMask to something not contsexpr.
No functional change.
After a session of tuning for King Psqt I got some new values, which was later
tweaked manually by me Fauzi, to result in an Elo-gain patch which seems to scale
pretty well:
STC: LLR: -2.96 (-2.94,2.94) [0.00,4.00]
Total: 100653 W: 22550 L: 22314 D: 55789 [Yellow patch]
LTC: LLR: 2.96 (-2.94,2.94) [0.00,4.00]
Total: 147079 W: 25584 L: 24947 D: 96548 [Green Patch]
Bench: 4669050
Introduce voting system for best move selction in multi-threads mode.
Joint work with Stefan Geschwentner, based on ideas introduced by
Michael Stembera.
Moves are upvoted by every thread using the margin to the minimum score
across threads and the completed depth.
First thread voting for the winner move is selected as best thread.
Passed STC, LTC. A further LTC test with only 4 threads failed with positive
score. A LTC with 31 threads was stopped with LLR 0.77 after 25k games to
avoid use of excessive resources (equivalent to 1.5M STC games).
Similar ideas were proposed by Michael Stembera 2 years ago #507, #508.
This implementation seems simpler and more understandable, the results
slightly more promising.
Further possible work:
1) Tweak of the formula using for assigning votes.
2) Use a different baseline for the score dependent part: maximum score
or winning probability could make more sense.
3) Assign votes in `Thread::Search` as iterations are completed and use
voting results to stop search.
4) Select best thread as the threads voting for best move with the highest
completed depth or, alternatively, vote on PV moves.
Link to SPRT tests
[stopped LTC, 31 threads 20+0.02](http://tests.stockfishchess.org/tests/view/5b61dc090ebc5902bdb95192)
LLR: 0.77 (-2.94,2.94) [0.00,5.00]
Total: 25602 W: 3977 L: 3850 D: 17775
Elo: 1.70 [-0.68,4.07] (95%)
[passed LTC, 8 threads 20+0.02](http://tests.stockfishchess.org/tests/view/5b5df5180ebc5902bdb9162d)
LLR: 2.96 (-2.94,2.94) [0.00,5.00]
Total: 44478 W: 7602 L: 7300 D: 29576
Elo: 1.92 [-0.29,3.94] (95%)
[failed LTC, 4 threads 20+0.02](http://tests.stockfishchess.org/tests/view/5b5f39ef0ebc5902bdb92792)
LLR: -2.94 (-2.94,2.94) [0.00,5.00]
Total: 29922 W: 5286 L: 5285 D: 19351
Elo: 0.48 [-1.98,3.10] (95%)
[passed STC, 4 threads 5+0.05](http://tests.stockfishchess.org/tests/view/5b5dbf0f0ebc5902bdb9131c)
LLR: 2.97 (-2.94,2.94) [0.00,5.00]
Total: 9108 W: 2033 L: 1858 D: 5217
Elo: 6.11 [1.26,10.89] (95%)
No functional change (in simple threat mode)
Use operator const T&() instead of operator T() to avoid possible
costly hidden copies of non-scalar nested types.
Currently StatsEntry has a single member T, so assuming
sizeof(StatsEntry) == sizeof(T) it happens to work, but it's
better to use the size of the proper entry type in std::fill.
Note that current code works because std::array items are ensured
to be allocated in contiguous memory and there is no padding among
nested arrays. The latter condition does not seem to be strictly
enforced by the standard, so be careful here.
Finally use address-of operator instead of get() to fully hide the
wrapper class StatsEntry at calling sites. For completness add
the arrow operator too and simplify the C++ code a bit more.
Same binary code as previous master under the Clang compiler.
No functional change.
As a note, current 2 LMR conditions on stat score
could be simplified in a single line:
r -= ((ss->statScore >= 0) - ((ss-1)->statScore >= 0)) * ONE_PLY;
We keep them splitted in 2 "if" statements because are easier
to (immediately) read.
No functional change.
This is the first patch teaching Stockfish how to use the 7-pieces
Syzygy tablebase currently calculated by Bujun Guo (@noobpwnftw) and
Ronald de Man (@syzygy1). The 7-pieces database are so big that they
required a change in the internal format of the files (technically,
some DTZ values are 16 bits long, so this had to be stored as wide
integers in the Huffman tree).
Here are the estimated file size for the 7-pieces Syzygy files,
compared to the 151G of the 6-pieces Syzygy:
```
7.1T ./7men_testing/4v3_pawnful (ongoing, 120 of 325 sets remaining)
2.4T ./7men_testing/4v3_pawnless
2.3T ./7men_testing/5v2_pawnful
660G ./7men_testing/5v2_pawnless
117G ./7men_testing/6v1_pawnful
87G ./7men_testing/6v1_pawnless
```
Some pointers to download or recalculate the tables:
Location of original files, by Bujun Guo:
ftp://ftp.chessdb.cn/pub/syzygy/
Mirrors:
http://tablebase.sesse.net/ (partial)
http://tablebase.lichess.ovh/tables/standard/7/
Generator code:
https://github.com/syzygy1/tb/
Closes https://github.com/official-stockfish/Stockfish/pull/1707
Bench: 5591925 (No functional change if SyzygyTB is not used)
----------------------
Comment by Leonardo Ljubičić (@DragonMist)
This is an amazing achievement, generating and being able to use 7 men syzygy
on the fly. Thank you for your efforts @noobpwnftw !! Looking forward how this
will work in real life, and expecting some trade off between gaining perfect
play and slow disc Access, but once the disc speed and space is not a problem,
I expect 7 men to yield something like 30 elo at least.
-----------------------
Comment by Michael Byrne (@MichaelB7)
This definitely has a bright future. I turned off the 50 move rule (ala ICCF
new rules) for the following position: `[d]8/8/1b6/8/4N2r/1k6/7B/R1K5 w - - 0 1`
This position is a 451 ply win for white (sans the 50 move rule, this position
was identified by the generator as the longest cursed win for white in KRBN v KRB).
Now Stockfish finds it instantly (as it should), nice work 👊👍 .
```
dep score nodes time
7 +132.79 4339 0:00.00 Rb1+ Kc4 Nd6+ Kc5 Bg1+ Kxd6 Rxb6+ Kc7 Be3 Rh2 Bd4
6 +132.79 1652 0:00.00 Rb1+ Kc4 Nd2+ Kd5 Rxb6 Rxh2 Nf3 Rf2
5 +132.79 589 0:00.00 Rb1+ Kc4 Rxb6 Rxh2 Nf6 Rh1+ Kb2
4 +132.79 308 0:00.00 Rb1+ Kc4 Nd6+ Kc3 Rxb6 Rxh2
3 +132.79 88 0:00.00 Rb1+ Ka4 Nc3+ Ka5 Ra1+ Kb4 Ra4+ Kxc3 Rxh4
2 +132.79 54 0:00.00 Rb1+ Ka4 Nc3+ Ka5 Ra1+ Kb4
1 +132.7
```
This patch adds the tropism measure as a new term in the king danger variable.
Since we then trasform this variable as a Score via a quadratic formula, the
main effect of the patch is the positive correlation of the tropism measure
with some checks and pins information already present in the king danger code.
STC:
LLR: 2.96 (-2.94,2.94) [0.00,5.00]
Total: 6805 W: 1597 L: 1431 D: 3777
http://tests.stockfishchess.org/tests/view/5b5df8d10ebc5902bdb91699
LTC:
LLR: 2.96 (-2.94,2.94) [0.00,5.00]
Total: 32872 W: 5782 L: 5523 D: 21567
http://tests.stockfishchess.org/tests/view/5b5e08d80ebc5902bdb917ee
How to continue from there?
• it may be possible to use CloseEnemies=S(7,0)
• we may want to try incorporating other strategic features in the quadratic
king danger.
Closes https://github.com/official-stockfish/Stockfish/pull/1717
Bench: 5591925
The previous commit wouldn't compile on the Microsoft Virtual Studio C++ compiler. So use a more compatible style for the same idea (which we already use in numerous places of evaluate.cpp, for instance in line 563).
Under the Clang compiler, both versions generate exactly the same machine code (same md5 signatures for the two binaries).
No functional change.
Remove a popcount for HinderPassedPawn, and compensate by doubling
the bonus from S(4,0) to to S(8,0).
Maybe it was pure luck, but we got the idea of this Elo gaining patch by
seing the simplification attempt by Mike Whiteley in pull request #1703.
This suggests that whenever we have a passed evaluation simplification,
we should consider the possibility that the master bonus has become
slightly out of tune with time, and we should try a few Elo gaining [0..4]
tests by hand-tuning the master bonus.
STC:
LLR: 2.95 (-2.94,2.94) [0.00,4.00]
Total: 19136 W: 4388 L: 4147 D: 10601
http://tests.stockfishchess.org/tests/view/5b59be6f0ebc5902bdb8ac06
LTC:
LLR: 2.96 (-2.94,2.94) [0.00,4.00]
Total: 99382 W: 17324 L: 16843 D: 65215
http://tests.stockfishchess.org/tests/view/5b59d2410ebc5902bdb8afa8
Closes https://github.com/official-stockfish/Stockfish/pull/1710
Bench: 4688817
This tweak excludes files D and E from the KingFlank bitboard when our
king is on the A or H files respectively. As far as I can tell, this
affects two things: the calculation for CloseEnemies and PawnlessFlank.
Aside from filtering out slightly less relevant attacks in the flank,
I suspect this helps with king prophylaxis, avoiding attacks and moving
towards the center when the pawns start to come off.
STC
LLR: 2.95 (-2.94,2.94) [0.00,4.00]
Total: 56755 W: 12881 L: 12489 D: 31385
http://tests.stockfishchess.org/tests/view/5b58a94c0ebc5902bdb88c72
LTC
LLR: 2.95 (-2.94,2.94) [0.00,4.00]
Total: 130205 W: 22536 L: 21957 D: 85712
http://tests.stockfishchess.org/tests/view/5b58b7580ebc5902bdb89029
How to continue: Tweaking the two bonuses mentioned might give some
gain, although as far as I can tell, CloseEnemies is very sensitive to
even small changes.
Closes https://github.com/official-stockfish/Stockfish/pull/1705
Bench: 5026009
When evaluating threat by safe pawn and pawn push the same expression is used.
STC
LLR: 2.95 (-2.94,2.94) [0.00,5.00]
Total: 19444 W: 4540 L: 4309 D: 10595
http://tests.stockfishchess.org/tests/view/5b5a6e150ebc5902bdb8c5c0
Closes https://github.com/official-stockfish/Stockfish/pull/1709
No functional change.
--------------------
Comments by Stéphane Nicolet:
I don't measure any speed-up on my system, with two parallel benches at depth 22:
Total time (ms) : 74989
Nodes searched : 144830258
Nodes/second : 1931353
master
Total time (ms) : 75341
Nodes searched : 144830258
Nodes/second : 1922329
testedpatch
And anyway, like Stefan Geschwentner, I don't think that a 0.3% speed-up would
be enough to pass a [0..5] LTC test -- as a first approximation, we have this
rule of thumb that 1% speed-up gives about 1 Elo point.
However, considering the facts that the reformatting by itself is interesting,
that this is your first green test and that you played by the rules by running
the SPRT[0..5] test before opening the pull request, I will commit the change.
I will only take the liberty to change the occurrences of safe in lines 590 and
591 to b, to make the code more similar to lines 584 and 585.
So approved, and congrats :-)
This patch implements some idea by Alain Savard and Mike Whiteley taken from the perpertual renaming/reformatting thread.
This is a pure code cleaning patch (so no change in functionality), but I use it as a pretext to correct the bogus bench number that I introduced in the previous commit.
Bench: 4413383
This patch reverts the recent commit called "Tweak reductions formula, etc."
The decisions for the revert decision were as follows:
1) The original commit called "Tweak reductions formula: 0.88 * depth + 0.12"
showed bad scaling at in a Very Long Time Control (VLTC) test:
VLTC (180+1.8):
LLR: -1.59 (-2.94,2.94) [0.00,5.00]
Total: 14968 W: 2247 L: 2257 D: 10464
http://tests.stockfishchess.org/tests/view/5b559ffa0ebc5902bdb84f36
2) So there was a suspicion that the original fast passing LTC test which lead
us to accept the patch may have been a statistical accident, so we organized
a match against the previous master at LTC to get an Elo estimate for the
patch:
LTC match:
ELO: -1.83 +-2.1 (95%) LOS: 4.3%
Total: 36018 W: 6018 L: 6208 D: 23792
http://tests.stockfishchess.org/tests/view/5b55f8110ebc5902bdb8526f
3) Based on these results, we ran a simplification test with [-3..1] bounds
for the revert at LTC:
LTC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 41501 W: 7107 L: 7020 D: 27374
http://tests.stockfishchess.org/tests/view/5b5738670ebc5902bdb86932
4) So we revert.
Bench: 4491691
There seems to be some strange interaction between Overload and Connectivity.
Overload encourages us to not have too many defended and attacked pieces,
as this may expose us to various tactics. This feels somewhat like it is in
conflict with Connectivity, where pieces are defended preemptively.
Here I take the "pick one or the other" approach and just remove connectivity,
while strengthening the effect of Overload to compensate. The reasoning is that
if we defend our pieces preemptively, then it does get attacked, we want to do
something about it so we don't get penalized by Overload. On the other
hand, if it doesn't get attacked, then there's no need to defend it.
STC:
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 27734 W: 6174 L: 6064 D: 15496
http://tests.stockfishchess.org/tests/view/5b5073bd0ebc5902bdb7ba5c
LTC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 51606 W: 8897 L: 8827 D: 33882
http://tests.stockfishchess.org/tests/view/5b50aa900ebc5902bdb7bf29
Bench: 4658006
After some recent big tuning session, the values for King Protector were
simplified to only be used on minor pieces. This patch tries to further
simplify by just using a single value, since current S(6,5) and S(5,6)
are close to each other. The value S(6,6) ended up passing, although
S(5,5) was also tried and failed STC.
STC
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 14261 W: 3288 L: 3151 D: 7822
http://tests.stockfishchess.org/tests/view/5b4ccdf50ebc5902bdb77f65
LTC
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 19606 W: 3396 L: 3273 D: 12937
http://tests.stockfishchess.org/tests/view/5b4ce4280ebc5902bdb7803b
Bench: 5448998
Extend the bonus for Overload to cases where our side
has more than one attacker to a non pawn piece.
Based on an idea by Bryan in the forum. For instance,
now black gets the overload bonus in this position:
8/5R1k/6pb/p6p/P1N4P/1Pp5/2K3P1/2N4r b - - 6 46
because two black pieces are attacking the knight on c1
that is defended only by the king.
STC
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 57446 W: 12762 L: 12711 D: 31973
http://tests.stockfishchess.org/tests/view/5b4ca9970ebc5902bdb77a88
LTC
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 42113 W: 7295 L: 7209 D: 27609
http://tests.stockfishchess.org/tests/view/5b4ccea00ebc5902bdb77f69
Bench: 4667263
For the rationale to allow this, see commit
a66c73deef
This was broken when cuckoo hashing was added, and
subtly broke (for example) lichess' Android application,
thus illustrating the original judgement was sound.
No functional change.
Various king and pawn eval values tuned after 2 million games. Rounding
slightly adjusted.
LTC: http://tests.stockfishchess.org/tests/view/5b477a260ebc5978f4be3ed4
LLR: 2.95 (-2.94,2.94) [0.00,4.00]
Total: 32783 W: 5852 L: 5588 D: 21343
STC: http://tests.stockfishchess.org/tests/view/5b472d420ebc5978f4be3e4d
LLR: 3.23 (-2.94,2.94) [0.00,4.00]
Total: 44380 W: 10201 L: 9841 D: 24338
I think I reached the limit of the fishtest framework. It frequently
crashed at 2 million games already. The small values also moved a lot
throughout the entire tuning session though with smaller margin. The
passed danger and close enemies values seems the most sensitive (changing
close enemies alone to 6 failed before but now it passes), whether or not
they are close to optimal I don't know, but it seems some parameters are
also correlated to others.
Closes https://github.com/official-stockfish/Stockfish/pull/1670
bench: 5103722
In the current master, ThreatByKing is an array of two Scores, one for
when we have a single attack and one for when we have many. The latter
case is very rarely called during bench and was recently given a strange
negative value during a tuning run, as pointed out by @candirufish on
commit efd4ca2. Here, we simplify away this second case entirely, and
increase the remaining ThreatByKing to compensate.
Although I derived the parameter tweak independently, with the goal of
preserving the same average bonus, I later noticed that a very similar
Score had already been derived by an ongoing SPSA tuning session.
I therefore recognize @candirufish for first discovering these values.
I would also like to thank @Rocky640 for valuable feedback that pointed
me in the direction of ThreatByKing.
STC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 7677 W: 1772 L: 1623 D: 4282
http://tests.stockfishchess.org/tests/view/5b3db0320ebc5902b9ffe97a
LTC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 108031 W: 18329 L: 18350 D: 71352
http://tests.stockfishchess.org/tests/view/5b3dbf4b0ebc5902b9ffe9db
Closes https://github.com/official-stockfish/Stockfish/pull/1666
Bench: 4678861
In current master, the function make_bitboard() does nothing apart from
helping initialize the SquareBB[] array. This seems like an unnecessary
abstraction layer.
The advantage of make_bitboard() is we can define a bitboard, in a simple
and general way, not only from a single square but also from a list of
squares. It is more elegant, faster and readable than combining multiple
SquareBB explicitly, but the last complex use case in evaluation was
simplified away a few months ago.
If make_bitboard() becomes useful again to define complicated bitboards,
it will be easy enough to reintroduce it using this pull request as
an implementation reference.
No functional change.
Make sure each piece is not scored more than once as a passed pawn "hinderer",
by scoring only the blockers along the passed pawn path. Inspired by TCEC Game 29.
Passed STC as a simplification
http://tests.stockfishchess.org/tests/view/5b3016d00ebc5902b2e58552
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 75388 W: 16656 L: 16641 D: 42091
Passed LTC as a simplification
http://tests.stockfishchess.org/tests/view/5b302ed90ebc5902b2e587fc
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 49157 W: 8460 L: 8386 D: 32311
Current master was also counting the number of attacks along a passed pawn path,
which might be misleading:
a) a defender might be counted many times for the same pawn path. For example a
White rook on a1 attacking a black pawn on a7 would score the bonus * 6 but
would be probably better placed on a8
b) a defender might be counted on different pawn paths and might be overloaded. For
example a Ke4 or Qe4 against pawns on d6 and f6 would score the bonus * 6.
Counting each blocker or attacker only once is more complicated, and does not help
either: http://tests.stockfishchess.org/tests/view/5b2ff1cb0ebc5902b2e582b2
After this small simplification, there might be ways to increase the HinderPassedPawn
penalty.
Closes https://github.com/official-stockfish/Stockfish/pull/1661
Bench: 4520519
Silences the following warnings when compiling with GCC 8.
The fix is to use an intermediate pointer to anonymous function:
```
misc.cpp: In function 'int WinProcGroup::get_group(size_t)':
misc.cpp:241:77: warning: cast between incompatible function types from 'FARPROC' {aka 'long long int (*)()'} to 'fun1_t' {aka 'bool (*)(_LOGICAL_PROCESSOR_RELATIONSHIP, _SYSTEM_LOGICAL_PROCESSOR_INFORMATION_EX*, long unsigned int*)'} [-Wcast-function-type]
auto fun1 = (fun1_t)GetProcAddress(k32, "GetLogicalProcessorInformationEx");
^
misc.cpp: In function 'void WinProcGroup::bindThisThread(size_t)':
misc.cpp:309:71: warning: cast between incompatible function types from 'FARPROC' {aka 'long long int (*)()'} to 'fun2_t' {aka 'bool (*)(short unsigned int, _GROUP_AFFINITY*)'} [-Wcast-function-type]
auto fun2 = (fun2_t)GetProcAddress(k32, "GetNumaNodeProcessorMaskEx");
^
misc.cpp:310:67: warning: cast between incompatible function types from 'FARPROC' {aka 'long long int (*)()'} to 'fun3_t' {aka 'bool (*)(void*, const _GROUP_AFFINITY*, _GROUP_AFFINITY*)'} [-Wcast-function-type]
auto fun3 = (fun3_t)GetProcAddress(k32, "SetThreadGroupAffinity");
^
```
No functional change.
Compiling the current master with MSVC gives the following error:
```
search.cpp(956): error C2660: 'operator *': function does not take 1 arguments
types.h(303): note: see declaration of 'operator *'
```
This was introduced in commit:
88de112b84
We use a suggestion by @vondele to fix the error, thanks!
No functional change.
