The previous asymmetry measure of the pawn structure only used to
consider the number of pawns on semi-opened files in the position.
With this patch we also increase the measure by the number of passed
pawns for both players.
Many thanks to the community for the nice feedback on the previous
version, with special mentions to Alain Savard and Marco Costalba
for clarity and speed suggestions.
STC:
LLR: 2.96 (-2.94,2.94) [0.00,5.00]
Total: 13146 W: 3038 L: 2840 D: 7268
http://tests.stockfishchess.org/tests/view/5a91dd0c0ebc590297cc877e
LTC:
LLR: 2.96 (-2.94,2.94) [0.00,5.00]
Total: 27776 W: 4771 L: 4536 D: 18469
http://tests.stockfishchess.org/tests/view/5a91fdd50ebc590297cc879b
How to continue after this patch?
Stockfish will now evaluate more positions with passed pawns, so
tuning the passed pawns values may bring Elo. The patch has also
consequences on the initiative term, where we might want to give
different weights to passed pawns and semi-openfiles (idea by
Stefano Cardanobile).
Bench: 5302866
Move the first killer move out of the capture stage, combining treatment
of first and second killer move.
passed STC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 55777 W: 12367 L: 12313 D: 31097
http://tests.stockfishchess.org/tests/view/5a88617e0ebc590297cc8351
Similar to an earlier proposition of Günther Demetz, see pull request #1075.
I think it is more robust and readable than master, why hand-unroll the loop
over the killer array, and duplicate code ?
This version includes review comments from Marco Costalba.
Bench: 5227124
The previous asymmetry measure of the pawn structure only used to
consider the number of pawns on semi-opened files in the postions.
With this patch we also increase the measure by the number of passed
pawns for both players.
STC:
LLR: 2.96 (-2.94,2.94) [0.00,5.00]
Total: 13146 W: 3038 L: 2840 D: 7268
http://tests.stockfishchess.org/tests/view/5a91dd0c0ebc590297cc877e
LTC:
LLR: 2.96 (-2.94,2.94) [0.00,5.00]
Total: 27776 W: 4771 L: 4536 D: 18469
http://tests.stockfishchess.org/tests/view/5a91fdd50ebc590297cc879b
How to continue from there: Stockfish will now evaluate more positions
with passed pawns, so tuning the passed pawns values may bring Elo.
The patch also has consequences on the initiative term.
Bench: 5302866
When iid (Internal iterative deepening) is invoked, the prior value of ttValue is
not guaranteed to be VALUE_NONE. As such, it is currently possible to enter a state
in which ttValue has a specific value which is inconsistent with tte->bound() and
tte->depth(). Currently, ttValue is only used within the search in a context that
prevents this situation from making a difference (and so this change is non-functional,
but this is not guaranteed to remain the case in the future.
For instance, just changing the tt depth condition in singular extension node to be
tte->depth() >= depth - 4 * ONE_PLY
instead of
tte->depth() >= depth - 3 * ONE_PLY
interacts badly with the absence of ttMove in iid. For the ttMove to become a singular
extension candidate, singularExtensionNode needs to be true. With the current master,
this requires that tte->depth() >= depth - 3 * ONE_PLY. This is not currently possible
if tte comes from IID, since the depth 'd' used for the IID search is always less than
depth - 4 * ONE_PLY for depth >= 8 * ONE_PLY (below depth 8 singularExtensionNode can
never be true anyway). However, with DU-jdto/Stockfish@251281a , this condition can be
met, and it is possible for singularExtensionNode to become true after IID. There are
then two mechanisms by which this patch can affect the search:
• If ttValue was VALUE_NONE prior to IID, the fact that this patch sets ttValue allows
the 'ttValue != VALUE_NONE' condition of singularExtensionNode to be met.
• If ttValue wasn't VALUE_NONE prior to IID, the fact that this patch modifies ttValue's
value causes a different 'rBeta' to be calculated if the singular extension search is
performed.
Tested at STC for non-regression:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 76981 W: 17060 L: 17048 D: 42873
http://tests.stockfishchess.org/tests/view/5a7738b70ebc5902971a9868
No functional change
The razoring heuristic is quite a drastic pruning technique,
using a depth 0 search at internal nodes of the search tree
to estimate the true value of depth n nodes. This patch limits
this razoring to the case of internal nodes of depth 1.
Author: Jarrod Torriero (DU-jdto)
STC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 8043 W: 1865 L: 1716 D: 4462
http://tests.stockfishchess.org/tests/view/5a90a9290ebc590297cc86c1
LTC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 32890 W: 5577 L: 5476 D: 21837
http://tests.stockfishchess.org/tests/view/5a90c8510ebc590297cc86d5
Opportunities opened by this patch: it would be interesting to
know if it brings Elo to re-introduce razoring or soft razoring
at depth >= 2, maybe using a larger margin to compensate for the
increased pruning effect.
Bench: 5227124
This is one of the most difficult to understand but also
most important and speed critical functions of SF.
This patch rewrites some part of it to hopefully
make it clearer and drop some redundant variables
in the process.
Same speed than master (or even a bit more).
Thanks to Chris Cain for useful feedback.
No functional change.
Where variable names are explicitly incorrect, I feel morally obligated to at least
suggest an alternative. There are many, but these two are especially egregious.
No functional change.
As far as can tell, semiopenFiles are set if there is a pawn anywhere on
the file. The removed condition would be true even if the pawns were very
advanced, which doesn't make sense if we're looking for a trapped rook.
Seems the engine fairs better with this removed. My guess s that the
condition that mobility is 3 or less does this well enough.
Begs the question whether this is a mobility issue alone... not sure.
Should I do LTC test?
STC
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 13377 W: 3009 L: 2871 D: 7497
http://tests.stockfishchess.org/tests/view/5a855be40ebc590297cc8166
Passed LTC
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 16288 W: 2813 L: 2685 D: 10790
http://tests.stockfishchess.org/tests/view/5a8575a80ebc590297cc817e
Bench: 5006365
Make contempt dependent on the current score of the root position.
The idea is that we now use a linear formula like the following to decide
on the contempt to use during a search :
contempt = x + y * eval
where x is the base contempt set by the user in the "Contempt" UCI option,
and y * eval is the dynamic part which adapts itself to the estimation of
the evaluation of the root position returned by the search. In this patch,
we use x = 18 centipawns by default, and the y * eval correction can go
from -20 centipawns if the root eval is less than -2.0 pawns, up to +20
centipawns when the root eval is more than 2.0 pawns.
To summarize, the new contempt goes from -0.02 to 0.38 pawns, depending if
Stockfish is losing or winning, with an average value of 0.18 pawns by default.
STC:
LLR: 2.95 (-2.94,2.94) [0.00,5.00]
Total: 110052 W: 24614 L: 23938 D: 61500
http://tests.stockfishchess.org/tests/view/5a72e6020ebc590f2c86ea20
LTC:
LLR: 2.97 (-2.94,2.94) [0.00,5.00]
Total: 16470 W: 2896 L: 2705 D: 10869
http://tests.stockfishchess.org/tests/view/5a76c5b90ebc5902971a9830
A second match at LTC was organised against the current master:
ELO: 1.45 +-2.9 (95%) LOS: 84.0%
Total: 19369 W: 3350 L: 3269 D: 12750
http://tests.stockfishchess.org/tests/view/5a7acf980ebc5902971a9a2e
Finally, we checked that there is no apparent problem with multithreading,
despite the fact that some threads might have a slightly different contempt
level that the main thread.
Match of this version against master, both using 5 threads, time control 30+0.3:
ELO: 2.18 +-3.2 (95%) LOS: 90.8%
Total: 14840 W: 2502 L: 2409 D: 9929
http://tests.stockfishchess.org/tests/view/5a7bf3e80ebc5902971a9aa2
Include suggestions from Marco Costalba, Aram Tumanian, Ronald de Man, etc.
Bench: 5207156
This patch simplifies the time management code, removing the extra
thinking time for moves with draw PV and increasing thinking time
for all moves proportionally by around 4%.
Last time when the time management was carefully tuned was 1.5-2 years
ago. As new patches were getting added, time management was drifting out
of optimum. This happens because when search becomes more precise pv and
score are becoming more stable, there are less fail lows, best move is
picked earlier and there are less best move changes. All this factors are
entering in time management, and average time per move is decreasing with
more and more good patches. For individual patches such effect is small
(except some) and may be up or down, but when there are many of them,
effect is more substantial. The same way benchmark with more and more
patches is slowly drifting down on average.
So my understanding that back in October adding more think time for draw
PV showed positive Elo because time management was not well tuned, there
was more time available, and think_hard patch applied this additional time
to moves with draw PV, while just retuning back to optimum would recover Elo
anyway. It is possible that absence of contempt also helped, as SF9 is showing
less 0.0 scores than the October version.
Anyway, to me it seems that proper place to deal with draw PV is search, and
contempt sounds as much better solution. In time management there is little
additional elo, and if some code is not helping like removed here, it is better
to discard it. It is simpler to find genuine improvement if code is clean.
• Passed STC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 20487 W: 4558 L: 4434 D: 11495
http://tests.stockfishchess.org/tests/view/5a7706ec0ebc5902971a9854
• Passed LTC:
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 41960 W: 7145 L: 7058 D: 27757
http://tests.stockfishchess.org/tests/view/5a778c830ebc5902971a9895
• Passed an additional non-regression [-5..0] test at the time control
of 60sec for the game (sudden death) with disabled draw adjudication:
LLR: 2.95 (-2.94,2.94) [-5.00,0.00]
Total: 8438 W: 1675 L: 1586 D: 5177
http://tests.stockfishchess.org/tests/view/5a7c3d8d0ebc5902971a9ac0
• Passed an additional non-regression [-5..0] test at the time control
of 1sec+1sec per move with disabled draw adjudication:
LLR: 2.97 (-2.94,2.94) [-5.00,0.00]
Total: 27664 W: 5575 L: 5574 D: 16515
http://tests.stockfishchess.org/tests/view/5a7c3e820ebc5902971a9ac3
This is a functional change for the time management code.
Bench: 4983414
The 'eval' debugging command in Terminal did not initialize the Eval::Contempt
variable, leading to random output during debugging sessions (normal search
was unaffected by the bug).
Example of session where the two 'eval' commands should give the same output,
but did not:
./stockfish
position startpos
d
eval
go depth 20
d
eval
The bug is fixed by initializing Eval::Contempt to SCORE_ZERO in Eval::trace
No functional change.
The current logic in master is to continue return quiet moves if their
history score is above 0. It appears as though this check can be
removed, which is also more logically consistent with the “skipQuiets”
semantics used in search.cpp.
This patch may open new opportunitiesto get Elo by changing or
tuning the definition of 'moveCountPruning' in line 830 of search.cpp,
because obeying skipQuiets without checking the history scores makes
the search more sensitive to 'moveCountPruning'.
STC
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 34780 W: 7680 L: 7584 D: 19516
http://tests.stockfishchess.org/tests/view/5a79f8d80ebc5902971a99db
LTC
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 38757 W: 6732 L: 6641 D: 25384
http://tests.stockfishchess.org/tests/view/5a7afebe0ebc5902971a9a46
Bench 4954595
Enable link-time optimization in the Makefile when compiling with clang.
Also update travis.yml to use clang++-5.0 and llvm-5.0-dev.
No functional change.
For these recaptures, we’re are only considering those captures
that recapture the recapture square (small portion of all the
captures). Therefore, scoring all of the captures and pick_besting
out of the whole group is not necessary.
STC
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 85583 W: 18978 L: 18983 D: 47622
http://tests.stockfishchess.org/tests/view/5a717faa0ebc590f2c86e9a7
LTC
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 20231 W: 3533 L: 3411 D: 13287
http://tests.stockfishchess.org/tests/view/5a73ad330ebc5902971a96ba
Bench: 5023593
The difference between QCAPTURES_1 and QCAPTURES_2 quiescence search stages
boils down to a simple check of depth. The way it's being done now is
unnecessarily complex.
This patch is simpler, clearer, and easier to understand.
Passed SPRT[-3..1] test at STC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 99755 W: 22158 L: 22192 D: 55405
http://tests.stockfishchess.org/tests/view/5a71f41c0ebc590f2c86e9cb
No functional change.
It seems to be a waste of time to loop through all remaining root moves
after finishing each PV line. This patch skips this until we have reached
the last PV line (this is the way it was done in Glaurung and very early
versions of Stockfish).
No functional change in Single PV mode.
MultiPV=3 STC and LTC tests
LLR: 2.95 (-2.94,2.94) [0.00,5.00]
Total: 3113 W: 1248 L: 1064 D: 801
LLR: 2.95 (-2.94,2.94) [0.00,5.00]
Total: 2260 W: 848 L: 679 D: 733
Bench: 5023629
It does the following:
- If a TB win or loss value allows an alpha or beta cutoff, the cutoff is taken.
- Otherwise, the search of the current subtree continues. In PV nodes, the final value returned is adjusted to reflect that the position is a TB win (or loss).
The patch also fixes a potential problem caused by root_probe() and root_probe_wdl() dirtying the root-move scores.
This patch removes the limitation of current master that a mate is never found if the root position is not yet in the TBs, but the path to mate at some point enters the TBs. The patch is intended to preserve the efficiency and effectiveness of the current TB probing approach.
No functional change (withouth TB)
Set the default contempt value of Stockfish to 20 centipawns.
The contempt feature of Stockfish tries to prevent the engine from
simplifying the position too quickly when it feels that it is very
slightly behind, instead keeping the tension a little bit longer.
Various tests in November 2017 have proved that our current imple-
mentation works well against SF7 (which is about 130 Elo weaker than
current master) and than the Elo gain is an increasing function of
contempt, going (against SF7) from +0 Elo when contempt is set at
zero centipawns, to +30 Elo when contempt is 40 centipawns.
See pull request 1325 for details:
https://github.com/official-stockfish/Stockfish/pull/1325
This november discussion left open the decision of which "default"
value for contempt we should use for Stockfish, taking into account
the various uses ofStockfish (opening preparation for humans, computer
online tournaments,analysis tool for web pages, human/computer play,
etc).
This pull request proposes to set the default contempt value of SF
to twenty centipawns, which turns out to be the highest value which
is not a regression against current master, as this seemed to be a
good compromise between risk and safety. A couple of SPRT[-3..1]
tests were done to bisect this value:
Contempt 10: http://tests.stockfishchess.org/tests/view/5a5d42d20ebc5902977e2901 (PASSED)
Contempt 15: http://tests.stockfishchess.org/tests/view/5a5d41740ebc5902977e28fa (PASSED)
Contempt 20: http://tests.stockfishchess.org/tests/view/5a5d42060ebc5902977e28fc (PASSED)
Contempt 25: http://tests.stockfishchess.org/tests/view/5a5d433f0ebc5902977e2904 (FAILED)
Surprisingly, a test at "very long time control" hinted that using
contempt 20 is not only be non-regressive against contempt 0, but
may actually exhibit some small Elo gain, giving a likehood of superio-
rity of 88.7% after 8500 games:
VLTC:
ELO: 2.28 +-3.7 (95%) LOS: 88.7%
Total: 8521 W: 1096 L: 1040 D: 6385
http://tests.stockfishchess.org/tests/view/5a60b2820ebc590297b9b7e0
Finally, there was some concerns that a contempt value of 20 would
be worse than a value of 7, but a test with 20000 games at STC was
neutral:
STC:
ELO: 0.45 +-3.1 (95%) LOS: 61.2%
Total: 20000 W: 4222 L: 4196 D: 11582
http://tests.stockfishchess.org/tests/view/5a64d2fd0ebc590297903868
See the comments in pull request 1361 for the long, nice discussion
(180 entries :-)) leading to the decision to propose contempt 20 as
the default value:
https://github.com/official-stockfish/Stockfish/pull/1361
Whether Stockfish should strictly adhere to the Komodo and Houdini
semantics and add the UCI commands to force the contempt to be White
in the so-called "analysis mode" is still under discussion, and may
be or may not be the object of a future commit.
Bench: 5783344
1. avoid recursive call of verification.
For the interested side to move recursion makes no sense.
For the other side it could make sense in case of mutual zugzwang,
but I was not able to figure out any concrete problematic position.
Allows the removal of 2 local variables.
2. avoid further reduction by removing R += ONE_PLY;
Benchmark with zugzwang-suite (see #1338), max 45 secs per position:
Patch solves 33 out of 37
Master solves 31 out of 37
STC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 76188 W: 13866 L: 13840 D: 48482
http://tests.stockfishchess.org/tests/view/5a5612ed0ebc590297da516c
LTC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 40479 W: 5247 L: 5152 D: 30080
http://tests.stockfishchess.org/tests/view/5a56f7d30ebc590299e4550e
bench: 5340015
as discussed in issue #1349, the way pages are allocated with calloc might imply some overhead on first write.
This overhead can be large and slow down the first search after a TT resize significantly, especially for large TT.
Using an explicit clear of the TT on resize fixes this problem.
Not implemented, but possibly useful for large TT, is to do this zero-ing using all search threads. Not only would this be faster, it could also lead to a more favorable memory allocation on numa systems with a first touch policy.
No functional change.
The heuristic to avoid thread binding if less than 8 threads are requested resulted in the first 7 threads not being bound.
The branch was verified to yield a roughly 13% speedup by @CoffeeOne on the appropriate hardware and OS, and an earlier version of this patch tested well on his machine:
http://tests.stockfishchess.org/tests/view/5a3693480ebc590ccbb8be5a
ELO: 9.24 +-4.6 (95%) LOS: 100.0%
Total: 5000 W: 634 L: 501 D: 3865
To make sure all threads (including mainThread) are bound as soon as the total number exceeds 7, recreate all threads on a change of thread number.
To do this, unify Threads::init, Threads::exit and Threads::set are unified in a single Threads::set function that goes through the needed steps.
The code includes several suggestions from @joergoster.
Fixes issue #1312
No functional change
For efficiency reasons current master only allows for transposition table sizes that are N = 2^k in size, the index computation can be done efficiently as (hash % N) can be written instead as (hash & 2^k - 1). On a typical computer (with 4, 8... etc Gb of RAM), this implies roughly half the RAM is left unused in analysis.
This issue was mentioned on fishcooking by Mindbreaker:
http://tests.stockfishchess.org/tests/view/5a3587de0ebc590ccbb8be04
Recently a neat trick was proposed to map a hash into the range [0,N[ more efficiently than (hash % N) for general N, nearly as efficiently as (hash % 2^k):
https://lemire.me/blog/2016/06/27/a-fast-alternative-to-the-modulo-reduction/
namely computing (hash * N / 2^32) for 32 bit hashes. This patch implements this trick and now allows for general hash sizes. Note that for N = 2^k this just amounts to using a different subset of bits from the hash. Master will use the lower k bits, this trick will use the upper k bits (of the 32 bit hash).
There is no slowdown as measured with [-3, 1] test:
http://tests.stockfishchess.org/tests/view/5a3587de0ebc590ccbb8be04
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 128498 W: 23332 L: 23395 D: 81771
There are two (smaller) caveats:
1) the patch is implemented for a 32 bit hash (so that a 64 bit multiply can be used), this effectively limits the number of clusters that can be used to 2^32 or to 128Gb of transpostion table. That's a change in the maximum allowed TT size, which could bother those using 256Gb or more regularly.
2) Already in master, an excluded move is hashed into the position key in rather simple way, essentially only affecting the lower 16 bits of the key. This is OK in master, since bits 0-15 end up in the index, but not in the new scheme, which picks the higher bits. This is 'fixed' by shifting the excluded move a few bits up. Eventually a better hashing scheme seems wise.
Despite these two caveats, I think this is a nice improvement in usability.
Bench: 5346341
Current master can yield different staticEvals depending on the path
used to reach the position. The reason for this is that the evaluation after a
null move is always computed subtracting 2 * Eval::Tempo, while this is not
the case for lazy or specialized evals. This patch always adds tempo to evals,
which doesn't affect playing strength:
LTC
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 59911 W: 7616 L: 7545 D: 44750
STC
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 104947 W: 18897 L: 18919 D: 67131
Fixes issue #1335
Bench: 5208264
