Introduce a new search extension when pushing an advanced passed pawn is
also suggested by the first killer move. There have been previous tests
which have similar ideas, mostly about pawn pushes, but it seems to be
overkill to extend too many moves. My idea is to limit the extension to
when a move happens to be noteworthy in some other way as well, such as
in this case, when it is also a killer move.
STC:
LLR: 2.96 (-2.94,2.94) [0.50,4.50]
Total: 19027 W: 4326 L: 4067 D: 10634
http://tests.stockfishchess.org/tests/view/5cac2cde0ebc5925cf00c36d
LTC:
LLR: 2.94 (-2.94,2.94) [0.00,3.50]
Total: 93390 W: 15995 L: 15555 D: 61840
http://tests.stockfishchess.org/tests/view/5cac42270ebc5925cf00c4b9
For future tests, it looks like this will interact heavily with passed
pawn evaluation. It may be good to try more variants of some of the more
promising evaluations tests/tweaks.
Bench: 3666092
The current update only by main thread depends on the luck of
whether main thread sees any/many changes to the best move or not.
It then makes large, lumpy changes to the time to be
used (1x, 2x, 3x, etc) depending on that sample of 1.
Use the average across all threads to get a more reliable
number with a smoother distribution.
STC @ 5+0.05 th 4 :
LLR: 2.95 (-2.94,2.94) [0.50,4.50]
Total: 51899 W: 11446 L: 11029 D: 29424
http://tests.stockfishchess.org/tests/view/5ca32ff20ebc5925cf0016fb
STC @ 5+0.05 th 8 :
LLR: 2.96 (-2.94,2.94) [0.50,4.50]
Total: 13851 W: 2843 L: 2620 D: 8388
http://tests.stockfishchess.org/tests/view/5ca35ae00ebc5925cf001adb
LTC @ 20+0.2 th 8 :
LLR: 2.95 (-2.94,2.94) [0.00,3.50]
Total: 48527 W: 7941 L: 7635 D: 32951
http://tests.stockfishchess.org/tests/view/5ca37cb70ebc5925cf001cec
Further work:
Similar changes might be possible for the fallingEval and timeReduction calculations (and elsewhere?), using either the min, average or max values across all threads.
Bench 3506898
Shuffle detection procedure :
Shuffling positions are detected if
the last 36 moves are reversible (rule50_count() > 36),
the position have been already in the TT,
there is a still a pawn on the board (to avoid special endings like KBN vs K).
The position is then judged as a draw.
An extension is realized if we already made 14 successive reversible moves in PV to accelerate the detection of the eventual draw.
To go further : we can still improve the idea. The length of the tests need a lot of ressources.
the limit of 36 is logic but must be checked again for special zugzwang positions,
this limit can be decreased in special positions,
the limit of 14 moves for extension has not been tuned.
STC
LLR: -2.94 (-2.94,2.94) [0.50,4.50]
Total: 32595 W: 7273 L: 7275 D: 18047 Elo +0.43
http://tests.stockfishchess.org/tests/view/5c90aa330ebc5925cfff1768
LTC
LLR: 2.95 (-2.94,2.94) [0.00,3.50]
Total: 51249 W: 8807 L: 8486 D: 33956 Elo +1.85
http://tests.stockfishchess.org/tests/view/5c90b2450ebc5925cfff1800
VLTC
LLR: 2.96 (-2.94,2.94) [0.00,3.50]
Total: 137974 W: 20503 L: 19983 D: 97488 Elo +1.05
http://tests.stockfishchess.org/tests/view/5c9243a90ebc5925cfff2a93
Bench: 3548313
Adding a clamp function makes some of these range limitations a bit prettier and removes some #include's.
STC
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 28117 W: 6300 L: 6191 D: 15626
http://tests.stockfishchess.org/tests/view/5c9aa1df0ebc5925cfff8fcc
Non functional change.
always use the implementation of gives_check in position, no need to
hand-inline part of the implementation in search.
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 57895 W: 12632 L: 12582 D: 32681
http://tests.stockfishchess.org/tests/view/5c9eaa4b0ebc5925cfffc9e3
No functional change.
While looking at pruning using see_ge() (which is very valuable)
it became apparent that the !extension test is not adding any
value - simplify it away.
STC:
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 56843 W: 12621 L: 12569 D: 31653
http://tests.stockfishchess.org/tests/view/5c8588cb0ebc5925cffe77f4
LTC:
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 78622 W: 13223 L: 13195 D: 52204
http://tests.stockfishchess.org/tests/view/5c8611cc0ebc5925cffe7f86
Further work could be to optimize the remaining see_ge() test. The idea of less pruning at higher depths is valuable, but perhaps the test (-PawnValueEg * depth) can be improved.
Bench: 3188688
This is a functional simplification that removes the FutilityMoveCounts array with a simple equation using only ints.
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 14175 W: 3123 L: 2987 D: 8065
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 9900 W: 1735 L: 1597 D: 6568
Bench: 3380343
This is a non-functional patch which shrinks the reductions array.
This saves about 8Kb of memory.
The only slow part of master's reductions array is the calculation
of the log values, so using a separate array for those values and
calculating the rest real-time appears to be just as fast as master.
STC
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 63245 W: 13906 L: 13866 D: 35473
http://tests.stockfishchess.org/tests/view/5c7b571f0ebc5925cffdc104
No funcional change.
This removes the skipQuiets variable, as was done in an earlier round by
@protonspring, but fixes an oversight which led to wrong mate
announcements. Quiets can only be pruned when there is no mate score, so
set moveCountPruning at the right spot.
tested as a fix at STC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 66321 W: 14690 L: 14657 D: 36974
http://tests.stockfishchess.org/tests/view/5c74f3170ebc5925cffd4b3c
and as the full patch at LTC:
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 25903 W: 4341 L: 4228 D: 17334
http://tests.stockfishchess.org/tests/view/5c7540030ebc5925cffd506f
Bench: 3292342
This is a somewhat different patch. It fixes blindspots for
two knights vs pawn endgame.
With local testing starting from random KNNvKP positions where the
pawn has not advanced beyond the 4th rank (thanks @protonspring !)
at 15+0.15 (4 cores), this went +105=868-27 against master. All except
two losses were won in reverse.
The heuristic is simple but effective - the strategy in these endgames
is to push the opposing king to the corner, then move the knight that's
blocking the pawn in for the checkmate while the pawn is free to move
and prevents stalemate. This patch gives SF the little boost it needs
to search the relevant king-cornering mating lines.
See the discussion in pull request 1939 for some more good results for
this test in independant tests:
https://github.com/official-stockfish/Stockfish/pull/1939
Bench: 3310239
A simple idea, but it makes sense: in current master the search is extended
for checks that are considered somewhat safe, and for for this we use the
static exchange evaluation which only considers the `to_sq` of a move.
This is not reliable for discovered checks, where another piece is giving
the check and is arguably a more dangerous type of check. Thus, if the check
is a discovered check, the result of SEE is not relevant and can be ignored.
STC:
LLR: 2.96 (-2.94,2.94) [0.50,4.50]
Total: 29370 W: 6583 L: 6274 D: 16513
http://tests.stockfishchess.org/tests/view/5c5062950ebc593af5d4d9b5
LTC:
LLR: 2.95 (-2.94,2.94) [0.00,3.50]
Total: 227341 W: 37972 L: 37165 D: 152204
http://tests.stockfishchess.org/tests/view/5c5094fb0ebc593af5d4dc2c
Bench: 3611854
This patch removes line 875 of search.cpp, which was updating pvHit after IID.
Bench testing at depth 22 shows that line 875 of search.cpp never changes the
value of pvHit at NonPV nodes, while at PV nodes it often changes the value
from true to false (and never the reverse). This is because the definition of
pvHit at line 642 is :
```
pvHit = (ttHit && tte->pv_hit()) || (PvNode && depth > 4 * ONE_PLY);
```
while the assignment after IID omits the ` (PvNode && depth > 4 * ONE_PLY) `
condition. As such, unlike the other two post-IID tte reads, this line of code
does not make SF's state more consistent, but rather introduces an inconsistency
in the definition of pvHit. Indeed, changing line 875 read
```
pvHit = (ttHit && tte->pv_hit()) || (PvNode && depth > 4 * ONE_PLY);
```
to match line 642 is functionally equivalent to removing the line entirely, as
this patch does.
STC
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 62756 W: 13787 L: 13746 D: 35223
http://tests.stockfishchess.org/tests/view/5c446c850ebc5902bb5d4b75
LTC
LLR: 3.19 (-2.94,2.94) [-3.00,1.00]
Total: 61900 W: 10179 L: 10111 D: 41610
http://tests.stockfishchess.org/tests/view/5c45bf610ebc5902bb5d5d62
Bench: 3796134
stopOnPonderhit is used to stop search quickly on a ponderhit. It is set by mainThread as part of its time management. However, master employs it as a signal between mainThread and the UCI thread. This is not necessary, it is sufficient for the UCI thread to signal that pondering finished, and mainThread should do its usual time-keeping job, and in this case stop immediately.
This patch implements this, removing stopOnPonderHit as an atomic variable from the ThreadPool,
and moving it as a normal variable to mainThread, reducing its scope. In MainThread::check_time() the search is stopped immediately if ponder switches to false, and the variable stopOnPonderHit is set.
Furthermore, ponder has been moved to mainThread, as the variable is only used to exchange signals between the UCI thread and mainThread.
The version has been tested locally (as fishtest doesn't support ponder):
Score of ponderSimp vs master: 2616 - 2528 - 8630 [0.503] 13774
Elo difference: 2.22 +/- 3.54
which indicates no regression.
No functional change.
The new form is likely to trigger a bit more at LTC. Given that LTC
appears to be an improvement, I think that is fine.
The change is not very invasive: it does the same as before, use
potentially less time for moves that are very stable. Most of the
time, the full bonus was given if the bonus was given, so the gradual
part {3, 4, 5} didn't matter much. Whereas previously 'stable' was
expressed as the last 80% of iterations are the same, now I use a
fixed depth (10 iterations). For TCEC style TC, it will presumably
imply some more moves that are played quicker (and thus more time
on the clock when it potentially matters). Note that 10 iterations
of stability means we've been proposing that move for 99.9% of search
time.
passed STC
http://tests.stockfishchess.org/tests/view/5c30d2290ebc596a450c055b
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 70921 W: 15403 L: 15378 D: 40140
passed LTC
http://tests.stockfishchess.org/tests/view/5c31ae240ebc596a450c1881
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 17422 W: 2968 L: 2842 D: 11612
No functional change.
Introducing new concept, saving principal lines into the transposition table
to generate a "critical search tree" which we can reuse later for intelligent
pruning/extension decisions.
For instance in this patch we just reduce reduction for these lines. But a lot
of other ideas are possible.
To go further : tune some parameters, how to add or remove lines from the
critical search tree, how to use these lines in search choices, etc.
STC :
LLR: 2.94 (-2.94,2.94) [0.50,4.50]
Total: 59761 W: 13321 L: 12863 D: 33577 +2.23 ELO
http://tests.stockfishchess.org/tests/view/5c34da5d0ebc596a450c53d3
LTC :
LLR: 2.96 (-2.94,2.94) [0.00,3.50]
Total: 26826 W: 4439 L: 4191 D: 18196 +2.9 ELO
http://tests.stockfishchess.org/tests/view/5c35ceb00ebc596a450c65b2
Special thanks to Miguel Lahoz for his help in transposition table in/out.
Bench: 3399866
This was inspired after reading about
[Multi-Cut](https://www.chessprogramming.org/Multi-Cut).
We now do non-singular cut node pruning. The idea is to prune when we
have a "backup plan" in case our expected fail high node does not fail
high on the ttMove.
For singular extensions, we do a search on all other moves but the
ttMove. If this fails high on our original beta, this means that both
the ttMove, as well as at least one other move was proven to fail high
on a lower depth search. We then assume that one of these moves will
work on a higher depth and prune.
STC:
LLR: 2.96 (-2.94,2.94) [0.50,4.50]
Total: 72952 W: 16104 L: 15583 D: 41265
http://tests.stockfishchess.org/tests/view/5c3119640ebc596a450c0be5
LTC:
LLR: 2.95 (-2.94,2.94) [0.00,3.50]
Total: 27103 W: 4564 L: 4314 D: 18225
http://tests.stockfishchess.org/tests/view/5c3184c00ebc596a450c1662
Bench: 3145487
The `&& (ss-1)->killers[0] ` conditions are there seemingly to protect
accessing ss-5.
This is unneeded and not so intuitive (as the killer is checked for equality
with currentMove, and that one is non-zero once we're high enough in the stack,
this protects access to ss-5). We can just extend the stack from ss-4 to ss-5,
so we can call update_continuation_histories(ss-1, ..) always in search.
This goes a bit further than #1881 and addresses a comment in #1878.
passed STC:
http://tests.stockfishchess.org/tests/view/5c1aa8d50ebc5902ba127ad0
LLR: 3.12 (-2.94,2.94) [-3.00,1.00]
Total: 53515 W: 11734 L: 11666 D: 30115
passed LTC:
http://tests.stockfishchess.org/tests/view/5c1b272c0ebc5902ba12858d
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 140176 W: 23123 L: 23192 D: 93861
Bench: 3451321
this patch fixes a rare but reproducible segfault observed playing a
multi-threaded match, it is discussed somewhat in fishcooking.
From the core file, it could be observed that the issue was in qsearch, namely:
````
ss->pv[0] = MOVE_NONE;
````
and the backtrace shows the it arrives there via razoring, called from the rootNode:
````
(gdb) bt
alpha=-19, beta=682, depth=DEPTH_ZERO) at search.cpp:1247
````
Indeed, ss->pv can indeed by a nullptr at the rootNode. However, why is the
segfault so rare ?
The reason is that the condition that guards razoring:
````
(depth < 2 * ONE_PLY && eval <= alpha - RazorMargin)
````
is almost never true, since at the root alpha for depth < 5 is -VALUE_INFINITE.
Nevertheless with the new failHigh scheme, this is not guaranteed, and rootDepth > 5,
can still result in a depth < 2 search at the rootNode. If now another thread,
via the hash, writes a new low eval to the rootPos qsearch can be entered.
Rare but not unseen... I assume that some of the crashes in fishtest recently
might be due to this.
Closes https://github.com/official-stockfish/Stockfish/pull/1860
No functional change
On November 30th, @xoto10 experimented with removing this threshold,
but the simplification barely failed LTC. I was inspired to try various
[0, 4] tweaks to increase its value, which would narrow the effects of
this threshold without removing it entirely. Various values repeatedly
led to Elo gains at both STC and LTC, most of which were insufficient
to pass.
After a couple of weeks, I tried again to find an Elo-gaining tweak
but noticed that I could raise the threshold higher and higher without
regression. I decided to try removing it entirely--forgetting that
@xoto10 had already attempted this. However, this now performs much
better at both STC and LTC, producing a STC Elo gain and also potentially
a smaller LTC one.
The reason appears to be a recent change in master (e8ffca3) near
this code, which interacts with this patch. This simplification
governs the conditions under which that patch's effects are applied.
Something non-obvious about that change has significantly improved
the performance of this simplification.
I recognize and thank @xoto10, who originally had this idea. Since
I ran several LTCs recently (to determine whether to open this PR,
or one for a related [0, 4]), I would also like to acknowledge the
other developers and CPU donors for their patience. Thank you all!
STC:
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 13445 W: 3000 L: 2862 D: 7583
http://tests.stockfishchess.org/tests/view/5c11f01b0ebc5902ba11a6b8
LTC:
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 33868 W: 5663 L: 5563 D: 22642
http://tests.stockfishchess.org/tests/view/5c11ffe90ebc5902ba11a8a9
Closes https://github.com/official-stockfish/Stockfish/pull/1870
Bench: 3343286
STC:
LLR: 2.96 (-2.94,2.94) [0.00,5.00]
Total: 13323 W: 3015 L: 2818 D: 7490
http://tests.stockfishchess.org/tests/view/5c00a2520ebc5902bcedd41b
LTC:
LLR: 2.96 (-2.94,2.94) [0.00,5.00]
Total: 52294 W: 9093 L: 8756 D:34445
http://tests.stockfishchess.org/tests/view/5c00b2c40ebc5902bcedd596
Some obvious followups to this are to further tune this PSQT, or
try 8x8 for other pieces. As of now I don't plan on trying this
for other pieces as I think the majority of the ELO it brings is
for pawns and kings.
Looking at the new values, the differences between kingside and
queenside are quite significant. I am very hopeful that this a
llows SF to understand and plan pawn structures even better than
it already does. Cheers!
Closes https://github.com/official-stockfish/Stockfish/pull/1839
Bench: 3569243
I've gone through the RENAME/REFORMATTING thread and changed everything I could find, plus a few more. With this, let's close the previous issue and open another.
No functional change.
Remove the F[] array which I find unhelpful and rename `improvingFactor` to
`fallingEval` since larger values indicate a falling eval and more time use.
I realise a test was not strictly necessary, but I ran STC [-3,1] just to
check there are no foolish errors before creating the pull request:
STC:
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 35804 W: 7753 L: 7659 D: 20392
http://tests.stockfishchess.org/tests/view/5bef3a0c0ebc595e0ae39c19
It was then suggested to clean the constants around `fallingEval`
to make it more clear this is a factor around ~1 that adjusts time
up or downwards depending on some conditions. We then ran a double
test with this simplification suggestion:
STC:
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 68435 W: 14936 L: 14906 D: 38593
http://tests.stockfishchess.org/tests/view/5c02c56b0ebc5902bcee0184
LTC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 37258 W: 6324 L: 6230 D: 24704
http://tests.stockfishchess.org/tests/view/5c030a520ebc5902bcee0a32
No functional change
On November 16th, before the removal of the depth condition, I tried
revising castling extensions to only handle castling moves, rather than
moves that change castling rights generally. It appeared to be a slight
Elo gain at STC but insufficient to pass [0, 4] (+0.5 Elo), but what I
overlooked was that it made pos.can_castle(us) irrelevant and should
have been a simplification. Recent discussion with @Chess13234 and
Michael Chaly (@Vizvezdenec) inspired me to take a second look, and
the simplification continues to pass when rebased on the current master.
This replaces two conditions with one, because type_of(move) == CASTLING
implies pos.can_castle(Us), allowing us to remove the latter condition.
STC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 110948 W: 24209 L: 24263 D: 62476
http://tests.stockfishchess.org/tests/view/5bf8f65c0ebc5902bced3a63
LTC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 88283 W: 14681 L: 14668 D: 58934
http://tests.stockfishchess.org/tests/view/5bf994a60ebc5902bced4349
Bench: 3939338
Don't do an extra TT update in case of a fail-high,
but simply break off the moves loop and let the TT update
at the end of qsearch do this job.
Same workflow/logic as in our main search function now.
Tested for no regression to be on the safe side.
STC
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 30237 W: 6665 L: 6560 D: 17012
http://tests.stockfishchess.org/tests/view/5bf928e80ebc5902bced3f3a
LTC
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 51067 W: 8625 L: 8553 D: 33889
http://tests.stockfishchess.org/tests/view/5bf937180ebc5902bced3fdc
No functional change.
Because of aggressive time management and optimistic assumptions
about move overhead, it's still very easy to get Stockfish to forfeit
on time when we hit an endgame and have Syzygy EGTB on a spinning
drive. The latency from serving a few thousand EGTB probes (~10ms each),
of which there can currently be up to 4000 outstanding before a time
check, will easily overwhelm the default Move Overhead of 30ms.
This problem was first raised by Gian-Carlo Pascutto and some solutions
and improvements were discussed in the following pull requests:
https://github.com/official-stockfish/Stockfish/pull/1471https://github.com/official-stockfish/Stockfish/pull/1623https://github.com/official-stockfish/Stockfish/pull/1783
This patch is a minimal change proposed by Marco Costalba to lower
the impact of the bug. We now force a check of the clock right after
each tablebase read.
No functional change.
