Introducing new concept, saving principal lines into the transposition table
to generate a "critical search tree" which we can reuse later for intelligent
pruning/extension decisions.
For instance in this patch we just reduce reduction for these lines. But a lot
of other ideas are possible.
To go further : tune some parameters, how to add or remove lines from the
critical search tree, how to use these lines in search choices, etc.
STC :
LLR: 2.94 (-2.94,2.94) [0.50,4.50]
Total: 59761 W: 13321 L: 12863 D: 33577 +2.23 ELO
http://tests.stockfishchess.org/tests/view/5c34da5d0ebc596a450c53d3
LTC :
LLR: 2.96 (-2.94,2.94) [0.00,3.50]
Total: 26826 W: 4439 L: 4191 D: 18196 +2.9 ELO
http://tests.stockfishchess.org/tests/view/5c35ceb00ebc596a450c65b2
Special thanks to Miguel Lahoz for his help in transposition table in/out.
Bench: 3399866
This was inspired after reading about
[Multi-Cut](https://www.chessprogramming.org/Multi-Cut).
We now do non-singular cut node pruning. The idea is to prune when we
have a "backup plan" in case our expected fail high node does not fail
high on the ttMove.
For singular extensions, we do a search on all other moves but the
ttMove. If this fails high on our original beta, this means that both
the ttMove, as well as at least one other move was proven to fail high
on a lower depth search. We then assume that one of these moves will
work on a higher depth and prune.
STC:
LLR: 2.96 (-2.94,2.94) [0.50,4.50]
Total: 72952 W: 16104 L: 15583 D: 41265
http://tests.stockfishchess.org/tests/view/5c3119640ebc596a450c0be5
LTC:
LLR: 2.95 (-2.94,2.94) [0.00,3.50]
Total: 27103 W: 4564 L: 4314 D: 18225
http://tests.stockfishchess.org/tests/view/5c3184c00ebc596a450c1662
Bench: 3145487
This addresses partially issue #1911 in that it documents in our
Readme the command that users can use to verifying the md5sum of
their downloaded tablebase files.
Additionally, a quick check of the file size (the size of each
tablebase file modulo 64 is 16 as pointed out by @syzygy1) has been
implemented at launch time in Stockfish.
Closes https://github.com/official-stockfish/Stockfish/pull/1927
and https://github.com/official-stockfish/Stockfish/issues/1911
No functional change.
Delay legality check of castling moves at search time,
just before making the move, as is the standard with all
the other move types.
This should avoid an useless and not trivial legality check
when the castling is then not tried later. For instance due
to a previous cut-off.
The patch is also a big simplification and allows to entirely
remove generate_castling()
Bench changes due to a different move sequence out of MovePicker.
STC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 45073 W: 9918 L: 9843 D: 25312
http://tests.stockfishchess.org/tests/view/5c2f176f0ebc596a450bdfb3
LTC:
LLR: 3.15 (-2.94,2.94) [-3.00,1.00]
Total: 10156 W: 1707 L: 1560 D: 6889
http://tests.stockfishchess.org/tests/view/5c2e7dfd0ebc596a450bcdf4
Verified with perft both in standard and Chess960 cases.
Closes https://github.com/official-stockfish/Stockfish/pull/1929
Bench: 3559104
A single popcount in evaluate.cpp replaces all openFiles stuff in pawns. It doesn't seem to affect performance at all.
STC
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 28103 W: 6134 L: 6025 D: 15944
http://tests.stockfishchess.org/tests/view/5b7d70a20ebc5902bdbb1999
No functional change.
This custom predicate filter creates an unnecessary abstraction layer, but doesn't make the code any more readable. The code is clear enough without it.
No functional change.
The extra condition is used as a shortcut to skip the following 3 assignments:
```C++
Bitboard b1 = shift<UpRight>(pawnsOn7) & enemies;
Bitboard b2 = shift<UpLeft >(pawnsOn7) & enemies;
Bitboard b3 = shift<Up >(pawnsOn7) & emptySquares;
```
In case of EVASION with no target on 8th rank (the common case), we end up performing the 3 statements for nothing because b1 = b2 = b3 = 0.
But this is just a small micro-optimization and the condition is quite confusing, so just remove it and prefer a readable code instead.
STC
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 78020 W: 16978 L: 16967 D: 44075
http://tests.stockfishchess.org/tests/view/5c27b4fe0ebc5902ba135bb0
No functional change.
I tried to improve the Readme because many people in my local
chess club do not understand some of the UCO options properly.
Starting point of this was Cfish's Readme by Ronald de Man,
some internet resources and the Stockfish code itself.
Closes https://github.com/official-stockfish/Stockfish/pull/1898
Initial commit by user @erbsenzaehler, with help from users
Adrian Petrescu, @alayan-stk-2 and Elvin Liu.
No functional change
Co-Authored-By: Alayan-stk-2 <alayan-stk-2@users.noreply.github.com>
Co-Authored-By: Adrian Petrescu <apetresc@gmail.com>
Co-Authored-By: Elvin Liu <solarlight2@users.noreply.github.com>
Recent tests by @xoto10, @Vizvezdenec, and myself seemed to hint that Elo could
be gained by expanding the number of cases where king safety is applied. Several
users (@Spliffjiffer, @Vizvezdenec) have anticipated benefits specifically in
evaluation of tactics. It appears that we actually do not need to restrict the
cases in which we initialize and evaluate king safety at all: initializing and
evaluating it in every position appears roughly Elo-neutral at STC and possibly
a substantial Elo gain at LTC.
Any explanation for this scaling is, at this point, conjecture. Assuming it is
not due to chance, my hypothesis is that initialization of king safety in all
positions is a mild slowdown, offset by an Elo gain of evaluating king safety
in all positions. At STC this produces Elo gains and losses that offset each
other, while at longer time control the slowdown is much less important, leaving
only the Elo gain. It probably helps SF to explore king attacks much earlier in
search with high numbers of enemy pieces concentrating but not essentially attacking
king ring.
Thanks to @xoto10 and @Vizvezdenec for helping run my LTC!
Closes https://github.com/official-stockfish/Stockfish/pull/1906
STC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 35432 W: 7815 L: 7721 D: 19896
http://tests.stockfishchess.org/tests/view/5c24779d0ebc5902ba131b26
LTC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 12887 W: 2217 L: 2084 D: 8586
http://tests.stockfishchess.org/tests/view/5c25049a0ebc5902ba132586
Bench: 3163951
------------------
How to continue from there?
* Next step will be to tune all the king danger terms once more after that :-)
When iterating through 'SYSTEM_LOGICAL_PROCESSOR_INFORMATION_EX' structure, do not use structure member beyond known size.
API is guaranteed to provide us at lease one element upon successful, and no element in the structure can have a zero size.
No functional change.
This pull request fixes a rare crashing bug on Windows inside our NUMA code, first
reported by Dann Corbit in the following forum thread (thanks!):
https://groups.google.com/forum/?fromgroups=#!topic/fishcooking/gA6aoMEuOwg
The fix is to not use structure member beyond known size when iterating through
'SYSTEM_LOGICAL_PROCESSOR_INFORMATION_EX' structure. We note that the Microsoft
API is guaranteed to provide us at least one element upon successful, and no
element in the structure can have a zero size.
No functional change.
The `&& (ss-1)->killers[0] ` conditions are there seemingly to protect
accessing ss-5.
This is unneeded and not so intuitive (as the killer is checked for equality
with currentMove, and that one is non-zero once we're high enough in the stack,
this protects access to ss-5). We can just extend the stack from ss-4 to ss-5,
so we can call update_continuation_histories(ss-1, ..) always in search.
This goes a bit further than #1881 and addresses a comment in #1878.
passed STC:
http://tests.stockfishchess.org/tests/view/5c1aa8d50ebc5902ba127ad0
LLR: 3.12 (-2.94,2.94) [-3.00,1.00]
Total: 53515 W: 11734 L: 11666 D: 30115
passed LTC:
http://tests.stockfishchess.org/tests/view/5c1b272c0ebc5902ba12858d
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 140176 W: 23123 L: 23192 D: 93861
Bench: 3451321
Even when playing without endgame table bases, this particular endgame should
be a win 100% of the time when Stockfish is given a KRBK position, assuming
there are enough moves remaining in the FEN to finish the game without hitting
the 50 move rule.
PROBLEM: The issue with master here is that the PushClose difference per square
is 20, however, the difference in squares for the PushToCorners array is usually
less. Thus, the engine prefers to move the kings closer together rather than pushing
the weak king to the correct corner.
What happens is if the weak king is in a safe corner, SF still prefers pushing the
kings together. Occasionally, the strong king traps the weak king in the safe corner.
It takes a while for SF to figure it out, but often draws the game by the 50 move rule
(on shorter time controls).
This patch increases the PushToCorners values to correct this problem. We also added
an assert to catch any overflow problem if anybody would want to increase the array
values again in the future.
It was tested in a couple of matches starting with random KRBK positions and showed
increased winning rates, see https://github.com/official-stockfish/Stockfish/pull/1877
No functional change
* Update our continuous integration machinery
Ubuntu 16.04 can now be used with travis. Updating all the other stuff
when there.
Invoking the lld linker seems to save 5 minutes with clang on linux.
No functional change.
* fix
* Use a bit less code to calculate hashfull(). Change post increment to preincrement as is more standard
in the rest of stockfish. Scale result so we report 100% instead of 99.9% when completely full.
No functional change.
Although this is a compile-time constant, we stick the castlingSide into a CastlingRight, then pull it out again. This seems unecessarily complex.
No functional change.
We do not need to change the winnerKSq variable, so we can simplify
a little bit the logic of the code by changing only the loserKSq
variable when it is necessary. Also consolidate and clarify comments.
See the pull request thread for a proof that the code is correct:
https://github.com/official-stockfish/Stockfish/pull/1854
No functional change
~stronglyProtected is quite similar to ~attackedBy[Them][PAWN] & ~attackedBy2[Them],
the only difference appears to be that the former includes squares attacked twice
by both sides. The resulting logic is simpler, and the change appears to be at least
Elo-neutral at both STC and LTC.
STC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 35924 W: 7978 L: 7885 D: 20061
http://tests.stockfishchess.org/tests/view/5c14a5c00ebc5902ba11ed72
LTC:
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 37078 W: 6125 L: 6030 D: 24923
http://tests.stockfishchess.org/tests/view/5c14ae880ebc5902ba11eed8
Bench: 3646542
this patch fixes a rare but reproducible segfault observed playing a
multi-threaded match, it is discussed somewhat in fishcooking.
From the core file, it could be observed that the issue was in qsearch, namely:
````
ss->pv[0] = MOVE_NONE;
````
and the backtrace shows the it arrives there via razoring, called from the rootNode:
````
(gdb) bt
alpha=-19, beta=682, depth=DEPTH_ZERO) at search.cpp:1247
````
Indeed, ss->pv can indeed by a nullptr at the rootNode. However, why is the
segfault so rare ?
The reason is that the condition that guards razoring:
````
(depth < 2 * ONE_PLY && eval <= alpha - RazorMargin)
````
is almost never true, since at the root alpha for depth < 5 is -VALUE_INFINITE.
Nevertheless with the new failHigh scheme, this is not guaranteed, and rootDepth > 5,
can still result in a depth < 2 search at the rootNode. If now another thread,
via the hash, writes a new low eval to the rootPos qsearch can be entered.
Rare but not unseen... I assume that some of the crashes in fishtest recently
might be due to this.
Closes https://github.com/official-stockfish/Stockfish/pull/1860
No functional change
As noticed in the forum, a crash in extract_ponder_from_tt could result
if hash size is set before the ponder move is printed. While it is arguably
a GUI issue (but it got me on the cli), it is easy to avoid this issue.
Closes https://github.com/official-stockfish/Stockfish/pull/1856
No functional change.
On November 30th, @xoto10 experimented with removing this threshold,
but the simplification barely failed LTC. I was inspired to try various
[0, 4] tweaks to increase its value, which would narrow the effects of
this threshold without removing it entirely. Various values repeatedly
led to Elo gains at both STC and LTC, most of which were insufficient
to pass.
After a couple of weeks, I tried again to find an Elo-gaining tweak
but noticed that I could raise the threshold higher and higher without
regression. I decided to try removing it entirely--forgetting that
@xoto10 had already attempted this. However, this now performs much
better at both STC and LTC, producing a STC Elo gain and also potentially
a smaller LTC one.
The reason appears to be a recent change in master (e8ffca3) near
this code, which interacts with this patch. This simplification
governs the conditions under which that patch's effects are applied.
Something non-obvious about that change has significantly improved
the performance of this simplification.
I recognize and thank @xoto10, who originally had this idea. Since
I ran several LTCs recently (to determine whether to open this PR,
or one for a related [0, 4]), I would also like to acknowledge the
other developers and CPU donors for their patience. Thank you all!
STC:
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 13445 W: 3000 L: 2862 D: 7583
http://tests.stockfishchess.org/tests/view/5c11f01b0ebc5902ba11a6b8
LTC:
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 33868 W: 5663 L: 5563 D: 22642
http://tests.stockfishchess.org/tests/view/5c11ffe90ebc5902ba11a8a9
Closes https://github.com/official-stockfish/Stockfish/pull/1870
Bench: 3343286
STC:
LLR: 2.96 (-2.94,2.94) [0.00,5.00]
Total: 13323 W: 3015 L: 2818 D: 7490
http://tests.stockfishchess.org/tests/view/5c00a2520ebc5902bcedd41b
LTC:
LLR: 2.96 (-2.94,2.94) [0.00,5.00]
Total: 52294 W: 9093 L: 8756 D:34445
http://tests.stockfishchess.org/tests/view/5c00b2c40ebc5902bcedd596
Some obvious followups to this are to further tune this PSQT, or
try 8x8 for other pieces. As of now I don't plan on trying this
for other pieces as I think the majority of the ELO it brings is
for pawns and kings.
Looking at the new values, the differences between kingside and
queenside are quite significant. I am very hopeful that this a
llows SF to understand and plan pawn structures even better than
it already does. Cheers!
Closes https://github.com/official-stockfish/Stockfish/pull/1839
Bench: 3569243
I've gone through the RENAME/REFORMATTING thread and changed everything I could find, plus a few more. With this, let's close the previous issue and open another.
No functional change.
This reverts commit 33d9548218 ,
which crashed in DEBUG mode because of the following assert in position.h
````
Assertion failed: (is_ok(m)), function capture, file ./position.h, line 369.
````
No functional change
Remove the F[] array which I find unhelpful and rename `improvingFactor` to
`fallingEval` since larger values indicate a falling eval and more time use.
I realise a test was not strictly necessary, but I ran STC [-3,1] just to
check there are no foolish errors before creating the pull request:
STC:
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 35804 W: 7753 L: 7659 D: 20392
http://tests.stockfishchess.org/tests/view/5bef3a0c0ebc595e0ae39c19
It was then suggested to clean the constants around `fallingEval`
to make it more clear this is a factor around ~1 that adjusts time
up or downwards depending on some conditions. We then ran a double
test with this simplification suggestion:
STC:
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 68435 W: 14936 L: 14906 D: 38593
http://tests.stockfishchess.org/tests/view/5c02c56b0ebc5902bcee0184
LTC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 37258 W: 6324 L: 6230 D: 24704
http://tests.stockfishchess.org/tests/view/5c030a520ebc5902bcee0a32
No functional change
Exclude doubly protected by pawns squares when calculating attackers on
king ring. Idea of this patch is not to count attackers if they attack
only squares that are protected by two pawns.
STC
LLR: 2.95 (-2.94,2.94) [0.00,5.00]
Total: 70040 W: 15476 L: 15002 D: 39562
http://tests.stockfishchess.org/tests/view/5c0354860ebc5902bcee1106
LTC
LLR: 2.96 (-2.94,2.94) [0.00,5.00]
Total: 16530 W: 2795 L: 2607 D: 11128
http://tests.stockfishchess.org/tests/view/5c0385080ebc5902bcee14b5
This is third king safety patch in recent times so we probably need
retuning of king safety parameters.
Bench: 3057978
Official release version of Stockfish 10.
This is also the 10th anniversary version of the Stockfish project, which
started exactly ten years ago! I wish to extend a huge thank you to
all contributors and authors in our amazing community :-)
Bench: 3939338
The patch was tested for correctness by running bench with and
without the change against current master, and the tablebase hit
numbers were found to be identical in both cases. See the pull
request comments for details:
https://github.com/official-stockfish/Stockfish/pull/1826
No functional change.
On November 16th, before the removal of the depth condition, I tried
revising castling extensions to only handle castling moves, rather than
moves that change castling rights generally. It appeared to be a slight
Elo gain at STC but insufficient to pass [0, 4] (+0.5 Elo), but what I
overlooked was that it made pos.can_castle(us) irrelevant and should
have been a simplification. Recent discussion with @Chess13234 and
Michael Chaly (@Vizvezdenec) inspired me to take a second look, and
the simplification continues to pass when rebased on the current master.
This replaces two conditions with one, because type_of(move) == CASTLING
implies pos.can_castle(Us), allowing us to remove the latter condition.
STC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 110948 W: 24209 L: 24263 D: 62476
http://tests.stockfishchess.org/tests/view/5bf8f65c0ebc5902bced3a63
LTC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 88283 W: 14681 L: 14668 D: 58934
http://tests.stockfishchess.org/tests/view/5bf994a60ebc5902bced4349
Bench: 3939338
When running on a cloud VM (n1-highcpu-96) with several NVMe SSDs and
some non-SSDs for tablebases, I noticed that the average SSD request size was
more than 256 kB. This doesn't make a lot of sense for Syzygy tablebases,
which have a block size of 32 bytes and very low locality.
Seemingly, the tablebase access patterns during probing make the OS,
at least Linux, think that readahead is advantageous; normally, it
gives up doing readahead if there are too many misses, but it doesn't,
perhaps due to the fairly high overall hit rates. (It seems the kernel cannot
distinguish between reading a block that was paged in because the userspace
wanted it explicitly, and one that was read as part of readahead.)
Setting MADV_RANDOM effectively turns off readahead, which causes
the request size to drop to 4 kB. In the aforemented cloud VM test,
this roughly tripled the amount of I/O requests that were able to go
through, while reducing the total traffic from 2.8 GB/sec to 56 MB/sec
(moving the bottleneck to the non-SSDs; it seems the SSDs could have
sustained many more requests).
Closes https://github.com/official-stockfish/Stockfish/pull/1829
No functional change.
Don't do an extra TT update in case of a fail-high,
but simply break off the moves loop and let the TT update
at the end of qsearch do this job.
Same workflow/logic as in our main search function now.
Tested for no regression to be on the safe side.
STC
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 30237 W: 6665 L: 6560 D: 17012
http://tests.stockfishchess.org/tests/view/5bf928e80ebc5902bced3f3a
LTC
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 51067 W: 8625 L: 8553 D: 33889
http://tests.stockfishchess.org/tests/view/5bf937180ebc5902bced3fdc
No functional change.
Tropism in kingdanger was simplified away in this pull request #1821.
This patch reintroduces tropism in kingdanger with using quadratic scaling.
Passed STC http://tests.stockfishchess.org/tests/view/5bf7c1b10ebc5902bced1f8f
LLR: 2.96 (-2.94,2.94) [0.00,5.00]
Total: 52803 W: 11835 L: 11442 D: 29526
Passed LTC http://tests.stockfishchess.org/tests/view/5bf816e90ebc5902bced24f1
LLR: 2.96 (-2.94,2.94) [0.00,5.00]
Total: 17204 W: 2988 L: 2795 D: 11421
How do we continue from there?
I've recently tried to introduce tropism difference term in kingdanger which
passed STC 6 times but failed LTC all the time. Maybe using quadratic scaling
for it will also be helpful.
Bench 4041387
A recent LTC tuning session by @candirufish showed this term decreasing significantly. It appears that it can be removed altogether without significant Elo loss.
I also thank @GuardianRM, whose attempt to remove tropism from king danger inspired this one.
After this PR is merged, my next step will be to attempt to tune the coefficients of this new, simplified kingDanger calculation.
STC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 12518 W: 2795 L: 2656 D: 7067
http://tests.stockfishchess.org/tests/view/5befadda0ebc595e0ae3a289
LTC:
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 164771 W: 26463 L: 26566 D: 111742
http://tests.stockfishchess.org/tests/view/5befcca70ebc595e0ae3a343
LTC 2, rebased on Stockfish 10 beta:
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 75226 W: 12563 L: 12529 D: 50134
http://tests.stockfishchess.org/tests/view/5bf2e8910ebc5902bcecb919
Bench: 3412071
