This patch simplifies latest @MJZ1977 elo gainer. Seems like PvNode check in
condition of last capture extension is not needed. Note - even if this is a
simplification it actually causes this extension to be applied more often, thus
strengthening effect of @MJZ1977's patch.
passed STC
http://tests.stockfishchess.org/tests/view/5deb9a3eb7bdefd50db28d0e
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 80244 W: 17421 L: 17414 D: 45409
passed LTC
http://tests.stockfishchess.org/tests/view/5deba860b7bdefd50db28d11
LLR: 2.94 (-2.94,2.94) [-3.00,1.00]
Total: 21506 W: 3565 L: 3446 D: 14495
Bench: 5097036
As reported on the forum it is possible, on very rare occasions, that we are
trying to print a PV line with an invalid previousScore, although this line
has a valid actual score. This patch fixes output of PV lines with invalid
scores in a MultiPV search. This is a follow-up patch to 8b15961 and makes
the fix finally complete.
The reason is the i <= pvIdx condition which probably is a leftover from the
times there was a special root search function. This check is no longer needed
today and prevents PV lines past the current one (current pvIdx) to be flagged
as updated even though they do have a valid score.
8b15961349https://groups.google.com/forum/?fromgroups=#!topic/fishcooking/PrnoDLvMvro
No functional change.
the removed line is not needed, since with the conditions on SE, eval
equals ttValue (except inCheck), which must be larger than beta if the second condition
is true.
The removed line is also incorrect as eval might be VALUE_NONE at this
location if inCheck. This removal addresses part of https://github.com/official-stockfish/Stockfish/pull/2406#issuecomment-552642608
No functional change.
This patch measures how frequently search is exploring new configurations.
This is done be computing a running average of ttHit. The ttHitAverage rate
is somewhat low (e.g. 30% for startpos) in the normal case, while it can be
very high if no progress is made (e.g. 90% for the fortress I used for testing).
This information can be used to influence search. In this patch, by adjusting
reductions if the rate > 50%. A first version (using a low ttHitAverageResolution
and this 50% threshold) passed testing:
STC
LLR: 2.96 (-2.94,2.94) [-1.50,4.50]
Total: 26425 W: 5837 L: 5650 D: 14938
http://tests.stockfishchess.org/tests/view/5dcede8b0ebc5902563258fa
LTC
LLR: 2.96 (-2.94,2.94) [0.00,3.50]
Total: 32313 W: 5392 L: 5128 D: 21793
http://tests.stockfishchess.org/tests/view/5dcefb1f0ebc590256325c0e
However, as discussed in pull request 2414, using a larger ttHitAverageResolution
gives a better approximation of the underlying distributions. This needs a slight
adjustment for the threshold as the new distributions are shifted a bit compared
to the older ones, and this threshold seemingly is sensitive (we used 0.53125 here).
https://github.com/official-stockfish/Stockfish/pull/2414
This final version also passed testing, and is used for the patch:
STC
LLR: 2.95 (-2.94,2.94) [-1.50,4.50]
Total: 16025 W: 3555 L: 3399 D: 9071
http://tests.stockfishchess.org/tests/view/5dd070b90ebc5902579e20c2
LTC
LLR: 2.96 (-2.94,2.94) [0.00,3.50]
Total: 37576 W: 6277 L: 5998 D: 25301
http://tests.stockfishchess.org/tests/view/5dd0f58e6f544e798086f224
Closes https://github.com/official-stockfish/Stockfish/pull/2414
Bench: 4989584
The removed lines approximately duplicate equivalent logic in the movePicker.
Adjust the futility_move_count to componsate for some difference
(the movePicker prunes one iteration of the move loop later).
Passed STC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 8114 W: 1810 L: 1663 D: 4641
http://tests.stockfishchess.org/tests/view/5dc6afe60ebc5902562bd318
Passed LTC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 89956 W: 14473 L: 14460 D: 61023
http://tests.stockfishchess.org/tests/view/5dc6bdcf0ebc5902562bd3c0
Closes https://github.com/official-stockfish/Stockfish/pull/2407
Bench: 4256440
---------------------
How to continue from there?
It would be interesting to see if we can extract some Elo gain
from the new futility_move_count formula, for instance by somehow
incorporating the final -1 in the 5 constant, or adding a linear
term to the quadratics...
```
futility_move_count = (5 + depth * depth) * (1 + improving) / 2 - 1
```
Current master 648c7ec25d will generate an
incorrect mate score for:
```
setoption name Hash value 8
setoption name Threads value 1
position fen 8/1p2KP2/1p4q1/1Pp5/2P5/N1Pp1k2/3P4/1N6 b - - 76 40
go depth 49
```
even though the position is a draw. Generally, SF tries to display only
proven mate scores, so this is a bug.
This was posted http://www.talkchess.com/forum3/viewtopic.php?f=2&t=72166
by Uri Blass, with the correct analysis that this must be related to the
50 moves draw rule being ignored somewhere.
Indeed, this is possible as positions and there eval are stored in the TT,
without reference to the 50mr counter. Depending on the search path followed
a position can thus be mate or draw in the TT (GHI or Graph history interaction).
Therefore, to prove mate lines, the TT content has to be used with care. Rather
than ignoring TT content in general or for mate scores (which impact search or
mate finding), it is possible to be more selective. In particular, @WOnder93
suggested to only ignore the TT if the 50mr draw ply is closer than the mate
ply. This patch implements this idea, by clamping the eval in the TT to
+-VALUE_MATED_IN_MAX_PLY. This retains the TTmove, but causes a research of
these lines (with the current 50mr counter) as needed.
This patch hardly ever affects search (as indicated by the unchanged
bench), but fixes the testcase. As the conditions are very specific,
also mate finding will almost never be less efficient (testing welcome).
It was also shown to pass STC and LTC non-regression testing, in a form
using if/then/else instead of ternary operators:
STC:
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 93605 W: 15346 L: 15340 D: 62919
http://tests.stockfishchess.org/tests/view/5db45bb00ebc5908127538d4
LTC:
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 33873 W: 7359 L: 7261 D: 19253
http://tests.stockfishchess.org/tests/view/5db4c8940ebc5902d6b146fc
closes https://github.com/official-stockfish/Stockfish/issues/2370
Bench: 4362323
Make dynamic contempt weight factor dependent on static contempt so that higher
static contempt implies less dynamic contempt and vice versa. For default contempt
24 this is a non-functional change. But tests with contempt 0 shows an elo gain.
Also today is my birthday so i have already give to myself a gift with this patch :-)!
Further proceedings:
in the past we checked for default contempt that it doesn't regress against
contempt 0. Now that the later is stronger and the former is the same strength
this should be rechecked. Perhaps the default contempt have to be lowered.
It would be interesting to get some idea of the impact of this patch outside
of the 0-24 contempt range.
STC: (both with contempt=0)
LLR: 2.95 (-2.94,2.94) [-1.50,4.50]
Total: 21912 W: 3898 L: 3740 D: 14274
http://tests.stockfishchess.org/tests/view/5db74b6f0ebc5902d1f37405
LTC: (both with contempt=0)
LLR: 2.96 (-2.94,2.94) [0.00,3.50]
Total: 27172 W: 3350 L: 3126 D: 20696
http://tests.stockfishchess.org/tests/view/5db760020ebc5902d1f375d0
Closes https://github.com/official-stockfish/Stockfish/pull/2382
No functional change (for current default contempt 24).
This PR refactors update_quiet_stats, update_capture_stats and search to more clearly reflect what is actually done.
Effectively, all stat updates that need to be done after search is finished and a bestmove is found,
are collected in a new function ```final_stats_update()```. This shortens our main search routine, and simplifies ```update_quiet_stats```.
The latter function is now more easily reusable with fewer arguments, as the handling of ```quietsSearched``` is only needed in ```final_stats_update```.
```update_capture_stats```, which was only called once is now integrated in ```final_stats_update```, which allows for removing a branch and reusing some ```stat_bonus``` calls. The need for refactoring was also suggested by the fact that the comments of ```update_quiet_stats``` and ```update_capture_stats``` were incorrect (e.g. ```update_capture_stats``` was called, correctly, also when the bestmove was a quiet and not a capture).
passed non-regression STC:
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 75196 W: 16364 L: 16347 D: 42485
http://tests.stockfishchess.org/tests/view/5db004ec0ebc5902c06db9e1
The diff is most easily readable as ```git diff master --patience```
No functional change
- Cleanups by Alain
- Group king attacks and king defenses
- Signature of futility_move_count()
- Use is_discovery_check_on_king()
- Simplify backward definition
- Use static asserts in move generator
- Factor a statement in move generator
No functional change
This patch changes the base aspiration window size depending on the absolute
value of the previous iteration score, increasing it away from zero. This
stems from the observation that the further away from zero, the more likely
the evaluation is to change significantly with more depth. Conversely, a
tighter aspiration window is more efficient when close to zero.
A beneficial side-effect is that analysis of won positions without a quick
mate is less prone to waste nodes in repeated fail-high that change the eval
by tiny steps.
STC:
LLR: 2.96 (-2.94,2.94) [0.50,4.50]
Total: 60102 W: 13327 L: 12868 D: 33907
http://tests.stockfishchess.org/tests/view/5d9a70d40ebc5902b6cf39ba
LTC:
LLR: 2.95 (-2.94,2.94) [0.00,3.50]
Total: 155553 W: 25745 L: 25141 D: 104667
http://tests.stockfishchess.org/tests/view/5d9a7ca30ebc5902b6cf4028
Future work : the values used in this patch were only a reasonable guess.
Further testing should unveil more optimal values. However, the aspiration
window is rather tight with a minimum of 21 internal units, so discrete
integers put a practical limitation to such tweaking.
More exotic experiments around the aspiration window parameters could also
be tried, but efficient conditions to adjust the base aspiration window size
or allow it to not be centered on the current evaluation are not obvious.
The aspiration window increases after a fail-high or a fail-low is another
avenue to explore for potential enhancements.
Bench: 4043748
It is always used as a bool, so let's make it a bool straight away.
We can always redefine it as a Piece in a later patch if we want
to use the piece type or the piece color.
No functional change.
Simplification that eliminates ONE_PLY, based on a suggestion in the forum that
support for fractional plies has never been used, and @mcostalba's openness to
the idea of eliminating it. We lose a little bit of type safety by making Depth
an integer, but in return we simplify the code in search.cpp quite significantly.
No functional change
------------------------------------------
The argument favoring eliminating ONE_PLY:
* The term “ONE_PLY” comes up in a lot of forum posts (474 to date)
https://groups.google.com/forum/?fromgroups=#!searchin/fishcooking/ONE_PLY%7Csort:relevance
* There is occasionally a commit that breaks invariance of the code
with respect to ONE_PLY
https://groups.google.com/forum/?fromgroups=#!searchin/fishcooking/ONE_PLY%7Csort:date/fishcooking/ZIPdYj6k0fk/KdNGcPWeBgAJ
* To prevent such commits, there is a Travis CI hack that doubles ONE_PLY
and rechecks bench
* Sustaining ONE_PLY has, alas, not resulted in any improvements to the
engine, despite many individuals testing many experiments over 5 years.
The strongest argument in favor of preserving ONE_PLY comes from @locutus:
“If we use par example ONE_PLY=256 the parameter space is increases by the
factor 256. So it seems very unlikely that the optimal setting is in the
subspace of ONE_PLY=1.”
There is a strong theoretical impediment to fractional depth systems: the
transposition table uses depth to determine when a stored result is good
enough to supply an answer for a current search. If you have fractional
depths, then different pathways to the position can be at fractionally
different depths.
In the end, there are three separate times when a proposal to remove ONE_PLY
was defeated by the suggestion to “give it a few more months.” So… it seems
like time to remove this distraction from the community.
See the pull request here:
https://github.com/official-stockfish/Stockfish/pull/2289
In lazySMP it makes sense to prune a little more, as multiple threads
search wider. We thus increase the prefactor of the reductions slowly
as a function of the threads. The prefactor of the log(threads) term
is a parameter, this pull request uses 1/2 after testing.
passed STC @ 8threads:
LLR: 2.96 (-2.94,2.94) [0.50,4.50]
Total: 118125 W: 23151 L: 22462 D: 72512
http://tests.stockfishchess.org/tests/view/5d8bbf4d0ebc59509180f217
passed LTC @ 8threads:
LLR: 2.95 (-2.94,2.94) [0.00,3.50]
Total: 67546 W: 10630 L: 10279 D: 46637
http://tests.stockfishchess.org/tests/view/5d8c463b0ebc5950918167e8
passed ~LTC @ 14threads:
LLR: 2.95 (-2.94,2.94) [0.00,3.50]
Total: 74271 W: 12421 L: 12040 D: 49810
http://tests.stockfishchess.org/tests/view/5d8db1f50ebc590f3beb24ef
Note:
A larger prefactor (1) passed similar tests at STC and LTC (8 threads),
while a very large one (2) passed STC quickly but failed LTC (8 threads).
For the single-threaded case there is no functional change.
Closes https://github.com/official-stockfish/Stockfish/pull/2337
Bench: 4088701
Fixup: remove redundant code.
A curious feature of Stockfish's current extension code is its repeated
use of "else if." In most cases, this makes no functional difference,
because no more than one extension is applied; once one extension has
been applied, the remaining ones can be safely ignored.
However, if most singular extension search conditions are true, except
"value < singularBeta", no non-singular extensions (e.g., castling) can
be performed!
Three tests were submitted, for three of Stockfish's four non-singular
extensions. I excluded the shuffle extension, because historically there
have been concerns about the fragility of its conditions, and I did not
want to risk causing any serious search problems.
- Modifying the passed pawn extension appeared roughly neutral at STC. At
best, it appeared to be an improvement of less than 1 Elo.
- Modifying check extension performed very poorly at STC
- Modifying castling extension (this patch) produced a long "yellow" run
at STC (insufficient to pass, but positive score) and a strong LTC.
In simple terms, prior to this patch castling extension was occasionally
not applied during search--on castling moves. The effect of this patch is
to perform castling extension on more castling moves. It does so without
adding any code complexity, simply by replacing an "else if" with "if" and
reordering some existing code.
STC:
LLR: -2.96 (-2.94,2.94) [0.00,4.00]
Total: 108114 W: 23877 L: 23615 D: 60622
http://tests.stockfishchess.org/tests/view/5d8d86bd0ebc590f3beb0c88
LTC:
LLR: 2.96 (-2.94,2.94) [0.00,4.00]
Total: 20862 W: 3517 L: 3298 D: 14047
http://tests.stockfishchess.org/tests/view/5d8d99cd0ebc590f3beb1899
Bench: 3728191
--------
Where do we go from here?
- It seems strange to me that check extension performed so poorly -- clearly
some of the singular extension conditions are also very important for check
extension. I am not an expert in search, and I do not have any intuition
about which of the eight conditions is/are the culprit. I will try a
succession of eight STC tests to identify the relevant conditions, then try
to replicate this PR for check extension.
- Recent tests interacting with the castle extension may deserve retesting.
I will shortly resubmit a few of my recent castling extension tweaks, rebased
on this PR/commit.
My deepest thanks to @noobpwnftw for the extraordinary CPU donation, and to
all our other fishtest volunteers, who made it possible for a speculative LTC
to pass in 70 minutes!
Closes https://github.com/official-stockfish/Stockfish/pull/2331
Maintain best move counter at the root and allow there only moves which has a counter
of zero for Late Move Reduction. For compensation only the first three moves are excluded
from Late Move Reduction per default instead the first four moves.
What we can further do:
- here we use a simple counting scheme but perhaps some aging to fade out early iterations
could be helpful
- use the best move counter also at inner nodes for LMR and/or pruning
STC:
LLR: 2.95 (-2.94,2.94) [0.50,4.50]
Total: 17414 W: 3984 L: 3733 D: 9697
http://tests.stockfishchess.org/tests/view/5d6234bb0ebc5939d09f2aa2
LTC:
LLR: 2.96 (-2.94,2.94) [0.00,3.50]
Total: 38058 W: 6448 L: 6166 D: 25444
http://tests.stockfishchess.org/tests/view/5d62681a0ebc5939d09f2f27
Closes https://github.com/official-stockfish/Stockfish/pull/2282
Bench: 3568210
@Vizvezdenec array suggested that alternate values may be better than current
master (see pull request #2270 ). I tuned some linear equations to more closely
represent his values and it passed. These futility values seem quite sensitive,
so perhaps additional Elo improvements can be found here.
STC
LLR: 2.95 (-2.94,2.94) [0.50,4.50]
Total: 12257 W: 2820 L: 2595 D: 6842
http://tests.stockfishchess.org/tests/view/5d5b2f360ebc5925cf1111ac
LTC
LLR: 2.96 (-2.94,2.94) [0.00,3.50]
Total: 20273 W: 3497 L: 3264 D: 13512
http://tests.stockfishchess.org/tests/view/5d5c0d250ebc5925cf111ac3
Closes https://github.com/official-stockfish/Stockfish/pull/2272
------------------------------------------
How to continue from there ?
a) we can try a simpler version for the futility margin, this would
be a simplification :
margin = 188 * (depth - improving)
b) on the other direction, we can try a complexification by trying
again to gain Elo with an complete array of futility values.
------------------------------------------
Bench: 4330402
Non functional simplification when we find the passed pawns in pawn.cpp
and some code clean up. It also better follows the pattern "flag the pawn"
and "score the pawn".
-------------------------
The idea behind the third condition for candidate passed pawn is a little
bit difficult to visualize. Just for the record, the idea is the following:
Consider White e5 d4 against black e6. d4 can (in some endgames) push
to d5 and lever e6. Thanks to this sacrifice, or after d5xe6, we consider
e5 as "passed".
However:
- if White e5/d4 against black e6/c6: d4 cannot safely push to d5 since d5 is double attacked;
- if White e5/d4 against black e6/d5: d4 cannot safely push to d5 since it is occupied.
This is exactly what the following expression does:
```
&& (shift<Up>(support) & ~(theirPawns | dblAttackThem)))
```
--------------------------
http://tests.stockfishchess.org/tests/view/5d3325bb0ebc5925cf0e6e91
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 124666 W: 27586 L: 27669 D: 69411
Closes https://github.com/official-stockfish/Stockfish/pull/2255
No functional change
