UE Threat Inputs for AB | Stockfish | Page 6

rocky vigil Oct 16, 2025, 2:23 AM

#

idk how much a net spsa is worth

frosty imp Oct 16, 2025, 2:23 AM

#

spsa should be the last resort imo

#

it's absolutely detrimental to net training development

rocky vigil Oct 16, 2025, 2:24 AM

#

True

naive comet Oct 16, 2025, 2:24 AM

#

Stc -15 is not bad

rocky vigil Oct 16, 2025, 2:24 AM

#

If no spsa we are essentially relying on scaling

frosty imp Oct 16, 2025, 2:24 AM

#

hopefully there's good scaling

twilit oriole Oct 16, 2025, 2:26 AM

#

rocky vigil idk how much a net spsa is worth

8 elo

#

was measured

rocky vigil Oct 16, 2025, 2:26 AM

#

Issue is master training run pre-spsa also is like -5 elo

twilit oriole Oct 16, 2025, 2:26 AM

#

well. just test vs that and if it beats it at LTC then spsa

#

its not good enough rn anyways. needs work

frosty imp Oct 16, 2025, 2:28 AM

#

well we need factorizer anyways

twilit oriole Oct 16, 2025, 2:28 AM

#

fix the factoriser, scale the L1 on the training side. that 30 elo is too low

rocky vigil Oct 16, 2025, 2:28 AM

#

Yeah let’s see how 1280 does w/o factorizer

#

Maybe it was actually worth a ton of elo

frosty imp Oct 16, 2025, 2:28 AM

#

why not add the factorizer

rocky vigil Oct 16, 2025, 2:29 AM

#

And I am a fool for not figuring out how to add it

frosty imp Oct 16, 2025, 2:30 AM

#

what you do is to add an extra 768 inputs at the end of the loaded indices. those are psq features that are active regardless of king position

rocky vigil Oct 16, 2025, 2:30 AM

#

Btw viren regarding like training longer

#

Since stages 4/5 don’t appear to help much

#

Do we just try like 1200SB for stages 1, 2, 3?

twilit oriole Oct 16, 2025, 2:31 AM

#

ig

#

factoriser has benefits even with unlimited data and should be as simple as adding extra features?

rocky vigil Oct 16, 2025, 2:32 AM

#

frosty imp what you do is to add an extra 768 inputs at the end of the loaded indices. thos...

Lemme check how halfkav2hm implements it

#

Yeah

twilit oriole Oct 16, 2025, 2:33 AM

#

only thing is to make sure it doesnt clip when it gets combined in main net weights

frosty imp Oct 16, 2025, 2:33 AM

#

rocky vigil Lemme check how halfkav2hm implements it

also modify get_feature_factors in full threats

rocky vigil Oct 16, 2025, 2:33 AM

#

I thought nnue-pytorch handled the coalescing

frosty imp Oct 16, 2025, 2:34 AM

#

well yes but you need to define the mapping

twilit oriole Oct 16, 2025, 2:34 AM

#

yeah idk how it does things. but in bullet i would need to half the scale

frosty imp Oct 16, 2025, 2:34 AM

#

twilit oriole yeah idk how it does things. but in bullet i would need to half the scale

nnue-pytorch doesn't clip ft weights iirc

rocky vigil Oct 16, 2025, 2:34 AM

#

Oh ok

frosty imp Oct 16, 2025, 2:35 AM

#

frosty imp also modify get_feature_factors in full threats

basically
return [idx] when threats index (no virtual feature)
return [idx, virtual idx] when pst index

#

could use a better system which would also make training faster

rocky vigil Oct 16, 2025, 2:38 AM

#

ouch

#

maybe i was too optimistic in how much things were worth

#

or like i thought we could get 80% speed of master

#

when it turns out it's closer to 70%

#

that 10% difference is indeed 20 elo

prime mica Oct 16, 2025, 2:45 AM

#

☹️

rocky vigil Oct 16, 2025, 2:54 AM

#

@frosty imp this look good?

#

on the data loader side

frosty imp Oct 16, 2025, 2:58 AM

#

rocky vigil <@453859636890828802> this look good?

are 533-547 pasted from halfkav2hm? if so then lgtm

rocky vigil Oct 16, 2025, 2:59 AM

#

yeah

#

lmao

#

decided that indexing the features probably costs 0 speed

#

in comparison to actually training

#

bruh you were right about not being able to add multiple features

rocky vigil Oct 16, 2025, 3:02 AM

#

frosty imp basically return [idx] when threats index (no virtual feature) return [idx, virt...

like this?

frosty imp Oct 16, 2025, 3:23 AM

#

Lgtm

rocky vigil Oct 16, 2025, 3:37 AM

#

is there anything else

#

to change

stray reef Oct 16, 2025, 4:10 AM

#

I'm a bit lost on where the test vs master is, and what has been tested since then

if it's -15 plus 5 elo from smallnet plus some other small speedups plus training improvements that's great honestly

rocky vigil Oct 16, 2025, 4:11 AM

#

-20 + 5 elo from smallnet + some other small speedups + training

#

actually on vondele's machines it's like -25 but we don't talk about that

#

https://tests.stockfishchess.org/tests/view/68eef18c28e6d77fcff9fe6e (test vs master)

#

https://tests.stockfishchess.org/tests/live_elo/68f007aa28e6d77fcffa0062 (smallnet)

stray reef Oct 16, 2025, 4:14 AM

#

big error bars

#

SF needs verbatim nets

#

I think it's essential for this

rocky vigil Oct 16, 2025, 4:15 AM

#

rocky vigil like this?

btw what on earth is self.get_factor_base_feature("A")

#

this seems wrong

#

when used for threats

#

ah nvm

#

i read the def

#

it seems correct

#

extremely overengineered though

#

pushed to branch

#

@frosty imp can you give it a try (the factorized features)

stray reef Oct 16, 2025, 4:31 AM

#

https://www.sp-cc.de/ Unfortunately here the TI version is 0 +- 4 elo relative to non-TI

#

could have been better, but no regression at least, still a partial success

naive comet Oct 16, 2025, 4:41 AM

#

stray reef <https://www.sp-cc.de/> Unfortunately here the TI version is 0 +- 4 elo relative...

me when reading this:

https://youtu.be/Qq_q6tp3vI0

violet badger Oct 16, 2025, 4:49 AM

#

so stage 5 finished as well, maybe another 1 Elo or so..

stray reef Oct 16, 2025, 4:50 AM

#

bring back the 13 stage training for threat inputs Kappa

violet badger Oct 16, 2025, 4:52 AM

#

That's 8 more Elo it seems.. 😉

#

currently training are l1=128 and l1=1280. The former can be used if there would be a need for a threats smallnet (idk), and the latter might give a hint on larger sizes, even though it is a small increment.

#

now, at scale this might look already interesting..

--------------------------------------------------
Results of master vs patch (60+0.6, 288t, 16000MB, UHO_Lichess_4852_v1.epd):
Elo: -5.79 +/- 18.13, nElo: -16.23 +/- 50.76
LOS: 26.54 %, DrawRatio: 74.44 %, PairsRatio: 0.77
Games: 180, Wins: 45, Losses: 48, Draws: 87, Points: 88.5 (49.17 %)
Ptnml(0-2): [0, 13, 67, 10, 0], WL/DD Ratio: 1.09
--------------------------------------------------

rocky vigil Oct 16, 2025, 4:56 AM

#

hmmm

violet badger Oct 16, 2025, 4:56 AM

#

just a teaser I think ...

prime mica Oct 16, 2025, 4:56 AM

#

what kind of system has so many threads wtf

violet badger Oct 16, 2025, 4:56 AM

#

fitbit

prime mica Oct 16, 2025, 4:57 AM

#

right...

rocky vigil Oct 16, 2025, 4:58 AM

#

stray reef <https://www.sp-cc.de/> Unfortunately here the TI version is 0 +- 4 elo relative...

ouch

violet badger Oct 16, 2025, 4:58 AM

#

that's at c2396284 ... so shawn's nn-598188c9a702.nnue branch

rocky vigil Oct 16, 2025, 5:01 AM

#

i mean the stage 5 is only like 2 elo (maybe)

violet badger Oct 16, 2025, 5:03 AM

#

plus small net another 5..

rocky vigil Oct 16, 2025, 5:03 AM

#

ah but does it scale

#

i guess it increases nps by a few %

frosty imp Oct 16, 2025, 5:18 AM

#

rocky vigil <@453859636890828802> can you give it a try (the factorized features)

hmm I won't have a gpu computer around soon

naive comet Oct 16, 2025, 5:21 AM

#

https://github.com/xu-shawn/Stockfish/pull/12 @frosty imp I suggest you test this on your hardware first before merging cuz I am not super confident in this one

#

maybe after this one we can reprofile

frosty imp Oct 16, 2025, 5:22 AM

#

maybe put it on fishtest?

#

incremental threat was 0 on my speedtest and +10 sprt

rocky vigil Oct 16, 2025, 5:23 AM

#

frosty imp incremental threat was 0 on my speedtest and +10 sprt

what be this :skull;

naive comet Oct 16, 2025, 5:24 AM

#

ok

frosty imp Oct 16, 2025, 5:24 AM

#

rocky vigil what be this :skull;

idk what went wrong

naive comet Oct 16, 2025, 5:25 AM

#

I submitted from phone xd

rocky vigil Oct 16, 2025, 5:26 AM

#

xd

#

i can also give it a test in my

#

or hangon

#

i have -3% but uh yeah

#

big noise

#

ig

#

let's just see how fishtest works

naive comet Oct 16, 2025, 5:35 AM

#

it's over 😔

rocky vigil Oct 16, 2025, 5:35 AM

#

💀

naive comet Oct 16, 2025, 1:27 PM

#

guys i'm not getting something

#

https://github.com/cj5716/stockfish/tree/threat_inputs_4 I have this

#

when we add or remove a piece from a square, we have to recompute the attacks through / blocked by it

#

the idea here is, on captures, we remove from a square then add back to that same square

#

so we prevent any recomputation and also reduce the number of extra updates to do

#

but,

#

this does more updates than the current best TI branch

#

???

plain flower Oct 16, 2025, 1:37 PM

#

naive comet the idea here is, on captures, we remove from a square then add back to that sam...

i call this operation a piece mutation, because this doesn't affect sliders

naive comet Oct 16, 2025, 1:38 PM

#

yeah exactly so taking into account piece mutations makes us have more updates somehow xd

plain flower Oct 16, 2025, 1:38 PM

#

so i have add-piece, remove-piece, mutate-piece

#

not sure, that doesn't make much sense

regal steeple Oct 16, 2025, 1:43 PM

#

naive comet yeah exactly so taking into account piece mutations makes us have more updates s...

I think the idea works, I had that idea as well and only noticed you already had that idea and implemented it before me too late, https://github.com/rn5f107s2/Stockfish/compare/40e85bebee329ac27018bc0ca80e247df80235dd...rn5f107s2:Stockfish:7c47493cd258aa3ff18d092a2ec01e5418eb0cc2 this is my branch, I get an average of
6.74802 updates for your branch
6.11372 for threat_inputs and
5.88847 for mine,
so Im pretty sure it works in theory

#

I started a test https://tests.stockfishchess.org/tests/live_elo/68f091a228e6d77fcffa0128 but stopped it after I saw you already had this idea

naive comet Oct 16, 2025, 1:45 PM

#

hmm so the issue is I implemented it poorly

#

I'd suggest you continue your test

regal steeple Oct 16, 2025, 1:48 PM

#

Could someone benchmark this https://tests.stockfishchess.org/tests/live_elo/68f0cc5428e6d77fcffa0196 btw? Im not sure how trustworthy my hardware is, I saw a lot of discrepancy in nps when running the speedup test

naive comet Oct 16, 2025, 1:49 PM

#

that is actually hilarious

#

actually you could use attackers_to() function for that cant you

#

oh but you reduce computation

#

nvm ignore me

#

OHHHH wait I think I know why yours is better

#

you remove before swap

#

I swap before remove

regal steeple Oct 16, 2025, 1:53 PM

#

Do you want to run yours then again? Im fine with stopping mine

naive comet Oct 16, 2025, 1:54 PM

#

I think you should keep yours

naive comet Oct 16, 2025, 2:00 PM

#

regal steeple I think the idea works, I had that idea as well and only noticed you already had...

I got 6.51335 for threat_inputs_4
and 6.79114 on threat_inputs

#

this is on standard bench

#

yours: 6.51507

#

I mean mine is microscopically less cuz I implemented it for castling too

#

next step is promotions I guess

#

I tried that but for some reason it changes my bench

#

I'll look into it tmr

violet badger Oct 16, 2025, 3:22 PM

#

so, that's a 128 L1 smallnet https://gitlab.com/cscs-ci/ci-testing/webhook-ci/mirrors/5137461961076608/2926829081096545/-/jobs/11734905338/artifacts/browse/step_671050ccc813/

#

Not sure it is useful, but better have it available.

frosty imp Oct 16, 2025, 3:27 PM

#

could run a simplification against the smallnet branch

violet badger Oct 16, 2025, 3:29 PM

#

yeah.

#

did we already run a stage 5 net test, with all improvements so far merged?

#

(e.g. fixed nodes test against master)

rocky vigil Oct 16, 2025, 3:42 PM

#

Lemme pr smallnet to Shawn’s branch

#

where's the copy button

#

@frosty imp https://github.com/xu-shawn/Stockfish/pull/13

violet badger Oct 16, 2025, 4:09 PM

#

this one is also bound to pass.. https://tests.stockfishchess.org/tests/view/68f0cc5428e6d77fcffa0196

rocky vigil Oct 16, 2025, 4:18 PM

#

naive comet I got 6.51335 for `threat_inputs_4` and 6.79114 on `threat_inputs`

this reduction is still good even if it adds bit of extra overhead because it increases the chances that 1280 is better than 1024

rocky vigil Oct 16, 2025, 6:14 PM

#

frosty imp could run a simplification against the smallnet branch

you can try this if you get it to work ig

frosty imp Oct 16, 2025, 7:30 PM

#

violet badger did we already run a stage 5 net test, with all improvements so far merged?

stage 5 not run yet since stage 4 test is ongoing

#

rn5 speedup not merged yet

violet badger Oct 16, 2025, 7:31 PM

#

ah, you mean on fishtest.

#

sure. We have fairly good estimates of the stages nevertheless.

#

let me paste them

frosty imp Oct 16, 2025, 7:31 PM

#

oh cool

violet badger Oct 16, 2025, 7:32 PM

#

5: Elo: -27.08 +/- 1.82, nElo: -50.97 +/- 3.40

#

4: Elo: -25.29 +/- 1.82, nElo: -47.45 +/- 3.40

#

3: Elo: -30.98 +/- 1.82, nElo: -58.16 +/- 3.40

#

2: Elo: -33.88 +/- 1.84, nElo: -63.06 +/- 3.40

#

1: Elo: -74.37 +/- 1.83, nElo: -142.55 +/- 3.40

#

so that would suggest 4 is right now the strongest.. but within error of that test

#

testing is end of the pipeline btw https://gitlab.com/cscs-ci/ci-testing/webhook-ci/mirrors/5137461961076608/2926829081096545/-/jobs/11736875984

frosty imp Oct 16, 2025, 7:36 PM

#

I see

frosty imp Oct 16, 2025, 7:38 PM

#

rocky vigil <@453859636890828802> can you give it a try (the factorized features)

segfault

rocky vigil Oct 16, 2025, 7:38 PM

#

Bruh

#

Knew smth like this would happen

#

Yeah idt that test includes smallnet either

#

Combined the two are probably worth 10 elo or so

rocky vigil Oct 16, 2025, 7:40 PM

#

frosty imp segfault

Is it a memory error?

frosty imp Oct 16, 2025, 7:41 PM

#

not sure

rocky vigil Oct 16, 2025, 7:42 PM

#

I’ll be back in ~20 min or so

frosty imp Oct 16, 2025, 7:44 PM

#

will run a threatnet test after rn5 passes

rocky vigil Oct 16, 2025, 7:44 PM

#

Yeah fair

#

With stage 4?

#

Or 3

frosty imp Oct 16, 2025, 7:45 PM

#

I mean smallnet

rocky vigil Oct 16, 2025, 7:45 PM

#

Oh I thought you meant stc estimation vs master

frosty imp Oct 16, 2025, 7:45 PM

#

But progress check would be good as well

rocky vigil Oct 16, 2025, 7:45 PM

#

Which is probably close to -10 at fishtest rn

frosty imp Oct 16, 2025, 7:45 PM

#

Yeah

rocky vigil Oct 16, 2025, 7:45 PM

#

Getting closer to -5

#

Which is what we can get w/o spsa

#

Just need factorizer to work…

rocky vigil Oct 16, 2025, 7:55 PM

#

frosty imp segfault

ok i can start code staring rn

#

do you know which code the error comes from?

frosty imp Oct 16, 2025, 8:05 PM

#

Not sure

#

Happens during sanity checking

rocky vigil Oct 16, 2025, 8:06 PM

#

where is sanity checking?

#

like what does it check

#

ah it happens at the beginning of traning

#

sanity checking dataloader

#

oh shoot @frosty imp i might be stupid

#

forgot to expose the factorized features

#

ok that should be fixed

#

didn't find any other errors during code staring

frosty imp Oct 16, 2025, 8:36 PM

#

rocky vigil didn't find any other errors during code staring

/workspace/nnue-pytorch/training_data_loader.cpp:548:2: error: expected ‘;’ after struct definition
  548 | }
      |  ^
      |  ;

#

also needs to expose on line 1269

rocky vigil Oct 16, 2025, 8:40 PM

#

Bruh

#

I just left my laptop

#

Oop

rocky vigil Oct 16, 2025, 8:41 PM

#

frosty imp ```c++ /workspace/nnue-pytorch/training_data_loader.cpp:548:2: error: expected ‘...

Can you test make changes locally for now and see if it works?

#

I won’t be back for a few hours; misread time of event

frosty imp Oct 16, 2025, 8:42 PM

#

i'll make a PR. works now as far as I can test

#

seems that my gpu doesn't have enough memory to run the training

candid ivy Oct 16, 2025, 8:44 PM

#

not sure if the preloading goes to the gpu or cpu memory, you can try lowerng that

violet badger Oct 16, 2025, 8:44 PM

#

or for testing just reduce l1 ...

candid ivy Oct 16, 2025, 8:44 PM

#

or that is probably the easiest 😄

frosty imp Oct 16, 2025, 8:45 PM

#

ah right

#

smallnet trains fine

violet badger Oct 16, 2025, 8:47 PM

#

for me l1=1024 seems to be need about 6GB GPU mem

frosty imp Oct 16, 2025, 8:50 PM

#

https://tests.stockfishchess.org/tests/view/68f0cc5428e6d77fcffa0196

rocky vigil Oct 16, 2025, 8:50 PM

#

gg

#

Should stack with smallnet

#

ig wait for @regal steeple pr

frosty imp Oct 16, 2025, 8:52 PM

#

ah I merged it now

rocky vigil Oct 16, 2025, 8:53 PM

#

That also works

prime mica Oct 16, 2025, 8:55 PM

#

frosty imp https://tests.stockfishchess.org/tests/view/68f0cc5428e6d77fcffa0196

what is this speedup test script you have that gives a probability of significance

rocky vigil Oct 16, 2025, 8:56 PM

#

frosty imp i'll make a PR. works now as far as I can test

Merged now

#

So I guess for the new training run

#

Can we try 1200 SB for stages 1, 2, 3?

#

Also

#

Or will it take too long

violet badger Oct 16, 2025, 8:58 PM

#

prime mica what is this speedup test script you have that gives a probability of significan...

https://github.com/hazzl/pyshbench or patched local version of this.

frosty imp Oct 16, 2025, 8:58 PM

#

https://tests.stockfishchess.org/tests/view/68f15c8128e6d77fcffa030f

prime mica Oct 16, 2025, 8:58 PM

#

thx!

lofty cedar Oct 16, 2025, 11:23 PM

#

Do we start LTC now?

#

Is it finished?

frosty imp Oct 16, 2025, 11:24 PM

#

well we could, but I believe we still have factorizers left to try

rocky vigil Oct 16, 2025, 11:25 PM

#

yeah

#

factorizers + longer stage 1, 2, 3

#

since the threat inputs are sparser

#

and no good factorization scheme

lofty cedar Oct 16, 2025, 11:26 PM

#

Anyway... threat input could benefit a lot from fast incremental threat calculation.

#

Which was implemented in the clockwork HCE engine.

#

If someone wanna take a look.

rocky vigil Oct 16, 2025, 11:26 PM

#

lofty cedar Anyway... threat input could benefit a lot from fast incremental threat calculat...

we have this, reasonably fast, already

frosty imp Oct 16, 2025, 11:27 PM

#

adopting clockwork's scheme is way too much change

rocky vigil Oct 16, 2025, 11:27 PM

#

though of course more speedups are welcome

lofty cedar Oct 16, 2025, 11:27 PM

#

Yeah...

frosty imp Oct 16, 2025, 11:27 PM

#

not practical short to medium term

twilit oriole Oct 16, 2025, 11:27 PM

#

https://tests.stockfishchess.org/tests/view/68f15c8128e6d77fcffa030f this has all the improvements right

rocky vigil Oct 16, 2025, 11:27 PM

#

like if we get another rn5 level speedup we just win

rocky vigil Oct 16, 2025, 11:27 PM

#

twilit oriole https://tests.stockfishchess.org/tests/view/68f15c8128e6d77fcffa030f this has al...

yeah, so far

frosty imp Oct 16, 2025, 11:27 PM

#

twilit oriole https://tests.stockfishchess.org/tests/view/68f15c8128e6d77fcffa030f this has al...

right

#

stage 5 net untested

#

so not in there

rocky vigil Oct 16, 2025, 11:28 PM

#

vondele says stage 5 is basically neutral (if not slightly worse) with stage 4

#

so what's next is factorizer

#

maybe 50% longer stages 1, 2, 3

#

idk

twilit oriole Oct 16, 2025, 11:28 PM

#

It's a fixed nodes test right? not reliable exactly

rocky vigil Oct 16, 2025, 11:28 PM

#

twilit oriole It's a fixed nodes test right? not reliable exactly

this is stc

twilit oriole Oct 16, 2025, 11:29 PM

#

The stage 5 net was tested STC?

frosty imp Oct 16, 2025, 11:29 PM

#

maybe better scaling from higher wdl

frosty imp Oct 16, 2025, 11:29 PM

#

twilit oriole The stage 5 net was tested STC?

not on fishtest

rocky vigil Oct 16, 2025, 11:29 PM

#

vondele did 40k games stc locally

frosty imp Oct 16, 2025, 11:29 PM

#

it's in the ci pipeline that the test was made

twilit oriole Oct 16, 2025, 11:29 PM

#

I see

frosty imp Oct 16, 2025, 11:29 PM

#

#1336647760388034610 message

rocky vigil Oct 16, 2025, 11:30 PM

#

rocky vigil + maybe 50% longer stages 1, 2, 3

@twilit oriole are you expecting this to be worth the time

#

so move from 800 SB to 1200

frosty imp Oct 16, 2025, 11:30 PM

#

is there a factorizer net in the training pipeline?

rocky vigil Oct 16, 2025, 11:30 PM

#

idt vondele has started one yet

#

lemme check

twilit oriole Oct 16, 2025, 11:31 PM

#

Oh u got factoriser working?

frosty imp Oct 16, 2025, 11:31 PM

#

it's working now

rocky vigil Oct 16, 2025, 11:31 PM

#

yeah

rocky vigil Oct 16, 2025, 11:31 PM

#

frosty imp it's working now

did you check like when the loader runs it indeed loads the correct factorized psq

#

i mean i copy pasted this from halfka code

#

so it probably just works

frosty imp Oct 16, 2025, 11:32 PM

#

no but let me check right now

rocky vigil Oct 16, 2025, 11:32 PM

#

yeah before we start real run

#

nice to do

frosty imp Oct 16, 2025, 11:32 PM

#

📎 message.txt

rocky vigil Oct 16, 2025, 11:33 PM

#

eh

#

did u load correct feature set?

frosty imp Oct 16, 2025, 11:34 PM

#

oops

#

📎 message.txt

rocky vigil Oct 16, 2025, 11:35 PM

#

well i can confirm popcount for sample 0 matches

#

probably some more effort to verify the psq

frosty imp Oct 16, 2025, 11:35 PM

#

cool

#

well that's probably more training time

twilit oriole Oct 16, 2025, 11:36 PM

#

https://tests.stockfishchess.org/tests/view/68f0cc5428e6d77fcffa0196 does plentychess have this trick

frosty imp Oct 16, 2025, 11:36 PM

#

@stray reef

rocky vigil Oct 16, 2025, 11:37 PM

#

oh yea

#

should be good as well

frosty imp Oct 16, 2025, 11:39 PM

#

threat smallnet not great btw https://tests.stockfishchess.org/tests/view/68f1719f28e6d77fcffa0353

rocky vigil Oct 16, 2025, 11:44 PM

#

💀

rocky vigil Oct 16, 2025, 11:45 PM

#

frosty imp

gimme a bit, i have setup a framework in desmos that should let me verify each position by hand relatively quickl

lofty cedar Oct 16, 2025, 11:45 PM

#

I guess let's try LTC then. Perhaps a progression test capstone.

rocky vigil Oct 16, 2025, 11:46 PM

#

sample 0 correct

#

sample 1 correct

#

sample 2 correct

#

i think this should be pretty good yeah

rocky vigil Oct 16, 2025, 11:50 PM

#

frosty imp cool

there an easy way to check the python function

#

for the coalescer

frosty imp Oct 16, 2025, 11:51 PM

#

if only the serializer is deterministic 💀

rocky vigil Oct 16, 2025, 11:51 PM

#

like i'm pretty sure get_feature_factors works

#

is there any way for you to isolate it individually and call it

#

like as a standalone python function

#

it shouldn't be that hard?

#

to make

frosty imp Oct 16, 2025, 11:52 PM

#

well you have the get_coalesced_ft

rocky vigil Oct 16, 2025, 11:52 PM

#

no i just wanna be sure get_feature_factors works

frosty imp Oct 16, 2025, 11:52 PM

#

which merges virtual weights

rocky vigil Oct 16, 2025, 11:52 PM

#

i think

#

i gonna assume the actual coalescer work

rocky vigil Oct 16, 2025, 11:53 PM

#

rocky vigil no i just wanna be sure get_feature_factors works

this could be an individual function no?

frosty imp Oct 16, 2025, 11:53 PM

#

i don't think there is

#

yeah

rocky vigil Oct 16, 2025, 11:53 PM

#

replace the constants

frosty imp Oct 16, 2025, 11:53 PM

#

well you already have a featureset object

#

in the model

#

so just call feature.get_feature_factors

rocky vigil Oct 16, 2025, 11:54 PM

#

hangon lemme just quickly test it in an isolated

#

python file

#

ok yeah it works on sample 0

#

i think that's good lol

frosty imp Oct 17, 2025, 12:04 AM

#

@violet badger any chance to get a factorized run in the near future?

rocky vigil Oct 17, 2025, 12:05 AM

#

twilit oriole Oh u got factoriser working?

yep, did some final sanity checks just now

#

so am pretty confident it works

rocky vigil Oct 17, 2025, 12:21 AM

#

i guess the question reduces to is factorizer woth 5 elo?

#

i would hope so...

frosty imp Oct 17, 2025, 12:22 AM

#

it's worth a lot in #engines-dev engines

#

not sure about stockfish

rocky vigil Oct 17, 2025, 12:22 AM

#

viren claims factorizer still helps even with unlimited data

#

so we'll see

twilit oriole Oct 17, 2025, 12:22 AM

#

U have to remember the threats act as a pseudo factoriser it's not so simple

frosty imp Oct 17, 2025, 12:23 AM

#

true

rocky vigil Oct 17, 2025, 12:23 AM

#

ah right

#

ngl i was definitely too optimistic about the gains

#

the actual results are a lot more close

#

speaking of scaling we should have 1280 in a couple days

#

or like, stage 3 of 1280 in like half a day

sharp sail Oct 17, 2025, 1:10 AM

#

frosty imp adopting clockwork's scheme is way too much change

true.
we should port SF NNUE to clockwork

rocky vigil Oct 17, 2025, 1:35 AM

#

@stray reef do you think using your lookup scheme to index threats would be measurably faster?

naive comet Oct 17, 2025, 2:32 AM

#

I have a smol idea I will try later

naive comet Oct 17, 2025, 3:02 AM

#

also rn5 that speedup is :xdd:

violet badger Oct 17, 2025, 4:04 AM

#

frosty imp <@713871252246495262> any chance to get a factorized run in the near future?

sure, if it is working. Can start later this weekend, would need to sha to insert in the threats recipe. I'll also see if I can slightly improve the current recipe by varying the lr/gamma of the current recipe.

frosty imp Oct 17, 2025, 4:05 AM

#

latest commit here should be working https://github.com/sscg13/nnue-pytorch/tree/threat-inputs

violet badger Oct 17, 2025, 4:10 AM

#

Pipeline should appear here https://gitlab.com/cscs-ci/ci-testing/webhook-ci/mirrors/5137461961076608/2926829081096545/-/pipelines/2104852394

naive comet Oct 17, 2025, 4:30 AM

#

maybe we can try horizontal mirroring for the threat inputs? not sure how worth it would be

frosty imp Oct 17, 2025, 4:57 AM

#

I believe we already do that

naive comet Oct 17, 2025, 5:18 AM

#

oh oopz

naive comet Oct 17, 2025, 5:37 AM

#

guys latest bench is wrong

frosty imp Oct 17, 2025, 5:38 AM

#

Oops

#

Pushed empty bench commit

amber fern Oct 17, 2025, 6:02 AM

#

You guys any where close to making Thread inputs stronger than main yet? How many elo off are we? 🙂

naive comet Oct 17, 2025, 6:06 AM

#

15ish I think?

stray reef Oct 17, 2025, 6:08 AM

#

twilit oriole https://tests.stockfishchess.org/tests/view/68f0cc5428e6d77fcffa0196 does plenty...

not yet, will implement today

stray reef Oct 17, 2025, 6:09 AM

#

rocky vigil <@415167192296849409> do you think using your lookup scheme to index threats wou...

i think it was maybe 3 STC elo

#

worth trying

naive comet Oct 17, 2025, 6:12 AM

#

also this https://github.com/official-stockfish/Stockfish/commit/40e85bebee329ac27018bc0ca80e247df80235dd

frosty imp Oct 17, 2025, 6:13 AM

#

amber fern You guys any where close to making Thread inputs stronger than main yet? How man...

https://tests.stockfishchess.org/tests/view/68f15c8128e6d77fcffa030f

naive comet Oct 17, 2025, 6:14 AM

#

zack

stray reef Oct 17, 2025, 6:14 AM

#

naive comet also this <https://github.com/official-stockfish/Stockfish/commit/40e85bebee329a...

https://furybench.com/test/3372/
not yet merged tho (2+0.02)

naive comet Oct 17, 2025, 6:14 AM

#

I see, nice

stray reef Oct 17, 2025, 6:15 AM

#

idk why i never thought about just using magics for that one part instead of keeping all threats, like in rn5s patch

naive comet Oct 17, 2025, 6:16 AM

#

I mean incremental is just intuitively faster lol

amber fern Oct 17, 2025, 7:29 AM

#

frosty imp https://tests.stockfishchess.org/tests/view/68f15c8128e6d77fcffa030f

-10 yoo!

#

So what does UE Thread Inputs actually mean, hows it better despite to my understanding being slower? Sorry I'm not a dev lol

stray reef Oct 17, 2025, 7:32 AM

#

The network explicitly receives all piece interactions (threats on enemies, defenses on own pieces) as input, so it has a lot more important information directly available

#

It is a lot stronger given equal nodes for this reason, but since keeping the threat information up to date is relatively expensive, it's slower overall

prime mica Oct 17, 2025, 7:36 AM

#

what's the distribution of the threat weights?

#

I'm wondering whether a light quantization on the threat weights would be less detrimental than quantizing the main NN

lofty cedar Oct 17, 2025, 7:38 AM

#

Well, quantization saves memory, but adds cost in interpretation.

#

So, not sure if it helps.

prime mica Oct 17, 2025, 7:38 AM

#

I'm pretty sure add/sub is grotesquely memory bandwidth bound

#

but idk I'll try it out at somep oint

lofty cedar Oct 17, 2025, 7:38 AM

#

Like... even if you imagined that you could losslessly compress the data... should you?

prime mica Oct 17, 2025, 7:39 AM

#

we'll find out

amber fern Oct 17, 2025, 7:39 AM

#

I tried running sf17.1 with this new nnue file, but I got this error in arena:

lofty cedar Oct 17, 2025, 7:39 AM

#

prime mica but idk I'll try it out at somep oint

I mean... I tried clustering the weight with k-nn. So, each weight would be represented by something else.

prime mica Oct 17, 2025, 7:40 AM

#

hm

lofty cedar Oct 17, 2025, 7:40 AM

#

So, instead of storing each weight, we store cluster weight and the cluster membership of each one.

prime mica Oct 17, 2025, 7:40 AM

#

the lookup table would be fairly expensive tho

lofty cedar Oct 17, 2025, 7:40 AM

#

But since my re-quantization code was botched, it got like -1000 elo at fixed node.

prime mica Oct 17, 2025, 7:41 AM

#

lol

#

beautiful

stray reef Oct 17, 2025, 7:41 AM

#

amber fern I tried running sf17.1 with this new nnue file, but I got this error in arena:

yeah you'll have to build the dev branch yourself currently. sf 17.1 does not implement the new architecture

lofty cedar Oct 17, 2025, 7:42 AM

#

prime mica the lookup table would be fairly expensive tho

I mean... instead of feature_length times the input, it's feature length times cluster plus one index per input.

lofty cedar Oct 17, 2025, 7:42 AM

#

prime mica the lookup table would be fairly expensive tho

What's expensive about it?

#

Though my quantization code is botched currently so I can't try.

amber fern Oct 17, 2025, 7:45 AM

#

Do I have to build sf17.1 using this repository? https://github.com/xu-shawn/Stockfish/tree/threat_inputs

GitHub

GitHub - xu-shawn/Stockfish at threat_inputs

A free and strong UCI chess engine. Contribute to xu-shawn/Stockfish development by creating an account on GitHub.

naive comet Oct 17, 2025, 7:52 AM

#

https://tests.stockfishchess.org/tests/view/68f1f57728e6d77fcffa0416

@frosty imp @regal steeple trying this rn

lofty cedar Oct 17, 2025, 7:54 AM

#

Oh, I see what was wrong.

#

The packing code contained the idea that didn't work.

amber fern Oct 17, 2025, 8:01 AM

#

Did I just get rate limited for checking out the website too much? LOL

candid ivy Oct 17, 2025, 8:02 AM

#

happens when there's a test with a big diff

lofty cedar Oct 17, 2025, 8:04 AM

#

Considering threat input adds significant complexity, what kind of elo do we need to merge?

#

5 elo maybe?

amber fern Oct 17, 2025, 8:12 AM

#

Yay, I successfully build the sf using the threads branch as suggested, but I was wondering why the default nnue file was different to the current 'best' that yall told me to download: nn-bf4519f857f4.nnue, you guys got me to download: nn-598188c9a702

#

ahh, it seems like the default hasnt been updated on the github yet, even though the new 598188 is +2 elo on fishtest: https://tests.stockfishchess.org/tests/live_elo/68efb98928e6d77fcff9ffd9

naive comet Oct 17, 2025, 9:34 AM

#

https://tests.stockfishchess.org/tests/view/68f20dab28e6d77fcffa045a

#

@frosty imp @rocky vigil if this one is real it's gg

stray reef Oct 17, 2025, 9:38 AM

#

nice idea

#

tracking threats_by_square is essentially free in your impl

lofty cedar Oct 17, 2025, 9:38 AM

#

Does Stockfish TI training data include positions with check?

stray reef Oct 17, 2025, 9:39 AM

#

afaik vondele just uses the (almost)master net training pipeline, so no

candid ivy Oct 17, 2025, 9:40 AM

#

lofty cedar Does Stockfish TI training data include positions with check?

doesn't matter anyway since the code hasn't been changed to call eval in check positions I guess?

stray reef Oct 17, 2025, 9:43 AM

#

https://furybench.com/test/3386/ rn5's speedup in plenty (needed to change back some stuff that already relied on threat information)
not 2+0.02 this time but 8+0.08

lofty cedar Oct 17, 2025, 9:48 AM

#

candid ivy doesn't matter anyway since the code hasn't been changed to call eval in check p...

Precisely what mattered. I was thinking of testing if I could remove that if threat input became good enough at detecting random checks.

candid ivy Oct 17, 2025, 9:52 AM

#

lel

candid ivy Oct 17, 2025, 9:53 AM

#

lofty cedar Precisely what mattered. I was thinking of testing if I could remove that if thr...

something for later imo

lofty cedar Oct 17, 2025, 10:14 AM

#

#

Doesn't look good on my machine... at least currently... at VVLTC.

candid ivy Oct 17, 2025, 10:15 AM

#

bruh just wait till we run this on fishtest

formal smelt Oct 17, 2025, 10:18 AM

#

lofty cedar

9 games sssposting is crazy

lofty cedar Oct 17, 2025, 10:21 AM

#

It's current. Maybe things will change.

stray reef Oct 17, 2025, 10:32 AM

#

no way, it will end at 247.9 elo

naive comet Oct 17, 2025, 10:35 AM

#

lofty cedar

it's not even an even number of games

lofty cedar Oct 17, 2025, 10:37 AM

#

Having some 100-games sanity test... not saying that they would be significant in the grand scheme of things.

candid ivy Oct 17, 2025, 10:39 AM

#

but rather useless

lofty cedar Oct 17, 2025, 10:40 AM

#

Want to see it playing, except I don't even know how well it plays because Stockfish plays are inscrutable to mere mortals like me.

twilit oriole Oct 17, 2025, 11:41 AM

#

prime mica I'm wondering whether a light quantization on the threat weights would be less d...

It is easier to quantise a threat net yes. Already we (monty) did this, with i8 weights in the FT

twilit oriole Oct 17, 2025, 11:43 AM

#

lofty cedar Precisely what mattered. I was thinking of testing if I could remove that if thr...

No. Already been tested. This is a waste of time

lofty cedar Oct 17, 2025, 11:44 AM

#

Oh... I see.

naive comet Oct 17, 2025, 11:44 AM

#

yeah I recall linrock trying that

twilit oriole Oct 17, 2025, 11:47 AM

#

lofty cedar So, not sure if it helps.

Can we not fill the thread with noise like this. Quantising the threat net further than the main net absolutely is promising

#

The way we did it is just use i8 FT weights but keep the calculation in i16. This just halves the mem bandwidth used to fetch FT weights and works very well

#

Was only -5 fixed nodes for us

stray reef Oct 17, 2025, 11:50 AM

#

how much faster?

twilit oriole Oct 17, 2025, 11:51 AM

#

https://tests.montychess.org/tests/view/68b5d37756f229dd4390d7a1 and this is with a relatively small value net which took only a fraction of total time

#

I guess it helps even if the weights are in L3 cache

#

The value L1 there is 3072 so it's pretty good indication it will work at least for SF

#

Rounding the weights when quantising them is crucial

stray reef Oct 17, 2025, 11:54 AM

#

okay cool

naive comet Oct 17, 2025, 12:03 PM

#

honestly memory bandwidth is not an issue with mmap is it

formal smelt Oct 17, 2025, 12:05 PM

#

well the linked monty test is with mmap

#

halving the number of bytes loaded is beneficial regardless

stray reef Oct 17, 2025, 12:10 PM

#

stray reef <https://furybench.com/test/3386/> rn5's speedup in plenty (needed to change bac...

passed with 3.26 +- 2.04 (95%)

rocky vigil Oct 17, 2025, 12:33 PM

#

prime mica what's the distribution of the threat weights?

I am pretty sure i8 works

#

At least a vast majority should work

#

Oh viren already said so

#

If we can get it to work and add sub is mem bandwidth bound as you said we should be able to shave off a significant amount of the 25%(?) runtime that it currently uses

#

btw shawn i think https://tests.stockfishchess.org/tests/view/68ef157828e6d77fcff9fead could also be merged

#

actually question

#

i8 quantizing threat weights means you should not do the x2 scaling

#

on loading the weights

#

@prime mica 81765782 weights losslessly quantizable to i8 out of 81772544

#

aka close to 99.99% of them

twilit oriole Oct 17, 2025, 12:54 PM

#

Yeah but what's getting clipped are the most important weights :p might need training scale change

rocky vigil Oct 17, 2025, 12:54 PM

#

true

#

but training scale change = re-spsa search

#

which is annoying

twilit oriole Oct 17, 2025, 12:55 PM

#

Not really, you can always scalar multiple the eval

rocky vigil Oct 17, 2025, 12:55 PM

#

that works?

rocky vigil Oct 17, 2025, 12:55 PM

#

rocky vigil i8 quantizing threat weights means you should not do the x2 scaling

there is also this

#

i guess the solution is to do (psq + 2 * threat)

twilit oriole Oct 17, 2025, 12:56 PM

#

We just i8 quantised the entire FT

rocky vigil Oct 17, 2025, 12:56 PM

#

ah interesting

#

since our accumulators are separate we could keep i16 for psq

#

if needed

rocky vigil Oct 17, 2025, 12:57 PM

#

rocky vigil <@418667403396775936> `81765782 weights losslessly quantizable to i8 out of 8177...

i.e. this is only out of the threat features

#

if i ran psq features probably would be nowhere near as good

twilit oriole Oct 17, 2025, 12:58 PM

#

There may be a problem because the multilayer is quantised so aggressively it makes it harder to quantise the FT. Because we have the inverse, can't do fast multilayer after quantising the FT to i8

rocky vigil Oct 17, 2025, 12:59 PM

#

i mean it is definitely worth a try to quantize threat only

#

how much of a bottleneck of memory bandwidth are we talking about?

#

in comparison to compute

twilit oriole Oct 17, 2025, 1:01 PM

#

Don't have comparable numbers because we don't have UE

rocky vigil Oct 17, 2025, 1:01 PM

#

would it be worth it to try and decompress 8 bit format live

twilit oriole Oct 17, 2025, 1:01 PM

#

No

rocky vigil Oct 17, 2025, 1:01 PM

#

ok

#

actually lemme check with the other net

#

this is bf4

#

it shouldn't make a difference

twilit oriole Oct 17, 2025, 1:03 PM

#

Well I would just test the inference speedup at least. I expect significant but single digit %

rocky vigil Oct 17, 2025, 1:04 PM

#

twilit oriole Was only -5 fixed nodes for us

this includes psq quantization right?

twilit oriole Oct 17, 2025, 1:04 PM

#

Yes but multilayer far less quantised

rocky vigil Oct 17, 2025, 1:04 PM

#

i guess we'll see later

#

btw is there a way to disable the net sha check on comp

#

it takes like 5 seconds

#

which adds up :p

regal steeple Oct 17, 2025, 1:08 PM

#

https://tests.stockfishchess.org/tests/view/68f23eac28e6d77fcffa04b3
I think this should be compatible with all pending patches, could someone approve?

rocky vigil Oct 17, 2025, 1:08 PM

#

81759318 weights losslessly quantizable to i8 out of 81772544 nn-598

#

viren could (approve)

#

i think

#

if he's heere

rocky vigil Oct 17, 2025, 1:16 PM

#

twilit oriole Well I would just test the inference speedup at least. I expect significant but ...

gonna give this a quick test

#

actually leb makes this annoying

#

gimme a second to write it in (little) endian

regal steeple Oct 17, 2025, 1:31 PM

#

regal steeple https://tests.stockfishchess.org/tests/view/68f23eac28e6d77fcffa04b3 I think thi...

I missed that updates can still be fused if the moved piece was not taken, submitted an improved version
https://tests.stockfishchess.org/tests/view/68f244ee28e6d77fcffa04bc

rocky vigil Oct 17, 2025, 1:39 PM

#

actually how do you get the vec_t to interpret it as i16

#

if it's i8 originally

#

to my understanding reinterpret_cast will just fill it with 2x the amount of i8 values

#

cvtepi8_epi16

#

ok

naive comet Oct 17, 2025, 1:47 PM

#

https://tests.stockfishchess.org/tests/view/68f20dab28e6d77fcffa045a stopped this, read comment for info

naive comet Oct 17, 2025, 1:48 PM

#

regal steeple I missed that updates can still be fused if the moved piece was not taken, submi...

bwoahohohohohoh

#

huge speedup

#

also a new one https://tests.stockfishchess.org/tests/view/68f2499a28e6d77fcffa04c6

rocky vigil Oct 17, 2025, 1:55 PM

#

twilit oriole Well I would just test the inference speedup at least. I expect significant but ...

how do you do this in monty? I'm losing like 5% with all the mm_cvtepi8_epi16s and stuff

#

like the i8 to i16 simd conversion

stray reef Oct 17, 2025, 1:57 PM

#

just convert to i16 at startup to save some instructions Kappa Kappa Kappa

rocky vigil Oct 17, 2025, 1:58 PM

#

actually from startpos it seems to be a few %

#

but like

#

i am def doing it wrong

#

bc the pv is cooked

naive comet Oct 17, 2025, 2:01 PM

#

https://tests.montychess.org/tests/view/68b5d37756f229dd4390d7a1

rocky vigil Oct 17, 2025, 2:02 PM

#

bahhh rust makes this so different

#

Nodes/second : 1029999 for current vs Nodes/second : 1021921 for i8 test

#

so um

#

i am not doing this correctly

#

anyways i leave it to simd experts to try this later

rocky vigil Oct 17, 2025, 2:08 PM

#

naive comet huge speedup

a lot of things are huge speedups if you accidentally confuse patch and master Kappa

#

wait cj does this actually work

#

ah because enemy only matters when the piece types are the same

#

nice find

#

btw it appears that "real compression" performs better on threat inputs

#

i.e. zipping master net gives 60 MB, and zipping l1=1024 threat net gives 60 MB as well

#

asdadasds l1=1280 too large for fishtest

#

btw 1280 is like, 15% slower

#

than 1024

#

from bench

#

lemme try speedtest

#

ok bench is just slow with 1280

rocky vigil Oct 17, 2025, 2:29 PM

#

rocky vigil lemme try speedtest

Nodes/second : 933288, ie 10% slower

rocky vigil Oct 17, 2025, 2:51 PM

#

...      Stockfish TI-1280 playing White: 145 - 76 - 279  [0.569] 500
...      Stockfish TI-1280 playing Black: 77 - 128 - 295  [0.449] 500
...      White vs Black: 273 - 153 - 574  [0.560] 1000
Elo difference: 6.3 +/- 14.0, LOS: 80.8 %, DrawRatio: 57.4 %
SPRT: llr 0 (0.0%), lbound -inf, ubound inf
1000 of 1000 games finished.``` stage 4 or 5 (or factorizer) need to bring serious improvement if we want this to be viable

frosty imp Oct 17, 2025, 2:51 PM

#

Fixed nodes?

rocky vigil Oct 17, 2025, 2:51 PM

#

yep

#

20k

#

ofc stc would be better

#

but it exceeds file size limit on fishtest

frosty imp Oct 17, 2025, 2:51 PM

#

how about 896 HL

rocky vigil Oct 17, 2025, 2:52 PM

#

let's wait for 1280 to finish

frosty imp Oct 17, 2025, 2:52 PM

#

surely if 1024->1280 is bad then 1024->896 is good

rocky vigil Oct 17, 2025, 2:52 PM

#

and then see

#

maybe stage 4 is where the breakthrough happens

#

larger net being slower to train and all that

rocky vigil Oct 17, 2025, 2:54 PM

#

rocky vigil `Nodes/second : 933288`, ie 10% slower

also https://tests.stockfishchess.org/tests/view/68f091a228e6d77fcffa0128 should help with this

#

if it adds overhead but decreases avg threat features updated

rocky vigil Oct 17, 2025, 2:57 PM

#

frosty imp how about 896 HL

btw shawn are you going to merge removing fusing

frosty imp Oct 17, 2025, 2:57 PM

#

merged

rocky vigil Oct 17, 2025, 2:57 PM

#

oh nice

#

yeah i guess yoshie was right that fusing is not a gain

rocky vigil Oct 17, 2025, 3:01 PM

#

frosty imp merged

(fix bench)

frosty imp Oct 17, 2025, 3:01 PM

#

oh bruh

#

fixed

twilit oriole Oct 17, 2025, 3:17 PM

#

Those error bars are too large. A 10 Elo fixed nodes improvement is fine

#

https://furybench.com/test/3274/ reminder this is what a 25% HL increase looked like for plenty

rocky vigil Oct 17, 2025, 3:21 PM

#

twilit oriole Those error bars are too large. A 10 Elo fixed nodes improvement is fine

yeah bc i'm running this on my laptop only

#

not vondele's 288 core fitbit or whatever he uses to do the local tests

twilit oriole Oct 17, 2025, 3:21 PM

#

yes but your statement a "serious improvement" is needed is incorrect

rocky vigil Oct 17, 2025, 3:22 PM

#

strange

rocky vigil Oct 17, 2025, 3:22 PM

#

twilit oriole https://furybench.com/test/3274/ reminder this is what a 25% HL increase looked ...

also the ltc was neutral

#

i'm pretty sure

#

https://furybench.com/test/3273/ ye merged on scaling vibes

#

again let's wait for stage 4/5

twilit oriole Oct 17, 2025, 3:23 PM

#

It's a 6 +- 14 test. serious improvement to number of games is what is needed is my point lol there's no indication it is underperforming where it needs to be

rocky vigil Oct 17, 2025, 3:24 PM

#

do u have hardware free to test

#

i can set up a branch if that's what you want

twilit oriole Oct 17, 2025, 3:24 PM

#

sure

rocky vigil Oct 17, 2025, 3:24 PM

#

would've put it on fishtest to check stc but we hit the size limit

twilit oriole Oct 17, 2025, 3:25 PM

#

will it require me getting the net and all that crap. cos im doing through ssh takes time

rocky vigil Oct 17, 2025, 3:25 PM

#

rocky vigil would've put it on fishtest to check stc but we hit the size limit

.

#

no auto download

twilit oriole Oct 17, 2025, 3:25 PM

#

ah

#

well its fine if i can just wget it or smth

#

got 384 thread machine free in an hour, when will branch be done?

#

ig I can just do the TC tests also

rocky vigil Oct 17, 2025, 3:28 PM

#

https://gitlab.com/cscs-ci/ci-testing/webhook-ci/mirrors/5137461961076608/2926829081096545/-/jobs/11722693716/artifacts/download might work

#

for getting a zipped version

#

branch is up now

twilit oriole Oct 17, 2025, 3:29 PM

#

link

rocky vigil Oct 17, 2025, 3:29 PM

#

https://github.com/sscg13/Stockfish/tree/threat-1280

twilit oriole Oct 17, 2025, 3:29 PM

#

cool

rocky vigil Oct 17, 2025, 3:30 PM

#

not even exceeding size limit by much even

#

130 MB vs 128 MB limit I'm pretty sure

twilit oriole Oct 17, 2025, 3:31 PM

#

well changing the limit is just changing a number in the nginx config on fishtest server

#

its easy to do

rocky vigil Oct 17, 2025, 3:31 PM

#

lmao

#

if local tc looks promising prob can get it done on fishtest

twilit oriole Oct 17, 2025, 3:32 PM

#

can u make/link branch for 1024 net also

stray reef Oct 17, 2025, 3:32 PM

#

rocky vigil <https://furybench.com/test/3273/> ye merged on scaling vibes

passed against main but didn't wanna waste games against L1=512

rocky vigil Oct 17, 2025, 3:33 PM

#

twilit oriole can u make/link branch for 1024 net also

ah just get https://github.com/xu-shawn/Stockfish/tree/threat_inputs

#

wait

#

underscore indeed

rocky vigil Oct 17, 2025, 3:33 PM

#

stray reef passed against main but didn't wanna waste games against L1=512

fair lol

twilit oriole Oct 17, 2025, 3:34 PM

#

i do this test after my meeting ig lol

#

when will stage 4 complete

rocky vigil Oct 17, 2025, 3:35 PM

#

uh

#

half a day?

twilit oriole Oct 17, 2025, 3:35 PM

#

cool

rocky vigil Oct 17, 2025, 3:36 PM

#

rocky vigil btw it appears that "real compression" performs better on threat inputs

btw idea

#

like to reduce also i.e. lichess load

#

oh wait it's not compatible with mmap

#

maybe we do it for the releases only

#

factorizer is surprisingly not that much of a slowdown

#

so far at least

#

still holding in mid-80 its/sec

frosty imp Oct 17, 2025, 4:25 PM

#

rocky vigil 130 MB vs 128 MB limit I'm pretty sure

https://github.com/official-stockfish/fishtest/pull/2405

rocky vigil Oct 17, 2025, 4:27 PM

#

aha

#

that also works

#

might take a couple days

#

who knows

rocky vigil Oct 17, 2025, 5:08 PM

#

twilit oriole i do this test after my meeting ig lol

ah have you had a chance to run fixed nodes / tc

#

or are you going to wait for stage 4

twilit oriole Oct 17, 2025, 6:07 PM

#

Wait probably. Should ask @violet badger to increase the net size limit on fishtest server also

violet badger Oct 17, 2025, 6:07 PM

#

twilit oriole Wait probably. Should ask <@713871252246495262> to increase the net size limit o...

#1336647760388034610 message

twilit oriole Oct 17, 2025, 6:08 PM

#

Ah nice u already read lol

violet badger Oct 17, 2025, 6:08 PM

#

well not merged and not me.. but yes

#

doesn't adjust the memory estimates for the workers though so is not fully complete.

twilit oriole Oct 17, 2025, 6:10 PM

#

I think there's some nginx limit or smth to adjust also. Outside of PR

rocky vigil Oct 17, 2025, 6:10 PM

#

yeah ppigazzini is the fishtest maintainer right

violet badger Oct 17, 2025, 6:11 PM

#

yes, and indeed could need a bit more.

#

https://tests.stockfishchess.org/tests/live_elo/68f2499a28e6d77fcffa04c6 ... seems promising?

rocky vigil Oct 17, 2025, 6:14 PM

#

higher than what the listed speedup suggests

#

but indeed should be very good

violet badger Oct 17, 2025, 6:15 PM

#

so that's on top of what the 10k STC test tested, right..

rocky vigil Oct 17, 2025, 6:15 PM

#

yes

violet badger Oct 17, 2025, 6:16 PM

#

would imply near parity with master..

rocky vigil Oct 17, 2025, 6:17 PM

#

once this finishes i plan to do both stc / ltc 10k games

lofty cedar Oct 17, 2025, 6:23 PM

#

Hmm? Why not SPRT against master right away? After it finishes?

#

If it gains now, then it ends here.

rocky vigil Oct 17, 2025, 6:24 PM

#

hmm

#

not expecting pass @ stc yet

#

ltc would be different story

#

idk what others think but i only wanted to do sprt vs master once we iron everything out and finalize it

#

since that will include a LTC SMP

lofty cedar Oct 17, 2025, 6:25 PM

#

Maybe...

#

I mean, a change this drastic needs VVLTC gainer.

violet badger Oct 17, 2025, 6:27 PM

#

I agree with finalizing before doing more advanced tests.

#

I expect also some net improvements could be found.

rocky vigil Oct 17, 2025, 6:28 PM

#

it is expected* to scale with both time and threads (*known in plentychess, we'll see how it goes in stockfish relatively soon, it seems?)

#

yeah patience is good

lofty cedar Oct 17, 2025, 6:28 PM

#

Yes... there's a good chance we could gain now at LTC.

But we could finalize things first.

rocky vigil Oct 17, 2025, 6:29 PM

#

things are moving very fast actually

violet badger Oct 17, 2025, 6:29 PM

#

I also still see warnings like:

position.cpp:1060:18: warning: declaration of 'threatened' shadows a previous local [-Wshadow]
 1060 |         Bitboard threatened = ray & qAttacks & occupied;
      |                  ^~~~~~~~~~
position.cpp:1030:14: note: shadowed declaration is here
 1030 |     Bitboard threatened = attacks_bb(pc, s, occupied) & occupied;
      |              ^~~~~~~~~~
position.cpp: In instantiation of 'void Stockfish::Position::update_piece_threats(Stockfish::Piece, Stockfish::Square, Stockfish::DirtyThreats*) [with bool put_piece = true]':
position.h:346:35:   required from here
position.cpp:1060:18: warning: declaration of 'threatened' shadows a previous local [-Wshadow]
 1060 |         Bitboard threatened = ray & qAttacks & occupied;
      |                  ^~~~~~~~~~
position.cpp:1030:14: note: shadowed declaration is here
 1030 |     Bitboard threatened = attacks_bb(pc, s, occupied) & occupied;
      |              ^~~~~~~~~~
position.cpp: In instantiation of 'void Stockfish::Position::update_piece_threats(Stockfish::Piece, Stockfish::Square, Stockfish::DirtyThreats*) [with bool put_piece = false]':
position.h:353:36:   required from here
position.cpp:1060:18: warning: declaration of 'threatened' shadows a previous local [-Wshadow]
 1060 |         Bitboard threatened = ray & qAttacks & occupied;
      |                  ^~~~~~~~~~
position.cpp:1030:14: note: shadowed declaration is here
 1030 |     Bitboard threatened = attacks_bb(pc, s, occupied) & occupied;
      |              ^~~~~~~~~~

#

probably easy fixes

rocky vigil Oct 17, 2025, 6:29 PM

#

in comparison to the 6 month hiatus before

lofty cedar Oct 17, 2025, 6:29 PM

#

Because once it passes, there would be more things to do like search tune and so on that could lock in the decisions.

#

Though I'd agree we should finalize things first.

#

So we don't waste compute on VVLTC search tunes on incomplete net.

rocky vigil Oct 17, 2025, 6:31 PM

#

our goal is to pass this (preferably, with a comfortable gain) without having to resort to spsa

#

which looks a lot more doable with the new speedups

#

https://tests.stockfishchess.org/tests/live_elo/68f091a228e6d77fcffa0128 has been flying under the radar

#

but it might also help with increasing L1

#

according to data above this reduces the avg. number of features updated which is more improvement with higher L1

violet badger Oct 17, 2025, 6:33 PM

#

hard to track what is actually the most up-to-date branch 😉

rocky vigil Oct 17, 2025, 6:34 PM

#

i think shawn has been trying to keep updated as tests pass

lofty cedar Oct 17, 2025, 6:34 PM

#

Someone mentioned that like 99+% of the feature transformer weights losslessly compress to i8.

#

Though the rest could remain problematic.

rocky vigil Oct 17, 2025, 6:34 PM

#

either clamping it directly does not work, or I have done something wrong elsewhere in the inference

#

in any case I gave it a quick try and got ~~neutral in speedtest

#

though someone more knowledgeable with simd could of course do better I presume

lofty cedar Oct 17, 2025, 6:35 PM

#

anematode is probably the most knowledgeable we have at this.

rocky vigil Oct 17, 2025, 6:36 PM

#

it helps with memory bandwidth yes, but adds many extra cvtepi8_epi16 calls, unless I am missing something

#

and also an extra addition pass

#

because the x2 trick doesn't work anymore

lofty cedar Oct 17, 2025, 6:37 PM

#

But then... what about our ambition to deduplicate the net? Now, do we have to start over again?

rocky vigil Oct 17, 2025, 6:37 PM

#

rocky vigil it helps with memory bandwidth yes, but adds many extra cvtepi8_epi16 calls, unl...

maybe instead doing i8 * (i8 = 2) -> i16 multiplication instead of this works better

violet badger Oct 17, 2025, 6:38 PM

#

these things will get sorted eventually, it is more or less orthogonal to the threats arch.

lofty cedar Oct 17, 2025, 6:39 PM

#

I'm glad the net deduplication didn't contain a bunch of arch-specific hacks to get it working.

rocky vigil Oct 17, 2025, 6:39 PM

#

i think threats make it more possible

#

since you gain more speed ostensibly

#

and less quantization penalty

#

since the threat feature weights are much less in abs value

lofty cedar Oct 17, 2025, 6:40 PM

#

Oh, it already gained like 40 elo in fishtest condition.

#

But there were some problems that made it not viable to be merged right away.

rocky vigil Oct 17, 2025, 6:41 PM

#

oh i was referring to i8 quantizatio

#

sorry

lofty cedar Oct 17, 2025, 6:41 PM

#

Oh, I see.

rocky vigil Oct 17, 2025, 6:42 PM

#

lofty cedar Oh, it already gained like 40 elo in fishtest condition.

this is not really "40 elo improvement" (in the normal sense) but rather "tests can be done X% faster"

twilit oriole Oct 17, 2025, 6:43 PM

#

I'm tracking who has contributed, we are gonna end up with a PR with 20 coauthors or smth lol

rocky vigil Oct 17, 2025, 6:43 PM

#

i think it's 10 so far right?

twilit oriole Oct 17, 2025, 6:43 PM

#

Nah it's way more

rocky vigil Oct 17, 2025, 6:43 PM

#

really

twilit oriole Oct 17, 2025, 6:43 PM

#

There's a lot of ppl that contributed earlier who are no longer active here

#

But I remembered

rocky vigil Oct 17, 2025, 6:44 PM

#

I have u, jw, (ravenslofty - since yoshie's impl is inspired from yukari), yoshie, disservin, linrock, vondele, me, shawn, rn5, cj

#

were there others?

twilit oriole Oct 17, 2025, 6:44 PM

#

Yes

#

But anyways not critical rn

rocky vigil Oct 17, 2025, 6:44 PM

#

will look funny if it happens

#

20 ppl authored and (vondele/disservin) merged

rocky vigil Oct 17, 2025, 6:45 PM

#

twilit oriole But I remembered

your memory ig is better than mine lol, bc I definitely forgot over 6 months

lofty cedar Oct 17, 2025, 6:45 PM

#

Though... well... this is Stockfish. 20+ people on months of work for maybe like 10 elo gain.

rocky vigil Oct 17, 2025, 6:45 PM

#

ehhhh

#

most of the work happened in march

twilit oriole Oct 17, 2025, 6:46 PM

#

More like 2 months work

rocky vigil Oct 17, 2025, 6:46 PM

#

and in the last month

#

so that would be ~2 months

#

yeah

prime mica Oct 17, 2025, 6:46 PM

#

so hyped

violet badger Oct 17, 2025, 6:46 PM

#

and we're still working on it...

#

could be another two months for all I know 😉

prime mica Oct 17, 2025, 6:46 PM

#

lol

violet badger Oct 17, 2025, 6:47 PM

#

but it is fun..

#

and actually nice it is many people contributing.

lofty cedar Oct 17, 2025, 6:49 PM

#

And now that we have our training infrastructure back up, we could try new archs.

rocky vigil Oct 17, 2025, 6:50 PM

#

there are a lot of cool ideas, note that a full net still takes several days though...

lofty cedar Oct 17, 2025, 6:50 PM

#

I've calculated that a subnet made up of only pawn/minor piece/major piece/diagonal piece inputs all could be cached with like 80% hit rate if not more.

rocky vigil Oct 17, 2025, 6:50 PM

#

so right now what will happen is basically experimentation with L1 and training schedule

violet badger Oct 17, 2025, 6:50 PM

#

https://github.com/official-stockfish/nnue-pytorch/pull/352 needs debugging ..

rocky vigil Oct 17, 2025, 6:50 PM

#

not really big arch changes

violet badger Oct 17, 2025, 6:51 PM

#

would speedup testing/training 2x.

rocky vigil Oct 17, 2025, 6:51 PM

#

thought it required 2x the hardware :p

violet badger Oct 17, 2025, 6:51 PM

#

(I think the actual data loader is probably OK; but something else is not).

#

HW is there.

lofty cedar Oct 17, 2025, 6:55 PM

#

With 2048 cache entries, a subnet with only pawn or minor pieces could be cached with about 90% hit rate.

#

Well, not sure what can we make of it.

rocky vigil Oct 17, 2025, 6:55 PM

#

this probably belongs in a separate like thread

lofty cedar Oct 17, 2025, 6:55 PM

#

Yeah...

#

Though mentioning it because... well... we could now try...

rocky vigil Oct 17, 2025, 6:56 PM

#

let's wait, there are more important things for now

#

anyways if we merge this into master this thread will be abandoned and further discussion will just be in nnue-dev

lofty cedar Oct 17, 2025, 6:57 PM

#

Yeah...

rocky vigil Oct 17, 2025, 7:05 PM

#

anyways there's not much to do now while waiting, if someone wants to try a potential improvement would be to update the threats lazily as well, since we don't use them for anything other than the nnue

#

but this is a lot of effort

twilit oriole Oct 17, 2025, 7:06 PM

#

Oh damn they aren't even lazy updated kekw

#

Isn't that like a decent speed boost

rocky vigil Oct 17, 2025, 7:11 PM

#

should be

prime mica Oct 17, 2025, 7:28 PM

#

regarding the i8 quantization, it'll probably be pretty machine dependent whether it's faster or not

#

but ideally it can just be optional

#

while maintaining a consistent bench

rocky vigil Oct 17, 2025, 7:34 PM

#

prime mica while maintaining a consistent bench

this can only be the case if we force the i16 version to also abide by the i8 limits no?

#

which will incur some (hopefully minor) loss

prime mica Oct 17, 2025, 7:35 PM

#

right

#

would be a balancing act

#

although, if the exceeding elements are extremely rare, we can do a scalar cleanup for those rows

#

without much overhead

#

(I tried this with the main net but the exceeding elements are too common for it to work)

twilit oriole Oct 17, 2025, 7:40 PM

#

The i8 speedup is more so on high thread counts

#

And makes net smaller

#

Smaller than master even I think

prime mica Oct 17, 2025, 7:41 PM

#

on my machine, I got a speedup even single threaded

#

but my computer has proven very weird in terms of perf characteristics

#

so probably wouldn't generalize

violet badger Oct 17, 2025, 7:49 PM

#

zen5 is however modern, and we should prioritize moving forward IMO.

rocky vigil Oct 17, 2025, 7:53 PM

#

how modern are the majority of fishtest workers btw

twilit oriole Oct 17, 2025, 7:54 PM

#

prime mica but ideally it can just be optional

Don't like this tbh, a big benefit is smaller net

rocky vigil Oct 17, 2025, 7:55 PM

#

less memory pressure on fishtest workers should help a lot as well yeah

#

especially with mmap

prime mica Oct 17, 2025, 7:55 PM

#

twilit oriole Don't like this tbh, a big benefit is smaller net

smaller as in the distributed binary, or smaller as in memory footprint?

twilit oriole Oct 17, 2025, 7:55 PM

#

Binary size

prime mica Oct 17, 2025, 7:55 PM

#

I see

rocky vigil Oct 17, 2025, 7:55 PM

#

note that lichess / chess.com would also like it

#

if our nets were smaller

prime mica Oct 17, 2025, 7:56 PM

#

I mean if you're willing to not use LEB128 then you could get it nearly the same size, e.g. use -127..=127 as literal i8 values and -128 as a prefix byte

#

but yeah there is an elegance about making it all i8

rocky vigil Oct 17, 2025, 7:56 PM

#

actually curious how we should continue to have the 7mb smallnet

#

like for websites

twilit oriole Oct 17, 2025, 7:58 PM

#

Not a major concern I think, they already use a custom binary

prime mica Oct 17, 2025, 7:58 PM

#

how does stockfish wasm even work? I don't see anything webassembly-specific in the repository...

rocky vigil Oct 17, 2025, 7:58 PM

#

lichess has a dedicated stockfish wasm repo?

#

idk what they do there

prime mica Oct 17, 2025, 7:58 PM

#

Ohhh ok

twilit oriole Oct 17, 2025, 7:59 PM

#

They zstd compress the net and that iirc

rocky vigil Oct 17, 2025, 7:59 PM

#

oh ok if they do it on their side it's good

#

it shouldn't impact them too much then

#

threat inputs compress better which will offset the raw size (assuming i16)

#

though i8 is obviously preferable

formal smelt Oct 17, 2025, 8:06 PM

#

rocky vigil how do you do this in monty? I'm losing like 5% with all the mm_cvtepi8_epi16s a...

i16::from

#

It just works™

rocky vigil Oct 17, 2025, 8:06 PM

#

yeah i saw and realized

#

it wouldn't be helpful lol

formal smelt Oct 17, 2025, 8:07 PM

#

This seems like the kind of thing that definitely doesn’t need manual simd

rocky vigil Oct 17, 2025, 8:07 PM

#

honestly to get around mulhi trick

#

i think if we wanted to do that

#

i8 * (i8 = 2) -> i16 mul is better

frosty imp Oct 17, 2025, 8:10 PM

#

rocky vigil 20 ppl authored and (vondele/disservin) merged

do we coauthor everyone or just code contributors?

#

because non-code contributors were not usually coauthored

#

but mentioned in the PR

green moat Oct 17, 2025, 8:10 PM

#

twilit oriole I'm tracking who has contributed, we are gonna end up with a PR with 20 coauthor...

sscg13, shawn_xu, cj5716, Yoshie2000, Viren, jw, vondele, anematode, rn5f107s2......who else?

frosty imp Oct 17, 2025, 8:12 PM

#

frosty imp because non-code contributors were not usually coauthored

for this I count sscg13, shawn_xu, cj5716, rn5, Yoshie2000, vondele

#

many more would need credits in the PR

rocky vigil Oct 17, 2025, 8:12 PM

#

yeah it might depend on how we do coauthor/credit split

#

a lot of ppl need credits yeah

frosty imp Oct 17, 2025, 8:14 PM

#

I guess disservin actually contributed to the original threat-inputs branch

prime mica Oct 17, 2025, 8:14 PM

#

green moat sscg13, shawn_xu, cj5716, Yoshie2000, Viren, jw, vondele, anematode, rn5f107s2.....

I didn't contribute lol, just kibitzing

twilit oriole Oct 17, 2025, 8:15 PM

#

Don't need to be posting lists lol. I already have it and none of the posted ones are complete anyways

rocky vigil Oct 17, 2025, 8:16 PM

#

yeah let's figure this out at the end

frosty imp Oct 17, 2025, 8:16 PM

#

are we getting too ahead of ourselves here Kappa

rocky vigil Oct 17, 2025, 8:16 PM

#

indeed

twilit oriole Oct 17, 2025, 8:17 PM

#

All we need to agree, I get to make the PR Kappa

rocky vigil Oct 17, 2025, 8:17 PM

#

though i wouldn't think it is wrong to be feeling pretty good about it

#

the scary time was before rn5 speedup

prime mica Oct 17, 2025, 8:18 PM

#

dumb question, why isn't threat information (or something equivalent) already encoded somehow in the main network through training

frosty imp Oct 17, 2025, 8:18 PM

#

can we clean up the different horizontal mirroring scheme btw

rocky vigil Oct 17, 2025, 8:19 PM

#

frosty imp can we clean up the different horizontal mirroring scheme btw

yeah i stuck to bullet mirroring bc i was too paranoid about it

frosty imp Oct 17, 2025, 8:19 PM

#

prime mica dumb question, why isn't threat information (or something equivalent) already en...

well the network can't extract those info efficiently

prime mica Oct 17, 2025, 8:20 PM

#

gotcha ok

upbeat pewter Oct 17, 2025, 8:25 PM

#

it's really hard to generalise that information just from the PST inputs without a lot of layers

daring wren Oct 17, 2025, 8:29 PM

#

prime mica dumb question, why isn't threat information (or something equivalent) already en...

this same argument can be used to "remove" king buckets

prime mica Oct 17, 2025, 8:32 PM

#

true

rocky vigil Oct 17, 2025, 8:33 PM

#

conversely, you cannot figure out psq information from threat information

#

the reason why this stuff doesn't work is like

#

you have to think of nnues as not really deep networks

#

so they are very contained by the additive structure

#

of the first layer

#

which carries most of the info

upbeat pewter Oct 17, 2025, 8:34 PM

#

part of why I wanted to play with threat inputs is that I always wanted to wire the attack table information I already had into the eval

#

and up until yoshie cracked it, I was the only AB engine that could do so without being majorly crippled in performance (though I did need to quarter my net width)

rocky vigil Oct 17, 2025, 8:37 PM

#

frosty imp are we getting too ahead of ourselves here <:Kappa:436339616866369553>

yeah cj speedup was displaying insane sss numbers earlier, in actuality it's probably closer to 5 elo which is what is predicted from the speedup amount listed

prime mica Oct 17, 2025, 8:37 PM

#

sss

#

I love this emoji btw

rocky vigil Oct 17, 2025, 8:37 PM

#

we still have work to do

violet badger Oct 17, 2025, 8:38 PM

#

yeah 5-10 Elo still needed I think

rocky vigil Oct 17, 2025, 8:38 PM

#

frosty imp can we clean up the different horizontal mirroring scheme btw

how would you want me to do it
it is relatively easy for the already trained nets, by repermuting the FT

#

i'll do it when we get closer to pass it hink

frosty imp Oct 17, 2025, 8:39 PM

#

yeah and also on the trainer side ig

lofty cedar Oct 17, 2025, 9:01 PM

#

Maybe someone could also use something similar to splat_moves to update threats faster?

#

Maybe...

#

Though the ideal byteboard... would be pretty hard.

prime mica Oct 17, 2025, 9:02 PM

#

I mean does threat updates still take a serious amoutn of time after cj/sscg/shawn's work?

rocky vigil Oct 17, 2025, 9:04 PM

#

the big gain still todo

#

is compute threat updates lazily

#

overall it should still be like high single digit % of runtime

lofty cedar Oct 17, 2025, 9:09 PM

#

How do we update lazily?

#

When the threat update depends on the board state.

rocky vigil Oct 17, 2025, 9:12 PM

#

we postpone the threat updates to when we need them

#

(i.e. on eval)

lofty cedar Oct 17, 2025, 9:13 PM

#

Yeah... I know, but each threat update depends on the board state... and updating the entire thing means... well... wait... tracking every added and removed piece? It could be as much as recomputing the entire threat...

#

Could be faster... IDK.

frosty imp Oct 17, 2025, 9:22 PM

#

prime mica Oct 17, 2025, 9:22 PM

#

update_accumulator_incremental 😩

frosty imp Oct 17, 2025, 9:23 PM

#

frosty imp

this is cj's branch btw

prime mica Oct 17, 2025, 9:23 PM

#

I'll try the i8 compression on ur branch later today

frosty imp Oct 17, 2025, 9:24 PM

#

🙏

frosty imp Oct 17, 2025, 9:45 PM

#

uh oh

#

info depth 39 seldepth 56 multipv 1 score cp -190 nodes 56649998 nps 1583021 hashfull 515 tbhits 0 time 35786 pv f8f5 b1c2 a6a5 d7d8 g8g7 d8a5 g7g8 a5a6 g8g7 c2c3 g7f7 c3b4 f7g7 b4a3 g7h7 a6a7 h7h8 a3a2 h8g8 a2b1 f5f1 b1c2 f1f5 a7b8 g8g7 c2c3 g7h7 b8c8 h7g7 c3b4 g7h7 b4a3 h7g7 c8b8 g7h7 a3a2 h7g7 b8d6 g7h7 a2a3 f5f7 a3b4 f7f5 d6g3
info depth 40 currmove f8f5 currmovenumber 1
stockfish: nnue/nnue_accumulator.cpp:115: void Stockfish::Eval::NNUE::AccumulatorStack::push(const Stockfish::DirtyBoardData&): Assertion `size + 1 < psq_accumulators.size()' failed.

#

position fen r4rk1/1b2bp2/p2p4/1p3pNp/4P2P/1P1Q1Pq1/1P6/1K1R2R1 b - - 1 25
go

twilit oriole Oct 17, 2025, 9:48 PM

#

Which branch is that

frosty imp Oct 17, 2025, 9:49 PM

#

cj branch

#

https://tests.stockfishchess.org/tests/view/68f2499a28e6d77fcffa04c6

frosty imp Oct 17, 2025, 9:50 PM

#

frosty imp ``` info depth 39 seldepth 56 multipv 1 score cp -190 nodes 56649998 nps 1583021...

wait wrong fen

#

position fen 5rk1/3Q4/p5p1/1p5p/8/1P6/1P6/1K6 b - - 0 37
go

rocky vigil Oct 17, 2025, 9:58 PM

#

Huh how does it have anything to do with threats

frosty imp Oct 17, 2025, 9:59 PM

#

it's broken on master actually

#

submitting issue rn

twilit oriole Oct 17, 2025, 10:02 PM

#

What the heck kek

frosty imp Oct 17, 2025, 10:03 PM

#

will bisect in a moment 🙃

lofty cedar Oct 17, 2025, 10:25 PM

#

https://tests.stockfishchess.org/tests/live_elo/68f2c1e828e6d77fcffa057d

Need to be quick and get my patch in as well!

stray reef Oct 17, 2025, 10:35 PM

#

lofty cedar https://tests.stockfishchess.org/tests/live_elo/68f2c1e828e6d77fcffa057d Need t...

how often does this trigger

lofty cedar Oct 17, 2025, 10:35 PM

#

About 12%.

prime mica Oct 17, 2025, 10:35 PM

#

Interesting

#

Could there be even more if we sorted them

#

Each eliminated pair is worth quite a lot

lofty cedar Oct 17, 2025, 10:36 PM

#

The data is quite structured.

#

As in, the first few almost always pair with the last few.

#

In order.

#

But I'm not sure if the cost of checking would outweight it so I only check one pair.

prime mica Oct 17, 2025, 10:41 PM

#

gotcha

rocky vigil Oct 17, 2025, 10:54 PM

#

frosty imp will bisect in a moment 🙃

How did it go

frosty imp Oct 17, 2025, 10:55 PM

#

the assert is wrong

#

the logic is correct

prime mica Oct 17, 2025, 10:55 PM

#

ok that makes sense

#

otherwise it would have crahsed by now

rocky vigil Oct 17, 2025, 11:10 PM

#

frosty imp

which commit is this

frosty imp Oct 17, 2025, 11:19 PM

#

rocky vigil which commit is this

cj ongoing test

rocky vigil Oct 17, 2025, 11:19 PM

#

oh ok

#

so it looks like tracking and indexing threats each take up ~5% of the runtime

#

after cj speedup

#

so lazy tracking could be a couple % gain

lofty cedar Oct 17, 2025, 11:28 PM

#

Though... well... there's this patch that needs aprxval. This one is a low-hanging fruit.

rocky vigil Oct 17, 2025, 11:30 PM

#

actually why is it that we get duplicate features in both added and removed

lofty cedar Oct 17, 2025, 11:37 PM

#

Well, I think the feature got added in one move but then it didn't use the net so it got removed later on.

rocky vigil Oct 17, 2025, 11:38 PM

#

huh shouldn't incremental always be one-move updates

#

strange

#

stage 1 validation loss 0.00305 w/ factorizer compared to 0.0031 from old run

#

not sure how much this can be read into

lofty cedar Oct 17, 2025, 11:45 PM

#

rocky vigil strange

It's one-move, but sometimes the eval is skipped.

#

Has anyone added the weight permutation or something to the system?

rocky vigil Oct 18, 2025, 12:04 AM

#

the only weight permutation to be done is re-indexing the threats to be efgh mirrored (so as to remain consistent with the psq mirroring)

#

this does not functionally change the evaluation, so I'm delaying it to finishing touches

rocky vigil Oct 18, 2025, 12:07 AM

#

lofty cedar It's one-move, but sometimes the eval is skipped.

hmm, I thought the incremental updates were done move-by-move though
... I won't question it

#

let's see how it fares on fishtest

#

how big are added / removed on average? if intersection is significant it might be worth it to search for cancellations

frosty imp Oct 18, 2025, 12:45 AM

#

rocky vigil huh shouldn't incremental always be one-move updates

yeah

naive comet Oct 18, 2025, 12:53 AM

#

old profile

naive comet Oct 18, 2025, 12:53 AM

#

frosty imp

new profile

frosty imp Oct 18, 2025, 12:54 AM

#

naive comet old profile

old and new together Kappa

naive comet Oct 18, 2025, 12:54 AM

#

rocky vigil how big are added / removed on average? if intersection is significant it might ...

total delta averages ~7 I think

#

goat

frosty imp Oct 18, 2025, 12:55 AM

#

why is there an intersection

#

a slider piece moving along its ray or something?

naive comet Oct 18, 2025, 12:57 AM

#

when you add a piece then remove it then there's an extra + and - from the slider attacking the piece behind it

frosty imp Oct 18, 2025, 12:58 AM

#

I wonder if you can do something about it then?

#

just hardcode threat updates for each movetype maybe?

naive comet Oct 18, 2025, 1:01 AM

#

I mean we already had a threat deduplication patch going

#

from rn5

twilit oriole Oct 18, 2025, 1:01 AM

#

https://tests.stockfishchess.org/tests/view/68f2c1e828e6d77fcffa057d so is this neutral?

naive comet Oct 18, 2025, 1:01 AM

#

it reduced updates by ~0.7 on avg I think

frosty imp Oct 18, 2025, 1:02 AM

#

twilit oriole https://tests.stockfishchess.org/tests/view/68f2c1e828e6d77fcffa057d so is this ...

can we test this on top of the other speedups

naive comet Oct 18, 2025, 1:02 AM

#

twilit oriole https://tests.stockfishchess.org/tests/view/68f2c1e828e6d77fcffa057d so is this ...

that directly interacts with rn5's patch lol

#

@lofty cedar

twilit oriole Oct 18, 2025, 1:03 AM

#

Well a dbg on repeat would be useful ig

naive comet Oct 18, 2025, 1:04 AM

#

naive comet it reduced updates by ~0.7 on avg I think

correction: 0.3

twilit oriole Oct 18, 2025, 1:05 AM

#

lofty cedar About 12%.

👀

frosty imp Oct 18, 2025, 1:05 AM

#

two updates for 12% of the time

#

that's 0.24

naive comet Oct 18, 2025, 1:07 AM

#

ok well gg I guess

frosty imp Oct 18, 2025, 1:39 AM

#

@naive comet plz pr

lofty cedar Oct 18, 2025, 1:40 AM

#

naive comet that directly interacts with rn5's patch lol

What interaction?

lofty cedar Oct 18, 2025, 1:57 AM

#

It looks like my patch speeds up more than I expected.

#

It measured barely 1% on my machine.

rocky vigil Oct 18, 2025, 2:02 AM

#

interesting

#

well

#

any and all elo is good 😄

prime mica Oct 18, 2025, 2:03 AM

#

🚀

rocky vigil Oct 18, 2025, 2:08 AM

#

frosty imp <@1082450465301733376> plz pr

btw while you wait for this I'm going to start both stc + ltc vs master

lofty cedar Oct 18, 2025, 2:12 AM

#

Are you merging my patch too?

rocky vigil Oct 18, 2025, 2:13 AM

#

lemme keep it to patches that have passed fishtest...

#

there's plenty of time to be patient

lofty cedar Oct 18, 2025, 2:15 AM

#

Oh, okay.

rocky vigil Oct 18, 2025, 2:22 AM

#

alright now we wait

rocky vigil Oct 18, 2025, 2:50 AM

#

prime mica 🚀

~~nooo how could your cores do threat inputs dirty like that !!!~~

prime mica Oct 18, 2025, 3:21 AM

#

lololol

#

the truth hurts

twilit oriole Oct 18, 2025, 3:23 AM

#

https://tests.stockfishchess.org/tests/view/68f15c8128e6d77fcffa030f this was the previous PT

prime mica Oct 18, 2025, 3:23 AM

#

😩

#

what's the ELO gain fixed nodes again?

twilit oriole Oct 18, 2025, 3:24 AM

#

30 but it is misleading

#

Because of how threat inputs work

prime mica Oct 18, 2025, 3:25 AM

#

elaborate?

twilit oriole Oct 18, 2025, 3:27 AM

#

Different game phases have a very varying speed diffs and fixed nodes differential. The fixed nodes gain occurs in the positions with the most slowdown

#

You have to just read the STC and LTC the fixed nodes does not tell about expected scaling

prime mica Oct 18, 2025, 3:27 AM

#

gotcha

twilit oriole Oct 18, 2025, 3:32 AM

#

Well the new speedups don't appear to help in PT much

#

So that's a big issue

#

What a terrible result kek

rocky vigil Oct 18, 2025, 3:47 AM

#

this is so cooked

#

what

prime mica Oct 18, 2025, 3:47 AM

#

😭

twilit oriole Oct 18, 2025, 3:48 AM

#

Well it may be related to the fact you rebased on master

#

Which is optimising for master net

#

Gainer patched there may not necessarily translate

#

Either that or the branch is fucked in some way

naive comet Oct 18, 2025, 3:51 AM

#

@frosty imp https://github.com/xu-shawn/Stockfish/pull/16

#

@rocky vigil @regal steeple remember to rebase

twilit oriole Oct 18, 2025, 3:54 AM

#

https://github.com/official-stockfish/Stockfish/compare/master...sscg13:Stockfish:cj-latest-speedup

#

It is rebased. That's the PT diff

naive comet Oct 18, 2025, 3:55 AM

#

ok nice

twilit oriole Oct 18, 2025, 3:57 AM

#

At best if the test got super unlucky in both STC and LTC it's equal to previous PT

#

We can just attribute it to that these new tests didn't have Shawn's blessings

#

Kappa

#

I think it is actually likely the previous PT was just super lucky

#

Since the jump to -10 didn't add up from the previous approx -25

prime mica Oct 18, 2025, 4:07 AM

#

sss

naive comet Oct 18, 2025, 4:08 AM

#

I have a good idea for more speed, I will :prayge: this works

#

once I get back home

rocky vigil Oct 18, 2025, 4:17 AM

#

twilit oriole Since the jump to -10 didn't add up from the previous approx -25

wasn't it -20 + (around 10) = -10
idk anymore

#

to be fair what I see in sprts

#

one side can apparently just randomly gain 5 elo

#

then lose it

#

so idk anymore

twilit oriole Oct 18, 2025, 4:24 AM

#

Well I think this PT was too early so not enough gainers to overcome error bars. There is still actual gain

#

It's just not visible enough yet

#

Adding sprt Elos is not valid lol

#

You have to assume on the low end for all of them

rocky vigil Oct 18, 2025, 4:29 AM

#

is 10k games even enough to check scaling

#

ig I can extend

#

if we want more

twilit oriole Oct 18, 2025, 4:30 AM

#

I don't think you can extend it

#

Fishtest "feature"

#

Try to if you want to see what I mean

rocky vigil Oct 18, 2025, 4:31 AM

#

eh whatever

#

oh yeah

#

"unable to modify number of games in a fixed game test" lmao

#

actual const int games = 10000

twilit oriole Oct 18, 2025, 4:32 AM

#

It's dumb, I disabled the check on our instance lol

prime mica Oct 18, 2025, 4:32 AM

#

rocky vigil "unable to modify number of games in a fixed game test" lmao

undefined behavior

twilit oriole Oct 18, 2025, 4:33 AM

#

It's just a check in the code that throws that message when you try

#

We used to be able to till a few years ago

#

Like it's intentional design choice to not allow users to do it now

frosty imp Oct 18, 2025, 4:34 AM

#

merged

naive comet Oct 18, 2025, 4:41 AM

#

bam

frosty imp Oct 18, 2025, 4:42 AM

#

💥

#

https://tests.stockfishchess.org/tests/view/68f31c6c28e6d77fcffa0611

naive comet Oct 18, 2025, 4:52 AM

#

whoo

frosty imp Oct 18, 2025, 4:53 AM

#

sanity check factorized vs unfactorized stage 1

frosty imp Oct 18, 2025, 5:18 AM

#

hmm seems alright

#

stopping the test

lofty cedar Oct 18, 2025, 5:20 AM

#

What's this factorization?

frosty imp Oct 18, 2025, 5:20 AM

#

just bringing back factorized weights for psq inputs

lofty cedar Oct 18, 2025, 5:22 AM

#

What's factorizing weight in the first place?

frosty imp Oct 18, 2025, 5:24 AM

#

https://github.com/official-stockfish/nnue-pytorch/blob/master/docs/nnue.md#feature-factorization

#

we add an extra bucket active regardless of ksq, then merging that bucket to all other buckets after training

#

help with convergence in rarer buckets

lofty cedar Oct 18, 2025, 5:25 AM

#

Oh, I see.

#

Though the elo in the test can be misleading.

#

I thought we were like only 10 elo away but after a few more patches it's further.

#

Though still within the error bar.

twilit oriole Oct 18, 2025, 5:41 AM

#

Well stage 1 ofc will be much better. It won't be that huge difference in the end

violet badger Oct 18, 2025, 9:49 AM

#

so, the equivalent test of https://tests.stockfishchess.org/tests/view/68f2f8d528e6d77fcffa05d6 run locally, but with 72t:

Results of master vs patch (10+0.1, 72t, 32000MB, UHO_Lichess_4852_v1.epd)
  # PLAYER    :  RATING  ERROR  POINTS  PLAYED   (%)
   1 master    :     0.0   ----  6518.5   12800    51
   2 patch     :    -6.6    4.4  6281.5   12800    49

naive comet Oct 18, 2025, 9:50 AM

#

yikes

violet badger Oct 18, 2025, 9:51 AM

#

well, not infinitely far from beating it.

#

3-4% speedup, or a clear improvement in the training.

candid ivy Oct 18, 2025, 10:22 AM

#

if i checked correctly then apply sometimes does remove/add the same threats can that be?
like the value in the added list also exists in the removed list ? so we are doing some unnecessary ops no?

frosty imp Oct 18, 2025, 10:23 AM

#

yeah that’s what people are now optimizing

candid ivy Oct 18, 2025, 10:23 AM

#

ah 👍

green moat Oct 18, 2025, 10:35 AM

#

@violet badger
Unexpected EOF in the Factorizer pipeline
🙄
https://gitlab.com/cscs-ci/ci-testing/webhook-ci/mirrors/5137461961076608/2926829081096545/-/jobs/11750723588

violet badger Oct 18, 2025, 10:35 AM

#

already restarted

#

https://gitlab.com/cscs-ci/ci-testing/webhook-ci/mirrors/5137461961076608/2926829081096545/-/pipelines/2107030038

violet badger Oct 18, 2025, 2:05 PM

#

btw, I wonder what that 'factorized' pipeline is actually using, as it is setup to use just --features=Full_Threats .. not --features=Full_Threats^ ?

rocky vigil Oct 18, 2025, 2:08 PM

#

Huh

violet badger Oct 18, 2025, 2:09 PM

#

so, you agree that it should be using the latter?

#UE Threat Inputs for AB