Even the later layers are sparse. If you could use this information to speed up the NNUE eval somehow, that could be a good idea.
#The NNUE is sparse
tidal raft
violet lodge
the later layers are so minimal that significant improvement is needed for a overall speedup can be measured
tidal raft
Maybe that could be the case...
But what about we could make the later layers bigger?
tidal raft
the later layers are so minimal that significant improvement is needed for a ove...
I actually tried replacing the affine transform with affine transform sparse input but it got an unstable bench.
violet lodge
yes because the later layers are too small for even the entire L1 output to fit fully into an avx512 vector