#fide-google-efficiency-chess-ai-challenge

1 messages · Page 1 of 1 (latest)

soft berry
zinc blade
zinc blade
#

Would anyone happen to know whether it's possible to pip install a package on the remote env?

cobalt ermine
#

At submission zone there is an option, packages, there you can write all pip install that needs to be installed, even internet off

zinc blade
cobalt ermine
#

Ups. True.

#

Actually "Create agents to play chess with resource constraints" doesn't mean frezze environment?

cedar coral
#

Hey everyone,
I am pretty new to AI/ML and wanna participate in this competition so can anyone guide me on:

  1. What should I learn to build a chess-playing AI?
  2. How to make it efficient?
  3. Any beginner-friendly resources to get started?
    Thanks a lot!
south dew
#

You can also search for TCEC 4K, which is a contest in a similar format (but even more restricted in file size)

zinc blade
#

Funny thing is that some packages require more than 5MB to be imported so it doesn't even matter that they're available

languid barn
# 
kaggle_submissions/
  main.py
  <other files>

You can run tar -czf submission.tar.gz -C kaggle_submissions . and upload submission.tar.gz

torn vortex
#

Does teaming up increases number of allowed submission per day?

languid barn
torn vortex
pseudo hound
#

Can someone please explain this constraint further:

Constraints

  • 5 MiB of RAM**
  • Dedicated CPU: a single 2.20GHz core
  • 64KiB compressed submission size limit

During the RAM allocation, I used psutil to monitor memory usage while running the script, but the basic package imports were too large.

Any suggestions would be appreciated.

languid barn
#

We also already have Chessnut imported

wind pecan
#

Need clarification on timing as I don't play with clock. Is 10s the total time for all your moves in a game? And does .1s delay mean you get a "free" .1s per move?

languid barn
pseudo hound
zinc blade
#

@languid barn Thank you for taking the time to help out here and in the discussion forum btw

wind pecan
# zinc blade <@1128370977743581305> Thank you for taking the time to help out here and in the...

It just needs to have "def chess_bot(obs):" as the last function. It doesn't even need to be named main.py if you're uploading 1 file. Take a look at this or other examples in Code tab. https://www.kaggle.com/code/vyacheslavbolotin/chess-bot-starter-step-0-0-0-1

Chess Bot | Starter - step 0.0.0.1

Explore and run machine learning code with Kaggle Notebooks | Using data from FIDE & Google Efficient Chess AI Challenge

zinc blade
wind pecan
neat sapphire
#

mh I have the feeling the agent logs are sometimes not able to capture what has gone wrong.. im having some issues with a submission and the json log is an invalid json and just

[

well not the feeling, it is this way

languid barn
languid barn
neat sapphire
#

yikes

#

the OOM refering to your 5MB limit ? or literally a big OOM?

languid barn
#

yeah 5MB limit

#

if process is OOM the OS will kill it

neat sapphire
#

meh, i shouldnt technically oom according to my calculations but I will check thanks

languid barn
#

Checking with folks on this

vagrant fractal
#

how often will the program be restarted?
is it,

  1. every game
  2. every turn
  3. never at all
smoky mesa
#

Hey everyone! New here and still figuring things out. I put together my first bot, and after a few games, it managed to hit rank 7, but I’ve noticed some teething issues as it went down the ladder. I’d love to connect with others—pros or newcomers alike—to collaborate, share ideas, and learn from each other

image.png
mortal dock
#

Does the CPU support avx2?

neat sapphire
fossil vault
#

hello everyone!
i have a general question, how is the evaluation done?
i mean my code should produce a pgn/fen for every move?
how is the piece movement evaluated ?
i am trying to understand what the output my code should be for every move
thanks in advance

neat sapphire
fossil vault
#

hm okay, i see, i meant in what format, seems helpful, thank you very much!

neat sapphire
#

uci format, "e2e4", "e7e8q" etc

zinc blade
languid barn
mortal dock
#

It's no longer about ML and more about pure optimization

languid barn
# mortal dock I feel like this might ruin the point of the competition? This just turns the co...

We definitely don't want it to be a stockfish comp. The 64 KiB submission limit along with the 5MB RAM limit guard against this.

Stockfish has many components:

  1. tree search (cpp files)
  2. nnue evaluator (10.5 million params)
  3. opening books
  4. position evaluation cache
  5. end game solvers

any of these pieces are larger than 64 KiB and many are so large they won't fit at all. In addition none of these pieces are designed to fit in 5 MB RAM.

heavy pewter
#

I am somewhat very new, maybe its a stupid question, can RL algos run under the given constraints?

candid thicket
# languid barn We definitely don't want it to be a stockfish comp. The 64 KiB submission limit ...
  1. You can just simply compile the binary. Afaik @neat sapphire already got it down to low 70KiB.
  2. Cutdown HCE from last stockfish with HCE will do. (Stockfish classical is still 3600 CCRL engine. Good luck beating that)
  3. Not part of stockfish
  4. Transposition Table as its called can be made really really small very easily
  5. Those are called tablebases. They are not part of stockfish either xD

Some are but are purely optional and can be worked around.

The 5MB limit is truly enough for cutdown stockfish xD

If you dont want it to become stockfish competition or clone competition as a whole. I recommend simply banning cutting down an existing engine or using code from existing engines. Let the developers be actually creative.

obsidian spoke
neat sapphire
#

If you dont want it to become stockfish competition or clone competition as a whole. I recommend simply banning cutting down an existing engine or using code from existing engines. Let the developers be actually creative.
which requires extra validation during the competition to not allow such entries

candid thicket
sturdy basin
#

It's not a significant issue. A custom solution will always beat a hacked together one. There is plenty of time (3 months)

candid thicket
#

I kinda disagree. You can SPSA tune the hacked code stockfish for this.

sturdy basin
#

I don't think the winning engines will even be HCE. You should wait to see what people are capable of instead of being immediately dismissive

candid thicket
#

It wont be. I know what you mean exactly 😉

We shall see but it doesnt hurt to be little more careful of this issue.

languid barn
vagrant fractal
clear swallow
#

im still new to kaggle

#

can somebody walk me throught the whole submitting process

#

is it just an agent we submit or what

clear swallow
#

so its legit just a raw agent

#

btw, how are yall managing with making sure ur bot knows what color it is

#

for specific piece instructions

#

im wondering if I should group both pieces from both sides into classes and code for that class itself

#

also

#

whats the deal with notebooks and jypter and stuff

#

i am still trying to figure out how it works lol

#

i THINK its supposed to be like some sort of frontend backend system??

#

as in there are files linked to specific cells

#

but i can't access them

vagrant fractal
candid thicket
#

@languid barn In case of C binary. What are the libraries installed in the enviroment ? math.h and pthread.h are there ?

neat sapphire
#

Why would you even need pthread ?

candid thicket
#

👀

neat sapphire
#

Unless the one core actually means that we are allowed to use 2 threads in which case yeah

languid barn
rugged python
#

how is remainingOverageTime calculated between turns? when I substract two remainingOverageTimes in the replay log, I'm using more time than the duration in my agent log

candid thicket
#

You should not be using average time at all

#

Your engine should be time managing from the left over time

rugged python
#

The description says this time is “Total remaining banked time (seconds) that can be used in excess of per-step actTimeouts -- agent is disqualified with TIMEOUT status when this drops below 0.”

#

My understanding is that I can use the 0.1 second plus this value of time for my next move

rugged python
candid thicket
#

Yea. Its the remaining time. You cannot use all the time per 1 move...

rugged python
#

Right. If I subtract two consecutive remaining time, I should get the duration I spent on a move. But the result I got doesn’t feel right

pastel creek
#

Is our agent only allowed to take the game as an argument or can we somehow pass the remaining time in the game?

neat sapphire
pastel creek
#

so you can just take the remainingOverageTime with obs.remainingOverageTime?

neat sapphire
#

yes

pastel creek
#

thanks

#

also people are not allowed to clone stockfish and use it in the competition right?

neat sapphire
#

Scroll up and read the chat

#

@languid barn

  1. Would it be possible if we somehow get information on what the "Err" was when games were played?
  2. We kinda need the move which the opponent played in the observation as well, otherwise it's annoying to figure out if a position will repeat itself and thuse be a 3 fold repetition. If we don't, we need to add "complex" logic to figure which move the enemy played.
#

This position is a mate, but got marked as a tie...

pastel creek
#

Also is anyone using a minimax algorithm or a trained model?

quasi portal
#

btw does anyone know if the top submission models are made on c wrapped to python?

hushed summit
#

We are able to create notebooks not on kaggle right? I was wanting to work on vscode

#

I am trying but am getting a unicodeDecodeError

quasi portal
compact tapir
quasi portal
#

but i dont knowww c 😭

waxen narwhal
#

Hey guys, does anyone know if we can actually upload compressed zip files with a main.py and additonal files as long as its under 64 KB? It seems to say we can but ive had no luck with it

compact tapir
waxen narwhal
#

Thanks!

obsidian spoke
warm jacinth
floral apex
#

can anybody tell me what format my code output should be in? like should it be a movelog notation? something like Ra7? or should it be like an FEN notation? or anything else

pastel creek
#

you should return a move like this e2e4 which is in the Coordinate Notation

arctic flint
#

More well known as UCI notation

floral apex
#

ohhk thankss

pastel creek
#

Can we submit a bot that is coded in another langauage like c++

#

or do we have to wrap it in python?

arctic flint
pastel creek
#

can we submit multiple files

arctic flint
#

Yea

candid thicket
candid thicket
neat sapphire
#

It needs time to get back to where it was. The scoring only works reliable if there are many agents evenly spread. Then it can get back to where it was quickly. If you are unlucky you run into an early tie because of missing threefold repetition

#

Which will reduce your progress

candid thicket
#

Hmmm fair

#

Tho i would think 7 hours is enough

mortal dock
#

idk man the grader is just borked

compact tapir
#

the grader seems to be some kind of elo system. elo ratings are just relative to the population so as new bots get submitted elo of the existing bots will also change

#

also i suspect people resubmitting bots has some downward push on elo since their rating gets reset

candid thicket
#

The grader is borked. We submitted 2 identical agents. 1 is 100 score higher than the other.

compact tapir
#

thats not particularly surprising. most chess bots aren't deterministic. plus there will be some variation in the opponents that they go against. overtime their ratings will probably converge but theres bound to be a difference

candid thicket
#

@languid barn If i may suggest. PLEASE use an opening book. You need thousands upon thousands of openings to get a real sense of the engine elo. if you dont you will just repeat the same game over and over and over again as engines are deterministic. 1 opening is played twice. Each engine plays black and white. When we do engine development we use UHO lichess opening book. Thats standard basically across entire engine dev scene

Also figure out a better rating system. Its borked and not representive of the actual strength at ALL. 2 identical agents are more then 120score points apart now. That is SERIOUSLY BAD.

candid thicket
#

Unless you screw something up

#

If your engine is non deterministic thats BAD.

compact tapir
#

if you put two engines against each other they won't play the same game over and over

candid thicket
#

from what we saw. There are 10 opening positions. I cannot even say how bad that is without using swear words

candid thicket
compact tapir
#

theres a lot of noise involved. sometimes the engine might get 1 depth deeper and change its move

candid thicket
#

Oh but you are using time

#

Deterministic means that with the same amount of nodes. they will say the same move

compact tapir
#

yes thats true. for the same nodes they will, but they wont always get to the same amount with iterative deepening

candid thicket
#

also 99% of the time even time doesnt matter. Since the speed doesnt change

candid thicket
#

Time adds minor but 99% of the time insignificant variance

obsidian spoke
#

i think ratings take a lot longer to converge with the current system

#

e.g. rafbill's rating takes half a day to go down

vagrant fractal
#

but even then if it's not close to a thousand it's not good

candid thicket
#

UHO has what like 2M positions ?

wind pecan
#

What's UHO?

candid thicket
#

Opening book

dusk pelican
viral inlet
tough ether
#

I’d like to participate in this competition, but I also want to learn the Julia programming language. I know that to submit this project, you need a main.py. Any workarounds to this?

vagrant fractal
#

run a compiled exe from the python script

candid thicket
#

not exe but a linux binary

quasi portal
#

are people using the chessnut library they give?

pastel creek
#

so would u make a c++ file, then a wrapper file, then a setup py file , and then finally execute your code in the main.py file?

#

To incorporate other languages?

hushed summit
wheat glen
#

Hello all,

As we know that the chessnut library will be pre loaded in the environment and hence its import wont be counted in the memory utilised.
But will the various functions which we use from the chessnut library also **not included **in the memory calculations or they would count towards ram utilisation?

arctic flint
hushed summit
arctic flint
#

made my own

hushed summit
#

I'm starting to think I should of been paying attention in my intro to c class

dire pond
#

Hello everybody, completely new to Kaggle, I want to participate in chess competition, Where should I begin? any examples?

pastel creek
mortal dock
#

how much disk space do we have?

pastel creek
#

To submit the agent or in the notebook?

pastel creek
#

If you want to run the code in another language like c or c++ do we have to make our own chess libraries or is it possible to import them?

wind pecan
arctic flint
obsidian spoke
#

the entire library contradicts the contest's focus on "efficiency and strategic thinking"

serene swallow
pastel creek
#

@obsidian spoke Are you writing code in python or is it wrapped in a faster language?

mortal dock
#

like when it's actually running

pastel creek
#

I think you only have 64 KB of compressed files
It's on the main page tho

wheat glen
#

whats the meaning of terms in this log file?
here stdout stands for the output given by my engine?

image.png
pastel creek
#

I think it means what you have printed out during the match.

slate sundial
arctic flint
#

Hm interesting. Have you ensured that your linux binary is compiled with ubuntu 20.04? That was an issue for me and some other people

#

Also @slate sundial , my script sends allocated time in milliseconds (in accordance with UCI protocol), so ensure that your program is using milliseconds to calculate how to use its time

#

If your program is interpreting it as seconds it could be timing out

#

Another issue that affected me is:

Make sure that when you tar your files you do something like:

tar -czf KaggleSubmission.tar.gz main.out main.py

not

tar -czf KaggleSubmission.tar.gz src_folder

arctic flint
slate sundial
arctic flint
#

hm ok. From what I understand they have Intel Xeon Haswell CPUs

slate sundial
#

I'm using make build ARCH=x86-64 in ubuntu 20.04 but still getting the timeout status on step 1. @wind pecan let me know if you ever find a solution to the timeout status

#

Is it possible to debug this kind of stuff in a kaggle notebook? (I just have a python script and an executable)

pastel creek
#

I have three files -- how do I submit them because my main.py needs my other file which needs my last file

wind pecan
#

@slate sundial No, I looked at @arctic flint 's solution and changed my code to mirror that. Still the same, works in docker but not in submit.

#

@arctic flint Compiling inside the docker container should do it. I didn't do that at first and I did see a "wrong format" error, so not seeing that and compiling in docker should be good. Although if nothing else explains it, it may be the only reason left.

wheat glen
pastel creek
#

Does anyone know if we are allowed to submit in a tar.xz file format

#

I cannot compress my solution into a tar.gz

candid thicket
#

7z doesnt work either

grand cloak
#

hello, im currently trying start and have little experience of kaggle environments, does anybody have similar error? it works for everything else but chess? apologies if its an FAQ

Screenshot_2024-11-25_at_12.17.09.png
craggy remnant
#

check kaggle-environments version
make sure you are using the latest version

grand cloak
arctic flint
wheat glen
#

compiling a c code before hand results in a binary of 140kb.
compiling the code in main.py is running fine on local but is failing validation post submission.

#

any suggestions..?

im veryy new to c,
so should i try submitting the compiled binary in submission or is trying to compile the c code post submission worth a shot..?

#

i personally feel that compiling the code in the python script would allow me to pack more lines in the .c file pre submission .-.
but that would result in my code being compiled every time the script is run to output new move

what are ur views @arctic flint @wind pecan ?

torn vortex
#

I am trying to call obs.board, such that it will give me an FEN string. But i get this

image.png
arctic flint
#

It would definitely be much better space-wise if you can minify your source code

torn vortex
#

Is pawn promotion available?

languid barn
languid barn
wind pecan
# grand cloak yes i have it already

Yes, I've seen it before. If you create a cell with just ```%%capture

ensure we are on the latest version of kaggle-environments

!pip install --upgrade kaggle-environments``` as shown in your-first-chess-bot example, the latest version is ensured. Maybe you didn't have the %%capture part when you did it.

torn vortex
languid barn
grand cloak
grand cloak
pastel creek
#

What file formats are allowed for sumbission?

compact tapir
#

i don't think there are any others, but i could be wrong

pastel creek
#

Ok thx

wind pecan
pastel creek
#

Is there any c library that will return the legal moves of a certain fen

#

Or do we have to make that functionality ourselves because that seems like a daunting task

vagrant fractal
#

there is a c++ library

#

and a rust library

pastel creek
#

what are they

mortal dock
#

rust is cozychess iirc

vagrant fractal
pastel creek
#

Is there any thing for C?

wheat glen
pastel creek
#

So do we have to make our own program that gets the legal moves

vagrant fractal
#

make your own if you want your own design choices

pastel creek
#

I cannot fit the header file into the submission
Is there any way to fit the file into the submission

#

Because the size limit is only 64 while the file size is 180 kb

vagrant fractal
#

just remove stuff you don't use from the file

#

or make your own (which I personally recommend)

lethal fractal
#

hi, came from kaggle, i am ranked at 20-30th place in leaderboard

#

(it changes frequently)

gaunt edge
#

hello, I am doing this kind of challenge for the first time, can anyone tell me how to start in this challenge, just where should I begin with? a basic approach

rugged python
#

And you can turn on -Os to optimize for binary size

torn vortex
gaunt edge
pastel creek
pastel creek
#

!tar -czvf submission.tar.gz a.out main.py
When submitting the tar.gz file it keeps giving me an error saying that a.out is not defined but it is clearly defined.
Would this be because the OS is Linux on that competition's end?

slate sundial
#

Hey everyone, I created a guide to compile stockfish into a 60 KB executable and profile its memory usage. The 60 KB executable still uses ~25 MB of RAM (exceeding competition limits), but the guide shows how to run valgrind to identify stockfish components using the most memory which can be targeted to reduce memory usage. I included a pre-compiled 60 KB stockfish executable, modified stockfish codebase used to compile the executable, profiling tool, and explanation of how to build and profile.

I'm selling this for $5 because it took a significant amount of trial and error to get everything working:
https://payhip.com/b/bRt1l

neat sapphire
#

Im pretty sure this is against the rules.

slate sundial
#

Is there a moderator I can check with? I didn't see any rules related to this

neat sapphire
#

Pretty certain that in the rules this was stated that it’s not allowed to share your code or even like sell it to others

slate sundial
#

The executable included in the guide can't be used as-is for the competition since it uses 25 MB of RAM, so it's mainly for learning purposes- it will still take some innovation to reduce the memory usage to 5 MB

wide oak
#

for 0$ I'll sell you the advice that paying someone 5$ to see their potentially half-baked take at reducing Stockfish's file size won't actually help you very much.

vagrant fractal
quasi portal
slate sundial
slate sundial
vagrant fractal
#

so Stockfish Classical? or do you mean Stockfish 11

wide oak
# 
git clone https://github.com/official-stockfish/Stockfish
cd Stockfish
git checkout 9587eeeb5ed29f834d4f956b92e0e732877c47a7
cd src/
make build ARCH=x86-64-modern -j
strip stockfish
tar -czvf temp.tar.gz stockfish
#

174k. Go delete some random tables and voila, you have Ben Swain's version.

opal idol
#

Is it ok to ask but is this what the people at the top of the LB are doing? Shrinking down open source chess engines...more or less? Or writing thier own in C?

vagrant fractal
#

most are going the former, I'm going the latter

obsidian spoke
#

the people at the top have their own open source chess engines Kappa

vagrant fractal
#

yeah there's that too

opal idol
neat sapphire
#

3 months to match the worlds strongest chess engine authors is an ambitious goal : D

wide oak
#

so is taking an existing engine and out-tweaking the worlds strongest chess engine authors

opal idol
neat sapphire
#

haha

candid thicket
candid thicket
novel osprey
#

Is this a skill issue

#

(Yes I have the latest)

grand cloak
# novel osprey

some people told me to restart the kernel, but i just run jupyter notebook locally

novel osprey
#

Thats what the console told me too

#

Nevermind

#

Just did a code block with exit()

pastel creek
#

If the bot says ERR for a game and the logs show that it had 9+ seconds left, does that mean that the agent exceeded the RAM limit?

waxen narwhal
#

Am I able to share my chess website in here or are there more appropriate discord channels for such a thing?

strong prairie
left kernel
wide oak
#

only if none of the worlds strongest chess engine authors also try to optimize the engine.

#

I'm not trying. But there will be some that are ~top tier.

twilit pendant
#

out of all the available chess libraries in python, why was chessnut chosen?

wide oak
#

python-chess would have been the far superior choice.

twilit pendant
#

Chessnut breaks(timeout) with just a lookahead of one(or maybe my code is that bad)

novel osprey
#

i uploaded a notebook 15 times already and still wont work (count as error)

#

what am i doing wrong 😔

novel osprey
twilit pendant
novel osprey
#

I know, i meant when i submit the submission.py it just counts as an error now

twilit pendant
#

I am using an open source bitboard chess move generator, and the submission is not working, but I am not even able to understand where the error is.

#

Even the logs are not working

novel osprey
#

The logs are just []

twilit pendant
#

same here

novel osprey
#

I've added countless debugs

twilit pendant
#

Same

#

I noticed that when I remove the import part, the logs start working again

#

I tried keeping everything in one file, in separate files, in different folders, everything ended up in the same issue

#

I have no idea what is wrong

#

Chessnut is depressingly slow

#

And I do not have the time to make my own bitboard

novel osprey
#

I dont know but should the submission file just be purely a def

twilit pendant
#

At least I do not think so

#

My first bot had a dict outside

#

And my present bot works without import, it is just importing the chess file that causes everything to go blank

novel osprey
twilit pendant
#

This is my first competition after years and I am getting so many weird problems

novel osprey
#

Maybe i think when my bot is placed to fight itself

#

It goes time out

twilit pendant
#

My bot does perfectly fine in my own env

novel osprey
#

Both is outsmarting each other and not recognizing the remainingOverageTime

twilit pendant
#

in kaggle notebook

novel osprey
twilit pendant
#

even with itself

novel osprey
#

"When you upload a submission, we first play a validation episode where that submission plays against copies of itself to make sure it works properly. If the episode fails, the submission is marked as error and you can download the agent logs to help figure out why. Otherwise, we initialize the submission with μ0=600 and it joins the pool of for ongoing evaluation. At this time we also deactivate older agents if the total number of active agents is greater than three."

#

I dont get how the episode fails

twilit pendant
#

same here

#

and especially why the logs are empty

#

the bitboard implementation I am using is about 4k lines of code

#

When I randomly remove code, the logs give valid results(but ofc errors because I removed random lines of code)

#

I mean it is not [] anymore

#

Only if chessnut was faster

#

A single look ahead results in a timeout

novel osprey
#

Its possible to resign via bot right?

#

Should i return None?

twilit pendant
#

resign?

#

surrender?

novel osprey
#

Yes

twilit pendant
#

I don't think so

#

a lot of repeated moves already ends in a draw it seems

#

if log is [], it means that neither the agend went trough a single step

#

It could not even load the agent

#

In short the game did not even begin

#

Maybe it took too long to load the dependencies, which resulted in a timeout

novel osprey
#

i think ill just

#

make it that it uses the same 3 moves

#

just to visualize a surrender

twilit pendant
#

But in my local machine, I timed the imports

twilit pendant
novel osprey
twilit pendant
#

I have no idea

novel osprey
#

but when it comes to the notebook, its sad to see

#

is there paid plans in kaggle lol

twilit pendant
#

But that would not apply to the competition env 😔

twilit pendant
novel osprey
#

I think it might work

twilit pendant
novel osprey
#

So it results in a draw

#

and not a time out

twilit pendant
#

But if your logs is [], it means that a single step is not working

#

the game does not even start

#

and I have no idea why are the logs []

#

the dependencies take 0.1 second to load on my local, how bad can it be in the game env ?? 😔

#

If I add a print statement at the top of my code, even that does not print anything

twilit pendant
latent fiber
#

any advice on how to measure the ammount of memory, compute and data im using?

twilit pendant
latent fiber
twilit pendant
latent fiber
twilit pendant
#

Are you using chess nut?

#

There shall not be any issue if you are using that

#

(also if you are not using other intensive libs)

#

You can go by the kaggle chess env then

latent fiber
twilit pendant
#

If you are using chessnut, you are fine as long as your chess env in kaggle says so(mostly)

#

The env was updated to match the actual constriants a few days ago

latent fiber
#

ah ok so as long as I use the chess env and I dont get an error Ill be fine on the constraints regardless of what I use?

serene swallow
#

I'm honestly kinda disappointed with this competition a bit so far--the current obvious best route is to just clone one of the top engines and remove NNUE/TT/any other tables, switch out -O3 to -Os when compiling, and zip to sub-64kb and sub-5Mb RAM. Just commenting to start the conversation, not sure how can be solved if there's a follow-up competition though and maybe this is due to the competition just beginning (I'm sure it'll get more competitive with engine improvements as it gets closer to ending)

wide oak
#

Lower the limits even more 🙂

#

But the actual answer was for FIDE or Google to consult with literally anyone in the field before making the event. It's clear just from reading the event description that there is a gross lack of understanding of computer chess.

digital anvil
coral wraith
#

Anyone else getting "Unknown Enviorment Specification" when running make("chess")? Works fine for tictactoe

#

NVM - just restart kernel for anyone facing similar issue

coral wraith
vagrant fractal
#

with increment, no ponder, with a cpp/rust library if they still intended on using a fixed movegen, idk. so many things could have improved

pastel creek
#

How would you implement pondering to you engine?

vagrant fractal
#

the only method I know of is using 1 thread to search, and the second one to receive communications at the same time, but I'm sure theres a better method

pastel creek
#

but I thought the CPU is single threaded so threading will not work?

digital anvil
#

single core != single thread

coral wraith
#

Oh is it 1c2t for the competition?

verbal stratus
#

That doesn’t matter, you can run multiple threads on any x86‐64 CPUs, regardless of literally anything else.

verbal stratus
wide oak
# coral wraith Ideally how should the event be set up in your eyes?

I don't know exactly what could be done in order to accomplish Google/Fide's goals of people doing something "original" -- but I know enough to say with absolute certainty that all 5 prize winning entities will be Stockfish, or another already-worked-on-for-years open-source engine.

torpid maple
#

I noticed that kaggle permits different file formats, including compression formats like .gz, .7z, .zip, and so on.

I uploaded a .gz file and it failed with no errors logs , so I compressed to .7z, it failed but this time their were error logs to download, and I noticed that it is trying to load thr 7z file as a script to be ran instead of extracting the contents

twilit pendant
torpid maple
twilit pendant
#

7z gave me errors too

#

I had tried to include the src code of a bitboard based chess move generator in python, although it was below 64kb and it ran fine in the env, submitting the code always resulted in error regardless of any compressions.

#

Actually not even errors

#

The logs were empty

#

Maybe they were not able to load the dependencies

torpid maple
twilit pendant
torpid maple
# twilit pendant thats so close

Bruh, I had to do hardcore and aggressive size optimisation to even get my binaries down to 128KB before using a compression format

When I originally built, it was 520KB

twilit pendant
#

how long have you been working on it?

torpid maple
twilit pendant
torpid maple
twilit pendant
#

the only thing left is compression?

torpid maple
#

There's a discussion about this 7z compression tho, discussion 547145.

I already tried .tar.gz format, and it failed without any error logs even

torpid maple
twilit pendant
torpid maple
torpid maple
bitter socket
#

anyone looking for teammate?

verbal stratus
#

The tutorial teaches how to submit a simple C engine, so if anyone is stuck trying to figure out how to do that, maybe the tutorial can be helpful.

pastel creek
#

If there are random timeouts in the submission(Like when the agent goes from 9.3 seconds to -0.1), does that mean I exceeded the RAM limit?

verbal stratus
sharp salmon
#

Okay so I'm just starting to look at this competition and it seems interesting. Looking at the the chat so far, I'm a bit hesitant to start working on this immediately since it seems there are some gimmicks people have been trying. Specifically trying to just use a barebones stockfish w/o nnue, compress the binary, and find ways to reduce RAM. Is this something serious to think about? Like the top submissions will all be barebones stockfish and some novelties on reducing ram? Should I just wait to work on this project until later in the competition when the rules may change?

languid barn
coral wraith
sturdy basin
#

There are some custom engines in the works. Supposedly they look promising but it's taking time to improve

dusk oriole
# wide oak Lower the limits even more 🙂

@languid barn is there any chance of it actually happening, or does FIDE/Google actually care?

I mean I don't want a precise answer, just an insight into whether any of the obvious concerns about the potential direction of this comp. have anyhow came to the hosts attention?

wide oak
#

I only said that comment because I believe lowering the limits more benefits me and hurts tho who are willing to put in more effort than I am.

#

( But still, lower the limits even more 🙂 )

dusk oriole
#

Haha, yeah, I referenced this as it seemed the only mentioned potential solution - maybe there are others.

wide oak
#

I would bet money that all 5 top submissions will be existing engines, as long as at least 5 people care enough to try.

dusk oriole
#

So I'm just asking if there is any chance of this changing

wide oak
#

What does "this" mean in your sentence?

dusk oriole
#

The first "this" refers to Your comment about lack of understanding from the hosts

#

The second 'this' refers to the current situation

wide oak
#

No one stopping Fide/Google from emailing me, or any other open source author, or anyone on the stockfish team.

#

I don't think there is really anything that can be done to ensure the top submissions are actually "interesting/novel", without just ruthlessly dictating what people can and cannot do. But the irony of the situation here is that the 64kb/5MB limits heavily favour existing approaches. Things like Lc0 don't exist under those conditions.

dusk oriole
#

But the irony of the situation here is that the 64kb/5MB limits heavily favour existing approaches.

Yup, thats what Im talking about

wide oak
#

Lower the limits further, then you favour AB even more. Raise the limits, then you allow existing tech, which makes it even harder for novelties to exist.

dusk oriole
#

AB ?

wide oak
#

Idk. Maybe its just $50,000 to see if anyone will come up with anything interesting. I'm sure someone will. they just won't win.

#

AlphaBeta.

languid barn
wide oak
#

No amount of lowering the limits meaningfully stops someone like me from submitting Ethereal or CFish. Unless you go psycho and down to like 4kb binaries.

#

And if you did go to 4kb binaries... then there exists already an entire class of engines that play at that constraint lol.

dusk oriole
#

I'd say going very low on the computational power fosters innovative solutions the most, because leveraging the efficient usage of CPU has been done for so many years on a world-class level (and almost always won by stockfish)

#

Let the code limit stay the same, but reducing move/game times and memory to even 10x less

wide oak
#

I think that change also helps existing engines.

dusk oriole
#

Would even disable AB

wide oak
#

Ethereal can play super human chess with 1s + 10ms.

dusk oriole
#

Does it do AB?

wide oak
#

Yes

dusk oriole
#

Then lets go for even less

wide oak
#

There is a bit of propaganda from alphazero that misleads people into thinking that modern chess engines are just "brute force" super calculators.

#

Engines are highly selective, and dynamic in their decisions to explore the search tree. Calling them brute force is essentially a pejorative.

dusk oriole
#

Sure thing

wide oak
#

Most of the non-alphabeta solutions are the ones that require greater time and resources.

#

CPUCT engines like A0/Lc0 require massive networks.

#

MCTS engines require memory to dynamically build search trees and not just iterate and toss them.

#

An LLM approach (lol) obviously requires a lot of memory, and probably compute for inference time.

dusk oriole
#

Exactly, how about constant computational complexity? Im thinking rule-based approaches, tabular ones even, FUNCTIONAL models, but ofc optimized locally to find interesting patterns that havent been found before

wide oak
#

Anyway, this is not just me ragging on the rule set. There is a very clear status quo, and forcing something outside that is hard. I'm sure there is a silly analogy to some other domain. If you run a motorcycle competition, you are going to get mostly combustion engine solutions.

#

the definition of constant complexity is bendable.

#

It takes constant time to solve chess.

#

Less silly, it takes constant time to do a depth 10 search, because you can bound it.

dusk oriole
wide oak
#

okay my lack of domain knowledge makes the analogy bad.

#

Maybe my bad anaolgy is the perfect analogy here. Since it seems Google does not understand the domain 😆

#

I just fell prey to the same thing 😦

dusk oriole
#

How do You constrain sth not to search in the move tree?

#

Because I was thinking with another analogy (a closer one maybe) - AlphaGo was a success mainly because it found patterns that no huma knew before, which can now be used by humans, I mean - no tree search (either direct or probabilistic)

proper stream
wide oak
#

anything that can be done iteratively can be done recursively and vice versa.

dusk oriole
#

Im pretty sure this can be done for chess, so ofc seeing the gain somewhere in the tree, but then being able to make a pattern from it, being albe to discover them from just the board setup

#

AlphaGo was a blackbox, right? I mean eventually, because of the computational complexity, no matter the approach actually

#

MCTS/RL/NN as well as simple AB are eventually black boxes

#

Yet what we got was human players know more about tactics

#

Some patterns being so easily explainable even to amateur-class players

#

Can't we get that in chess?

dusk oriole
#

You can actually lock recursion in Python, even disallow using loops pretty easily...

dusk oriole
#

How about that ? No time constrains then, no problem with the unstable environment/hardware

#

Just pure functional responses, no recursion, no loops

#

I'm thinking outloud, IDK, sb please tell me when my msgs are no longer valuable 😛

#

Maybe even just a table+mathatical (and logical) operators

proper stream
#

I'm sure there are lots of ways to introduce programming constraints that could make for an interesting (i.e. not already solved by current techniques) competition. But the question is what FIDE/Google are trying to get out of the competition, and what constraints would help to achieve that.

dusk oriole
#

Well said, the question is if we can rely on the official goal of the competition: to emcourage novel approaches, unseen strategies - thats what they say the're trying to get out of it, right?

verbal stratus
#

At the same time, it’s very difficult to get people who are familiar with chess engines to “think outside the box” to come up with a completely new approach, and people who aren’t familiar at all are unlikely to be able to have good ideas for it in general. I think you either need someone really creative, or maybe someone who has some understanding, but isn’t quite very experienced.

proper stream
#

Agreed. Also, it's hard to develop new ideas in just 3 months, when you're facing a problem that has ~50 years of research already!

minor nova
#

Does anyone know what version of kaggle environments should be used. When I run make("tictactoe") my code works but when I switch to chess I'm running into error saying the environment is known. My assumption is that im not im when I pip install/upgrade im not getting the right version of kaggle-environments, any suggestions would be helpful

languid barn
sturdy basin
#

I think removing the submission size limit and dropping CPU further would lead to very novel solutions

#

I dont want to say what those are in case it ever happens Kappa But it would be quite far from current engines

torpid maple
# sturdy basin I think removing the submission size limit and dropping CPU further would lead t...

Just as one can argue that decreasing the constraints even further can increase the likelihood of introducing novelty, one can also argue that increasing/relaxing the constraints increases the likelihood of introducing some novel approach.

It goes both ways but personally, I believe all this shouldn't matter. I see people complaining about constraints, and how it's either too small or too large. The bottom line is the constraints will not change so suck it up while you still can and tailor your solutions to fit right in.

verbal stratus
#

I think it’s mostly a ubiquituous take (at least among people who have some experience) that making it more constrained would lead to more novel ideas.

#

I don’t think anyone is really advocating that making the constraints looser would be better, at least not without making other constraints tighter.

#

It’s just a matter of figuring out which constraints should be tighter, and I agree with Virenz that throttling the CPU would probably be helpful to encourage novel ideas.

torpid maple
#

At the end of the day, it's all just opinions.

But hey, It doesn't actually matter

verbal stratus
#

Well, it’s not just opinions in the sense that there are sets of constraints that require people to come up with new approaches, and there are sets of constraint that allow people to just reuse existing engines. Though it’s not obvious to which extent Google and/or FIDE want innovation.

#

And if you want for people to actually come up with innovations, this is definitely something that warrants being conversed.

minor nova
#

@languid barn thank you for your help

quasi portal
#

Hi, are people using shared c libraries or is that no possible omly c executables? I am not very advanced on this sorry

torpid maple
torpid maple
# verbal stratus And if you want for people to actually come up with innovations, this is definit...

If you want novel approaches and Innovations that center around minimal compute resource consumption, then yeah, sure strictly enforcing the constraints or even reducing it is the way to go

If you want a novel approach that doesn't necessarily center around minimal resources, but rather performance in the game, relaxing some of the constraints will promote/favour that.

Your approach can be novel and still not fit within those constraints, just as it can also be novel and fit right in.

Relaxed constraints and novelty are not mutually exclusive like you think.

#

As at the time NNUE became a thing, it was quite novel, at least within the context of chess engines, but it's not exactly a good fit if you compare that to bots that can work under the stipulated constraints of this competition.

vagrant fractal
#

the whole issue is that under these constraints, novel ideas will not beat conventional negamax approaches

wide oak
#

Maybe its all fine. It just means that Google will have to look at the 40th best entry to scout for novel ideas, and just ignore the first 39.

torpid maple
torpid maple
wide oak
#

I enjoy the prospect of google getting 100 stockfish submissions.

vagrant fractal
#

I think small things will be found, but larger innovations won't happen

opal bolt
wide oak
#

👏8 👏 opening👏 lines👏 in👏 the👏 book👏

vagrant fractal
#

it will be changed at some point tho

wide oak
#

in my experience, when you start piling up things that will be changed eventually, a handful of them slip through.

torpid maple
#

Is there anyone here able to achieve a depth of 10-15 (assuming you're using minimax or a variant w/o PV search and Alpha-beta pruning)

wide oak
#

Pocket math on a depth 10 search using pure minimax. You explore all nodes. Lets say there are 30 moves per position on average. 30^10 = 590,490,000,000,000. You have about 200ms per move on averge let us say. So you need to search 2,952,450,000,000,000 nodes per second. I believe that is ~3 quadrillion nodes per second? I don't care how much A0 propaganda you've read, engines ain't that brute forced.

torpid maple
wide oak
#

Possible with what? In general? Yes of course, the AB engines do it easily.

# 
$ ./main.out
position startpos
go depth 15
info depth 15 seldepth 19 score cp 16 time 73 nodes 192516 nps 2601000 hashfull 527 pv e2e4 c7c5 g1f3 b8c6 d2d4 c5d4 f3d4 d8b6 d4c6 b7c6 b1c3 e7e6 e4e5
bestmove e2e4 ponder c7c5
#

73 ms to reach depth 15. But "depth" is a very subjective word.

torpid maple
# wide oak ``` $ ./main.out position startpos go depth 15 info depth 15 seldepth 19 score c...

Interesting.

A lot of pruning is really going on here.

I'm working on an AB engine myself, alongside an MCTS engine (just because...) and another that combines AB with iterative deepening for PV search.

The first and last one explore as many as 90k nodes in 2x this time, yet it's not nearly as deep as this (it reaches a depth of 6)

I guess this has something to do with my evaluation function.

torpid maple
wide oak
#

I can easily reach depth 1000 if I just prune everything except 1 move per node.

shrewd drum
#

First things, first. There is a chessboard. An 8 x 8 grid. A 64 dimensional tensor. King, Queen, Rook, Horse, Tower, Pawn. 6 dimensional "profiles" for every dimensional tensor, at any point in time. We cooking now. Thinking out loud.

#

24576 varities of: input layer to LSTM

#

It's a Discrete problem, btw, imho

wide oak
#

and you are already out of memory 🙂

shrewd drum
#

powerful algorithms, as cookie cutters, are approaches to problems, rather than cookie cutter solutions

verbal stratus
#

I can’t claim I have all the answer, but it observably, it seems tightening constraints seems to favor more innovation one way or another, since it makes existing approaches prohibitively expensive (sizewise or speedwise). A simple example is TCEC 4K, on which existing engines simply cannot fit, so new engines have to be written from scratch.

I suspect if there is a CPU speed constraint, that would require novel techniques even more, but of course that is only a suspicion. I think it’s at least fair to say that if you constrain the CPU enough, existing approaches won’t work well at all, though it’s difficult to say that people would be able to come up with anything better.

wide oak
#

the example you gave is another one where the answer still ends up being the existing methods.

#

Okay it sucks to get an engine to 4kb instead of 64kb. but it does not create innovation.

shrewd drum
#

We start here. An ANN with 64 input layers. The Chessboard.

#

Each input layer, a Square on the grid. Of state to Date.

#

A bit of a complex number, perhaps, as each Square encodes, binarily, its Spacetime content. Tis rook or knight, tis, King or Queen.

wide oak
torpid maple
# shrewd drum 24576 varities of: input layer to LSTM

If at all one wants to go down this route, to handle constraints, you'd have to use something more efficient than an LSTM, something like mamba, since the mapping from input to hidden to output is linear, you can perform an associative scan, which will make it a lot more faster than LSTMs

But associative scan consumes more memory than just doing things iteratively.

Also, your technique would have to be very well optimized for CPUs since GPUs aren't an option here.

Personally I'd say it's worth a try, but not for this competition.

mighty scarab
#

Hi I am new and have just started using C++. Has anyone had success finding any existing good C++ bitboard implementations?

I want to use a fast C++ library to turn the fen string into a bitboard and then be able to get all the possible moves and such.

torpid maple
neat sapphire
#

default version fits within the constraints you just have to trim it down a bit if you want to competitive

torpid maple
#

Yeah, true

Thanks

mighty scarab
#

So I could download the files in the src files and then trim it down myself?

torpid maple
#

chess.hpp

#

The one with Almost 4k lines

#

Write your strategy/strategies, when you're done remove the parts your implementation doesn't depend on

mighty scarab
#

Kk, thanks for your help!

torpid maple
#

Sure

merry kestrel
#

I have tried using that library but it seems like it uses too much memory. Could this be true?

mighty scarab
#

It might. I’m hoping I can just remove all the parts of the code I don’t need but haven’t tried yet

shrewd drum
#

You have a board. 8 x 8, or a grid of 64. Each grid point may be occupied by a piece (pawn, castle, horse, etc.), or no piece at all. Each piece , as it is, or as it's moved, changes the board. Does it threathen your opponents pieces, does it protect a piece, is it a barrier. Also, limiting the freedom of the King is paramount.

fresh02.png
#

these may be encoded as binary arrays of each grid point: its piece (6+1 possibles), its characteristics/ current contributions (binary classifications too), to then be used by an ANN to weigh moves (based of learning). What do you think about this approach @wide oak ?

#

C++ is not going to give you an edge. The problem has nothing to do with rapid calculations.

artworks-000145112972-9w4jxc-t500x500.png
wide oak
#

C++ not going to give you an edge over what?
Over everything except C and maybe Rust it certainly will.

#

I have exactly zero thoughts on your hypothetical net architecture that probably won't fit in 64kb.

neat sapphire
shrewd drum
pastel creek
#

Well how will you calculate the best move for a game that you only have ten seconds for?

#

You need to be very fast

shrewd drum
pastel creek
#

But if you use NNUE, you will go over the 64 KB limit

neat sapphire
#

No

shrewd drum
wide oak
#

You can easily use NNUE and be under 64kb.

#

C/C++ are only hard to debug if you are not very smart.

#

RAM is not the limit to the nets, its the submission size.

shrewd drum
#

sorry, i was focused on how much ram my system can use. my bad

wide oak
#

Your net can happily run with 5MB of memory, but you are not putting more than 64kb of weights in. Or 2x/3x that with some lucky/novel compression.

pastel creek
#

But how good can your model be, because to limit into a 64 KB compression size, it will be a small model right?

wide oak
#

You can beat hand-crafted evaluation functions with micro nnues very easily.

shrewd drum
#

well, 64KB is the compressed file submission

wide oak
#

Are they super jaw-droppingly awesome and competetive with SF's NNUE? No.

shrewd drum
#

i can fit 16,384 floats into a 64kb files. I ike them odds

#

More then enough for an ANN

shrewd drum
wide oak
#

feels like an undergrad CS take ngl.

#

debugging c or c++ is not meaningful different than anything else. you have debuggers, you have tools to step through code, to check the values of variables/registers at points, produce repeatadble core dumps, ...

#

Its all the same stuff.

shrewd drum
wide oak
#

Sure. I was never arguing against that.

verbal stratus
shrewd drum
# verbal stratus “Exhaustive” feels like an overexaggeration. But you definitely need some kind o...

You don't need to do a search. You need to encode the board. Every square has either a piece, or no piece. And the piece (or lack of) has some relationship to the other pieces/ squares on the board. So their are 64 input vectors to your ANN, composed of these binary encoding categories. Encoding such details as, a pawn to the diagonal left or the right of a Horse, etc. Or empty space. The system then weighs movements of pieces

wide oak
#

you gonna have to search if you want to place top 25 🙂

shrewd drum
#

I like this project

verbal stratus
verbal stratus
shrewd drum
#

i don't care about the prize on this one

verbal stratus
#

Maybe you could investigate e.g. Lc0, which sounds like what you are describing (i.e. networks that can play above human grandmaster level on policy alone, without search), though you probably won’t be able to train a good enough network that fits in 64KB. I was told by one of the Lc0 developers that he is looking for someone to figure out how to train a small network to replace their “trivial” backend, and he wants to use this competition as an incentive for it.

shrewd drum
#

so, as input to an ANN, i can include, grid, piece (or lack), relationship to the whole (protects a piece, backed by a piece, threathens, sacrificial, etc.) and map this to smart moves without searches

#

all inputs being binary vectors

verbal stratus
#

Normally neural networks for chess just use one hot encoding (i.e. one input for each piece type and for each square) and let the network figure it out.

#

That is probably better than trying to encode heuristics ultimately.

shrewd drum
verbal stratus
#

Well, not “its impact” in any way. But yes, just boolean vectors.

shrewd drum
#

Impact; a piece in square may have an impact: encoding that as booleans too, is what i meant: checkmate is the big one 🙂

verbal stratus
#

Well, sure, you can explore novel ideas like those, but I’m just describing how existing strong engines go about it.

wide oak
#

lc0 without search is super human, although terrible in engine terms. but that requires 100+MB nets.

shrewd drum
# verbal stratus Well, sure, you can explore novel ideas like those, but I’m just describing how ...

Sure. Now look at it this way: you move a piece: it may back up another, be supported by another, threaten your opponent, be used as a sacrifice to save another piece, etc. These can be booleans to describe a piece. The vector resultant is rich with meaning. What it is and what it can do. The board is a landscape of these. Every square is a piece, absence of piece being a piece too. An ANN gets fed these vectors (64 sets of booleans) and learns to weigh next moves. No search involved.

verbal stratus
#

Well, in non‐“size constrained” engines, the network can just learn nuances of the position without needing to have it be part of the input.

proven quarry
#

I am looking at a team

torpid maple
# shrewd drum Pointers are nightmares

Well, you'll get used to it. I'm guessing you're just new to C++ that's why.

If you can't properly handle raw pointers or are too lazy to always malloc and free, I suggest you go for smart pointers (assuming you're using C++)

Shared pointer, unique pointers and weak pointers.

I use them a lot these days, in my MCTS implementation, I store shared pointers of children nodes and each node holds a pointer to their corresponding parent, to avoid circular referencing, use weak pointers for parents (they only reference but cannot own an object)

#

That being said, you should probably avoid pointers when they're not needed.

torpid maple
torpid maple
torpid maple
neat sapphire
torpid maple
#

I must've read that wrong

shrewd drum
#

A grid square on a Chess Board, is defined by the piece on it (or lack thereof). And that piece, threatens and/or supports other pieces. Or threatened or supported given current board state. Also, it commands space, as potential next move of occuying piece (if there is one). It also affects the opponents King's freedom of latitude in movement, at all times. And a checkmate denies this. 🤔. @torpid maple , @pastel creek , @wide oak , @verbal stratus , what do you think of this?

twilit pendant
shrewd drum
verbal stratus
shrewd drum
twilit pendant
shrewd drum
twilit pendant
#

but if you post it in the official discussions forum a lot more people can see 🙂

shrewd drum
#

So there is also 'Command of Space" for a grid square, Qafig. It represents all grids a piece (in that gird) can move to, given a turn. So every grid square (with piece) becomes defined as threatens/ supports, commands space, supported, threatened (which can be encoded as booleans) , when next move potentials are weighed and passed to an ANN. And all focused on cutting off the King's freedom of movement, under attack (a checkmate). This is the vision.

twilit pendant
twilit pendant
shrewd drum
twilit pendant
#

One particular problem in representing a chess board for input to NNs is how would you represent a grid

#

idk tho

#

maybe it might work out with proper selection of inputs.

#

also, what shall the model output?

#

or maybe I am not getting what you are trying to say

shrewd drum
shrewd drum
twilit pendant
shrewd drum
shrewd drum
twilit pendant
#

evolving values?

#

I think to make such an nn work, you need multiple layers with tons out parameters

#

also the output of the NN would be a move?

twilit pendant
#

how will an ANN output chess moves(like what is the format)?

shrewd drum
#

Just out of curiousity. What's your take on some of the ideas I've dropped

#

and anything else

shrewd drum
shrewd drum
twilit pendant
#

To tune such a complex neural network I imagine you would need to train it for millions of games.(unless you train it on data of chess matches, which is better but still)

quasi portal
twilit pendant
#

I am not sure tho

#

Maybe I am wrong

quasi portal
twilit pendant
quasi portal
#

I mean definately not good for this competition constraints anyways

quasi portal
twilit pendant
twilit pendant
quasi portal
twilit pendant
quasi portal
twilit pendant
quasi portal
#

what it thinks is the best move

twilit pendant
#

but if we pass the next state as input, it would generate the next to next move

quasi portal
#

sure and then evaluate that one, similarly to minimax ig

twilit pendant
quasi portal
#

but you can have an evaluation function at the end of the iterations

blazing sedge
torpid maple
#

Maybe there are NNUE networks that are smaller than that anyways... with reasonable accuracy even after being quantized

However the RAM itself is not the major issue with using a neural network for this, it's the submission size that is the problem, compressing millions of weight params into a file less than 64KB is hell...

pastel creek
#

Also with the NNUE, I am under the impression that there has to be an engine that takes those evaluations of positions into account to make the best move.

blazing sedge
#

and that is exactly what will be happening

#

and I am pretty sure the trainers know about quantization

#

all that is happening is you are turning floats in [0, 1] to ints in [0, n]

blazing sedge
#

when I say you can fit 95% of the weights in i8 without accuracy loss I mean that the usual i16 quantization process which takes weights from (-2, 2) to (-512, 512) or so results in usually around 95% of them in the range (-128, 127)

torpid maple
torpid maple
#

Well, there's always IEEE 754 half precision compression format

Not like it has an edge over quantization

torpid maple
#

For 50k params with i8bits quantisation, your weights would be atleast 50KB

Atleast coz there's usually extra serialisation data as well.
So you'd have to build your code binaries to be less than 14KB, or even less

torpid maple
stone otter
#

Is it possible to return algebraic notation? (a3) instead of (a2a3)

verbal stratus
#

No.

clear swallow
pastel creek
#

If we use the Chessnut library to generate legal moves, does that count toward the RAM limit?
Would it be better if we had our own move gen?

ionic escarp
pastel creek
#

Does that mean it doesn't count toward RAM usage or not?

shrewd drum
languid barn
shrewd drum
languid barn
#

As far as the question of "should I my make own move gen" that depends on how often you are calling it and how much you think you could improve on the existing.

wide oak
#

What have you raised the RAM limit to, as the (hopeful) stop-gap on better limit checking.
People are reporting using quite a bit

#

Your comment a few minutes ago is 1MB. But I saw claims of people using 8MB+.

shrewd drum
languid barn
vagrant fractal
#

why?

obsidian spoke
#

can we raise RAM by another 3MB 🥹 😛

vagrant fractal
#

please no

#

was a vote held before this change was made?

wide oak
vagrant fractal
#

so it should be reduced back when the RAM usage is enforced better?

languid barn
neat sapphire
#

while we investigate RAM usage issues
so this means the environment itself has some issues, not strictly related to the agent?

slate sundial
#

It's pretty great when you fix a small bug in your code and go from 140th place to 20th place

languid barn
slate sundial
#

Hoping my new submission goes like this (removing ankle weights for more power)

#

it might reach the top 10 this time

slate sundial
#

Just made it to 9th place- i can’t believe it

verbal stratus
#

Very cool! 😄 Congrats!

quasi portal
#

Hi does someone know if they added obs.LastMove?

languid barn
viral inlet
#

you should also consider using a (much) larger book. should be easy to do, and it will improve the significance of the leaderboard no end

vagrant fractal
pulsar aurora
#

Hi everyone
I just wanted to try a sample submission with an existing engine but my submit fails... do you know if there is a way to get log or understand why the submit fails even if everythings seems to work on my notebook ?

merry kestrel
#

Most common reason is that it takes too much memory, or the directory you are loading the engine from is wrong (directory is agent_simulations if run outside of notebook(

pulsar aurora
#

@merry kestrel so the path of the directoy is not "/kaggle_simulations/agent/" but "/agent_simulations" or "/agent_simulations/agent", right ?

wide oak
# 
engine_name = "main.out"
engine_file_path = "./" + engine_name

if not os.path.exists(engine_file_path):
    engine_file_path = "/kaggle_simulations/agent/" + engine_name
pulsar aurora
#

ah ... so my file was already managing this... so I suppose it is linked to the ram usage .... Will check your link, thanks 🙂

slate sundial
#

Does anyone have a good way to score their agent locally?

verbal stratus
#

c-chess-cli, fastchess, cutechess

quasi portal
#

Hi does anyone know which cpus are or will be used to test the engines? (or where to look)

slate sundial
mighty scarab
#

I have a C++ bot and I am trying to submit. My bot runs locally with my main.py file but when I submit it I get this error and I don't know how to interpret.

Anyone else run into anything similar and figure it out?

Screenshot_2024-12-22_at_12.17.15_AM.png
verbal stratus
mighty scarab
mighty scarab
verbal stratus
#

Ah, that makes sense. You need to compile on Linux (with a glibc old enough to match the one in their container). Or conversely cross‐compile, but that would be less straightforward.

neat sapphire
mighty scarab
neat sapphire
#

you sure ? using g++ on mac is a bit unlikely

#

i assume it is actually clang 15

mighty scarab
#

Ohh, looks like you're right. Clang 15

neat sapphire
#

well and i ran it like you would run any program, ./myprogram

#

and compiled on ubuntu 20, to have an old glibc/cxx dependency

#

and uploaded the compiled program

mighty scarab
#

Should I get a virtual machine to run ubuntu 20?

opal bolt
#

@languid barn Have you looked at https://www.kaggle.com/competitions/fide-google-efficiency-chess-ai-challenge/discussion/550339 ? I hate being intusive and pinging you directly, but I assume you are the main moderator of this competition. Please look at this. The submission placement is currently all over the place, the swings are +- 100 elo, and I don't mean the initial ramp-up. A whole day after submission our engine can place 2nd, or it can place 6th, in a matter of few hours.

#

It is a fundamental flaw of how the elo (score) is being calculated

neat sapphire
mighty scarab
neat sapphire
#

okay well not so much cross compilation just change the compilation target to be x86-64 and hope that glibc will work

verbal stratus
#

I doubt Mac has glibc at all.

#

Probably the ELF headers are different too.

sullen nimbus
#

Is there a way to measure RAM usage locally?

mighty scarab
shrewd drum
quasi portal
pastel creek
quasi portal
pastel creek
dire pond
#

After downloading new version to Fide Chess , I got Error=Validation Episode failed, size of main.py is 7kib,When clicked on notebook name I got the replay that ended in mate and following text: Agent exit status/reward/time left:
DONE / 1 / 1.689153000000002
DONE / 0 / 5.775771 . Need help!

terse gull
#

has anyone managed to get stockfish to fit in the RAM constraints

proven quarry
#

no

quasi portal
quasi portal
shrewd drum
quasi portal
shrewd drum
#

It's not gonna happen. Stockfish uses massive search trees to analyze postions and come up with a move. It's not gonna fit in a 64KB file, even compressed.

quasi portal
#

stripped down version of it, and without the nnue version it can definately fit

quasi portal
slate sundial
#

Has anyone gotten NNUE to fit within the filesize and memory constraints?

wide oak
#

one could say that all engines that have a PSQT, have an NNUE. So sure.

slate sundial
#

Mainly curious if anyone has compressed the NNUE weights and biases file without reducing the skill level below that of stockfish classic

terse gull
#

does anyone have the source code for what obs actually is

pastel creek
quasi portal
#

Hi does someone know if they are enforcing correctly the 5Mib RAM? Im pretty sure my engine is using more but it is working

slate sundial
#

I think I've reached the best solution I can while having 0 chess knowledge. If any chess masters want to team up, I might be open to it

verbal stratus
#

You don’t need to have particularly good chess knowledge to come up with a good engine.

#

Knowing how to play chess well and knowing how to make a good chess engine are almost entirely orthogonal skills. There is almost no overlap.

quasi portal
verbal stratus
#

I suppose, but ideally you’d be looking into Elo test results, rather than individual games.

quasi portal
#

hi can someone help: have people managed to ponder, like to have their compiled engine think during the opponents turn?

#

im really struggling with it :(imgoinginsane

merry kestrel
dire pond
#

when tried to test all possible positions when when starting with 2e-e4, I have 20 possible moves, so im using two nested for loops , its like for every 20 moves I have 20 moves for every initial move, when submitting this logic I have an error, but if I limit every loop to 10, I can submit , but the moves are not so strong.

solemn night
#

unless it's some sort of PUCT

quasi portal
#

Hi does someone know if to promote a pawn you send for example c7c8q or c7c8, I am having issues when sending a move to promote and receiving obs['lastMove']. Like can you promote to a knight?

verbal stratus
#

The (lowercase) letter of piece you want to promote to is attached to the move name.

#

q -> queen, r -> rook, b -> bishop, n -> knight

#

E.g. c7c8n to promote to a knight.

smoky musk
#

Hi, is the CPU used by kaggle for validation too slow compared to a normal laptop? On local machine the agent uses 2-3 secs to finish a game but runs out of time (10 secs) in validation within a few moves. No use of threads.

#

Wanted to check if others are seeing it too. It seems 5-6x slower than a laptop cpu core

stone otter
#

Hi, I'm trying to submit a C agent, this is my main.py

from subprocess import Popen, PIPE
def main(obs):
    print("Starting main", obs.board)
    process = Popen(['./main.out', obs.board], stdout=PIPE)
    stdout, _ = process.communicate()
    return stdout.decode('utf-8').splitlines()[-1]

But the submission errors out before running the validation game, this is the error I get

[
  [
    {
      "duration": 0.001298,
      "stdout": "",
      "stderr": "Traceback (most recent call last):\n  File \"/usr/local/lib/python3.10/dist-packages/kaggle_environments_chess/agent.py\", line 44, in get_last_callable\n    code_object = compile(raw, path, \"exec\")\n  File \"/kaggle_simulations/agent/main.py\", line 1\n    PK\u0003\u0004\u0014\n    ^\nSyntaxError: invalid syntax\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/usr/local/lib/python3.10/dist-packages/kaggle_environments_chess/agent.py\", line 169, in act\n    action = self.agent(*args)\n  File \"/usr/local/lib/python3.10/dist-packages/kaggle_environments_chess/agent.py\", line 135, in callable_agent\n    agent = get_last_callable(raw_agent, path=raw) or raw_agent\n  File \"/usr/local/lib/python3.10/dist-packages/kaggle_environments_chess/agent.py\", line 65, in get_last_callable\n    raise InvalidArgument(\"Invalid raw Python: \" + repr(e))\nkaggle_environments_chess.errors.InvalidArgument: Invalid raw Python: SyntaxError('invalid syntax', ('/kaggle_simulations/agent/main.py', 1, 1, "
    }
  ]
]

It's apparently something to do with my python file? I compress both main.py and main.out in a bot.7z file

bot.7z
- main.py
- main.out

I can't see what I'm doing wrong

verbal stratus
#

People have reported not being able to use 7z. Instead, gzip a tarball, and it should work.

#

Also, note that the file name in the actual Kaggle environment can’t be relative like you expect with ./main.out.

stone otter
shrewd drum
quasi portal
shrewd drum
quasi portal
slate sundial
quasi portal
#

Oh then no. I think training your own smaller nnue would be a better apporach no?

quasi portal
slate sundial
#

I wonder how many of the teams in the top 20 are hardcoding for the 8 openings used in the dev phase leaderboard

quasi portal
#

I dont think that many actually. It doesnt matter that much anyways

neat sapphire
wide oak
#

because of how insanely dumb it it, I'm guessing at least a few people are doing it LOL

slate sundial
#

The 1st place team has been pretty consistent for the past few weeks, I wonder what they're doing differently than everyone else

bright cradle
bright cradle
#

Are custom bitboard functions more efficient than ChessNut library ?

wide oak
#

They are simply repeating the same methodology that anyone would do for engine development.

bright cradle
#

Can't we use stockfish engine ?

quasi portal
wide oak
#

openbench, fishtest, playing local games with cutechess/fastchess/c-chess-cli, whatever.

#

there is a very clear, already beaten path, to iteratively improving a chess playing entity.

#

However, based on the leaderboard, I only thing two teams are actually doing it.

#

The rest are just varying degrees of chopped up SF, some doing a better/worse job than others, IMO.

steep estuary
#

in the current state of the competition the "conventional path" is overrated and perhaps not worth the effort

#

if you are at the point where you need to test for fine grained changes then you should already be placing very very high (roughly be in contention for 1st place i'd say)

steep estuary
quasi portal
steep estuary
#

well, you can't blame partecipants for going with the most obvious route

#

given the constraints there isn't much in the way of ideas that isn't covered by tradtional engines and given the lack of constraint using the best engine you can get your hands on makes sense

wide oak
#

have to start somewhere. starting with sf beats every other starting point. foolish not to. even if you rip much of it away and add more in

solemn night
#

surprisingly enough, last we tested.. the (8 openings) "book" almost didn't compress a progression test Elo gap obtained on a lichess UHO books..
I suspect it might be due to how weak the engine is compared to a top modern chess engine.

#

that being said, I wish more openings would be added anyways

steep estuary
#

if this is the complaining about things moment i'd really like 1mb less of ram and increments

#

and if i'm allowed to dream BayesElo wouldn't be terrible

wide oak
#

I believe we have increments now.

#

At least, my only update to Ethereal in a month is that, after it was posted about having been changed.
A long with the 6m. "Temporary" ram limit

steep estuary
#

which introduces the extra ram and doesn't mention real increments

#

you might be thinking of the 3fold change

#

or i'm just working off of outdated info

bright cradle
#

anyone tried/trying Sunfish NNUE version ??

solemn night
#

It's impossible to fit without making your own nnue arch

bright cradle
solemn night
#

Omg I'm blind.. I saw it as stockfish

#

Sorry nvm

#

Never heard of it

mental flume
#

if you're going to take a pre-existing engine, just do what the people at the top of the leaderboard are doing and take stockfish/cfish/etc and strip them down

quasi portal
wide oak
#

to learn, the best way is to forget about the competition, and just begin your engine dev journey.
Your first step is PERFT completeness 🙂

lunar inlet
#

I'm trying to submit multiple files ... zipped 48kb ... contains main.py and model.pkl ... but I keep getting validation errors

[[{"duration": 0.001201, "stdout": "", "stderr": "Traceback (most recent call last):\n File \"/usr/local/lib/python3.10/dist-packages/kaggle_environments_chess/agent.py\", line 44, in get_last_callable\n code_object = compile(raw, path, \"exec\")\n File \"/kaggle_simulations/agent/main.py\", line 1\n PK\u0003\u0004\u0014\n ^\nSyntaxError: invalid syntax\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/usr/local/lib/python3.10/dist-packages/kaggle_environments_chess/agent.py\", line 169, in act\n action = self.agent(*args)\n File \"/usr/local/lib/python3.10/dist-packages/kaggle_environments_chess/agent.py\", line 135, in callable_agent\n agent = get_last_callable(raw_agent, path=raw) or raw_agent\n File \"/usr/local/lib/python3.10/dist-packages/kaggle_environments_chess/agent.py\", line 65, in get_last_callable\n raise InvalidArgument(\"Invalid raw Python: \" + repr(e))\nkaggle_environments_chess.errors.InvalidArgument: Invalid raw Python: SyntaxError('invalid syntax', ('/kaggle_simulations/agent/main.py', 1, 1, "}]]

Code looks like this

` with open("/kaggle_simulations/agent/model.pkl", "rb") as f:
print("Starting model load...")
model_state = pickle.load(f)
if isinstance(model_state, dict):
model.load_state_dict(model_state)
else:
model = model_state
print("Model loaded successfully!")
except Exception as e:
print(f"Error loading model: {e}")
return model

model = load_model()
model = model.to(device)
model.eval()

def chess_bot(obs):
"""
Chess bot using the simplified board representation.
Returns the best move in UCI format (e.g., 'e2e4').
"""

# Get FEN and color from the board
fen = obs.board

`

bright cradle
#

got similar error

shrewd drum
#

Here’s the streamlined general approach:
1. Upload Data: Prepare your data and create a private dataset on Kaggle.
2. Access Data: Use the dataset path (/kaggle/input/...) in your notebook to load and use it.

Done! Simple and direct. 😊

lunar inlet
shrewd drum
wide oak
#

It do be as simple as following that post.

#

If not running a binary, then you now see the import/untarball structure.

lunar inlet
# shrewd drum Let's start more simply. What Are you trying to acheive, exactly?

submitting my agent. My compression has two files main.py and model.pkl ... of size 48kb ... I load the pickle file in the main file like this

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")    

def load_model():
    import sys
    # Register the class in the main module
    sys.modules['__main__'].NeuralNet = NeuralNet

    model = NeuralNet()
    try:
        with open("/kaggle_simulations/agent/model.pkl", "rb") as f:
            print("Starting model load...")
            model_state = pickle.load(f)
            if isinstance(model_state, dict):
                model.load_state_dict(model_state)
            else:
                model = model_state
            print("Model loaded successfully!")
    except Exception as e:
        print(f"Error loading model: {e}")
    return model


model = load_model()
model = model.to(device)
model.eval()
shrewd drum
lunar inlet
bright cradle
#

excuse me guys theres a Stockfish_ubuntu model in kaggle but i m unable to use it in a notebook

#

its saying permission denied

shrewd drum
#

Submitted code would access weights at runtime, and plug these into neural net

#

The 64kb limit bites. I get why: build a minimal system; but it bites, imho

bright cradle
#

getting this error

image.png
#

maybe cause its too slow ?

shrewd drum
#

works properly is a bit vague, but perhaps it is things such as trying to make illegal moves for a piece, etc.

lunar inlet
bright cradle
#

timeout most probably

pastel creek
bright cradle
shrewd drum
# bright cradle ok n i m gettting

OK. It's telling you Validation Episode failed, for your failed submission. Figure out how to get one that doesn't fail, to get over this hurdle. Relax. Until final date to submit, you can submit imperfect solutions. During last days of competition, leaderboard is reset, and submissions are locked.

shrewd drum
shrewd drum
gritty void
#

It is unrealistic to develop a complete and innovative chess engine within two months. From this perspective, this competition is actually tailored for current chess engine developers. However, based on the current situation, it seems that not many chess engine developers are willing to spend time participating. Is there any specific reason for this? For example... difficulties with debugging?

wide oak
#

10k prize money is not very much, if you are talking about employed software developers in the US.
There is also not particular utility in the results. As soon as this ends, every dev will rm -rf their working dir.

bright cradle
steep estuary
#

and the way the competition is structured doesn't leave much space for not doing that, unless you are willing to work at a severe disadvantage

#

it's more an excercise in compression and code rewriting compared to actually improving on modern day chess engines, it's just boring

bright cradle
steep estuary
#

you could have 1m as the first prize

bright cradle
#

(but i g there's no way to give feedback to them like they improved a Lil bit Ram constraints)

steep estuary
#

and it would still be about cloning sf and cutting down its search

#

more money means more partecipation but not more innovation

bright cradle
steep estuary
#

you'd have to either accept the fact this is about nnue engines, and force everyone to use the same default one, or make things very interesting and cut down on ram so much there's no chance to fit SF

#

1MB or less

wide oak
#

I'll still fit SF in 1Mb ngl

bright cradle
wide oak
#

I mean, it won't be functionally equal, but it would be something.

steep estuary
#

at that point the work you'd have to do is equal or more taxing than doing something from scratch

wide oak
#

but all of giuseppes points holds. Why do I give a hoot to work on some random code base thats not mine

steep estuary
#

but ig the only real solution is moving onto Ataxx

#

chess is flawed because the design space is incredibly well explored, that's all

wide oak
#

The text of the competition points out quite clearly that FIDE was unaware how explored it was.

steep estuary
#

fwiw my preferred solution would be having a base engine and then it all becoming about NNUE, it makes more sense in the context of a ML competition

#

Yes, they'd have greatly benefitted from consulting with an actual dev prior to this

wide oak
#

Yup.

#

NNUE training only event would be intersting. You would get some competition about novel archs paired with novel compression techniques.

bright cradle
steep estuary
wide oak
#

alphazero is dogwater in comparison to modern engines.

steep estuary
#

A0 loses to the top submissions of this exact competition

#

let alone "unrestrained" modern top engines

wide oak
#

Little bit of free money, probably for a handful of chess devs for very limited amounts of work, and thats about it. Its a net good, but its not all that much to write home about.

steep estuary
#

it is overall a net positive for an hobby that is notoriously populated by people starving for hw, and i think something interesting might come out of it, so it's definitely not all bad

#

thanks fide, if you ever want to do a nnue only one i'm available as a consultant.

dire pond
#

Im struggling with slow code execution, ** get_moves** and apply_moves are the slowest to execute, is there a way to use another faster function too get legal moves and then apply moves, Can I use another chess engine, like** Chess** instead of** Chessnut**?

bright cradle
quasi portal
dire pond
dire pond
shrewd drum
#

@dire pond , A possible solution: Refined Mobility Maps

The refined mobility map is a crucial concept for determining where a chess piece can move, factoring in the constraints of the current board state. This process (below) is computationally efficient and involves two primary steps:

Step 1: Generate the Unrefined Mobility Map
• What it is:
• A precomputed lookup table for each type of chess piece.
• Contains vectors representing all potential moves a piece can make, unrestricted by obstacles.
• How it works:
• For a piece at a given square, retrieve its possible move vectors based on its movement rules (e.g., a bishop’s diagonal paths).
• This map assigns a value of 1 to squares the piece can potentially move to and -1 to all other squares.

Step 2: Calculate the Refined Mobility Map
• What it is:
• A dynamically calculated map that modifies the unrefined mobility based on the board state.
• How it works:
1. Iterate through each vector from the unrefined map:
• For example, a rook’s horizontal or vertical movement vectors.
2. Determine constraints along each vector:
• Stop if a friendly piece is encountered.
• Include a square if it contains an opponent’s piece but stop further exploration along that vector.
• Respect the edges of the board as natural boundaries.
3. Assign refined values:
• Replace -1 with 1 up to the constrained index along each vector.

#

Optimization of the Algorithm
• Initial State:
• Start with a 64-dimensional vector for the board, initialized to -1 values.
• Efficient Processing:
• Use the unrefined map’s vectors as guides, reducing unnecessary checks.
• Dynamically calculate the refined mobility by:
• Iterating over the vector only until a constraint is met.
• Minimizing operations by leveraging precomputed patterns.
• Final Output:
• A refined 64-dimensional vector, with 1 values marking the squares the piece can legally move to and -1 elsewhere.

#

The system assumes all values for 64 dimensional vector are -1. It determines where to add ones, based on a piece vectors. It includes just the ones that are legal moves.

glossy nest
# lunar inlet my agent logs is not helpful...

Have you found a solution to this? I've been having the same issue. Only the Replay has any data to analyze but it doesn't look like the replay information is good for debugging the error.

dire pond
# glossy nest Have you found a solution to this? I've been having the same issue. Only the Rep...

try clicking on "Agent 0 logs" you must something like this: [[{"duration": 0.341105, "stdout": "Elapsed time: 0.3017001152038574\nbest move= d2d4\n", "stderr": ""}],
[{"duration": 0.348884, "stdout": "Elapsed time: 0.3420417308807373\nbest move= b1c3\n", "stderr": ""}],
[{"duration": 0.333992, "stdout": "Elapsed time: 0.3260173797607422\nbest move= c3a4\n", "stderr": ""}],
[{"duration": 0.345103, "stdout": "Elapsed time: 0.3381059169769287\nbest move= a2b3\n", "stderr": ""}],
[{"duration": 0.443004, "stdout": "Elapsed time: 0.43607473373413086\nbest move= a4c5\n", "stderr": ""}],
[{"duration": 0.418567, "stdout": "Elapsed time: 0.41050052642822266\nbest move= c1f4\n", "stderr": ""}],
[{"duration": 0.485729, "stdout": "Elapsed time: 0.47602176666259766\nbest move= b3b4\n", "stderr": ""}],
[{"duration": 0.421666, "stdout": "Elapsed time: 0.41419196128845215\nbest move= a1c1\n", "stderr": ""}],
[{"duration": 0.411412, "stdout": "Elapsed time: 0.4040870666503906\nbest move= c5a6\n", "stderr": ""}],
[{"duration": 0.340503, "stdout": "Elapsed time: 0.33300304412841797\nbest move= a6c5\n", "stderr": ""}],
[{"duration": 0.903928, "stdout": "elapsed: 0.9038302898406982\n", "stderr": ""}]]

glossy nest
dire pond
torpid maple
torpid maple
terse gull
#

yo

#

if i get timeout as a message on the very first move does that mean that im OOMing

#

or is another error being thrown for OOM

#

my submission plays fine when i run it on a kaggle notebook but it doesnt seem to play a single move in the validation test

#

it just times out

verbal stratus
#

Yes, timeouts can indicate RAM limit issues.

pastel creek
slate sundial
wide oak
#

I think I probably spent around 20 hrs at this point for Kaggle. So for that investment, 10k is great. $500 an hr. But to invest another 80 hrs? 180 hrs? yuck.

solemn night
#

probably the competitive part plays into it also, some try to make the most of it if they are already in a competition
for me personally as someone who is involved in chess engines if I wouldn't know I would score good I probably wouldn't have submitted
others might do it for fun or probably as an opener to chess engines world

#

I think that my teammate already got into chess engine development because of a previous "similar" competition on youtube

gritty void
#

Many Kaggle competition participants are not in it for the money; if they were relying on the prize money to live, they'd have starved by now. Most people are in it just for fun or learning. For this competition, developing a chess engine sounds pretty cool to those who haven't been involved in this field.

wide oak
#

Yeah I don't disagree with that. The question I responded to originally was "why are engine devs not interested". And for them, its a drag-and-drop for their existing hobby, with marginal gain.

blazing sedge
#

NNUE compression alone would be more interesting

#

lobotimizing Stockfish is much less interesting

solemn night
#

We've expanded the possible opening positions to include 2000 different openings! This will be the set used during the final evaluation unless there are significant bugs or issues.
I think there is an issue here, if the set is the same one as the final one because pre-prepared book moves and hardcoded solutions would emerge..

#

although it could still be very hard to achieve...

red lark
#

Guys quick question, what's better in terms of speed/ram. a subprocess from the main python script, or an imported extension module?

#

Or if there's a third option please do let me know

gritty void
steep estuary
# torpid maple Pretty possible

it was less of a comment about feasibility and more of a comment about the fact it would be terrible.
You can do it, it's gonna perform terribly at vstc, with those constraints, against nnue + a/b engines

#

Note that i'd absolutely love to be proven wrong and for the top 30 entries to not be all A/B + nnue

quasi portal
#

Hi, does someone know if it is better to input starpos <moves> or the fen directly?, at the start i feel just moves is better, but when there are like 100 moves fen would be better right?

quasi portal
#

How is this a good starting position DOWN A BISHOP😭 I think they got too unbalanced positions.. Stockfish rates it -4.8 lmao

1cdb2c91-f341-4728-8214-1420d51b56fa.jpg
steep estuary
#

if not you can just parse the latest fen

#

the actual parsing of the moves itself from the command isn't (or at least shouldn't be) particularly taxing compared to everything else that goes on

dire pond
#

Hello, where can I find the instruction how to use external modules in FIDE competition.I have to copy them to Kaggle? How? Thanks

red lark
steep estuary
red lark
#

What's the point of competing with something you didn't build?

#

other than potential prize money if it's actually good

steep estuary
#

for most people? having an entry with a chance to place at a decent spot

#

if 20 people re-use SF and you have to write something from scratch, your theoretical best placement is 21st

#

if 50 people re-use SF, well, good luck

quasi portal
solemn night
#

Well beginning with some SF clone can probably get you top 5 but there is a chance that the pain of being randomly getting excluded from top 5 because of random luck and unstable ranking would overweight it.
We have a +250 tracked commit on top some clone.

steep estuary
#

now imagine not starting from a ready-made engine but having to read up on how to write movegen, good luck competing against that

red lark
#

Well at the very least you can look up a good chess library for everything related to the game itself

solemn night
#

most libraries are too big tho.. tailored functions would be needed eventually..

#

People try to remove dependancy on standard libraries

steep estuary
#

you can see SF as a library if you squint hard enough

gritty void
# red lark Well at the very least you can look up a good chess library for everything relat...

In competitions hosted by communities like Kaggle, this kind of situation is inevitable. As a Kaggle user (like me), the goal is to "climb the leaderboard" using any legitimate means, and copying Stockfish (SF) might be the best approach. Of course, during this process, one is forced to learn a lot of knowledge. Personally, I believe that if you want to start developing your own chess engine, it's best to first become proficient at using someone else's engine.

graceful flower
#

Anyone who has successfully implemented NNUE? I’m working on ethereal and disabled the nnue and tbprobing but the engine is not performing well (elo 1600)

gritty void
#

So you must have done something wrong. If you think the error is with NNUE/HCE, you can go back and cross-validate with an earlier version of Ethereal that hasn't implemented NNUE yet, to pinpoint where the issue is.

wide oak
thorny loom
quasi portal
wide oak
#
  1. I don't know. I refuse to actually measure pondering elo values since its troll.
  2. This hardware / config is a real unknown. So lots of noise there.
  3. Does it really take that long to "stop pondering"? I don't think so.
  4. If you have a good ponderhit rate, then the stop comment is null.
  5. Strictly its gotta be > 0 elo right?
quasi portal
#

It will be a very likely outcome between top engines, like its almost useless I feel

red lark
#

Wdym waiting to stop? you can just interuppt it

quasi portal
wide oak
#

There is no public Ethereal NNUE. So yes I mean Ethereal HCE.

#

I don't understand :

It will be a very likely outcome between top engines, like its almost useless I feel

quasi portal
#

"Does it make sense to include such unbalanced positions? Given the relatively small differences in engine strength among the top few hundred competitors, starting a game with such a disadvantage often makes the outcome almost inevitable. While unbalanced positions are often used in engine development to avoid draws, to my best knowledge they typically fall within a range of [-200, +200] centipawns. Positions like the one above fall far outside that range."

wide oak
#

I thought that was in response to me.

Yeah I don't disagree. Although for the weak engines, + a bishop might not matter. So who is to say all openings should cater to the top of the board.

red lark
#

If they made it so the elo gain is affected more highly when losing/winning with an unbalanced starting position it can be a trick to speed up the tournament and discern the top from the rest

quasi portal
wide oak
#

Game-pairs are played ( in theory ), so it just becomes a drawn pair, and its fine. It just compresses the result a bit.

However, I notice often that 1 but not both of the games error out on both sides. If that occurs, then there are problems. But Its unclear if that will get fixed before all is said and done.

So you are wrong, but also right.

quasi portal
quasi portal
red lark
#

This unbalanced starting position can cause trouble if the positions chosen for games are random, someone getting that position more often is more likely to draw the match (1-1), which can either boost him down or up unfairly from his real level

#

Overall, i don't think it matters that much either way

wide oak
blazing sedge
wide oak
#

If you make a competition for a domain that has zero already established best practices ... then maybe no one will care in the first place. So its almost inevitable that any competition results in tinkering on-top of state of the art practices.

slate sundial
#

I think it's impressive that atom123 was able to reach top 5 without having worked on chess engines before- we have similar approaches to this competition but I only reached 8th place at best, and now dropped to 20th place since I haven't found any improvements over the past few weeks of tinkering, although I have found many different ways to break the chess engines