Brotato SDK Integration v0.1

[twin temple]

I started work on implementing the neuro-sdk into Brotato. The goal of this is to make it possible for neuro to fully complete and play Brotato. Future ambitions include also supporting Coop but that is rightnow a bit out of scope.

the project can be found at: https://github.com/Adesii/Neuro-sdk-Brotato-Integration

The current state is basically just the Connecting to the sdk and getting through the initial menus.

No actual gameplay or gameloop works yet. But im confident it will work based on my ideas.
Movement during stages will be handled by giving neuro the ability to pick from movement patterns or specific goals.
i.e. (circle_move,spiral_move,standing_still,move_to(x,y)) etc.
Menu navigation and such should be easy to implement. Just getting the context and such correct will be difficult

a video of gary using qwen3:4b going through the initial menus. i didn't turn off thinking thats probably the reason its slow

[twin temple]

very small update as i get rolling with this integration. I've finally properly configured everything on my end now so i can test using gary more... reliably.
before it seemed to just randomly pick a action and not really a smart one. or would just spam the same one (i've tested the balatro integration where i noticed this behaviour the most)

mostly just tried many different models and configurations. adjusted some context etc.
but now it actually thinks a bit more before just blindly choosing a weapon.

hoping to get it to be able to beat the first wave within this week

for anyone interested... im using gary with llama-3.1-8b-instruct-Q4_K_L.gguf now with 0.7 temp, 8k context window,flash attn and yapping allowed. and enforce_schema

[twin temple]

been a bit since the last update. but now i got movement working. somewhat.. not sure if its smart to send a updated context every second.. but it works with llama. likely needs tweaking, not sure how neuro handles constant context spam.

[loud dock]

It's probably "fine" if the context is sent as "silent". I think, but looks very cool

[twin temple]

yeah. sending it as silent is the plan. tho i don't think gary supports silent context so i can't really test that

[spring quest]

might be too often based on this

[twin temple]

i see... i guess i could do every 5 seconds. or maybe just when something happens. like low hp or reached a destination etc. with a soft cap of every 10 seconds

[stone walrus]

is this abandoned?