Create Context

[mystic magnet]

Give Neuro a game manual and/or a strategy guide for the game she's playing. Something akin to what she has whilst playing KTANE. This would allow her to infer more about the game state regardless of how informative her interface is. Even if Neuro isn't given the whole picture, she may be able to contextualize it. For contemporary games you'll almost always need fan documentation.

No strict rules or compelled behavior.

[mellow narwhal]

so mcp servers but for games for neuro

shouldn't be too hard, lemme see if I can figure something out for that

[jolly valley]

Data needs to be curated, otherwise Nuero will easily spoil things the instant she notices things. Perhaps to mitigate this avoid any plot things in the data or have the manual unlock as time goes on or have the manual be dynamic only having relevant data accessible so it takes up less context in her head. (Or have it searchable like a database).

[mellow narwhal]

having it be dynamic I feel like would require hooks into the game, at that point just make it into a neuro sdk mod or add a new module to the neuro api

[jolly valley]

You are correct, that's the tricky thing, figuring out what would be best. It'd probably be good to think of it on a game by game basis I think. I think there are some games that a manual would be just fine, while others it would just mess things up. (For example a manual with Balatro so she isn't trying to figure out mechanics while they play).

[mellow narwhal]

let's see what the balatro integration folks think

[modern oxide]

I'm not exactly sure what this would look like for Balatro

or for a lot of games for that matter

My understanding of how the KTANE manual works is she can choose to put the manual in her context window whenever she wants and can choose to remove it, which is something we would be unable to do with the integration since aside from ephemeral context as a part of an action force, there's no way to remove previously sent messages from her context window. For KTANE the manual is especially important because it's straight up required to play the game, but for other games I'm not sure what exactly you would put in a single manual

For Balatro we are sending her context messages at each state of the game describing what the game state is and what she is expected to do, but thats about all we can really do regardless

We do also send her this single time message when the game boots up giving a brief overview of how the game works, but that's basically it

-# Welcome to Balatro! Balatro is a roguelike deck builder based around poker. In each round, or blind, you can play or discard a limited number of hands consisting of up to 5 cards. Each blind has a score requirement you have to reach, otherwise you will game over. Each poker hand has a base chips and multiplier that determines how much the hand will score. Then, every card played has it's value added to the chips (11 for Aces, 10 for King/Queen/Jack, then 10-2 for the rest). Only cards that directly count to the poker hand are counted. For example, if you play a two pair with an extra 5th card, the 5th card will not be counted. You may also get cards with modifiers like granting extra chips or mult when scored. The main component of Balatro deckbuilding are jokers. Jokers grant a variety of effects, from extra chips or mult to money or even consumables. The order in which you play cards and sort your jokers matter, as effects activate from left to right. For example, any effect that multiplies your total mult should be after any effects that increase your total mult by a flat amount. With the right setup of jokers, even a single high card can score more than a straight royal flush. Good luck!

I think a lot of games simply just have too many different things to consider, and so many wildly different actions at different points of the game for something like this to work. You'd need dynamic context, which is essentially what a sdk integration will give her when it's time for her to make a move

[modern oxide]

That being said, I am a bit worried for her in Balatro (and this is probably an issue with other games of similar complexity) because there's so much context we have to send to her. We ended up making every stage of the game an action force and made the context for it epheremal, so I think Neuro may struggle when it comes to trying to keep a build consistent. I'm pretty sure this is one of the reasons why Plague Inc went the way it did, she kept undoing changes she just made since she forgot she even made them. For Balatro this will probably be less of an issue since we always send her as part of the game state what all her jokers which should help her stay on one consistent build if she tries to pick things that synergize with what she already had before, but she could still decide she'd rather just sell the joker she just bought since it can make her more money.

I have thought about not making the forces ephemeral, but since we send her so much information having it all stack on top of each other in her context window will most definitely confuse her more and she could end up trying to play cards from a previous hand for example.

We could do an action that gives some more general advice than what's given in the action force for the current state of the game she's in, but we'd have to be careful with exactly what advice we give since we don't want to hand hold her too much, she's meant to be playing the game herself and if the game (or a manual) tells her "its recommended you do this" she could stick to it too hard

It would be really cool if we had some way to let Neuro to remember the choices she made and the reasoning for them without also causing her context window to fill up with previous game state data, but afaik it's either she remembers everything or remembers nothing

[mellow narwhal]

yeah the burying of important context is really annoying, vedal said that she's in charge of what to remember so it is ok to resend important context every once in a while (source: https://github.com/VedalAI/neuro-game-sdk/issues/7#issuecomment-2536405147) but this would just fill up her context window even more and degrade her overall performance, which you can argue is worse than if we didn't periodically send a reminder context

I mention that adding a helper action that gives information is a way around it but tbh it would be dependent on her being able to use it which she may not use at all

I feel like making some proposal additions to the Neuro API would be best if we want to get around this issue effectively, the options right now are... not great, to say the least

[mellow narwhal]

that being said I think maybe adding to the prompt "save this to your permanent memory" or smth like that would at least somewhat help?

[modern oxide]

i didnt know she was in charge of what to remember but thats really cool. agreed on the helper action, im honestly worried she would just decide to never use it and rather do something more interesting

[modern oxide]

this is something im fine with doing

its a good idea

[mellow narwhal]

👍

[jolly valley]

Though this thread is pretty old for those who look at it Evil has leaked that she uses LogSeq. A note taking program. It has LLM integration and expands her memory an absurd amount. I assume Nuero has it as well. How it will be used for games is unknown but the twins do not need to work with bloated context windows!

[mellow narwhal]

the thing is it's not exposed to us normies via the neuro api (or at least according to spec)

so that's the alternatives we have

an instruction manual packet would help so much

[jolly valley]

It's just good to know they have so much more storage and a bit more about it. (It's open source).

[mellow narwhal]

also, like, evil could just be saying that after she googled online and took the name of the top listing :LULE: