Idea to learn any game

[trim dirge]

https://www.youtube.com/watch?v=fzuYEStsQxc

"curiosity is not only a great replacement for extrinsic rewards,
the two are often aligned, but curiosity, in some cases, is even superior to that."

[primal lance]

not sure if this is easily generalisable to other games

you need to keep a list of what states/positions/events the AI has experienced
for games like minecraft, this can get quite big

[drifting cloud]

interesting

[trim dirge]

wait i might be cooking. vedal said he was gonna do this language model style. So maybe we can use verbal descriptions instead. If you have verbal description like "we are near a plant, a tree, and some sheep", that might be enough information.

[primal lance]

eh if we are using a language model, I'm not sure this is relevant - the language model probably has enough randomness. It also hasn't been trained on that specific game, so it won't fall in a local minima.
i remember seeing this a few years ago and it was a pretty cool result

[trim dirge]

i meant using vision. Instead of keeping features as like pixels, with hundreds of them and millions of frames, you add a layer of abstraction. The lump of pixels becomes stored and manipulated in memory as language.

If you can make vision extract the relevant features

Well, that's kinda of how vision neuro networks work anyway, by adding layers of abstraction each step

so they can finally caption the image