integrating gpt-2?

[zenith hedge]

hello! i'm writing a research paper (for high school) and my research question is about dialogue generation in video games. i'm comparing text generation done with markov chains and gpt-2; both of these are meant to occur at run-time (kinda like a controlled variable tbh) so i can't just pre-generate dialogue and paste it into my code. does anyone have any tips for integrating gpt-2 into a game so that it 'works'/outputs dialogue at run-time? thank you so much!

[fiery pine]

If you are running this model locally, then I would look into a part of Unity AI called Inference Engine(formerly Sentis) running in the cloud you would a web request to send the prompt to gpt.

I would also check out Inworld and Convai. Both are cloud services that specialize in no dialogue and interactions

[zenith hedge]

i looked at sentis a little bit before but it seems really heavy, so i'm not sure how well it would work on my mac haha

[fiery pine]

It can technically run any onnx model found on huggingface. It's just matter of finding one that suits your needs.

[median slate]

LLM stuff is always incredibly heavy and wasteful compared to a simple thing like a Markov chain.

[zenith hedge]

alright, thank you! i'll see what i can do :)

[zenith hedge]

absolutely lol thats the conclusion im hoping to prove :D

[river sedge]

If that's what you want to prove why would you use an extremely outdated model like gpt 2? Sounds like a biased research...

I don't think markov chains are good for anything beyond one sentence background exclamations by npcs. Definitely not something you can call a "dialog". Might be wrong though, as I've never used or encountered them in games.

[fiery pine]

I think a better comparison would be the rules based Alice chatbots compared to modern GPTs

[zenith hedge]

im using gpt2 cos its the easiest to fine-tune i think? in terms of it being open-source etc

also its the model that my supervisor recommended and i kinda have to do what he says lol

[zenith hedge]

ur right, but im focusing more on the technical linguistic quality of the sentences produced; it's more of a probability thing for my essay? like, discussing the probability of having a correct (context- and grammar-wise) sentence and the implications of that on user experience

[zenith hedge]

idk i got really attached to markov chains 😭 and it's a bit too late to change now tbh

[fiery pine]

Maybe context aware Markov chains exist nowadays. But it's been over a decade since I looked into them for chatbots

[river sedge]

You can fine tune as easily any other model. And there are plenty of modern open source models.
It's just that the conclusions of your research would be irrelvent today if you're gonna make them based on the results of gpt2.

[zenith hedge]

yeah thats fair

it's alright i'll just talk about that in my evaluation i spose