#Any sort of local framework for generating chats?

2 messages · Page 1 of 1 (latest)

molten sluice
#

I would like to create a discord bot for some friends to chat with, but have no clue how to interface with generation. I don't know whether to use the UI or one of the loaders by themselves. Any guidance?

#

For reference, I get the best perf with GPTQ-for-LLaMa, and I looked at the github page for it and don't really understand how to format anything, which is why the webui mgiht be useful