where can I access convo-6B-8bit model

[quartz parcel]

You mean Gpt-j-6B?

It's a finetune of that

[simple portal]

I'm not sure if I hosted the current weights to HF, but the secret sauce is in the set up, not the model.
Which I don't have access too

[timid oracle]

https://huggingface.co/hakurei/convogpt-staging/tree/main/6b-uft

this but quantized

chen cfg https://github.com/harubaru/eliza/blob/main/config/discord_chen.json

[simple portal]

https://huggingface.co/Ryex/lotus-fixed-Q4_K_M-GGUF

Here's the weights currently running

[timid oracle]

lotus 12b?

[simple portal]

yup.

[simple portal]

Keep in mind, this is a very old build, the bots currently run on something similar that was built off that. I don't think haru ever pushed the re-write

[simple portal]

https://huggingface.co/Ryex/Lotus-12B-GPTQ
I made it much smaller.

oh wait, you weren't the one asking...

[simple portal]

@sonic bough

[quartz parcel]

Heck yeah

[simple portal]

actually, don't pull that yet

I need to re-make it

[quartz parcel]

Okay

[simple portal]

should be alright? who knows

[quartz parcel]

You just gotta test it man

[simple portal]

it actually loads into vLLM now, so it's better than the first attempt

[quartz parcel]

Well that's cool,but how is it?

[simple portal]

dunno I'll test it when I have time. it was more just something for people who had no room

[quartz parcel]

Oh

That's nice actually

Maybe get some people in this server to test it?

[simple portal]

well it needs a modified tokenizer so far...