WSL help for @Hueman Instrument

[charred locust]

You get it?

You can make it role play like a girl

and talk like that

etc

[long sorrel]

if it's something it's unfamiliar with, shouldn't it remember the dataset of that

[charred locust]

No

It could, but its not in your control

[long sorrel]

wait why not, what are these vtuber AI's doing then

[charred locust]

That's real tuning

That's what meta did with 15 trillion tokens

[long sorrel]

how do I do "real tuning"

no these people are college kids doing this

vtuber things

[charred locust]

what things?

show me one example

[long sorrel]

just custom models I guess

[charred locust]

Show me

1 example

of what things you speak about

so I understand you

[long sorrel]

computer is being slow

[charred locust]

Use windows performance window

and keep eye on memory , cpu and vram usage

if it is too much pressure, you need to lower per_device_train_batch_size

1 / 2 / 4 etc.

I run 2

on rtx 4090

anywao bro I gtg

u can google and reddit all these topics

plenty of info

[long sorrel]

https://youtu.be/1qjIqkceiVs?t=495

[charred locust]

lemme see

That's role playing ai probably

He fine tuned it

[long sorrel]

exactly

[charred locust]

but the dataset is not

what it replies

it replies a style

a behaviour

[long sorrel]

please stop repeating that haha

I understand it's not going to directly quote the dataset every single instance

[charred locust]

lol but this is what i described already this video

it wont quote it at all

maybe if you're lucky

or if you overfit it

at the cost of it losing quality

[long sorrel]

it's just the LoRA I'd be working with anyways, it only uses like 5% of the full model right?

[charred locust]

It can talk more like you want and remember the dataset but at the cost of losing intelligence and other knowledge randomly

you can pick

r = 8,
lora_alpha = 16,

increase the rank

and get more parameters to train

but takes longer and more resources

if you increase to 128, and set alpha as 64 for example

you get very large parameters to train

but then you need big dataset

[long sorrel]

I'm not seeing
r = 8,
lora_alpha = 16,

Anywhere

[charred locust]

its in model = FastLanguageModel.get_peft_model(

[long sorrel]

no results for that either

[charred locust]

checki n mytrain.py

not mybot.py

[long sorrel]

err maybe I found it now

[charred locust]

If you increase rank

you get more trainable parameters here

[long sorrel]

niceeeeee

[charred locust]

but it will affect the underlying knowledge it already has more too

[long sorrel]

if i made like 30,000 questions is should be able to retain it's intelligence I imagine

[charred locust]

There's no secret sauce

that's what makes it fun

experiment 🙂

[long sorrel]

I will be doing that for sure but

I would like to know more about RAG too

seems like I'll be trying to use both in the end

[charred locust]

RAG is like feeding it documents and it can search those documents for you

you can do that with already done models

its not for training

not like GPT where you attach stuff, that's in context

youtube / reddit has plenty of it

cua bro 😛 gl hf

[long sorrel]

can I do rag with ML Studio?

[charred locust]

You combine it

LM Studio starts the API Server and you need third party tool to connect to the api etc

never used it but its doable yes.

[long sorrel]

but I can inference within LM Studio too tho

can I find some way to do it there?

I'm not good with api

I don't know any third party tools

[charred locust]

https://community.amd.com/t5/ai/how-to-enable-rag-retrieval-augmented-generation-on-an-amd-ryzen/ba-p/670670

There u go

[long sorrel]

amd?

I'm intel

X )

[charred locust]

same shit

[long sorrel]

k

[charred locust]

good tutorial for lm studio

ez

^^

BB! 😄

[long sorrel]

this is giving me an error

@charred locust hey question

This ^

and this

both give the same answer....

is the LoRA really suppose to do like.. literally nothing?

[long sorrel]

This ended up being the proper setup!!!!!

THE MODEL IS RESPONDING AS IF IT WAS TRAINED NOW!!!!