What are the limitations of the fine-tuning API?

[gaunt tundra]

Have little knowledge so pls treat me like a complete noob.

I am trying to ascertain whether I could essentially create my own version of chatGPT with the fine tuning APIs? Or at least could I with a wrapping API / no more AI work?

If not, will that functionality ever be available?

Thank you

[lofty elbow]

I'd say the limitations are cost. You would have to take advantage of embeddings first and then rely on your fine tuning afterwards

[gaunt tundra]

Maybe I'm missing something, but my current understanding is that I can only do step 1 in the flow described in the chatGPT blog?

There is currently no way to do step 2 and 3? Or is there and I am being blind?

Thank you

[lofty elbow]

Look for weight biases in the Fine Tuning docs

But just keep in mind that for each time the person adds something, you will need to send a request with the whole conversation each time. For $0.12/1k tokens that can add up really fast

[umbral plinth]

There is no ChatGPT fine-tuning, mind you. There is also no ChatGPT API, only GPT-3.

If your question is about fine-tuning GPT-3 into something like ChatGPT, the answer is: it’s plausible, but would be very expensive and not guaranteed at all.

Better wait until ChatGPT becomes available as an API

[gaunt tundra]

@lofty elbow Weights & Biases? Looks like an external service to help with fine tuning?

@umbral plinth - was thinking if you could train something like ChatGPT that was specialised in a certain subject....I just can't see how after fine-tuning, I then do the "Reinforcement Learning from Human Feedback (RLHF)" - which I understand it was seeing what it is currently outputting and telling it what it good / bad and then getting it to improve?

[umbral plinth]

I see what you want to do. As of now, there is no feature for training using RLHF. The only way to train OpenAI models is through fine tuning the GPT family of models (except for ChatGPT, which is not available for inference or fine tuning)