Is there any limit? Because we want to use one fine-tuned model per client and there are hundreds. Will it work?
#Limit on how many fine-tuned models
hollow wadi
dusty terrace
What is your use case?
hollow wadi
We have an employeeportal where we have data like employees, transcribts of trainingvideos and many more information from that company. Now we want to build a ChatGPT-Bot into the App where the Bot has access to all the information we have in the employeeportal, like all the employees for example.
Our Approach was to fine-tune a model per client with all theire data
But if there's a limit on how many fine-tuned models we have, that will not work
dusty terrace
You'll want to use embeddings and semantic search for this
You can't teach the model any new information with fine tuning
hollow wadi
We tried embeddings but there we have to pass every information for every request and the information from everything can be couple hundred thousands words (because we also transcript every video and so on), so that's way to expensive if we have to "retrain" model for each request
dusty terrace
With embeddings you generate the embeddings from your data set once
You store those embeddings in a database (weaviate, pinecone, etc)
One someone wants to send a message you find the most similar embeddings to their message and add them as the context for the request
hollow wadi
How would we handle if we want to update one employees name in the embeddings, can we find which number it is, delete it and create only that new dataset or do we have to regenerate the whole dataset?
dusty terrace
You'll want to have metadata for the embeddings when you add then, you'd want the text it maps to, employee info and whatever other information you need. You'd be able to search for that embedding, delete it and insert a new one with the correct information
hollow wadi
Would you be able to get on a quick google meet call to help us? We would PayPal you 100 USD 😀