#Deciding which AI LLM to use based on text.

35 messages · Page 1 of 1 (latest)

stoic cedar
#

Im trying to build a function that based on your text will output which model you should use (gpt-4 for complex tasks, or something like llama-3-70b for a simple hello, or a un moderated model for other things.) But i have no idea where i should start, i tried building a simple text classifier but it didnt work out to well

unborn flume
#

First part is hardware requirements and whether a model need to be fine tuned. You can eliminate a lot of models just by those two questions.

stoic cedar
#

i need to make something that can help decide which model to use for my api

vapid sonnet
robust matrix
#

you pretty much want to do the same thing as openAis gpt4 ,their gpt4 is made of a lot of models, each has a profound expertise in a field,depending the input prompt ...

robust matrix
#

and have each llms best at something specific

stoic cedar
unborn flume
stoic cedar
#

i assume you didnt read or look at anything i said or sent

stoic cedar
unborn flume
#

I did, it just doesn't make sense

#

no need to be rude.

#

toggling between functions/actions, ok I get that. But swapping between entire models is a costly thing, especially as an API. Unless you're attempting what the other person said, creating your own chatGPT "app" library

robust matrix
#

well u both are right , using the better model is always best for example gpt4

#

this would be useful when you have smaller open source models that are finetuned to do something

#

or if you like decided that a input doesnt need gpt 4 so you settle for a low level model

robust matrix
#

XD

robust matrix
#

then setup like a metric to measure

#

it

stoic cedar
unborn flume
#

Why not?

#

It would reduce the latency and offers you more control over just having an API wrapper like the people at HumaneAI pin or Rabbit R1

unborn flume
stoic cedar
#

also you can't self host claude 3 opus or gpt 4

unborn flume
#

Proprietary models not withstanding, self hosting models is a good idea because you can exercise control. This sounds better like a hobby project, but not something to invest resources for as a full business. Lots of competition and open alternatives.

tranquil bane
#

Seems like it's relevant to ask what the goal of having this router capability actually is intended to achieve. at the frontier, all the proprietary options are going to give you effectively the same performance and you could probably just choose at random. On the other hand, if you have a set of sample data that matches what would be the ultimate usecase, you could build up a dataset where the optimal preference is labeled and then maybe do something like a finetune of flan t5 small to decide on.

stoic cedar
tranquil bane
#

Something I've done as a simple way to approach this type of thing is to present it to the user. It may not make sense in your case but, as an example, on the UI I have a selector the user can click if they want to toggle between chat, zero-shot task completion, or continued generation. Doing it that way, the user is going to build out the dataset on your behalf and if you wanted to automate the task switching later you'd have the ground truth's based on their behaviors. AS to the question of model switching itself, this is technically what OpenAI is currently doing in their chat interace today. They let the user switch models at any point in the conversation. And you can bet that they're tracking every time a user does that so they can identify the model weakpoints.