Defining a custom model client | AG2 (formerly AutoGen) | Page 1

blazing hull Mar 24, 2025, 1:15 PM

#

Im not capable of defining a custom model client with the http call of my custom llm, ¿can someone explain me how its done?

placid pilot Mar 26, 2025, 10:25 PM

#

Also looking for answers on this. I tried putting in my custom model and OpenAI API key, and base url, but it keeps saying that the model doesn't exist

#

in the llm_config list

#

I'm not using ollama/local. I'm using an endpoint i am trying to ping on a cloud server.

neon vortex Mar 27, 2025, 3:31 AM

#

Can you provide an example of the LLMConfig or llm config dictionary? Mainly looking at the key names and types of values.

placid pilot Mar 27, 2025, 1:30 PM

#

llm_config = {
"config_list": [{
"model": "unknown",
"api_key": os.environ["OPENAI_API_KEY"],
"base_url": "https://inference-api.exampleurl.com/Llama-3.3-70B-Instruct/v1",
}],
}

neon vortex Mar 28, 2025, 7:59 PM

#

Thanks @placid pilot, would you be able to paste the error you're getting about the model not existing so I can see if it's something we're triggering or the openai package.

placid pilot Mar 28, 2025, 9:55 PM

#

I think I figured out that I actually need to list the correct model name for the infrastructure that I am pinging. I think it is mostly working now that I am naming the model correctly. I didn't know I needed to have the correct model name, but it seems it is expected. Because I'm using a custom endpoint, I can't necessarily use all of the functionality out of the box of the agent classes in Autogen, but that's okay--figuring this part out too.

#Defining a custom model client