TypeError: create() got an unexpected keyword argument 'api_type'

[outer garnet]

I'm getting this error when trying to run on my Local LLM

Traceback (most recent call last): File "e:\AI Tools\AutoGen\Basic.py", line 26, in <module> user_proxy.initiate_chat( File "C:\Users\Sagnik.THRAWN\.conda\envs\ChatDev_conda_env\lib\site-packages\autogen\agentchat\conversable_agent.py", line 556, in initiate_chat self.send(self.generate_init_message(**context), recipient, silent=silent) File "C:\Users\Sagnik.THRAWN\.conda\envs\ChatDev_conda_env\lib\site-packages\autogen\agentchat\conversable_agent.py", line 354, in send recipient.receive(message, self, request_reply, silent) File "C:\Users\Sagnik.THRAWN\.conda\envs\ChatDev_conda_env\lib\site-packages\autogen\agentchat\conversable_agent.py", line 487, in receive reply = self.generate_reply(messages=self.chat_messages[sender], sender=sender) File "C:\Users\Sagnik.THRAWN\.conda\envs\ChatDev_conda_env\lib\site-packages\autogen\agentchat\conversable_agent.py", line 962, in generate_reply final, reply = reply_func(self, messages=messages, sender=sender, config=reply_func_tuple["config"]) File "C:\Users\Sagnik.THRAWN\.conda\envs\ChatDev_conda_env\lib\site-packages\autogen\agentchat\conversable_agent.py", line 631, in generate_oai_reply response = client.create( File "C:\Users\Sagnik.THRAWN\.conda\envs\ChatDev_conda_env\lib\site-packages\autogen\oai\client.py", line 260, in create response = self._completions_create(client, params) File "C:\Users\Sagnik.THRAWN\.conda\envs\ChatDev_conda_env\lib\site-packages\autogen\oai\client.py", line 346, in _completions_create response = completions.create(**params) File "C:\Users\Sagnik.THRAWN\.conda\envs\ChatDev_conda_env\lib\site-packages\openai\_utils\_utils.py", line 303, in wrapper return func(*args, **kwargs) TypeError: create() got an unexpected keyword argument 'api_type'

[graceful thunder]

Replace api_base with base_url

[outer garnet]

Thanks

[outer garnet]

Can autogen 0.2 run on open LLM?

[graceful thunder]

Not sure, I installed autogen yesterday. I am using mistral 7b from openrouter API. I had a similar issue which you had and found the fix for this.
But now I am getting another error """AttributeError: 'NoneType' object has no attribute 'prompt_token"""

@supple orbit does autogen 0.2 support open source LLM?

[supple orbit]

I'm on vacation. I'll take a note of this question. <@&1180519647552999444> do you remember who has made mistral work?

[ocean copper]

@final chasm and @twin loom and @outer tiger have posted comments regarding their attempts to use mistral in autogen. Did any of you get it working, or hit the prompt_token error that @graceful thunder is seeing?

[oblique thistle]

@honest idol has made zephyr work, which I think was based on mistral.

[honest idol]

I have had a bit of success using Mistral7B models while running them using vllm. vllm provides an openai compatible endpoint that should be easily interchangeable with the openai models.
It is possible that openrouter has made some changes to their end point breaking compatibility with the openai library (their documentation seems to suggest compatibility though). Can you confirm if you have this error for all openapi router models or only mistral 7b?

Edit: I did not try any function calling

[twin loom]

I did have similar nonetype errors using a local llm. I'd wager there is a field in the json response passing null or none or something causing the issue. Probably prompt_token.

[graceful thunder]

Thanks. Mistral is not working but Zephyr is working but not in the intended way. User proxy response are not showing and the assistant response is not complete. Need to explore different API providers and models.

[maiden shell]

Hello @honest idol @graceful thunder ! Have you get it to work or tried with instruct fine-tuned models? Setting up the agents with a system message failed with 'error', 'message': 'Conversation roles must alternate user/assistant/user/assistant/...' due to : https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-AWQ/blob/main/tokenizer_config.json#L42, I believe.

[outer tiger]

Can we run it with lm studio

[outer tiger]

import autogen

config_list_mistral = [
    {
        'base_url': "http://localhost:1234/v1",
        'api_key': "NULL"
    }
]

llm_config_mistral={
    "config_list": config_list_mistral,
}

coder = autogen.AssistantAgent(
    name="Coder",
    llm_config=llm_config_mistral
)

user_proxy = autogen.UserProxyAgent(
    name="user_proxy",
    human_input_mode="NEVER",
    max_consecutive_auto_reply=10,
    is_termination_msg=lambda x: x.get("content", "").rstrip().endswith("TERMINATE"),
    code_execution_config={"work_dir": "web"},
    llm_config=llm_config_mistral,
    system_message="""Reply TERMINATE if the task has been solved at full satisfaction.
Otherwise, reply CONTINUE, or the reason why the task is not solved yet."""
)

task="""
Write a python script to output numbers 1 to 100 and then the user_proxy agent should run the scrip and test it. If it works modify the script to prit the number 1 to 200.
"""

user_proxy.initiate_chat(coder, message=task)

this code work but i can't use a agentbuilder. use lm studio to host the LLM.