'str' has no attribute 'get' when running local LLM | AG2 (formerly AutoGen) | Page 1

alpine scroll Nov 2, 2023, 9:29 PM

#

The problem
When I run the python file described below, I get the error described in the next message (I reached the word limit)

My environment
OS: Windows 11 22H2
Python: 3.10.12

Reproduction

Install LM Studio
From the home page, search for and download TheBloke/zephyr-7B-beta-GGUF/zephyr-7b-beta.Q4_K_M.gguf
Select the downloaded model, and start the local inference server
Install Python 3.10 thorugh python.org or an Anaconda environment
Create a file thisisaproblem.py with the following content:

from autogen import AssistantAgent, UserProxyAgent

config_list = [{
    "api_type" : "open_ai",
    "api_base" : "http://localhost:1234/v1",
    "api_key" : "NULL"
}]

llm_config = {'config_list': config_list}

#create an instance of AssistanAgent
assistant = AssistantAgent(
    name = "assistant",
    llm_config=llm_config,
)

#create an instance of UserProxyAgent
user_proxy = UserProxyAgent(
    name="user_proxy",
    human_input_mode="NEVER",
    max_consecutive_auto_reply=2,
)

user_proxy.initiate_chat(
    assistant,
    message = """Write a short story about an old man"""
)

Use the conda environment or the bare 3.10 installation to run the following:

python -m pip install pyautogen

Run the python file:
python thisisaproblem.py

#

The error is:

Traceback (most recent call last):
  File "C:\dev\py\blockchain_capstone\autogen_test.py", line 44, in <module>
    user_proxy.initiate_chat(
  File "C:\Users\simen\anaconda3\envs\blackchain\lib\site-packages\autogen\agentchat\conversable_agent.py", line 531, in initiate_chat
    self.send(self.generate_init_message(**context), recipient, silent=silent)
  File "C:\Users\simen\anaconda3\envs\blackchain\lib\site-packages\autogen\agentchat\conversable_agent.py", line 334, in send
    recipient.receive(message, self, request_reply, silent)
  File "C:\Users\simen\anaconda3\envs\blackchain\lib\site-packages\autogen\agentchat\conversable_agent.py", line 464, in receive
    self.send(reply, sender, silent=silent)
  File "C:\Users\simen\anaconda3\envs\blackchain\lib\site-packages\autogen\agentchat\conversable_agent.py", line 334, in send
    recipient.receive(message, self, request_reply, silent)
  File "C:\Users\simen\anaconda3\envs\blackchain\lib\site-packages\autogen\agentchat\conversable_agent.py", line 464, in receive
    self.send(reply, sender, silent=silent)
  File "C:\Users\simen\anaconda3\envs\blackchain\lib\site-packages\autogen\agentchat\conversable_agent.py", line 334, in send
    recipient.receive(message, self, request_reply, silent)
  File "C:\Users\simen\anaconda3\envs\blackchain\lib\site-packages\autogen\agentchat\conversable_agent.py", line 462, in receive
    reply = self.generate_reply(messages=self.chat_messages[sender], sender=sender)
  File "C:\Users\simen\anaconda3\envs\blackchain\lib\site-packages\autogen\agentchat\conversable_agent.py", line 781, in generate_reply
    final, reply = reply_func(self, messages=messages, sender=sender, config=reply_func_tuple["config"])
  File "C:\Users\simen\anaconda3\envs\blackchain\lib\site-packages\autogen\agentchat\conversable_agent.py", line 606, in generate_oai_reply
    response = oai.ChatCompletion.create(
  File "C:\Users\simen\anaconda3\envs\blackchain\lib\site-packages\autogen\oai\completion.py", line 803, in create
    response = cls.create(
  File "C:\Users\simen\anaconda3\envs\blackchain\lib\site-packages\autogen\oai\completion.py", line 834, in create
    return cls._get_response(params, raise_on_ratelimit_or_timeout=raise_on_ratelimit_or_timeout)
  File "C:\Users\simen\anaconda3\envs\blackchain\lib\site-packages\autogen\oai\completion.py", line 224, in _get_response
    response = openai_completion.create(request_timeout=request_timeout, **config)
  File "C:\Users\simen\anaconda3\envs\blackchain\lib\site-packages\openai\api_resources\chat_completion.py", line 25, in create
    return super().create(*args, **kwargs)
  File "C:\Users\simen\anaconda3\envs\blackchain\lib\site-packages\openai\api_resources\abstract\engine_api_resource.py", line 155, in create
    response, _, api_key = requestor.request(
  File "C:\Users\simen\anaconda3\envs\blackchain\lib\site-packages\openai\api_requestor.py", line 299, in request
    resp, got_stream = self._interpret_response(result, stream)
  File "C:\Users\simen\anaconda3\envs\blackchain\lib\site-packages\openai\api_requestor.py", line 710, in _interpret_response
    self._interpret_response_line(
  File "C:\Users\simen\anaconda3\envs\blackchain\lib\site-packages\openai\api_requestor.py", line 775, in _interpret_response_line
    raise self.handle_error_response(
  File "C:\Users\simen\anaconda3\envs\blackchain\lib\site-packages\openai\api_requestor.py", line 428, in handle_error_response
    error_code=error_data.get("code"),
AttributeError: 'str' object has no attribute 'get'

I see that this error stems from the package openai, and I'm not sure if autogen really is the source of the problem

#

Here is a video showing what's happening

upper osprey Nov 2, 2023, 9:49 PM

#

@warm zealot Can you take a look at this error?

alpine scroll Nov 2, 2023, 9:52 PM

#

I tried setting 'request_timeout': 999999 in llm_config to see if there was a problem with the default timeout, however I ended up with the exact same error

potent falcon Nov 2, 2023, 10:27 PM

#

I'm running into the same error

warm zealot Nov 3, 2023, 5:21 AM

#

upper osprey <@1035469432258056202> Can you take a look at this error?

It looks like your local llm doesn't reply in the way compatible with openai package. You can try make a direct call with openai instead of calling from autogen to verify it.

signal lake Nov 3, 2023, 3:20 PM

#

I have experienced this issue on local LLMs (hosted by LM Studio) when the context window is full

alpine scroll Nov 3, 2023, 8:15 PM

#

Yeah I'm going to look into MemGPT to see if it expands the context window

smoky topaz Nov 4, 2023, 10:46 PM

#

getting the same error on LM Studio.

Tried:
"read_timeout": 300,
"max_retries": 50,
in llm_config of AutoGen

but it seems that LM Studio has some internal timeout of 60 because I'm getting this in the console:
openai.error.Timeout: Request timed out: HTTPConnectionPool(host='localhost', port=8000): Read timed out. (read timeout=60)

smoky topaz Nov 4, 2023, 10:48 PM

#

alpine scroll Yeah I'm going to look into MemGPT to see if it expands the context window

Did you manage?

I've tried MemGPT but then LM Studio did NOT work at all.

I got this back in the console:
C:\Users\irene!AI-LLM\AutoGenMemGPTLocalLLM.venv\Lib\site-packages\memgpt\autogen\memgpt_agent.py:212: RuntimeWarning: coroutine 'AgentAsync.step' was never awaited
(
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Traceback (most recent call last):
File "C:\Users\irene!AI-LLM\AutoGenMemGPTLocalLLM\app.py", line 84, in <module>
user_proxy.initiate_chat(
File "C:\Users\irene!AI-LLM\AutoGenMemGPTLocalLLM.venv\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 531, in initiate_chat
self.send(self.generate_init_message(**context), recipient, silent=silent)
File "C:\Users\irene!AI-LLM\AutoGenMemGPTLocalLLM.venv\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 334, in send
recipient.receive(message, self, request_reply, silent)
File "C:\Users\irene!AI-LLM\AutoGenMemGPTLocalLLM.venv\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 462, in receive
reply = self.generate_reply(messages=self.chat_messages[sender], sender=sender)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\irene!AI-LLM\AutoGenMemGPTLocalLLM.venv\Lib\site-packages\autogen\agentchat\conversable_agent.py", line 781, in generate_reply
final, reply = reply_func(self, messages=messages, sender=sender, config=reply_func_tuple["config"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\irene!AI-LLM\AutoGenMemGPTLocalLLM.venv\Lib\site-packages\memgpt\autogen\memgpt_agent.py", line 212, in _generate_reply_for_user_message
(
TypeError: cannot unpack non-iterable coroutine object

alpine scroll Nov 4, 2023, 10:53 PM

#

smoky topaz Did you manage? I've tried MemGPT but then LM Studio did NOT work at all. I go...

I have only read into the theory, but not yet tried to implement any MemGPT code. I'm going back and forth between multiple projects, so I can't give a definite time I can go back and continue on this

#'str' has no attribute 'get' when running local LLM