groupchat streaming error | AG2 (formerly AutoGen) | Page 1

token_count_utils throws NotImplementedError if 'stream': True is used in llm_config. This seems to be due to a lack of support for local LLMs in _num_token_from_messages().

Warning: model not found. Using cl100k_base encoding.
Traceback (most recent call last):
  File "OLlama_tests/app.py", line 154, in <module>
    user_proxy.initiate_chat(manager,
  File "autogen/agentchat/conversable_agent.py", line 544, in initiate_chat
    self.send(self.generate_init_message(**context), recipient, silent=silent)
  File "autogen/agentchat/conversable_agent.py", line 344, in send
    recipient.receive(message, self, request_reply, silent)
  File "autogen/agentchat/conversable_agent.py", line 475, in receive
    reply = self.generate_reply(messages=self.chat_messages[sender], sender=sender)
  File "autogen/agentchat/conversable_agent.py", line 887, in generate_reply
    final, reply = reply_func(self, messages=messages, sender=sender, config=reply_func_tuple["config"])
  File "autogen/agentchat/groupchat.py", line 174, in run_chat
    speaker = groupchat.select_speaker(speaker, self)
  File "autogen/agentchat/groupchat.py", line 91, in select_speaker
    final, name = selector.generate_oai_reply(
  File "autogen/agentchat/conversable_agent.py", line 619, in generate_oai_reply
    response = client.create(
  File "autogen/oai/client.py", line 244, in create
    response = self._completions_create(client, params)
  File "autogen/oai/client.py", line 286, in _completions_create
    prompt_tokens = count_token(params["messages"], model)
  File "autogen/token_count_utils.py", line 59, in count_token
    return _num_token_from_messages(input, model=model)
  File "autogen/token_count_utils.py", line 107, in _num_token_from_messages
    raise NotImplementedError(
NotImplementedError: _num_token_from_messages() is not implemented for model dolphin2.1-mistral:7b-q5_K_S.

#groupchat streaming error