streaming example | AG2 (formerly AutoGen) | Page 1

carmine arch Nov 15, 2023, 8:32 PM

#

I've seen that there is new streaming support added recently. (11/11) Is there an example I can refer on this? I want to get the responses one by one from a group chat while the conversation is still on-going rather than waiting until the entire thing finishes.

lucid snow Nov 15, 2023, 8:52 PM

#

Are you referring to this recent PR? https://github.com/microsoft/autogen/pull/597

GitHub

Enable streaming support for openai v1 by Alvaromah · Pull Request ...

Why are these changes needed?
The OpenAI API, along with other LLM frameworks, offers streaming capabilities that enhance debugging workflows by eliminating the need to wait for complete responses,...

#

This issue has more information on the work in progress. https://github.com/microsoft/autogen/issues/633

GitHub

Refractor streaming for oai client · Issue #633 · microsoft/autogen

#597 is merged as a temporal solution. The following changes are needed: support streaming for function call. move print outside oai function

carmine arch Nov 15, 2023, 9:12 PM

#

lucid snow Are you referring to this recent PR? https://github.com/microsoft/autogen/pull/5...

yes I'm referring to 597. I can't even get the temporary solution working unfortunately. (I'm using Azure OpenAI )

#

or any other alternative way to get a responses from group chats, as soon as one of the agents responsds, rather than waiting for a very long time to whole conversation finishes. This is purely from the UX point of view so the user will see the discussion happening while the group chat conversation goes on and on

#streaming example