FastAPI and web UI

[vast python]

Hello, I wanted to ask if it's possible to create a simple Web UI that could be used for communication with agents instead of the terminal. I'm fairly new in Python, coming from the web dev world but I was thinking about WebSockets with FastAPI maybe. So I created this script that should send the messages to the FE (Next.js) as they come in from the AutoGen agents.

[cunning canyon]

I am also trying to combine autogen with fastapi, but unfortunately I am also stuck.

do you already know how to implement ws with multiple agents?

[vast python]

whats ws? 😄

[cunning canyon]

websocket

[vast python]

oh 🤦‍♂️ no, i just want to send the messages from the agents to the client firstly, then implement the realtime user input functionality.

[torpid monolith]

Do you still see the same error when you try to return results on a simple post/get route?

I also see that you are importing autogen from flaml. AutoGen has moved to its own repo and there is a change some of the issues youare encountering are fixed already

pip install pyautogen

import autogen

https://github.com/microsoft/autogen

[cunning canyon]

I also got it working to response, but idk how to chat with it:

from fastapi import FastAPI, WebSocket
from agents import runAgent
from config import config

app = FastAPI()
config()

@app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
    await websocket.accept()
        
    while True:
        data = await websocket.receive_text()

        lastMessage = runAgent()

        print(lastMessage)
        await websocket.send_text(f"Message text was: {lastMessage}")



agents.py:
def runAgent():
    print("Running agent")

    config_list = config_list_from_json(
        "OAI_CONFIG_LIST",
        filter_dict={
            "model": ["gpt-3"],
        },
)

    assistant = AssistantAgent("assistant", llm_config={"config_list": config_list})
    user_proxy = UserProxyAgent("user_proxy", code_execution_config={"work_dir": "coding"})
    user_proxy.initiate_chat(assistant, message="Ask me questions about life, the universe, and everything.")

    return user_proxy.last_message()

[vast python]

Yes thanks, I didn't noticed. It didn't fix the issue tho. I actually managed to run the agent when the endpoint is triggered but it doesn't sends the messages as they come in the console.

What I also found is the user_proxy.chat_messages which is the list of messages send by agents.

[cunning canyon]

@vast python any progress so far?

my current problem is that i cannot send to the user_proxy via websocket because the agent is executed in the same thread/cmd? If it is possible to put the agent in the background..

a workaround would be to run the agent in a new cmd window and then paste in the message from the websocket. Idk it's kinda scuffed

[sharp rapids]

have you tried async chat?

a_send

a_receive

[cunning canyon]

@mossy barn

[mossy barn]

so i treid the chat messages as well^ didnt work. So i just cloned the terminal basically. I dont know if thats ecactly what youre trying to do, but, i made a ui to interact with autogen this way

to clone the terminal into my ui i basically did it in 2 parts (with the help of ai : ))

first partf is creating the function:

function start -----------------------

class CapturedOutput(io.StringIO):
def init(self):
super().init()

def write(self, text):
    super().write(text)
    _output.insert(tk.END, text)  # Insert captured text into _output
    _output.update_idletasks()  # Update the UI to show the captured text

function end -----------------------

then pt 2 would be right before user.proxy_initiate_chat, insert this to initalize the function:

    captured_output = CapturedOutput()
    sys.stdout = captured_output

[cunning canyon]

im fairly new to python, basically CapturedOutput will render a new terminal?

[mossy barn]

this allowed me to get all of the text outside of the termianal and into where i needed it to be. again im not sure if that is exaxtly what youre trying to do but maybe it could help?

[cunning canyon]

hm im not not using tk :/

[mossy barn]

its fine this isnt exclusive to tk

capturedoutput is just turning the teminal output into text basically

im then giving that text to tk

but esssentially, you should be able to give that text to anything

[cunning canyon]

ok ty i will try some stuff

[mossy barn]

why use fast api and not just plain python like are there any advantages? im curious as ive never heard of fastapi or i guess what are you trying to do?

[cunning canyon]

my first goal is i to build an api, so i can connect it to a frontend (nextjs)

[mossy barn]

hmmm i see i see...well if you want to get something like this running on the web you COULD use streamlit, woudlnt exactly be an api but you could get it up and running and could start using it if you want to

[cunning canyon]

do you know if streamlit has support for multiple agents?

[mossy barn]

If it can be done it python it can be done in streamlit i know that

when i went on the repository for autogen, a lot of it is just copy and paste and it works which is good it’s not a whole bunch of configuration on their end

[fallow swift]

This Vercel example should help:

https://vercel.com/templates/next.js/nextjs-flask-starter

[torpid monolith]

Great! I've been exploring something similar and I'll share what I have once I have a v1.

[cunning canyon]

nice would love to see what youre building

i will also give updates

[torpid monolith]

Same here, I'd love to see what you are building!!

[sharp rapids]

Do you have a plan on how to do it? I might be able to help. I am working on a hackathon and need to build a chatbot around this myself

[nimble tinsel]

normally you need a browser or a gradio interface / vercel something like that

[nimble tinsel]

you can see some examples in our repo : https://github.com/team-tonic-arena-hacks

[cunning canyon]

if i want to start a agent e.g. /start-agent the fastapi process will be on freeze - so my first goal is to put the user_agent in a subprocess so i can communicate with it

[cunning canyon]

Is it also your goal to run a web api with autogen?

[sharp rapids]

want to call APIs based on autogen. APIs are our own platform APIs. Want to make it like bingAI

[cunning canyon]

ok so for e.g. gpt-4 has an api that can be called etc. but autogen doesnt it will just open a new cmd. How would you solve this?

i think something like that should be implemented inside autogen but i haven't found anything

[sharp rapids]

was planning on writing a websocket to send and recieve messages

[cunning canyon]

same

[torpid monolith]

Hey folks ... wanted to share some updates on what I have so far..
Mainly I am focusing on a stateless fastapi endpoint called from a front end (I really like the websocket approach @cunning canyon is working on but have not gotten to it yet).

Current screenshot of the UI is attached (build with Next.js, static file export served by a fastapi endpoint) .
The roadmap I am thinking of is in line with much of the discussion we have been having here (attached screenshot)

What do folks think? Its very scattered right now, if folks are interested, I can share the repo (note: it is poorly organized at the moment though)

@charred hill , @jade lotus

[cunning canyon]

That's nice so far!
I have also created a repo: https://github.com/jaemil/autogen_websocket
@unreal carbon is helping, my python knowledge isn't that good :>
My focus is more on the frontend, so I would love to contribute @torpid monolith

[fallow swift]

Looks great… be nice to add a control panel to edit configs

[torpid monolith]

Great idea .. will put that on the roadmap

[inland rain]

hello

@torpid monolith

can we have a quick chat?

we were discussing your work with integrating the frontend with autogen

[torpid monolith]

Sure.

[inland rain]

can we have a call when you're free?]

[torpid monolith]

I am in a series of meetings till end of day today, but will try to reach out later once I have cycles. In the meantime, we can chat over dm?

[torpid monolith]

Here is what I have so far - https://github.com/victordibia/autogen-ui ... free free to fork, make changes, start discussions etc. There will be bugs here and there as I havent spent too much time on it.
I might be offline for a while but happy to respond when back

[inland rain]

Yess sent you a dm

[lunar pollen]

Hi! Just saw this thread! These are great sample repos.

If you want to have an out-of-the-box UI, you can try Chainlit (https://github.com/Chainlit/chainlit) - built using FastAPI, WebSockets and React frontend - which is async.

Here is an example by a member of the Autogen community https://github.com/AaronWard/generative-ai-workbook/tree/main/personal_projects/9.chainlit-autogen

Feedback on dev experience would be very much appreciated!!

[nimble tinsel]

Hey Dan are you working on this with @cunning canyon ?

[lunar pollen]

Nope.. what is emil working on?

[nimble tinsel]

Like … such a sick interface , drag and drop agents / teams that you link with your mouse

[rocky escarp]

It doesn't work with the latest chainlit, the user_message type passed in is an object and you can't do +=.
Also not sure why autoGen messages are not returned to the client. Is it because autoGen isn't using async?

Hmm, i tried to switch to the async API, chainlit UI still doesn't receive messages