CUPCAKEAGI

[half sparrow]

CupcakeAGI is an agent that aims to mimic human-like behavior and cognitive abilities to assist users in performing various tasks. CUPCAKEAGI can accept images, video, audio and text. It has access tools like search, calculator and more tools can be added in a modular fashion like react. It has unlimited memory. I've added features to let it dream and have random thoughts and store emotion parameters, it has a thought bubble that can be seen by user at all times. It also performs tasks using pre-defined ability functions and can make new abilities using python code.
https://github.com/AkshitIreddy/CUPCAKEAGI

[whole harbor]

You have a lot of similar goals to mine and are way way more developed. Have you thought of adding some more robotics-type-stuff? Or even just discussing it? I’m going to be using similar concepts to enable my ‘assistant’ to drive around on tank track wheels, and hope to add a few extra things down the line..

I haven’t tried cupcakeagi. I have my own thing going on. Does it all work as advertised? And you’re just offering this for free? What’s the catch?

Also, is it possible to do muffins?

[brisk pawn]

This project seems quite interesting! Although I don't typically engage with these types of things, I'm curious about the differences between this and AutoGPT. Based on the README, there appear to be some similarities with AutoGPT, but I'm sure there are also significant differences. Could you highlight some of the key distinctions between the two?

[pure obsidian]

Now that looks promising. Will definetly check it out. 🙂

[dire nebula]

Interesting, will follow the development. ~

[pure obsidian]

When i run pip install -r requirements.txt it gives me this Error:

ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: 'C:\home\conda\feedstock_root\build_artifacts\asttokens_1670263926556\work'

I did what i always do and tried to solve it with the help of gpt. But gpt recommends opening the txt and change lines. That cant be the Solution right. 😄 I tried and created this directory cause i didnt knew what else todo. But that just gives me another error and gpt says i should change lines in the txt again. So a little help would be appreciated. 🙂

[naive palm]

@half sparrow can I edit images too, or is it only a chatbot

Also I loved the demo

[half sparrow]

technically it can run python code and it has access to any file you upload so it can do any modification you want to the image and if you tell it where to store the image in your machine it will save it there

[half sparrow]

autogpt runs on a loop so it great for complex task, cucpakeagi does it in one go so it more suited for simple tasks, like you can tell cupcakeagi to send an email to someone after sometime based on what they sent to you, all it would need is the read_email and send_email ability defined in ability functions

[half sparrow]

i havent thought about adding robotics to this, maybe the llm can give instructions telling it to invoke certain functions like drive straight and each second is converted to a text description though im not sure if text description will be enough, if youre using gpt-4 it could work though since its multimodal it can take the image and then output directions,

[half sparrow]

oh😓 , i've fixed the requirements file now

[whole harbor]

At the moment I’ve got a basic concept that the llm will return stuff like ‘I move forward slowly for 1 second’ which goes through a function that strips unnecessary stuff and passes a sanitized string to an arduino. Really crude and untested but I hope it will be something that can be refined and improved..

Camera control would be like “I turn to look at the x” or “I turn my view to look left”

[half sparrow]

sounds fun

[brisk pawn]

when gpt4's image ability is released, feed the camera's input to your bot too and see what will happen👀

[whole harbor]

Definitely. It’ll be pricey though, so that’ll be a separate function. Until then, it’s going to be mostly input from a smart cam and occasional sending images to a captioning service, I think.

[brisk pawn]

by feeding it images directly, i think it will actually be able to navigate, maybe even wandering around in the house

[whole harbor]

It will have a gyro/magnetometer thing too, which gives really good accurate orientation data. So yeah, I’m hoping with that and the memory, it’ll figure out how to get around the house 🙂

[brisk pawn]

truly AI pet moment lmao

[whole harbor]

CupcakeAGI looks like a great start for someone wanting to do that. Just have to add extra functions.

[visual summit]

seem to be getting these errors

npm run dev

> assistant@0.1.0 dev
> next dev

'next' is not recognized as an internal or external command,
operable program or batch file.

any ideas on a fix?

[half sparrow]

Npm install next

[visual summit]

oh sorry, thanks

[pure obsidian]

So Cupcake is starting up for me now, but it doesnt work. Theres this Error i made a screen of (see at bottom).

And when i run the installs there are a few Errors too:

Errors when running "pip install -r requirements.txt":

Reason for being yanked: Contains an incorrect torch dependency
WARNING: The candidate selected for download or install is a yanked version: 'torchvision' candidate (version 0.15.0 at https://files.pythonhosted.org/packages/79/31/b4caf2c73296f1f12ae7af6647bf48b51442bd68fd8c40426e5a60f46dd3/torchvision-0.15.0-cp39-cp39-win_amd64.whl (from https://pypi.org/simple/torchvision/) (requires-python:>=3.8))
Reason for being yanked: Contains an incorrect torch dependency```

```Building wheel for hnswlib (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for hnswlib (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [5 lines of output]
      running bdist_wheel
      running build
      running build_ext
      building 'hnswlib' extension
      error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for hnswlib
Successfully built Chroma moviepy sentence-transformers wikipedia
Failed to build hnswlib
ERROR: Could not build wheels for hnswlib, which is required to install pyproject.toml-based projects```

Note: I saw this in the Error: "Microsoft Visual C++ 14.0 or greater is required." I checked but i have the newest of this. 

Error when running "uvicorn inference:app":

```ModuleNotFoundError: No module named 'fastapi'```

Additional Note: Started it a second time and the:"Next.js is up to date" from the Screen changed to: "Next.js (13.3.0) out of date".

[half sparrow]

Try git cloning the repo again, I made a couple of changes

I updated the steps in how to use section

If you're getting a module not found error, check if aagi env is activated

[pure obsidian]

Hmmm still the same errors. I get the feeling im just making a stupid mistake. Heres exactly what i did: Clone Git, added apis to env, then executed this commands in backend/Multi-Sensory Virtual AAGI:

conda activate aagi
pip install -r requirements.txt
uvicorn inference:app```

Then opened a second cmd in frontend/assistant and this commands:

```npm install next
npm run dev```

Open browser and navigated to localhost:3000.

I have the newest Version of Anaconda and Microsoft Visual C++ installed. Anything else im missing?

[half sparrow]

i made a couple of changes, now its environment.yml file, i've updated the readdme with new instructions, hope it works for you now😀

[crimson frigate]

i actually really liked this, pretty similar to the work of Dave Shap and what I'm aiming for awesome 😎👍

[whole harbor]

Does it require gpt4 api access? Could It be installed on a raspberry pi?

[half sparrow]

nope, it uses gpt-3.5

[half sparrow]

it can work with raspberry pi, you'll need to make a couple of changes though, the file_describe function use a huggingface model to convert image to text so youll need to host that function somewhere and make a request to it instead of running it locally, the audio to text uses whisper api so no problem with that

[naive palm]

Ooh gpt3.5turbo?