Real-time chat with a virtual avatar. "Access Point AI" application.

[daring jetty]

Access Point AI is basically an end-to-end audio-visual system that enables users to communicate with ChatGPT via natural language.

https://www.accesspoint-ai.com/

You speak into the microphone and the digital avatar responds to your questions. It's feels less like a request-response experience, but more like having a real dialogue with an intelligent entity.

The core of the application is a system of speech recognition (speech-to-text), ChatGPT response, and speech synthesis (text-to-speech). To bring the virtual avatars to life, we use Epic's Metahuman Creator.

At present, we have two options that we are working on.
Option 1: A personal learning and conversation space. The female avatar, Aida, can engage in general conversations or act as an expert tutor on a topic of the user's choice. Aida creates a topic outline based on the user's interest and guides them through a course-like process with small quizzes in between to reinforce their learning.
Option 2: A history mode option, which enables users to learn about history. Users can directly converse with a “person” from the past. We are working on a growing database of famous personalities from different eras, allowing users to select their preferred historical figure to learn about their life, culture, and events.

Some of the historical figures that are currently implemented, and a short demo of Queen Nefertiti...

[mild dew]

good evenung

[wide widget]

well done

[ivory sundial]

Cool - got a link?

[split olive]

May I ask, what are you using for TTS? ElevenLabs?

[daring jetty]

We are currently working on a streaming setup, so the users can just open the application in a browser on any device. I will put a link to the demo website when we have a beta ready.

[daring jetty]

We currently use Azure Speech for TTS. ElevenLabs is a bit expensive for what we have in mind. 😉

[mint creek]

Are you hiring?

[daring jetty]

No we are not hiring atm.

[mint creek]

ok

[snow hull]

I have a similar project using STT and TTS. But I'm super new to all of this.

How do you handle ending the request on the user side with the speech capture before pushing the text to the OpenAi api?

Man, I might not have even asked that in a way that you can answer. Hopefully you know what I'm asking lol.

[cursive inlet]

Do you have any suggestions for bringing down the latency of the system? working on a similar project and id love to hear how others have worked with this issue.

[daring jetty]

Yes I understand what you are asking. 😉
You typically have some start recording and finish recording function. How they are triggered depends on your application. You can use push-to-talk, or some keyboard klick event, or even an automated finish (e.g. 2 seconds after microphone input drops below a certain threshold).

[daring jetty]

One thing to consider would be cutting your audio into chunks. That's usually the biggest issue with latency...

[daring jetty]

Getting there...
https://www.youtube.com/watch?v=V7tAR0bWEJA

[grizzled sage]

/!aiadd

!aiadd

[daring jetty]

The Beta is now open for everyone to test. Would be awesome to get some feedback! 🙂

https://www.accesspoint-ai.com/

[tulip barn]

very cool. slightly uncanny valley. but i like the push to talk interface, over that used by other talking head chatbots.

[frigid slate]

Hi. It’s great! Only in English?

[daring jetty]

Thanks! Yes, the Metahuman characters are not out of the uncanny valley yet. Maybe in 2-3 years we'll get to really believable digital humans.

[daring jetty]

No, we can also implement other languages. For now its just German and English. More will be coming....

[tulip barn]

make this Avatar-as-a-service with a no-code workflow backend like flowise ai or botpress, so any numpty can lock down its response to specific domain... then this will be huge

i was messing around with github:the-full-stack/ask-fsdl bot just yesterday and have it ETL ingesting information from predefined videos, websites and documents (isnt nocode but is opensource)

or just make 'romantic' dlc like all the other talking heads. and you will make millions

[daring jetty]

Cool, thanks for the suggestions! Yes, this is just a rudimentary setup for now. We are looking into langchain and other things. The real value of such an AI comes from knowing specific information, and probably also long-term memory. Still a lot of stuff we would like to implement in the future.
And no, we dont want to go down the Replika road of romantic AI. 😅

[tulip barn]

yeah well you gotta check out botpress to see how its like <1min to setup domain knowledge chatbot now (including super easy workflow ui)