Chatbot

[vague oracle]

i was wondering if i can find someone with enough knowledge about chatbots to help me out? i have my graduation project about a chatbot dedicated to a supermarket and after i made everything my college professor decided to cancel everything and wanted me to recreate it in a dynamic way or rather static but acceptable for the terms he chose "supermarket" and idk what to do tbh anymore i kinda need some help from someone

[regal matrix]

Please provide more context on the problem statement. It's hard to understand without context of "dyanmic way" or "static but acceptable". Also, you have contradiction within you current paragraph, that your project is about supermarket, and your professor canceled it, and wanted you to make a supermarket chatbot again?

[vague oracle]

The bot is about a supermarket i supposed he wanted static answers aka using intents and so on but then he required it to be dynamic as in more self learning because intents are not as accurate and sometimes the responses are messed up because of the tokenizer taking only few words and answering depending on it such as "how are"
Counts as how old depending on the word how

I've made a similar project using intents alone but it's about a clothing store and my team kinda messed me up so i can't change the interface into a supermarket i can provide the source code as well if someone wants to help

Basically i just need to redo my project for a supermarket chatbot with text to speech and speech to text and I'm kinda lost in the process qwq

[regal matrix]

I’ve never built a chatbot myself, but I’d imagine that as long as you provide additional options like “not what you’re looking for? How about option A B C…”, where the options given are ranked by probabilities, record every time the user choose those options instead, and retrain everyday with those new options provided

[vague oracle]

the problem is that i tried to give it the options for each question asked.
Let's say i asked it
How Are you

it will read it as "How" "Are" "You"

the response will depend on that so if i add responses it will be either
("I'm fine", "Thank you for asking i'm doing good", "I'm glad you asked i'm doing good how about you?")

it's random

but it sometimes responds outside of that depending on if i complete the question of How are you or if i just ask How are

it will respond with other tags like "I'm 20 years old" or smth like that

do you think if i ask in #💬・chatgpt-llama-deepseek maybe people could help?

[regal matrix]

No that's a very different stuff. That's also not what I meant by options though. Let me write a better response

[vague oracle]

alright

if you want this is like the project i was working with that i'm stuck at changing the interface from a clothing store into a supermarket one
also the server part is something im not really talented in either like almost no info qwq
https://github.com/Jihad92/arabic-chatbot

at this point i give up so i don't mind sending it :(

i took this as a reference and i was trying to remake it but i failed horribly since my chatbot is not applicable with the professor

and i don't have much time either tbh

so relearning something rn is gonna end horribly especially since it's hard to find someone to explain something like that rn

[regal matrix]

So I read up Voiceflow basics (I recommend their blog posts in general): https://www.voiceflow.com/blog/nlu-design-how-to-train-and-use-a-natural-language-understanding-model

What you basically want to do is:

1. User input -> Classify intent -> Extract entities from user input -> Respond with template with entities as variables

2. Set threshold, if intent is not clear, ask again. If entities are not all present, ask for entities

3. You should still design to give other intents as possible options to continue to train your intent classifier

So say user ask:
"How much is Sealtest Milk 2 Litre?"

Your model recognized it's "Asking Price", you recognize "shopping_item" as entity, and respond with "Sealtest Milk 2 Litre is $X", with a "{shopping_item} is {shopping_item_price}". This also means you should have a catalog that is a dictionary of {shopping_item: shopping_item_price}. If, say, shopping_item is recognized by your NER, but not in your catalog, you can provide the most similar items based on text similarity.

If say user ask:
"How much is this?", you should still be able to recognize it's Asking Price, but "shopping_item" is not recognized, so you can respond with "I see you are asking price for an item. Please clarify which item you're asking the price of?"

[vague oracle]

this is an easy-made template tho right? like no coding used?

[regal matrix]

If say user ask:
"Hi, how are you?", you can also have an intent that is called "general chat" to work it out

Huh?

[regal matrix]

I mean I typed out everything imagining you're gonna code it out

I thought that's the requirement by your prof

[vague oracle]

ah yep i understand

yea i'm supposed to code it yep

[vague oracle]

so like decide the intents on voiceflow and organize them as in that

[regal matrix]

Oh I'm just using their blog post to show design principle

[vague oracle]

ah alright

[regal matrix]

They're a company that builds no-code chatbot yes

[vague oracle]

mhm

[regal matrix]

But it's just understanding what you'd need

[vague oracle]

i kinda got the basic understanding of it now

do you know the server part tho? like connect it with an interface and enable a server into it

[regal matrix]

Also

@vague oracle I just realized you're not using frameworks. Consider Rasa, which has open source framework to build chatbots: https://towardsdatascience.com/ai-chatbots-made-easy-courtesy-rasa-8ecdc2853e66

About server, I assume you mean deployment, you can find Deploy Rasa on AWS: https://medium.com/graymatrix/deploying-rasa-on-aws-ec2-and-using-the-model-stored-on-s3-9edfcfb15ffe

If this article doesn't work out, there should be others

[vague oracle]

i dont really understand from this point qwq

[regal matrix]

You can check out their YouTube channel: https://youtu.be/PfYBXidENlg

[vague oracle]

oh

[regal matrix]

It's like the number one mentioned framework when it comes to chatbot / assistant

[vague oracle]

so that's like an open source?

[regal matrix]

They have commercial version, but yes they have open sourced the framework

[vague oracle]

i could actually just redesign the open source on mine since i don't have much time tho

[vague oracle]

the problem is the server part hopefully this one could help out

[regal matrix]

I don't really get what you mean by server part problem

If you have everything coded up in a single repo, you can just spin up an EC2, clone your GitHub repo, start up your Python script, and share that EC2 URL to people

[vague oracle]

oo alr

[vague oracle]

@regal matrix Sorry to disturb you but i followed this video and created the chatbot through this video.
https://youtu.be/a37BL0stIuM
I created the interface and the chatbot but i couldn't clone it, apparently i forgot to clone and cd before starting the project i wanted to create an enviroment before doing it do you have any idea of how to do that using pycharm?

[regal matrix]

I don't use Pycharm, but I assume you should have access to the terminal still. Do you use pyenv or conda or Docker as your environment manager?

[vague oracle]

idk what those are :(

mind helping me like enable any of those?

[regal matrix]

You'd want to use Docker to run your application on a server. I don't know what do you mean by you couldn't clone it, but here's an example of using Docker and Pycharm: https://www.youtube.com/watch?v=bi0cKgmRuiA&ab_channel=PatrickLoeber

[vague oracle]

so i have to use docker to run the application on a server, but i don't have a server

does that allow me to use a virtual server?

[regal matrix]

You can run Docker locally

It's how you can use any OS on any machine, since the OS is virtualized

https://youtu.be/eGz9DS-aIeY Here's a good intro to Docker

[vague oracle]

so that basically helps me connect the bot to the interface and be able to run it right?

[regal matrix]

I know I'm throwing a lot.. but I also think your prof shouldn't ask you to build a whole system without these fundamental stuff

[vague oracle]

he didn't really give us much info tbh

do you think if i sharescreen for you with my team and show you what we need to be done?

[regal matrix]

Docker doesn't exactly do that, but I would imagine you're just launching a script?

[vague oracle]

yep

i can show you the project rn

[regal matrix]

So I don't understand what you can't do

[vague oracle]

i can't connect the bot to the interface right now

[regal matrix]

Post here.. if I continue to have time, I'll sprinkle commnets, but cna't promise dedicated help

[vague oracle]

alr

[regal matrix]

What does the error message say

[vague oracle]

1sec

it just doesn't work i don't know how to connect the bot to the interface at all

1sec

wait before asking about the interface part

do you know if i need a specific library to be able to add intents in arabic?

this is the intent

it doesn't read it

[vague oracle]

i think it's library based right?

[regal matrix]

You’d need an Arabic based language model. Are you still using Rasa?

[vague oracle]

i'm currently on python

[regal matrix]

You should try arabic models first: https://huggingface.co/asafaya/bert-base-arabic

[vague oracle]

so just add this to the code?

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("asafaya/bert-base-arabic")
model = AutoModelForMaskedLM.from_pretrained("asafaya/bert-base-arabic")

this?

[regal matrix]

I just realized you're not even using language models in your repo

It's honestly faster to start from scratch and follow someone's work, like those videos. That is, if you're solo

[vague oracle]

i tried doing that ^

https://youtu.be/a37BL0stIuM

this is what i followed rn

[regal matrix]

Yeah but you haven't showed error message when you say can't connect

But still, debugging is better than what you have on your GitHub

That's just a typical life

[vague oracle]

i skipped the github one

[regal matrix]

Plus it's a group project 😦

[vague oracle]

https://drive.google.com/file/d/11VdzQvUWm_XoEFqdNZFKMGeJe8ybEq8g/view?usp=sharing

this is the one i made using that vid

[regal matrix]

No...

I won't download anything like this

[vague oracle]

true :(

[vague oracle]

ah understandable

[regal matrix]

It should always be a GitHub repo

Or GitLab if you wish

[vague oracle]

mhm give me a sec

[vague oracle]

https://github.com/Tocash/testing

apologies for making you wait

i created this through the video

i didn't create a virtual enviroment for it tho..

my problem right now is that the javascript code is not working unsure why, can't use the interface properly and i can't enter arabic intents

whenever i enter an arabic intent it'd just return the "i do not understand" and ignore the intent entirely

@regal matrix i'm unsure of the problems since it doesn't give me an error

[regal matrix]

I don't know Javascript. But I imagine you should be able to write a request from Javascript. Typically Javascript runtime would send request to the component (think server) that handles inference, and send result back. I don't know how do you do local APIs. Now, I would again say build chatbots with existing chatbot framework rather than pure neural network with Bag of Words. Find one that can use Huggingface models, so that you can use other people's model: https://github.com/aub-mind/Arabic-Empathetic-Chatbot

But yeah, i would also get help from your school/mates. TAs are supposed to help too

[vague oracle]

i will be working with my mates i just need to figure out those parts that's all-

[vague oracle]

i could probably copy this model since i don't have much time to work all over again-

but i got to this error-

[regal matrix]

That's a warning not an error. So it's just working fine

[vague oracle]

but i can't use the bot tho

[regal matrix]

The model is large. You can try on Google Colab to make sure code is working, and find your way to deploy to a server etc.

[vague oracle]

ah alright

[vague oracle]

so thanks to your tips i was able to research on how to deploy on a server and create a virtual server and etc