I'm new to the whole RAG. But keen to learn about i have a planned hackathon where they want us to use ai to help the citezens I nthe municipality I thought of creating a rag chatbot called as ur municipality and it needs to using data from the db embeddings to answer questions or users the main seeling point is get ur answer in minutes not hours
#RAG chatbot
granite whale
potent radishBOT
Hi @granite whale. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
Absolutely, building a RAG (Retrieval-Augmented Generation) chatbot for your municipality is a great idea, especially for providing quick, accurate answers to citizens using AI and your own data.
What is RAG?
RAG combines a Large Language Model (LLM) with a search over your own knowledge base (like municipal documents, FAQs, regulations, etc.). When a user asks a question, the system:
- Searches your database for relevant information using embeddings (semantic search).
- Passes the most relevant context to the LLM.
- The LLM generates an answer based on both the user’s question and the retrieved context, making responses more accurate and up-to-date than a generic chatbot.
How to Build a RAG Chatbot with Convex
Convex provides all the building blocks you need:
- Vector Search: Store and search embeddings for your documents.
- RAG Component: Handles chunking, embedding, and searching your data.
- Agent Component: Manages chat threads, message history, and integrates with LLMs.
Basic Flow
- Ingest Data: Add your municipal documents to the RAG component. It will chunk and embed them for you.
- User Asks a Question: The chatbot receives a question.
- Semantic Search: The RAG component searches for the most relevant chunks using vector similarity.
- LLM Response: The context and question are sent to the LLM, which generates an answer.
- Return Answer: The answer is sent back to the user, often within seconds.
Example Code Snippet
Here’s a simplified example of how you might set up a search and answer action using Convex’s RAG component:
export const askQuestion = action({
args: {
prompt: v.string(),
},
handler: async (ctx, args) => {
const userId = await getAuthUserId(ctx);
const { text, context } = await rag.generateText(ctx, {
search: { namespace: userId, limit: 10 },
prompt: args.prompt,
model: openai.chat("gpt-4o-mini"),
});
return { answer: text, context };
},
});
This will search your indexed data and generate an answer using the LLM, returning both the answer and the context used for transparency and trustworthiness. You can organize your data by namespaces (e.g., per department or user) and filter as needed for your use case. For more details, see the RAG component documentation.
Key Features for Your Use Case
- Fast, accurate answers: Citizens get responses in minutes, not hours.
- Up-to-date information: Easily update your knowledge base as municipal data changes.
- Transparency: Optionally show users the sources used to answer their questions.
- Scalability: Convex handles real-time updates and can scale as your usage grows.
Getting Started
- Set up a Convex project.
- Install the RAG component:
npm install @convex-dev/rag - Ingest your municipal data.
- Build your chat UI and connect it to your Convex backend.
For a hands-on example, check out the RAG example repo and the official docs.
If you need more advanced workflows (like tool-based RAG, where the AI decides when to search), Convex supports that too. See the RAG approaches for details.
Good luck with your hackathon! If you have more specific questions as you build, feel free to ask.
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
potent radishBOT
Let me check the knowledge base for you again. One moment! :books:
Sorry @granite whale, I seem to be having a bit of trouble. Don't worry though, I've pinged my support and they will try to fix me up! :technologist:
Perhaps try again in 15mins?
granite whale
@potent radish for the chat UI and backend connection what should I be using
potent radishBOT
Let me check the knowledge base for you again. One moment! :books:
For connecting your chat UI to the backend in a Convex-powered RAG chatbot, you should use the Convex client libraries for your frontend framework (such as React) and call your backend functions (queries, mutations, or actions) via these libraries.
Frontend (UI):
- If you’re using React, use the
convex/reactpackage. This provides hooks likeuseQueryanduseMutationto interact with your Convex backend in real time. - For streaming chat (like a ChatGPT experience), you can use the
@convex-dev/persistent-text-streaming/reactpackage, which provides auseStreamhook for real-time streaming updates in your UI. This is especially useful for displaying AI-generated responses as they stream in, rather than waiting for the full response to complete.
Backend:
- Write your backend logic (such as message storage, RAG search, and LLM calls) as Convex functions (mutations, queries, or actions) in your
convex/directory. - Use the RAG component for semantic search and context retrieval, and connect it to your LLM of choice.
Example Frontend Integration:
import { useMutation } from "convex/react";
import { api } from "../convex/_generated/api";
const sendMessage = useMutation(api.chat.sendMessage);
<form onSubmit={async (e) => {
e.preventDefault();
await sendMessage({ user: NAME, body: newMessageText });
setNewMessageText("");
}}>
{/* ... */}
</form>
This pattern allows your UI to send messages to the backend and receive real-time updates as new messages arrive or as AI responses stream in. For streaming, you would use the useStream hook as shown in the persistent text streaming example.
Summary:
- Use Convex’s React client (
convex/react) for real-time queries and mutations. - Use
@convex-dev/persistent-text-streaming/reactfor streaming AI responses. - Backend logic lives in Convex functions, using the RAG component for retrieval and LLM integration.