🎍System Diagram

Summary: Here is the overall architectural overview diagram of the Telegram Chatbot System, designed to provide an automatic and intelligent interaction channel with users through the Telegram messaging platform. The system functions as a virtual assistant, allowing users to easily access information, perform tasks, or receive support through the familiar chat interface.

The system is capable of receiving and processing diverse types of interactions from Telegram users, including:

Pre-defined Commands (e.g., /start, /help, /feedback, /command, /account, ...).
Ordinary text messages using natural language.
Responses from users clicking on interactive buttons (Callback Query) within messages.

These three processors operate independently after the dispatching step, but they can all interact with common system components such as the Database or jointly use the Output Interface to send responses back to the user.

Upon receiving a request from a user, the system, via the API Gateway and Input Dispatcher, determines the type of interaction and routes it to the corresponding specialized processor:

1. Command Handler

Main Role: This module is responsible for handling structured requests, typically starting with a "/" character (e.g., /start, /help, /account, /command, etc.). These commands usually require the system to perform a specific, clearly defined task.

In-Depth Processing Flow Analysis:

Receiving Command: The system receives structured command data from the Input Dispatcher. This data includes the command name and may contain additional parameters (e.g., /start<user_id>).
Parsing & Command Identification: This module parses the command string to accurately extract the command name and any associated parameters (if present). Based on the command name, it identifies the specific action to be performed.
Internal Routing Logic: This is the main branching point within the Command Processor. Depending on the identified action:
- Branch 1: Interaction with Database: If the command requires dynamic data retrieval or manipulation (e.g., fetching user status, generating reports, updating configurations), the flow proceeds with:
  - Preparing and sending a query/request to the Database (Postgres DB).
  - Receiving the response from the Database.
  - Processing, filtering, and transforming the data from the Database to prepare it for display.
- Branch 2: Returning Preconfigured Results: If the command only requires returning static information or performing a simple function without needing a Database query (e.g., displaying /help or the /start greeting), the flow proceeds with:
  - Retrieving preconfigured information/content for that specific command.

Result Formatting: Results from both branches (either processed data from the DB or preconfigured content) are formatted into a message suitable for display on Telegram (e.g., plain text, Markdown, including inline buttons or a reply keyboard).

Sending Response:The formatted message is sent back to the user via the Output Interface.

Key Interactive Components: Database (Postgres DB), command parsing and routing logic, preconfigured result repository.

Purpose of This Structure: To ensure that structured, clearly defined requests are processed quickly, accurately, and efficiently, often involving system data retrieval or manipulation.

2. Text Handler - AI Agent

Main Role: This module processes ordinary text messages from users that are not commands or callback queries. This is where artificial intelligence (AI) comes into play to understand and respond to natural language.

Detailed Processing Flow:

Receive Message: Receives text message data from the Input Dispatcher.
Handover to AI Agent: The message is forwarded to the AI processing hub, the AI Agent Logic. This AI Agent acts as a brain, coordinating other AI components to process the request.
Interaction with Memory (Chat Memory): The AI Agent typically reads conversation history from the Chat Memory (Postgres DB) to understand the current conversation context. After processing the current request, it may also write the interaction to the Chat Memory to maintain context for subsequent interactions.
Interaction with AI Chat Model: The AI Agent sends the user's message (along with context from memory) to one or more AI Chat Models (In this project using: Llama 3.2 Chat Model) to:
- Understand the intent and entities in the query.
- Generate an initial response or determine if further information is needed.
Retrieval Augmented Generation (RAG) Process: If the user's request relates to specific information not available in the AI Chat Model's general knowledge, the AI Agent activates the Document Retrieval process:
- Uses an Embeddings Model (Embeddings Llama 3.2) to convert the user's query into numerical vectors.
- Searches for similar vectors in the Vector Store (Supabase Vector Store) containing vectors of internal documents/knowledge.
- Retrieves the most relevant document snippets.
- Feeds these retrieved document snippets (as "context") along with the original query back into the AI Chat Model (potentially a different instance - Llama 3.2) for the model to generate the final answer based on both the model's knowledge and the information from the documents.
Synthesis & Final Response Generation: The AI Agent synthesizes the result from the AI Chat Model (with or without RAG) and other information to create a complete answer.
Result Formatting: The answer is formatted into a Telegram message.
Send Response: The formatted message is sent out via the Output Interface.

Key Interacting Components: AI Agent Logic, AI Chat Model, Chat Memory (Postgres DB), Document Retrieval Logic, Embeddings Model, Vector Store (Supabase).

Purpose of this Structure: To enable the system to flexibly and intelligently understand and respond to natural language questions and requests from users, providing accurate and up-to-date information by combining the AI's conversational ability with the capability to retrieve specialized knowledge (RAG).

3. Callback Handler

Main Role: This module processes interactions resulting from users clicking on inline buttons within messages previously sent by the bot. Each button carries a small amount of data (callback_data) that specifies the action to be performed.

Detailed Processing Flow:

Receive Callback Query: Receives the identified callback query data from the Input Dispatcher. This data contains the callback_data and information about the original message as well as the user.
Extract Callback Data: Extracts the callback_data value from the received data.
Action Routing Logic: Similar to the Command Processor, this module determines the specific action to be performed based on the value of the callback_data. This is a branching point with many different possibilities, corresponding to each type of button.
Specific Action Processing Flows: Each branch from the router will execute a separate sequence of steps depending on the callback_data. Actions may include:
- Updating the content or interface of the original message the user interacted with.
- Sending a new message.
- Performing a business operation (e.g., confirming an order, voting, changing list pages). This operation may require interaction with the Database (Postgres DB) or calling other internal/external APIs.
- Displaying a small temporary notification (alert) to the user.
Acknowledge Callback Query: This is a crucial technical step. The system needs to send an acknowledgment response back to the Telegram API (using the answerCallbackQuery method) so that Telegram knows the query has been received and processed. This helps remove the loading state on the button on the user's side. This step should ideally be performed early after determining the action, but before executing time-consuming tasks.
Prepare & Format Response (if applicable): If the action requires sending or updating a message, the message content will be prepared and formatted.
Send / Update Message (if applicable): Executes sending a new message or updating the original message via the Output Interface.

Key Interacting Components: Callback data parsing and routing logic, specific business logic modules for each action, Database (depending on action), Telegram Bot API (for acknowledgment and updating/sending messages).

Purpose of this Structure: To allow users to interact directly with message content via buttons, providing a faster and more intuitive user experience compared to using text commands only, and to execute specific tasks assigned to each button.

4. Database

4.1. Database Replication

The database layer plays a central role in storing and managing data for the Telegram Chatbot system. We have chosen to use PostgreSQL as the primary database management system due to its high reliability, ability to handle structured and unstructured data (JSONB is very useful for chat/AI data), and strong community support.

Replication and Purpose of Use: To meet the requirements for High Availability (HA), Scalability, and High Load Capacity, we have implemented Database Replication for the PostgreSQL system. ==> Database Replication allows for the creation of data copies (replicas) on independent servers. This is particularly important for a chatbot system, where a large number of read requests (chat history queries, RAG data queries) can occur simultaneously.

Mechanism:

The database system is implemented using a Primary-Replica Replication model. One server is designated as the Primary (or Master), responsible for handling all write operations (INSERT, UPDATE, DELETE) to the database.
One or more other servers are configured as Replicas (or Standbys). The Replica servers continuously receive copies of the data from the Primary through a synchronization process (e.g., using PostgreSQL's Streaming Replication). This ensures that the data on the Replicas is nearly or completely identical to the data on the Primary.

4.2. Database Structure

After establishing a solid data infrastructure foundation with replication capability, we will continue to delve into the internal logical structure of the database. The next section will introduce the main tables in detail, where all important information is stored that helps shape and operate the chatbot's functionalities, from user identification to providing intelligent, data-driven responses.

Chat History table: This table stores the entire chat history between users and the chatbot.

Function in the System: This is a core data table, particularly important for the AI Agent functionality. The chat history is used to provide context for the AI Agent to better understand the user's intent and previous interactions, thereby generating more coherent and relevant responses. It can also be used for user behavior analysis or reviewing interactions.

Context info table: This table is designed to store metadata and reference information related to content used in the RAG technique, including embeddings, version information of the original content, and other descriptive data

Function in the System: This table is an essential component of the RAG pipeline. It links the embeddings (stored in the Vector Store - Supabase in your case) with the corresponding original content, source information, update timestamp, content version, etc. This allows the system not only to search based on semantics (via vectors) but also to retrieve and display the accurate original information, as well as manage the lifecycle and updates of the RAG data sources.

User table: This table is used to store identification information and related attributes about users interacting with the chatbot.

Function in the System: Contains basic information about Telegram users (e.g., Telegram User ID, username, active status, join time, etc.). Data from this table is crucial for user identification, personalizing the experience, and managing access.

Role table: This table defines different roles or permission groups that users can have within the system.

Role in the System: Supports access control and functional authorization. Each user can be assigned one or more roles, thereby determining which commands they can use, which types of information they can access, or whether they have certain administrative privileges. This table provides the system with flexibility in managing users with different levels of interaction.

Feedback table: This table records feedback, ratings, or suggestions from users regarding the quality or features of the chatbot.

Function in the System: Data in the feedback table provides valuable information to the development team to understand user satisfaction levels, detect bugs, gather new feature requests, and continuously improve the chatbot's service quality.

User Question table: This table is designed to store information about randomly selected questions (selected_questions) for individual users. Each record will be uniquely identified by the user's chat_id and the specific message_id associated with the interaction where the questions were presented.

Function in the System:

Store Randomly Generated Question Lists: When a user requests suggested questions, the system will randomly generate a list. This table then persists this specific list (selected_questions) for each user, uniquely identified by their chat_id and the message_id of the interaction where the suggestions were provided. This ensures that the user sees a consistent set of suggestions for that particular message.
Facilitate AI Analysis and Response: If the user selects a suggested question from the displayed list, the system will retrieve the historical question data stored in this table (using the chat_id and message_id). This historical context is then sent to the AI Agent for analysis and to generate an appropriate response, which is subsequently delivered back to the user.
Optimize Data Management: To maintain efficiency and optimize query performance, the suggested question lists have a defined validity period. Once this period expires, the system will automatically clear these records. This proactive data clearing helps reduce the overall data load and keeps the database streamlined.

5. Notification

The automatic notification function plays an essential role in the Telegram Chatbot system, allowing the system to proactively send important updates, alerts, or related information to users without requiring users to make direct requests. This helps maintain engagement, provides timely information, and enhances the user experience by keeping them constantly updated on important events from various sources (internal systems, on-chain, etc.).

Operational Flow:

Event Source: Events originate from various sources outside or inside the system, e.g., state changes on the blockchain (on-chain), updates from external APIs, or internal system events. This is the starting point of the notification flow.
Event Listener: This module is responsible for monitoring and listening for events from the configured Event Sources. When an event is detected, the Event Listener collects the event data and performs initial preprocessing steps if necessary (e.g., standardizing the event data format).
Queue: After being collected by the Event Listener, events are placed into a queue. The queue acts as a buffer, separating the event collection and event processing processes. This is particularly important so the system can effectively handle sudden bursts of events (event bursts) and ensure no events are lost during processing.
Notification Processor Service: This is the main business logic module, responsible for fetching events from the Queue for processing. For each event, the Notification Processor Service will:
- Query/Update Database (Postgresql DB): Interacts with the database to retrieve necessary information related to the event (e.g., details about the change, user information needing notification based on subscription configuration, related historical data). It may also update the processing status of the event in the DB.
- Process Notification Logic: Based on the event type and data from the Database, this module executes business logic to determine the specific notification content to be sent, format the notification, and identify the list of users to receive the notification.
- Prepare Output: Creates the final data structure or message string ready to be sent.
Telegram Sender: This module receives the finalized message content and recipient information from the Notification Processor Service. Its sole task is to execute sending the message to the Telegram API.
Output Interface: This is the system's common communication layer with the Telegram API. The Telegram Sender uses the Output Interface to call the necessary API methods (e.g., sendMessage) to push messages out to the Telegram platform.
Users: Finally, the automatic notification message is successfully sent and displayed on the user's Telegram application.

Integrating the automatic notification function into the chatbot's structure helps the chatbot system become proactive and provides continuous value to users.

6. RAG

To enhance the ability to answer user questions accurately, up-to-date, and based on specific information sources (beyond the general knowledge of the language model), our Telegram Chatbot system has integrated the Retrieval Augmented Generation (RAG) technique. RAG allows the Large Language Model (LLM) to retrieve relevant information from an external knowledge base before generating the final response. This is particularly useful when needing to answer questions about specialized, internal data or very new information that the model may not have been trained on.

Our chatbot's RAG system consists of two main pipelines:

Ingestion Pipeline: The process of preparing and loading data from various sources into the Vector Database.
Query Pipeline: The process of using data in the Vector Database to augment the LLM's answer when a user asks a question (already described in the Text Message Processor - AI Agent section).

This section will focus on detailing the Ingestion Pipeline.

6.1. Ingestion Pipeline

The Ingestion Pipeline is responsible for collecting, processing, and transforming data from various formats into semantically searchable numerical vectors, then storing them in the Vector Database. This process ensures that the knowledge base for RAG is always ready and up-to-date.

The Ingestion Pipeline flow is illustrated in the diagram "Ingestion Pipeline". The main steps include:

Raw Data Sources: This is the starting point, including diverse data sources such as documents (PDF, DOCX), spreadsheets (Excel), web links (URLs), or other data formats. The system is designed to be able to receive information from various sources to build a comprehensive knowledge base.
Data Loader: This module is responsible for reading and extracting content from the Raw Data Sources. It handles different file formats and loads the data into the system for further processing.
Data Preprocessor: The raw data after loading needs to be cleaned and standardized. This step includes removing unnecessary characters, handling formatting, and converting the content to plain text. In this pipeline, the AI Agent plays a supporting role in the preprocessing process, potentially helping extract important information, classify content, or perform more complex cleaning tasks to optimize the quality of the input data for subsequent steps.
Text Splitter: Documents are often long. This module splits the preprocessed text into smaller segments (chunks) of appropriate size. Splitting into segments helps ensure that each text segment, when converted into a vector, carries clear contextual meaning suitable for later search.
Embedding Creator: This module receives the text segments (chunks) from the Text Splitter. It sends these text segments to the Embedding Model Service to convert them into numerical vectors (embeddings) representing the semantic meaning of that text segment.
Embedding Model Service: This is an external AI service (e.g., using Google Gemini's Embeddings model or other models like Llama - as potentially mentioned in your diagram) specializing in creating embedding vectors from text. The Embedding Creator module interacts with this service via an API.
Vector Database (Supabase Vector Store): Finally, pairs of (original text segment, corresponding embedding vector) are stored in the Vector Database (using Supabase Vector Store). The Vector Database is optimized for efficient storage and similarity search of numerical vectors, serving as the foundation for the retrieval step in the Query Pipeline.

This Ingestion Pipeline process ensures that the RAG system always has an organized and updated knowledge base, ready to serve intelligent query requests from users via the AI Agent.

6.2. Query Pipeline

(This section can refer to the detailed description in Section 2. Text Message Processor - AI Agent, particularly the steps related to Retrieve Documents, interaction with the Vector Store and Embedding Model, and using the retrieval results to augment the AI Chat Model. You can briefly summarize here and refer back to the previous section.) When a user asks a text question, the AI Agent activates the Query Pipeline. The user's question is converted into a vector, and the Vector Database is searched for similar vectors to retrieve related text segments. These text segments are then fed along with the original question into the AI Chat Model to generate the final answer, combining the model's knowledge and the retrieved information.

The RAG system, with its robust Ingestion Pipeline and efficient Query Pipeline, is a core component that helps our chatbot provide accurate, reliable, and data-driven responses, significantly enhancing the user interaction experience.

PreviousLlama 3.2 model NextFeatures

Last updated 1 month ago