Architecture

Akcio is a chatbot system that utilizes a retrieval augmented large language model (LLM) to provide users with more accurate and relevant information. The system is designed to retrieve related information from a knowledge base through semantic search and then generate an answer using a large language model or service such as GPT-3.5.

The Akcio system comprises five modules: Agent, LLM, Embedding, Store, and DataLoader. If you want to learn more about each of these modules, you can refer to Modules. The entire architecture is constructed using these modules and can be divided into two main phases: the Insert phase and the Query phase. Now, let's take a closer look at each of these stages.

Insert

The Insert phase in the Akcio architecture is responsible for building the knowledge base by archiving documents in advance. This phase involves the following key components:

• DataLoader in this phase is responsible for preparing data by loading documents from a data source and splitting documents into smaller chunks.
• Embedding converts document chunks into vectors or embeddings.
• VectorStore and ScalarStore are used to save the document chunks with corresponding embeddings and metadata.

The DataLoader prepares the data for Embedding, which converts the document chunks into embeddings. These embeddings are then stored in the Stores along with metadata such as the document's source. This phase ensures that the knowledge base is well-structured and easy to access.

Query

The Query phase in the Akcio architecture is responsible for processing user queries, retrieving related information from the knowledge base, and generating accurate and relevant responses. This part involves the following key components:

• Vector store is responsible for retrieving information from the knowledge base through semantic search. It includes Embedding module to convert the user's query into an embedding first and then searches the vector database for document chunks with similar embeddings.
• Scalar Store in query stage is responsible for retrieving information from the knowledge base through keyword matching.
• Memory Store stores and provides chat history to better support context in conversation.
• LLM ensures the core functionality of conversation via calling a chat API powered by some large language model.

The query stage processes user queries and retrieves related information from the knowledge base through the Vector Store and Scalar Store. Additionally, it uses Memory Store to ensure that the conversation can provide contextually relevant responses. In order to generate better answers, this stage passes user query, retrieved information, and chat history to large language model or service such as GPT-3.5.

Overall, the architecture of Akcio ensures that the chatbot can provide users with a fluent conversation and more accurate answers. By retrieving information from a knowledge base, the factuality of LLM is improved, making Akcio a reliable and effective system for information retrieval.