RAG is a natural language querying approach for enhancing existing LLMs with external knowledge, so answers to questions are more relevant in case they require specific knowledge. It includes a retrieval information component that is used to fetch additional information from an external source, also known as “grounding context”, which is then fed to the LLM prompt to answer the desired question with higher accuracy.
This approach is the cheapest and most standard way to enhance LLMs with additional knowledge for the purposes of answering a question. In addition, it is shown to reduce the tendency of LLMs for hallucination, as the generation adheres more to the information from the context, which is generally reliable. Due to this nature of the approach, RAG emerged as the most popular way to augment the output of generative models.
To achieve Graph RAG for question answering, you need to select what part of the information that is available to you to send to the LLM. This is usually done by querying a database based on the intent in the user question. The most appropriate databases for this purpose are vector databases, which via embeddings capture the latent semantic meanings, syntactic structures, and relationships between items in a continuous vector space. The enriched prompt contains the user question together with the pre-selected additional information, so the generated answer takes it into account.
As simple as the basic implementation is, you need to take into account a list of challenges and considerations to ensure good quality of the results:
Data quality and relevance is crucial for the effectiveness of Graph RAG, so questions such as how to fetch the most relevant content to send the LLM and how much content to send it should be considered.
Handling dynamic knowledge is usually difficult as one needs to constantly update the vector index with new data. Depending on the size of the data this can impose further challenges such as efficiency and scalability of the system.
Transparency of the generated results is important to make the system trustworthy and usable. There are techniques for prompt engineering that can be used to stimulate the LLM to explain the source of the information included in the answer.
Graph RAG is an enhancement over the popular RAG approach. Graph RAG includes a graph database as a source of the contextual information sent to the LLM. Providing the LLM with textual chunks extracted from larger sized documents can lack the necessary context, factual correctness and language accuracy for the LLM to understand the received chunks in depth. Unlike sending plain text chunks of documents to the LLM, Graph RAG can also provide structured entity information to the LLM combining the entity textual description with its many properties and relationships, thus encouraging deeper insights facilitated by the LLM. With Graph RAG each record in the vector database can have contextually rich representation increasing the understandability of specific terminology, so the LLM can make better sense of specific subject domains. Graph RAG can be combined with the standard RAG approach to get the best of both worlds – the structure and accuracy of the graph representation combined with the vastness of textual content.
We can summarize several varieties of Graph RAG, depending on the nature of the questions, the domain and information in the knowledge graph at hand:
Graph as a Content Store: Extract relevant chunks of documents and ask the LLM to answer using them. This variety requires a KG containing relevant textual content and metadata about it as well as integration with a vector database.
Graph as a Database: Map (part of) the NL question to a graph query, execute the query and ask the LLM to summarize the results. This variety requires a graph that holds relevant factual information. The implementation of such a pattern requires some sort of NL-to-Graph-query tool and entity link
In conclusion, the Graph RAG approach represents a significant advancement in the enrichment of LLMs. By effectively combining the strengths of both retrieval-based and generative approaches, Graph RAG enhances the ability of LLMs to produce more accurate, relevant, and contextually informed responses. This technique not only improves the overall quality of outputs but also expands the capabilities of LLMs in handling complex and nuanced queries. As a result, Graph RAG opens up new possibilities in various applications, from advanced chatbots to sophisticated data analysis tools, making it a pivotal development in the field of natural language processing.
Business, governance, and adoption-focused material. Real-world implementations, case studies, and industry impact.
