05 Jul 2024 AI Engineering 4 min read

GraphRAG: Answering Global Questions with Knowledge Graphs

Microsoft has published GraphRAG: an indexing pipeline that uses an LLM to extract an entity graph from text, partitions it with the Leiden algorithm, and pre-summarizes every community. We explain how this answers global questions that vector RAG cannot, what the paper actually measured, and why indexing cost is the real trade-off.

Questions That Span an Entire Corpus

Retrieval-augmented generation, as commonly deployed, is a vector pipeline. Documents are split into chunks, chunks are embedded, and at query time the top-k most similar chunks are placed into the context window. This works well for local questions: the answer sits in a handful of passages, and semantic similarity finds them.

It fails for global questions. "What are the main themes in this dataset?" has no answer chunk. Top-k retrieval returns passages that merely resemble the question and produces a misleadingly confident summary of a random sample. The GraphRAG paper classifies such questions as query-focused summarization, not retrieval — and prior summarization methods do not scale to corpora of RAG size.

Entities and relations are extracted from every document. 1/4

What Microsoft Has Released

Microsoft Research introduced GraphRAG in a blog post on 13 February 2024, demonstrated on thousands of Russian and Ukrainian news articles from June 2023. On 24 April 2024 the team around Darren Edge and Jonathan Larson published the accompanying paper (arXiv:2404.16130), which announced an open-source Python implementation as forthcoming.

That implementation is now publicly accessible: the repository microsoft/graphrag is on GitHub under the MIT license, with documentation, an indexing engine, and two query modes — global and local search. Notably, there is no announcement post yet; the code is simply there. GraphRAG is not a vector-database replacement. It is an additional, more expensive index.

An LLM Extracts the Entity Graph

Indexing starts conventionally: source texts are split into chunks of around 600 tokens. Each chunk then passes through a multipart extraction prompt that returns entities, relationships between them, and short natural-language descriptions of both. Because a single pass misses items, the pipeline runs additional "gleaning" rounds, asking the model whether entities were overlooked, up to a configured maximum.

Chunk size is not cosmetic. The paper reports that 600-token chunks yielded almost twice as many entity references as 2400-token chunks on the same corpus. Duplicate descriptions of the same element are summarized into one. The result is a weighted, described entity graph — built without a predefined ontology and therefore domain-agnostic.

Leiden Communities as Index Structure

GraphRAG partitions this graph with the Leiden algorithm (Traag et al., 2019), a refinement of Louvain that guarantees well-connected communities. The partition is hierarchical and recursive: coarse root communities at level C0, progressively finer sub-communities at C1 through C3. Every node belongs to exactly one community per level.

This is the structural difference from vector search. A community partition covers all nodes, so every input text has influenced the index. Top-k retrieval samples the corpus; the community hierarchy describes it in full, at several resolutions.

Community Summaries and Map Reduce

During indexing, the LLM writes a report for every community — from element summaries at the leaf level, and from sub-community reports higher up. These reports exist before any question is asked. They function as a pre-computed, hierarchical summary of the corpus.

At query time, global search runs map-reduce. Map: the community reports of a chosen level are packed into context windows, and the model produces partial answers with helpfulness scores. Reduce: the highest-scoring partial answers are condensed into one final answer. Local search takes the other path — it resolves entities in the question and assembles their graph neighborhoods plus raw text.

What the Paper Measured

The evaluation used two corpora in the one-million-token range: podcast transcripts (about 1 million tokens) and news articles (about 1.7 million tokens), with 125 GPT-4-generated sensemaking questions per dataset and an LLM judge scoring comprehensiveness, diversity, empowerment, and directness in head-to-head comparisons.

GraphRAG won roughly 70–80% of comparisons against naive vector RAG on comprehensiveness and diversity. Naive RAG won on directness — its answers are terser. Against direct source-text summarization, root-level community summaries were competitive in quality while requiring over 97% fewer context tokens per query.

Approach	Global questions	Tokens per query	Index construction
Vector RAG (top-k)	fails	low	embeddings only — cheap
Source-text summarization	good answers	entire corpus — very high	none
GraphRAG (root level C0)	competitive quality	>97% below summarization	LLM extraction — expensive

The Bill Arrives at Indexing Time

The trade-off is stated plainly in the repository itself: indexing is warned to be an expensive operation. Mechanically, every 600-token chunk passes through the LLM at least once, gleaning rounds multiply that, element descriptions are summarized, and a report is generated for every community on every hierarchy level. For a million-token corpus this means thousands of LLM calls — orders of magnitude above the cost of embedding the same text.

Equally important is what GraphRAG does not do. It does not improve simple factoid lookups; naive RAG answers those more directly and far more cheaply. The released pipeline has no incremental updates — a changed corpus means re-indexing. Extraction errors propagate silently into the graph. And the evaluation rests on an LLM judge, two datasets, and one question class.

Outlook From June 2024

We expect the indexing cost to fall faster than skepticism suggests. Microsoft is already working on automatic tuning of extraction prompts, and smaller models specialized for extraction are an obvious next step. Ports into the LangChain and LlamaIndex ecosystems and managed graph-RAG offerings seem likely within months. The hierarchical community summary itself — a corpus that writes its own table of contents — is a pattern we expect to outlive this particular implementation.

Our working position at Blue IT Systems: treat the graph as a derived index, never as a second source of truth. Vector search remains correct for local questions. The graph index earns its cost where global questions are asked repeatedly against a stable corpus. Incremental indexing and rigorous evaluation standards are the open problems we will be watching.