Sizing chunks so Graph-RAG extraction never loses an edge

Chunking is usually discussed as a retrieval problem: cut the corpus into pieces that fit an embedding window and that a nearest-neighbour search can rank. Graph-RAG changes what the chunk is for. Before anything is retrieved, every chunk is handed to an information extraction (IE) step — the process that reads unstructured text and emits structured facts. The output unit of IE is a triple: a subject, a predicate, and an object, such as (Policy 4.2, supersedes, Policy 3.9). Triples become the nodes and edges of the graph, so the chunk is not a retrieval unit first. It is an extraction unit, and its boundaries decide which facts the graph is capable of containing at all.

The rule below encodes that reframing. It routes each document by total token count into one of four strategies, and attaches a quality floor every chunk has to clear before extraction is worth paying for.

The original note

Quick rule when Chunking for Graph-RAG

≤300 tokens → don’t chunk. One extraction unit (prepend a rich header: IDs, section title). Run sentence-level IE.

300–900 → sentence-aware chunks (~300 tokens, 15–20% overlap). Stop at sentence boundaries; ensure ≥2 entities + 1 predicate — merge forward if not.

900–3500 → section-aware chunks (≈400–700 tokens, 15–20% overlap). Keep headers with the chunk; never split tables/code blocks.

>3500 → hierarchical: section → paragraph micro-chunks (300–500) for extraction; maintain parent section IDs.

Chunking for a graph is not chunking for a vector index

In vector RAG, a bad boundary is a ranking problem. A chunk cut in half still sits in the index; it just embeds worse or arrives with less context. The damage is graded and repairable at query time — retrieve the neighbours, widen the window, re-rank.

In Graph-RAG the same boundary is not graded. If a cut lands between a subject and its predicate, one chunk holds an entity with no assertion attached and the next holds an assertion with no subject. Neither half satisfies the minimum shape of a triple. The extractor does not emit a weak edge — it emits no edge. If that mention was the only place the entity appeared in a relational context, it lands in the graph as an orphan node: present, connected to nothing, unreachable by any traversal starting elsewhere.

Three properties make this worse than it sounds:

The loss is silent. Nothing throws, and the pipeline reports a normal triple count. The missing fact surfaces much later as a question the graph cannot answer, with nothing pointing back at a boundary.
The loss is permanent until a full re-run. Extraction is a batch write, so a bad boundary is baked in; correcting it means re-extracting documents, not re-querying them.
The chunk is the coreference window. Resolving “it”, “the service”, or “this control” only works over the span the extractor can see. A boundary does not merely split a sentence; it deletes the antecedent.

The objective function changes accordingly. Vector chunking optimises for retrievability per chunk; graph chunking optimises for self-containment per chunk.

The rule, tier by tier

Tokens	Mode	Boundary
≤300	one unit	none
300–900	sentence-aware, ~300	sentence
900–3500	section-aware, 400–700	section
>3500	hierarchical, 300–500	paragraph

≤300 tokens — don’t chunk

At this size there is nothing to gain from splitting and a boundary to lose. The whole document becomes one extraction unit.

What the tier does add is a rich header: document ID, section path, section title. Headers matter because body text is routinely written for a reader who already knows the context. A ticket says “the timeout was raised to 30s” and never names the service; a policy clause says “this requirement applies to all vendors” and never names the policy. Without the header the extractor has no subject to bind to and either drops the fact or invents one. The header restores the subject, and it doubles as the provenance key that lets any edge in the finished graph be traced back to the span it came from.

The tier also specifies sentence-level IE: run extraction sentence by sentence with the header as standing context. Documents this short are cheap enough for per-sentence precision, which reduces the chance the extractor blends two adjacent facts into one wrong triple.

300–900 — sentence-aware chunks

Target roughly 300 tokens, stop only at sentence boundaries, and overlap 15–20%.

The sentence is the smallest span that reliably contains a complete predicate, so it is the smallest boundary that does not risk severing one. Cutting at a fixed token count instead puts the boundary wherever the arithmetic lands, which is mid-clause most of the time.

The 15–20% overlap makes the seam survivable. Roughly one to two sentences of the previous chunk are carried into the next, so a fact whose subject sits in one sentence and whose predicate sits in the following one appears intact in at least one of the two. The band has a ceiling for a reason: overlap duplicates text, duplicated text produces duplicate triples, and duplicate triples inflate any mention-count signal used for edge weight. Below the band the seam is naked; above it the same sentence starts appearing in three consecutive chunks.

900–3500 — section-aware chunks

Target 400–700 tokens with the same overlap band, but cut on sections rather than sentences.

A heading is an author-declared semantic boundary. Topic shifts and coreference chains tend to respect it, which makes it safer to cut on than any position chosen by token count. Two constraints ride along:

Keep headers with the chunk. For most of what follows a heading, the heading is the subject. A chunk that begins after “3.2 Retention requirements” without carrying that string is a list of assertions with no topic.
Never split tables or code blocks. A table row is only interpretable next to its column headers; split the table and the extractor receives untyped values, then either emits nothing or guesses. A code block cut mid-function loses the signature naming the thing described. Both are atomic units that pass through whole, even when they exceed the target size.

>3500 — hierarchical

Long documents get two levels: sections for structure and retrieval, paragraph micro-chunks of 300–500 tokens for extraction. Extraction runs on the small unit, where precision is highest, while the section remains the addressable container.

The load-bearing detail is maintaining parent section IDs. The micro-chunk is the extraction unit, but the section is the disambiguation unit. Carrying the parent ID does three things: triples can be re-attached to the section they came from, so retrieval returns coherent context rather than a fragment; entity resolution gets a scope, since two mentions of “the controller” under the same parent are probably one node while mentions under different parents probably are not; and orphan entities can be re-linked against their siblings instead of dropped. Without it, a long document decomposes into a pile of unrelated small texts and the author’s structure is thrown away.

The floor: two entities and one predicate

The rule attaches a quality gate to the sentence-aware tier: ≥2 entities + 1 predicate, merge forward if not met.

This is the minimum shape of a triple restated as a test on the input: two nodes and an edge between them. A chunk with one entity and no relational verb can at best contribute a bare node — no structure, no traversal value — while still costing a full extraction call.

The check is deliberately cheap: a lightweight named-entity pass and a predicate heuristic, run before the expensive call rather than by the extraction model itself. The remedy is merge forward, not backward, because forward preserves reading order and the common failure is a trailing subject whose predicate arrives in the next sentence. Merging backward would attach that subject to text preceding it and change what the chunk asserts.

Two details keep the merge safe. It must be bounded — it can only consume units that exist, so it terminates at the end of the document rather than spinning. And the final chunk has no forward neighbour, so a failing tail folds into its predecessor rather than being emitted as a chunk that cannot yield anything, or dropped.

A playbook

Measure the token distribution of the corpus before choosing a strategy. Histogram per-document token counts with the same tokenizer the extractor uses. A corpus mostly under 300 tokens needs almost no chunker; one clustered above 3500 needs the hierarchical path from day one.
Route per document, not per corpus. A single global chunk size is the failure the rule exists to prevent. The tier is a property of the document in front of the pipeline.
Normalise structure before applying size logic. Parse headings, tables, and fenced code into typed units first, so the never-split guard has something to guard.
Prepend the rich header to every chunk, including ones produced by splitting, with the parent section ID whenever the document took the hierarchical path.
Enforce the floor before the expensive call, and merge forward when it fails.
Validate with chunk QA, not just extraction QA. Sample chunks and check boundary properties directly: no chunk opens with an unresolved pronoun, no table or code block is split, the header is present, realised overlap sits inside the 15–20% band, and every chunk clears the floor.
Count orphan nodes after extraction. Orphans concentrated in one document type or tier are evidence of a chunking bug, not a model bug, and the fix is a boundary change rather than a prompt change.

Failure modes

Dangling pronouns across a boundary. A chunk beginning “It supersedes the prior control” has no resolvable subject. The extractor either drops the fact or binds “it” to whatever entity is nearest in the visible span, producing a confidently wrong edge. A fabricated edge is worse than a missing one, because it will be retrieved and cited.
Split tables and code blocks. Column headers land in one chunk and rows in another, so the rows become untyped strings. Same for a function split from its signature.
Headers stripped from chunks. Extraction still succeeds and the metrics look normal, but the triples have no stable subject and no provenance. Entity resolution then merges unrelated nodes whose only distinguishing context was in the header.
Overlap set too high. The same sentence appears in three chunks, the extractor emits the same triple three times, and edge weights derived from mention frequency inflate. Ranking then reflects chunk arithmetic rather than the corpus.

Implementation

The router turns the four thresholds into a small configuration object. Sizes inside each band are the midpoint of the stated range; overlap is the midpoint of 15–20%.

from dataclasses import dataclass
from typing import List, Tuple


@dataclass(frozen=True)
class ChunkPlan:
    """How one document should be cut before extraction runs."""

    mode: str             # whole | sentence | section | hierarchical
    target_tokens: int    # size aimed for, not a hard ceiling
    overlap_ratio: float  # 0.15-0.20, carried back across the seam
    boundary: str         # none | sentence | section | paragraph
    keep_parent_ids: bool


def choose_strategy(token_count: int) -> ChunkPlan:
    """Route a document to a chunking mode by total token count."""
    if token_count <= 300:
        # One extraction unit: no boundary, no severed predicate.
        return ChunkPlan("whole", token_count, 0.0, "none", False)
    if token_count <= 900:
        return ChunkPlan("sentence", 300, 0.175, "sentence", False)
    if token_count <= 3500:
        return ChunkPlan("section", 550, 0.175, "section", True)
    return ChunkPlan("hierarchical", 400, 0.175, "paragraph", True)

The header is built once per document and travels with every chunk cut from it; the atomic guard keeps tables and code blocks intact.

def build_header(doc_id, section_path, section_title, parent_id=None):
    """Build the rich header that travels with every chunk.

    The header exists so a chunk is self-identifying. Body text often
    says "the service" or "this control" and never names the subject;
    the header restores it, and it is also the provenance key used to
    trace an edge back to the span it came from.
    """
    parts = [
        f"doc={doc_id}",
        "path=" + " > ".join(section_path),
        f"section={section_title}",
    ]
    if parent_id:
        # Hierarchical mode: keeps a micro-chunk attached to a scope
        # for entity resolution and for re-assembly after extraction.
        parts.append(f"parent={parent_id}")
    return "[" + " | ".join(parts) + "]"


def is_atomic(unit: str) -> bool:
    """Never-split guard: tables and fenced code pass through whole."""
    text = unit.lstrip()
    if text.startswith("```") or text.startswith("~~~"):
        return True
    lines = [ln for ln in text.splitlines() if ln.strip()]
    if len(lines) >= 2 and all(ln.count("|") >= 2 for ln in lines):
        return True
    return False

The chunker fills to the target on sentence boundaries, merges forward when the floor fails, and steps back by the overlap budget without ever failing to advance.

def yields_a_triple(text, extract_entities, count_predicates) -> bool:
    """The floor: >=2 distinct entities and >=1 predicate.

    Below this floor a chunk cannot produce even one (subject,
    predicate, object) triple, so extraction spend on it is wasted.
    """
    return (len(set(extract_entities(text))) >= 2
            and count_predicates(text) >= 1)


def _next_start(start: int, end: int, sizes, overlap_budget: int) -> int:
    """Step back from `end` to carry overlap into the next chunk.

    Never steps below start + 1, so the outer loop always advances.
    """
    idx = end
    carried = 0
    while idx > start + 1 and carried + sizes[idx - 1] <= overlap_budget:
        idx -= 1
        carried += sizes[idx]
    return idx


def sentence_aware_chunks(units, plan, count_tokens, extract_entities,
                          count_predicates):
    """Cut an ordered list of units into half-open chunk spans.

    Atomic units are emitted alone. A span that fails the floor merges
    forward; a failing final span folds into its predecessor so that
    the tail is never dropped.
    """
    n = len(units)
    sizes = [count_tokens(u) for u in units]
    atomic = [is_atomic(u) for u in units]
    target = plan.target_tokens
    overlap_budget = int(round(target * plan.overlap_ratio))
    spans: List[Tuple[int, int]] = []

    start = 0
    while start < n:
        if atomic[start]:
            spans.append((start, start + 1))
            start += 1
            continue

        end, used = start, 0
        while end < n and not atomic[end]:
            if end > start and used + sizes[end] > target:
                break
            used += sizes[end]
            end += 1

        # Merge forward one unit at a time until the span can yield a
        # triple, or until an atomic unit or the end of the document
        # stops it. Bounded by n, so it cannot loop forever.
        while end < n and not atomic[end]:
            text = " ".join(units[start:end])
            if yields_a_triple(text, extract_entities, count_predicates):
                break
            end += 1

        spans.append((start, end))
        if end >= n:
            break
        start = _next_start(start, end, sizes, overlap_budget)

    if len(spans) > 1:
        s, e = spans[-1]
        prev_s, prev_e = spans[-2]
        tail = " ".join(units[s:e])
        mergeable = not atomic[s] and not atomic[prev_s]
        if mergeable and not yields_a_triple(
                tail, extract_entities, count_predicates):
            spans[-2] = (prev_s, e)
            spans.pop()
    return spans

The token counter and the entity/predicate probes are injected so the logic stands alone. A real deployment replaces the stubs with the extractor’s own tokenizer and a trained tagger; the interface does not change.

import re

# Stubs only. A real deployment swaps in the model tokenizer used by
# the extractor and a trained NER / relation tagger.

CAPITALISED = re.compile(r"\b[A-Z][A-Za-z0-9_.-]{2,}\b")
PREDICATES = ("is", "are", "was", "were", "has", "have", "uses",
              "supersedes", "replaces", "requires", "returns",
              "depends", "owns", "emits")


def count_tokens(text: str) -> int:
    """Whitespace proxy for a real tokenizer."""
    return max(1, len(text.split()))


def extract_entities(text: str) -> List[str]:
    """Cheap NER stub: capitalised spans stand in for entities."""
    return CAPITALISED.findall(text)


def count_predicates(text: str) -> int:
    """Cheap relation stub: count known predicate verbs."""
    words = [w.strip(".,;:()").lower() for w in text.split()]
    return sum(1 for w in words if w in PREDICATES)


def chunk_document(doc_id, section_title, section_path, units,
                   parent_id=None):
    """End to end: measure, route, cut, then prepend the header."""
    total = sum(count_tokens(u) for u in units)
    plan = choose_strategy(total)
    header = build_header(
        doc_id, section_path, section_title,
        parent_id if plan.keep_parent_ids else None,
    )
    if plan.mode == "whole":
        return [f"{header}\n" + " ".join(units)]
    spans = sentence_aware_chunks(units, plan, count_tokens,
                                  extract_entities, count_predicates)
    return [f"{header}\n" + " ".join(units[s:e]) for s, e in spans]

Closing

The rule encodes one idea: in Graph-RAG the chunk boundary is where facts are created or destroyed, so size should follow the document rather than be fixed across the corpus. Measure the distribution first, route each document to its tier, keep the header attached, refuse to split what cannot be split, and never send a chunk to the extractor that could not produce a triple even in the best case.