RAG finds the sentence. SPIRE remembers where it lives.
Most RAG pipelines flatten HTML into chunks and throw away the structure that makes evidence intelligible. SPIRE keeps the document tree, retrieves precise sentence-level paths, and adds context only when it is needed.
RAG is very good at finding a sentence. It is much worse at remembering why that sentence means what it means.
A sentence lifted from the middle of a list can be technically correct and still be useless without the list heading. A table cell without its row and column labels is not evidence; it is a number floating in space. A paragraph whose topic is established only by a section title becomes strangely vague the moment the title is removed.
The embedding did not necessarily fail. We asked the wrong object to speak.
Most retrieval systems turn an HTML page into a flat sequence of fixed-size chunks before indexing it. That makes the interface simple, but it also throws away the document's own map: sections, headers, lists, tables, links, and the relationships between them.
Mike Rainey, Umut Acar, and I wrote SPIRE around a different idea: retrieve a precise piece of evidence, keep its structural identity, and decide how much context to show only after we know what the query needs.
Precision in. Context when needed. Provenance preserved.
Select a stage to see what SPIRE stores, what it renders, and why the separation matters.
HTML tree
Keep sections, headers, lists, tables, and links as addressable structure.
D = document treeThe bug is not always the embedding
Imagine a question asks, "How many state parks are there in Virginia?"
A vector search can find the sentence:
Virginia has over 41 state parks with diverse landscapes.
That is a good hit. But now imagine the retrieved text is a bullet from a section called Key Takeaways, or a sentence whose meaning depends on the paragraph immediately above it. The sentence is still similar to the query. It is just no longer self-contained.
This is the recurring tradeoff in RAG:
- Retrieve tiny units and you get precision, but lose context.
- Retrieve large passages and you preserve context, but dilute relevance and waste the token budget.
Fixed-size chunking forces us to choose one compromise for the whole corpus. Real documents do not work that way. A sentence in a table needs different context from a sentence in a numbered list. A paragraph under an h3 needs a different amount of context from a paragraph under an h1.
The structure is already in the document. SPIRE keeps it.
Treat the document as a tree again
The first design decision is to represent an HTML document as a tree rather than as a long string. Every node receives a stable path made from child indices. A sentence, paragraph, list item, or table cell can then be addressed without copying it out of the document.
The retrieval unit is a subdocument: a document tree plus a set of paths that identifies the material we want.
D = the original HTML tree
P = the paths selected by retrieval
Link(D, P) = add the ancestors and descendants needed for a valid tree
Prune(D, P) = remove everything outside the selected paths
Subdoc(D, P) = Prune(D, Link(D, P))
This looks small, but it changes what the system can preserve. The selected sentence is not replaced by a copied string. It remains connected to the exact title, header, list item, or table cell from which it came.
The thing being retrieved is a view over the tree.
Click a node. The citation seed stays precise while global and local context remain available around it.
The distinction between a path set and a rendered excerpt is important. SPIRE can manipulate the path set cheaply while it is deciding what to retrieve. It materializes and renders a concrete subdocument only when an embedding model or a generative model actually needs to read it.
That is delayed materialization in a retrieval pipeline: keep identity cheap and stable; spend tokens only on the view that is needed.
Context has two different shapes
SPIRE separates context into two operations because "context" is not one thing.
| Context | What it adds | Why it matters |
|---|---|---|
| Global | Titles, active headers, list labels, table row and column labels | Makes a selection interpretable even when the relevant structure is not nearby in the tree |
| Local | Nearby sentences, paragraphs, list items, and block-level neighbors | Makes a precise hit complete enough for a query-conditioned decision |
Global context is non-local scaffolding. If a sentence is deep inside a page, the document title may be far away in the tree, but it still tells us what the sentence is about. A header can scope everything that follows it even when the HTML nesting is not a perfect representation of that scope. A table cell needs labels, not the whole table.
Local context is neighborhood. Starting from a sentence seed, SPIRE expands outward through complete sentences, enclosing blocks, and nearby document-order regions until it approaches a target budget. It is not a fixed window. It is a structure-aware walk around the seed.
A citation has a location, a neighborhood, and a budget.
Rotate, zoom, and hover the conceptual view of a seed moving through SPIRE's structural stages.
Conceptual coordinates, not additional experimental measurements.
The two operations can be composed, but they answer different questions:
- Global contextualization asks: What structural facts must travel with this node?
- Local contextualization asks: What nearby material should we inspect before deciding whether this node is useful?
That separation is the heart of the paper.
The retrieval pipeline
The implementation has two stages: high-recall embedding retrieval, followed by optional generative contextual filtering.
1. Seed the index with sentences
HTML gives us paragraphs, list items, and table cells, but those are often too coarse. A paragraph may contain several unrelated sentences. Splitting the paragraph into strings can destroy inline links and emphasis.
SPIRE runs sentence segmentation over the rendered text and writes the boundaries back into the original tree using zero-width sentinel elements. The sentinels do not duplicate or move content. They simply mark where each sentence begins and ends.
Each sentence becomes a path-addressable seed. The embedding text is globally contextualized and rendered to Markdown, but the indexed identity is still the original sentence path.
2. Search with structural context
At query time, the user query and the globally contextualized sentence views are embedded in the usual way. The important difference is that the system does not confuse the rendered embedding text with the citation identity.
The embedding sees enough context to understand the sentence. The retrieval system still knows the exact path that produced it.
3. Merge hits from the same document
This was one of my favorite details in the system.
Suppose the ranker returns two strong sentences from the same page. Both sentences may inherit the same document title and section header. If we render each result independently, we pay for the title and header twice. Shared scaffolding crowds out other evidence under the budget.
SPIRE groups candidates by their source document, unions their path sets, applies global contextualization once, and then charges the combined view once. If the result is too small, it admits more candidates and repeats the process.
The graph is doing useful work here. Two sentence nodes that look separate to the vector index share a parent document and a structural neighborhood. Aggregation turns that relationship into saved budget.
4. Expand and filter locally
Embedding similarity is a high-recall candidate generator, not a final judge. A sentence can look relevant in isolation and become irrelevant when its neighboring sentence introduces a negation or exception. The reverse also happens: a sentence can be ambiguous until the paragraph around it is visible.
SPIRE locally expands each candidate, wraps citable units with deterministic labels such as <lab_1> and <lab_2>, and asks a generative model to return only the labels whose contents are needed to answer the query.
The model reasons over the expanded view. The final citation maps back to the original path set.
query
-> retrieve sentence seeds
-> add global structural context
-> merge candidates by source document
-> expand each candidate locally
-> select useful labeled units
-> map labels back to original paths
The model is never asked to reproduce citation text and then have a second system recover the source by string matching. The label is a stable handle. The path is the provenance.
The experiment: same budget, different evidence
The paper evaluates on HotpotQA and ASQA using the HTML benchmark setup from HtmlRAG: 400 randomly sampled instances per benchmark, each augmented with HTML pages sourced through Bing search.
For the embedding-only comparison, BGE and SPIRE use the same BGE-Large-EN embedding model. That matters because it isolates the retrieval algorithm from the embedding model.
The fixed citation budget is 1,000 tokens. Each result below is helpful / total (ratio), where helpfulness is judged by Qwen 3 235B Instruct.
| System | HotpotQA | ASQA | What changes |
|---|---|---|---|
| BGE | 336 / 1514 (0.222) | 998 / 1658 (0.602) | Strong block-level baseline |
embedding_retrieval | 479 / 2225 (0.215) | 1236 / 2010 (0.615) | Sentence-level SPIRE retrieval, no generative filter |
embedding_retrieval_noctx | 836 / 6077 (0.138) | 767 / 1872 (0.410) | Global contextualization disabled |
| SPIRE full pipeline | 538 / 822 (0.655) | 1673 / 2058 (0.813) | Embedding retrieval plus contextual filtering |
The 1,000-token budget is fixed. The evidence is not.
Hover each point to compare total citations, helpful citations, and the ratio reported in the paper.
HotpotQA
ASQA
The embedding-only result is not a magical ratio improvement over BGE on HotpotQA: 0.215 is slightly below 0.222. The win is different. Sentence-level retrieval emits 2,225 citations within the budget instead of 1,514. It uses the same space to surface many more precise candidates without losing comparable per-citation quality.
On ASQA, the embedding-only system is slightly better on both dimensions: 0.615 versus 0.602, and 2,010 citations versus 1,658.
The ablation without global context makes the structural point visible. It emits even more citations on HotpotQA because it spends less budget on context, but its helpful ratio drops to 0.138. The system is retrieving more fragments and understanding fewer of them.
Context changes the slope of quality.
The same 1,000-token budget, with each point showing the helpful-citation ratio from the paper.
The full SPIRE pipeline is where the generative filter earns its place. On HotpotQA, the helpful ratio rises from 0.215 to 0.655. On ASQA, it rises from 0.615 to 0.813. The filter is not searching the corpus again. It is taking the same embedding-derived candidates, looking at their local structural neighborhoods, and throwing away evidence that does not actually answer the question.
That is a more interesting role for an LLM than asking it to summarize a giant retrieved block. Give it a small, structured excerpt. Ask it to select evidence. Keep the citation identity outside the model.
What I take from SPIRE
RAG has two jobs that are usually collapsed into one:
- Find candidate evidence with high recall.
- Present a small, interpretable, trustworthy view of that evidence.
Flat chunking makes one object do both jobs. It has to be small enough for matching and large enough for explanation. That is why every chunk size feels wrong.
SPIRE separates the identity of a retrieved object from the view shown to a model. A sentence path can stay exact while the rendered context grows or shrinks around it. The evidence can be filtered without losing its address. The budget can be spent on useful context instead of repeated scaffolding.
This is also the part that feels closest to how I think about graph-based memory. A node is not the same thing as the neighborhood you walk around it. The node gives you identity. The edges give you a way to recover the context that makes the node useful.
Caveats and the next graph to build
The paper is a retrieval framework, not a claim that every HTML page can be understood perfectly from a few structural rules.
The path representation is a high-level specification. In deeply nested documents, explicit root-to-node paths can cost space proportional to tree depth. A production implementation could reduce that overhead with compact identifiers and shared-prefix encodings while keeping the same semantics.
The benchmark evaluation also uses an LLM judge, so the ratios should be read as evidence of improved citation usefulness under this protocol, not as a universal measure of truth. The experiments cover two HTML QA benchmarks, not every language, website, or retrieval workload.
The next obvious direction is domain-specific context. Technical documentation has definitions and cross-references. Legal documents have clauses and exceptions. Scientific papers have figures, captions, equations, and citations. A future contextualization policy should know that a sentence mentioning Figure 4 needs the figure caption, not an arbitrary neighboring paragraph.
The larger idea is simple enough to keep: structure is not decoration around the text. Structure is part of the evidence.
I do not want a RAG system that merely retrieves words. I want it to tell me: this sentence came from this exact place, under this header, beside these neighbors, and this is the smallest context that makes it honest.
That is what SPIRE is trying to build.
Read the full paper on arXiv, and if this kind of retrieval problem is familiar, get in touch.