HiKEY

Hierarchical Multimodal Retrieval
for Open-Domain Document Question Answering

Joongmin Shin¹, Gyuho Shim³, Jeongbae Park¹, Jaehyung Seo^2‡, Heuiseok Lim^1,3‡

¹Human-inspired AI Research, Korea University ²CSE, Konkuk University ³CSE, Korea University

^‡Corresponding authors

ACL 2026

Figure 1. Overview of HiKEY. (1) Offline hierarchy-aware graph and index construction: each PDF is parsed into a DHP tree whose nodes (text/table/figure blocks) carry section-path metadata, then encoded into Doc_cards (global routing) and Sec_cards (fine retrieval). (2) Online hierarchical coarse-to-fine retrieval: Stage-1 prunes the corpus by Doc_card hybrid scoring; Stage-2 ranks sections via MaxSim over modality-specialized unit scores. (3) Ancestry-aware evidence subgraph assembly serializes the Anchor Unit with its Governing Headers plus sibling/semantic associates under a fixed token budget, then hands the subgraph to an LVLM reader.

Abstract

Retrieval-augmented generation (RAG) for document-based Open-domain Question Answering (ODQA) on large-scale industrial corpora faces two critical bottlenecks: routing failure in locating the correct document and evidence fragmentation in integrating scattered information. Existing approaches relying on flat text chunks or page-level images inherently struggle to (i) precisely pinpoint the target document among thousands of candidates and (ii) organically connect multimodal evidence — such as tables and figures — within a limited token budget.

We propose HiKEY, a hierarchical tree-based multimodal retrieval framework that elevates document hierarchy to a first-class retrieval signal. Instead of simple chunking, HiKEY reconstructs a logical heterogeneous graph via Document Hierarchical Parsing (DHP), explicitly encoding parent–child relationships. Adopting a hierarchical coarse-to-fine strategy, the framework (1) performs global routing to rapidly prune the search space using hierarchical indexing, and (2) conducts fine-grained retrieval to rank sections by employing a multimodal fusion strategy that captures the most discriminative evidence. Finally, HiKEY assembles a token-efficient evidence subgraph via a hybrid structural-semantic packing strategy.

Experiments on ODQA benchmarks demonstrate that HiKEY significantly outperforms page- and chunk-based baselines, improving retrieval recall by up to +12.9% and end-to-end QA performance by up to +6.8%.

Contributions

HiKEY framework. A hierarchical tree-based multimodal retrieval framework that elevates document hierarchy from passive metadata to a first-class retrieval signal. By reconstructing a logical graph via DHP, HiKEY structurally resolves the evidence fragmentation problem inherent in industrial documents — bridging the gap between flat retrieval and complex document layouts.

Hierarchical coarse-to-fine retrieval. A retrieval strategy that integrates global routing with local fine-grained ranking. A fine-grained multimodal fusion strategy identifies the most discriminative evidence within a section, coupled with a graph traversal mechanism that enriches visual units with their hierarchical upper context — ensuring robustness against visual ambiguity and missing captions.

Comprehensive experiments. Across industrial-aligned multi-page document ODQA benchmarks, HiKEY consistently outperforms state-of-the-art text-based and multimodal RAG baselines. Retrieval improves by 4.5%–12.9% and QA by 3.7%–6.8%, validating the efficacy of ancestry-aware subgraph assembly in token-limited settings.

Headline results

Improvement over the strongest chunk- / page-based baseline (MultiDocFusion · SimpleDoc).

+12.9%

peak retrieval recall gain

FRAMES strict All, MultiDocFusion → HiKEY (+11.4)

+6.8%

peak end-to-end QA gain

FRAMES ROUGE-L, MultiDocFusion → HiKEY

+4.5%

R@10 lift at 0.5K-token budget

where page-level methods can no longer fit

Method

HiKEY treats retrieval as a hierarchy-aware problem. Before retrieving, every document is rebuilt as a heterogeneous tree of multimodal blocks; at query time, the corpus is explored coarse-to-fine and a token-budgeted evidence subgraph is handed to the reader. The pipeline has four phases:

(a) Offline: Hierarchical Tree Graph Parsing

Each document d is processed by a Document Hierarchical Parsing (DHP) module — the M3DocDep parser trained on public hierarchy datasets — which produces a logical tree T(d) from page-level layout blocks. Every block (text, table, figure) becomes a node connected by parent–child tree edges E_tree and receives an ancestor section path (e.g. Title › §5 › §5.3). For visual units lacking captions, a tree traversal pulls the nearest preceding textual node as Upper Context ctx(c), reducing visual ambiguity.

(b) Offline: Card-based Index Generation

HiKEY emits two hierarchical card types per document. A Doc_card — x_d = Title(d) ∥ ⋃_s∈T(d) path(s) — concatenates the document title with all section paths into a separate hierarchy field used for global routing. A Sec_card stores each section's text and non-text units C_s = {c₁, …, c_n} as searchable items, with each unit tagged by its modality type τ(c) ∈ {Text, Table, Image}.

(c) Online: Stage-1 Hierarchical Document Routing

For a query q, HiKEY first narrows the search scope by ranking Doc_cards via a hybrid lexical + dense-text score:

S_doc(d, q) = α · mm(s_lex(x_d, q)) + (1−α) · mm(s_text(x_d, q))

with per-query min–max normalization mm(·). The separately-indexed hierarchy paths in x_d turn the table-of-contents into a directly searchable signal — the lever that resolves routing failure.

(d) Online: Stage-2 Section MaxSim Scoring + Subgraph Assembly

Within the routed candidate set, each unit gets a type-specific score (hybrid text scoring for Text units; weighted vision + Upper-Context text scoring for Tables/Images). A section's relevance is the MaxSim over its units, and the final ranking fuses document and section scores. The top-K_sec sections are then packed into the reader's token budget B_tok by inserting the Anchor Unit and its Governing Headers, then expanding with Sibling Units (same-subtree structural association) and Semantic Associates (high-similarity non-local units) — preserving logical coherence while capturing cross-section dependencies.

How a query flows through HiKEY

A worked example. The user asks a multi-hop, table-grounded question against a corpus of multi-page PDFs. We trace what each stage sees and emits.

Input query. A multi-hop question whose answer is split across two PDFs and partially lives in a table cell.

q: "In the Apollo 11 mission, who was the command-module pilot, and what was the lunar-module's payload mass at landing?"
Evidence type: Text + Table · Hops: 2 documents

Hierarchical document routing. Doc_cards index title + section paths in a separate hierarchy field. HiKEY scores every Doc_card with a lexical + dense-text hybrid, normalizes per query, and returns a candidate set D̂_q — typically a handful of docs out of thousands.

S_doc(d, q) = α · mm(s_lex) + (1−α) · mm(s_text)

D̂_q: apollo_11_mission.pdf · lunar_module_eagle.pdf · apollo_crew_bios.pdf
Field-separated hierarchy lifts Avg R@10 from 84.6 → 88.6 vs. concatenation.

Section MaxSim with type-specific scoring. Inside each candidate doc, every unit is scored under its modality. Tables/Images additionally use the Upper-Context text signal (the closest textual ancestor in the DHP tree) — fixing the case where a figure has no caption.

S_sec(s, q) = max_{c ∈ C_s} s(c, q)

Top sections:
• apollo_crew_bios.pdf › Crew › Command Module Pilot (Text anchor)
• lunar_module_eagle.pdf › Specifications › Mass & Payload (Table anchor + Upper Context)

Ancestry-aware subgraph. HiKEY packs the Anchor Unit + its Governing Headers, then fills the remaining budget with Sibling Units (same parent section) and Semantic Associates (high-similarity non-local units). The reader (LVLM) sees a token-efficient subgraph with the hierarchy intact.

Subgraph (≤ 0.5K tokens):
# Apollo 11 › ## Crew › ### Command Module Pilot
  Anchor: "Michael Collins served as the command module pilot…"
# Lunar Module Eagle › ## Specifications › ### Mass & Payload
  Anchor (Table): [landing mass row · descent-stage payload]
  Sibling: [propellant table — same parent §Specifications]
Budget-aware: HiKEY stays best even at 0.5K, where page-level methods cannot fit at all.

Schematic walkthrough; reproduces the running example from the paper's appendix.

Try it: Compare what each baseline retrieves

Reproduced from the paper's Apppendix G qualitative walkthrough and the three documented baseline failure modes (Appendix H, F1–F3). For the same Apollo 11 query, each baseline family produces its own evidence — switch baselines to see exactly how HiKEY's ancestry-aware subgraph differs.

Query

"Who were the crew members of Apollo 11, and when did they land on the Moon?"

Evidence: Text + Table · Sections: §3.1 Prime crew (crew) + §5.1 Landing (landing time)

Select a baseline family

Baseline

Chunk-RAG (LumberChunker)

Flat text chunks, no hierarchy field, corpus-wide similarity.

HiKEY (ours)

Ancestry-aware evidence subgraph

Anchor + Governing Headers, Sibling expansion within §3.1, Semantic Associate from §5.1 — packed under B_tok.

Result: Both answer-bearing units arrive in one budget-aware subgraph — crew table (Sibling) and landing-time table (Semantic Associate) — each tagged with its own section path so the LVLM never confuses scope.

HiKEY's subgraph reproduces the exact serialized prompt from Appendix G.2; baseline behaviors track the failure modes documented in Appendix H.

Results

Evaluated on two industrial-aligned ODQA benchmarks — M3DocVQA (multi-page mixed-modality QA) and FRAMES-PDF (re-rendered multi-document multi-hop QA) — under a fixed Qwen2.5-VL-7B reader with a 16K token budget.

Document-level retrieval — averaged over K = 1…10 (%)

	M3DocVQA		FRAMES		AVG
	Recall	All	Recall	All	Recall	All
Text chunk-based RAG
Page	80.6	66.4	65.4	34.4	73.0	50.4
LumberChunker	81.5	68.1	64.8	33.2	73.2	50.6
MultiDocFusion	83.0	70.5	66.2	34.8	74.6	52.6
Text-based GraphRAG
RAPTOR	81.8	68.4	64.9	33.3	73.4	50.9
HopRAG	82.3	69.3	65.8	34.2	74.1	51.8
Page-level multimodal RAG
M3DocRAG	75.6	61.4	56.8	25.0	66.2	43.2
VDocRAG	77.1	63.2	58.5	26.5	67.8	44.9
Multimodal GraphRAG
MoLoRAG	76.0	62.0	57.4	25.5	66.7	43.8
SimpleDoc	77.3	63.6	60.6	28.4	69.0	46.0
HiKEY (ours)	84.9	73.6	73.3	46.2	79.1	59.9

Largest gains under the strict All metric on FRAMES (+11.4 vs. MultiDocFusion) — where every supporting document must be retrieved.

End-to-end QA — ANLS / ROUGE-L (%)

	M3DocVQA		FRAMES
	ANLS	ROUGE-L	ANLS	ROUGE-L
LumberChunker	24.2	25.6	9.4	12.5
MultiDocFusion	25.9	27.3	10.8	13.9
HopRAG	25.7	27.1	10.7	13.8
M3DocRAG	27.0	28.2	5.9	9.0
VDocRAG	27.4	28.8	6.3	9.4
MoLoRAG	28.2	30.0	6.5	9.6
SimpleDoc	28.7	30.1	6.8	9.9
HiKEY (ours)	30.7	32.2	14.6	17.7

On FRAMES, HiKEY closes the gap that multimodal page-level methods open up vs. text — ROUGE-L +3.8 over MultiDocFusion.

Token-budget sensitivity — R@10 (%)

Method	0.5K	1K	2K	Avg
LumberChunker	72.8	73.2	75.3	73.6
MultiDocFusion	74.3	74.6	76.8	75.0
SimpleDoc (page)	fixed — cannot resize			69.0
HiKEY	78.8	79.1	81.3	79.5

HiKEY remains best at every budget — page-level units cannot dynamically resize.

Ablation — Avg R@10

Stage-1 · Hierarchy indexing

Body-only (no hierarchy) 81.2

+ Concat title/header 84.6

+ Field-separated (ours) 88.6

Stage-1 · Routing strategy

Sec-only routing 76.1

Doc-only routing 87.7

Doc → Sec coarse-to-fine (ours) 88.6

Stage-2 · Fusion signals

BM25 only 87.6

+ Text dense 88.2

+ Visual dense (full) 88.6

Why HiKEY beats every baseline family

Chunk-based text RAG. Weak document-level cues → field-separated hierarchy in Doc_card lifts Avg R@10 84.6 → 88.6.

Text-only GraphRAG. Tables/figures not first-class → Type-specific unit scoring + region-level visual embeddings (Table +2.9, Image +2.8 ANLS).

Page-level multimodal RAG. Whole-page units dilute evidence → block units + ancestry-aware packing; FRAMES strict All 34.8 → 46.2 (+11.4).

Multimodal page-graph GraphRAG. Connectivity without hierarchical scope → governing-headers + hybrid sibling/semantic expansion holds gains down to the 0.5K budget where page units no longer fit.

BibTeX

@article{shin2026hikey,
    title         = {HiKEY: Hierarchical Multimodal Retrieval for Open-Domain Document Question Answering},
    author        = {Shin, Joongmin and Shim, Gyuho and Park, Jeongbae and Seo, Jaehyung and Lim, Heuiseok},
    journal       = {arXiv preprint arXiv:2605.29606},
    year          = {2026},
    note          = {Accepted to ACL 2026 (Main)},
    url           = {https://arxiv.org/abs/2605.29606}
}