# Multimodal Document Retrieval with MINER Layer-Fused Embeddings

> Build a retrieval harness that wraps a transformer encoder with a lightweight multi-layer fusion probe following the MINER design, then benchmark it against a single-vector baseline on a small visual document dataset. The result is a scored comparison report showing how much internal-layer signal you recover without retraining the backbone.

- Canonical URL: https://agentry.press/tutorial/multimodal-document-retrieval-with-miner-layer-fused-embeddings/
- Type: Tutorial
- Published: 2026-05-27
- By: agentry
- Tags: retrieval, embeddings, transformers, multimodal, pytorch

---

## Why this matters

Dense single-vector retrievers are the workhorse of production document search: one embedding per page, fast ANN lookup, small index. But they consistently underperform late-interaction models like ColPali because they discard everything except the final transformer layer. MINER [1] makes a precise diagnosis of this problem: retrieval-relevant signal is spread across internal layers, not concentrated at the output. The fix is a lightweight plug-in module, no backbone retraining required, that probes each layer and fuses the surviving signals into a single compact vector.

In benchmarks across ViDoRe V1/V2/V3, MINER delivers up to 4.5% nDCG@5 improvement over its backbone and narrows the gap to strong late-interaction baselines to as little as 0.2 nDCG@5 [1]. For teams already running a dense retrieval stack, that is a drop-in quality gain with no index-size penalty. This tutorial implements the two core MINER stages (Retrieval-Aligned Layer Probing and Adaptive Sparse Multi-Layer Fusion) in plain PyTorch, wraps them around a small CLIP-style text encoder, and produces a head-to-head comparison report against the single-vector baseline.

## Prerequisites

- Python 3.11 or 3.12
- Familiarity with cosine-similarity retrieval and transformer hidden states
- Basic PyTorch (tensors, `nn.Module`, `no_grad`)
- No GPU required; all blocks run on CPU with a small synthetic dataset

## Setup

Install the required packages. The tutorial uses `sentence-transformers` for a pretrained encoder backbone, `torch` for the fusion module, and `scikit-learn` for evaluation metrics.

```bash
uv pip install torch sentence-transformers scikit-learn numpy
```

Verify the key packages loaded correctly:

```python
from importlib.metadata import version
for pkg in ["torch", "sentence-transformers", "scikit-learn"]:
    print(pkg, version(pkg))
print("env_ok")
```

## Step 1: Build the Layer-Probing Encoder

The first MINER stage, Retrieval-Aligned Layer Probing, attaches a lightweight linear probe at each transformer layer and scores how much retrieval-relevant information that layer carries [1]. In the full MINER paper the probes are trained with a contrastive loss on a labeled dataset. Here you will implement the same structural pattern: one probe per layer, each projecting the layer's `[CLS]` token down to a shared embedding dimension, then scored by their cosine similarity to the final-layer embedding (a proxy for retrieval alignment when labels are unavailable).

```python
# filename: miner_encoder.py
import torch
import torch.nn as nn
import torch.nn.functional as F
from sentence_transformers import SentenceTransformer
from typing import List, Optional


class LayerProbe(nn.Module):
    """Single linear probe that projects one layer's CLS token to embed_dim."""

    def __init__(self, hidden_size: int, embed_dim: int):
        super().__init__()
        self.proj = nn.Linear(hidden_size, embed_dim, bias=False)
        nn.init.orthogonal_(self.proj.weight)

    def forward(self, hidden: torch.Tensor) -> torch.Tensor:
        return F.normalize(self.proj(hidden), dim=-1)


class MINEREncoder(nn.Module):
    """
    Wraps a SentenceTransformer backbone and adds:
      1. Per-layer probes (Retrieval-Aligned Layer Probing)
      2. Adaptive sparse multi-layer fusion

    The backbone weights are frozen; only the probes are trainable.
    """

    def __init__(
        self,
        model_name: str = "sentence-transformers/all-MiniLM-L6-v2",
        embed_dim: int = 128,
        top_k_layers: Optional[int] = None,
    ):
        super().__init__()
        self.st_model = SentenceTransformer(model_name)
        # Freeze backbone
        for p in self.st_model.parameters():
            p.requires_grad_(False)

        # Introspect the underlying BERT/RoBERTa encoder
        self.transformer = self.st_model[0].auto_model
        cfg = self.transformer.config
        self.num_layers = cfg.num_hidden_layers
        self.hidden_size = cfg.hidden_size
        self.embed_dim = embed_dim
        self.top_k_layers = top_k_layers or self.num_layers

        # One probe per layer
        self.probes = nn.ModuleList(
            [LayerProbe(self.hidden_size, embed_dim) for _ in range(self.num_layers)]
        )
        # Learnable per-layer fusion weights (log-space for stability)
        self.log_layer_weights = nn.Parameter(torch.zeros(self.num_layers))

    # ------------------------------------------------------------------
    # Internal helpers
    # ------------------------------------------------------------------

    def _get_all_layer_cls(self, input_ids, attention_mask) -> List[torch.Tensor]:
        """Return the CLS token hidden state from every transformer layer."""
        outputs = self.transformer(
            input_ids=input_ids,
            attention_mask=attention_mask,
            output_hidden_states=True,
        )
        # hidden_states[0] is the embedding layer; [1..num_layers] are encoder layers
        layer_cls = [outputs.hidden_states[i + 1][:, 0, :] for i in range(self.num_layers)]
        return layer_cls

    def _probe_scores(self, layer_cls: List[torch.Tensor]) -> torch.Tensor:
        """
        Compute a retrieval-alignment score for each layer.

        Proxy: cosine similarity between each probe's output and the
        final-layer CLS embedding (no labels needed at inference time).
        Returns a (num_layers,) tensor of scores in [0, 1].
        """
        final_emb = F.normalize(layer_cls[-1], dim=-1)  # (B, H)
        scores = []
        for i, probe in enumerate(self.probes):
            probe_emb = probe(layer_cls[i])  # (B, embed_dim)
            # Project final_emb to same dim for comparison
            with torch.no_grad():
                final_proj = F.normalize(
                    self.probes[-1].proj(layer_cls[-1]), dim=-1
                )
            sim = (probe_emb * final_proj).sum(dim=-1).mean()  # scalar
            scores.append(sim)
        return torch.stack(scores)  # (num_layers,)

    # ------------------------------------------------------------------
    # Forward pass
    # ------------------------------------------------------------------

    def forward(self, sentences: List[str]) -> torch.Tensor:
        """
        Encode a batch of sentences into a single fused embedding per sentence.
        Returns a (B, embed_dim) tensor, L2-normalised.
        """
        # Tokenise
        tokenizer = self.st_model.tokenizer
        encoded = tokenizer(
            sentences,
            padding=True,
            truncation=True,
            max_length=128,
            return_tensors="pt",
        )
        input_ids = encoded["input_ids"]
        attention_mask = encoded["attention_mask"]

        with torch.no_grad():
            layer_cls = self._get_all_layer_cls(input_ids, attention_mask)

        # Stage 1: score layers
        alignment_scores = self._probe_scores(layer_cls)  # (num_layers,)

        # Stage 2: select top-k layers, apply sparse masking, fuse
        topk_vals, topk_idx = torch.topk(alignment_scores, self.top_k_layers)
        # Softmax over selected layer weights (adaptive fusion)
        fusion_weights = torch.softmax(self.log_layer_weights[topk_idx], dim=0)  # (k,)

        fused = torch.zeros(len(sentences), self.embed_dim)
        for rank, layer_i in enumerate(topk_idx):
            probe_emb = self.probes[layer_i](layer_cls[layer_i])  # (B, embed_dim)
            fused = fused + fusion_weights[rank] * probe_emb

        return F.normalize(fused, dim=-1)

    def baseline_encode(self, sentences: List[str]) -> torch.Tensor:
        """
        Single-vector baseline: final-layer CLS, projected to embed_dim.
        Equivalent to a standard dense retriever.
        """
        tokenizer = self.st_model.tokenizer
        encoded = tokenizer(
            sentences,
            padding=True,
            truncation=True,
            max_length=128,
            return_tensors="pt",
        )
        with torch.no_grad():
            layer_cls = self._get_all_layer_cls(
                encoded["input_ids"], encoded["attention_mask"]
            )
        final_proj = self.probes[-1](layer_cls[-1])  # (B, embed_dim)
        return F.normalize(final_proj, dim=-1)
```

## Step 2: Create a Synthetic Visual-Document Dataset

The tutorial uses a small in-memory dataset that mimics a visual document retrieval scenario: each "document" is a short text description of a page (as you would get from a captioning model or OCR pipeline), and each query is a natural-language question about one of those pages.

```python
# filename: dataset.py
from dataclasses import dataclass, field
from typing import List, Tuple


@dataclass
class RetrievalDataset:
    documents: List[str] = field(default_factory=list)
    queries: List[str] = field(default_factory=list)
    # relevance[i] = list of relevant doc indices for query i
    relevance: List[List[int]] = field(default_factory=list)


def build_synthetic_dataset() -> RetrievalDataset:
    documents = [
        # Financial reports
        "Q3 2024 earnings report: revenue grew 12% YoY to $4.2B, operating margin 18%.",
        "Balance sheet summary: total assets $22B, long-term debt $3.1B, cash $5.4B.",
        "Cash flow statement: operating cash flow $1.1B, capex $420M, free cash flow $680M.",
        # Technical manuals
        "Figure 3: transformer architecture diagram showing multi-head attention and feed-forward layers.",
        "Table 2: benchmark results on GLUE — BERT-base 79.6, RoBERTa-large 88.9, DeBERTa 91.1.",
        "Algorithm 1: gradient checkpointing reduces peak memory by recomputing activations on backward pass.",
        # Medical documents
        "Patient cohort: 1,240 adults, mean age 54, randomised 1:1 to treatment vs placebo.",
        "Primary endpoint: 28-day mortality reduced from 22% to 14% (p=0.003, HR 0.61).",
        "Adverse events: nausea 8% treatment vs 6% placebo; serious adverse events 3% vs 4%.",
        # Legal filings
        "Section 4.2: indemnification clause limits liability to direct damages not exceeding contract value.",
        "Exhibit A: intellectual property assignment covers all inventions made during employment.",
        "Governing law: disputes resolved under Delaware law, exclusive jurisdiction in Chancery Court.",
        # Scientific papers
        "Abstract: we propose a novel attention mechanism that reduces quadratic complexity to linear.",
        "Results: our method achieves 94.3 F1 on SQuAD 2.0, outperforming the previous SOTA by 1.8 points.",
        "Conclusion: the proposed sparse attention pattern generalises across six NLP benchmarks.",
    ]

    queries = [
        "What was the revenue growth in Q3 2024?",
        "How much free cash flow did the company generate?",
        "What does the transformer architecture diagram show?",
        "Which model scored highest on GLUE benchmarks?",
        "How many patients were in the clinical trial?",
        "What was the reduction in 28-day mortality?",
        "What law governs dispute resolution?",
        "What F1 score did the proposed method achieve on SQuAD?",
        "How does gradient checkpointing reduce memory usage?",
        "What does the IP assignment clause cover?",
    ]

    relevance = [
        [0],        # revenue growth -> Q3 earnings
        [2],        # free cash flow -> cash flow statement
        [3],        # transformer diagram -> figure 3
        [4],        # GLUE scores -> table 2
        [6],        # patient cohort -> cohort description
        [7],        # mortality reduction -> primary endpoint
        [11],       # governing law -> legal filing
        [13],       # SQuAD F1 -> results section
        [5],        # gradient checkpointing -> algorithm 1
        [10],       # IP assignment -> exhibit A
    ]

    return RetrievalDataset(documents=documents, queries=queries, relevance=relevance)
```

## Step 3: Implement the Evaluation Harness

The harness computes nDCG@5 and Recall@5 for both the MINER fused encoder and the single-vector baseline, then prints a comparison report.

```python
# filename: evaluate.py
import math
import numpy as np
from typing import List, Dict
import torch
import torch.nn.functional as F


def cosine_similarity_matrix(queries: torch.Tensor, docs: torch.Tensor) -> np.ndarray:
    """Returns (Q, D) similarity matrix."""
    sims = torch.mm(queries, docs.T)  # both are already L2-normalised
    return sims.numpy()


def ndcg_at_k(ranked_docs: List[int], relevant: List[int], k: int = 5) -> float:
    """Compute nDCG@k for a single query."""
    dcg = 0.0
    for rank, doc_id in enumerate(ranked_docs[:k]):
        if doc_id in relevant:
            dcg += 1.0 / math.log2(rank + 2)  # rank is 0-indexed
    # Ideal DCG: all relevant docs at top positions
    ideal_hits = min(len(relevant), k)
    idcg = sum(1.0 / math.log2(i + 2) for i in range(ideal_hits))
    return dcg / idcg if idcg > 0 else 0.0


def recall_at_k(ranked_docs: List[int], relevant: List[int], k: int = 5) -> float:
    hits = sum(1 for d in ranked_docs[:k] if d in relevant)
    return hits / len(relevant) if relevant else 0.0


def evaluate_retriever(
    query_embeddings: torch.Tensor,
    doc_embeddings: torch.Tensor,
    relevance: List[List[int]],
    k: int = 5,
) -> Dict[str, float]:
    sim_matrix = cosine_similarity_matrix(query_embeddings, doc_embeddings)
    ndcg_scores, recall_scores = [], []
    for q_idx, relevant in enumerate(relevance):
        ranked = np.argsort(-sim_matrix[q_idx]).tolist()
        ndcg_scores.append(ndcg_at_k(ranked, relevant, k))
        recall_scores.append(recall_at_k(ranked, relevant, k))
    return {
        f"nDCG@{k}": float(np.mean(ndcg_scores)),
        f"Recall@{k}": float(np.mean(recall_scores)),
        "per_query_ndcg": ndcg_scores,
        "per_query_recall": recall_scores,
    }
```

## Step 4: Run the Comparison and Generate the Report

This script ties everything together: it instantiates the MINER encoder with different `top_k_layers` settings, runs both the fused and baseline encoders over the synthetic dataset, and prints a formatted comparison report.

```python
# filename: run_comparison.py
import torch
from dataset import build_synthetic_dataset
from miner_encoder import MINEREncoder
from evaluate import evaluate_retriever


def encode_corpus(encoder: MINEREncoder, texts, batch_size: int = 8):
    """Encode a list of texts in batches, return stacked tensor."""
    all_embs = []
    for i in range(0, len(texts), batch_size):
        batch = texts[i : i + batch_size]
        emb = encoder.forward(batch)
        all_embs.append(emb.detach())
    return torch.cat(all_embs, dim=0)


def encode_corpus_baseline(encoder: MINEREncoder, texts, batch_size: int = 8):
    all_embs = []
    for i in range(0, len(texts), batch_size):
        batch = texts[i : i + batch_size]
        emb = encoder.baseline_encode(batch)
        all_embs.append(emb.detach())
    return torch.cat(all_embs, dim=0)


def print_report(results_baseline, results_miner_configs):
    k = 5
    print("\n" + "=" * 62)
    print(" MINER vs Single-Vector Baseline -- Retrieval Report")
    print("=" * 62)
    print(f"{'Method':<30} {'nDCG@5':>10} {'Recall@5':>10}")
    print("-" * 62)
    print(
        f"{'Baseline (final layer only)':<30} "
        f"{results_baseline['nDCG@5']:>10.4f} "
        f"{results_baseline['Recall@5']:>10.4f}"
    )
    for label, res in results_miner_configs:
        delta_ndcg = res["nDCG@5"] - results_baseline["nDCG@5"]
        delta_recall = res["Recall@5"] - results_baseline["Recall@5"]
        print(
            f"{label:<30} "
            f"{res['nDCG@5']:>10.4f} "
            f"{res['Recall@5']:>10.4f}  "
            f"(nDCG delta: {delta_ndcg:+.4f})"
        )
    print("=" * 62)
    print("\nPer-query nDCG@5 breakdown (MINER all-layers vs Baseline):")
    print(f"{'Query #':<10} {'Baseline':>10} {'MINER (all)':>12}")
    print("-" * 35)
    miner_all_res = results_miner_configs[-1][1]
    for i, (b, m) in enumerate(
        zip(results_baseline["per_query_ndcg"], miner_all_res["per_query_ndcg"])
    ):
        marker = " <" if m > b else ""
        print(f"  Q{i+1:<7} {b:>10.4f} {m:>12.4f}{marker}")
    print("\n< = MINER outperforms baseline on this query")


if __name__ == "__main__":
    dataset = build_synthetic_dataset()
    print(f"Dataset: {len(dataset.documents)} documents, {len(dataset.queries)} queries")

    # Instantiate encoder once; probes are randomly initialised (no training)
    # In production you would fine-tune the probes with contrastive loss.
    encoder = MINEREncoder(
        model_name="sentence-transformers/all-MiniLM-L6-v2",
        embed_dim=128,
    )
    encoder.eval()

    print("Encoding documents and queries...")
    doc_embs_baseline = encode_corpus_baseline(encoder, dataset.documents)
    query_embs_baseline = encode_corpus_baseline(encoder, dataset.queries)

    results_baseline = evaluate_retriever(
        query_embs_baseline, doc_embs_baseline, dataset.relevance
    )

    miner_configs = []
    for top_k in [2, 4, encoder.num_layers]:
        encoder.top_k_layers = top_k
        doc_embs = encode_corpus(encoder, dataset.documents)
        query_embs = encode_corpus(encoder, dataset.queries)
        res = evaluate_retriever(query_embs, doc_embs, dataset.relevance)
        label = f"MINER top-{top_k} layers"
        miner_configs.append((label, res))
        print(f"  {label}: nDCG@5={res['nDCG@5']:.4f}")

    print_report(results_baseline, miner_configs)
    print("report_complete")
```

## Step 5: Understand the Fusion Mechanics

Before running the full pipeline, it helps to inspect what the probes are doing to each layer's representation. This block prints the per-layer alignment scores for a single sentence, showing which layers carry the most retrieval signal according to the cosine-similarity proxy.

```python
import torch
from miner_encoder import MINEREncoder

encoder = MINEREncoder(
    model_name="sentence-transformers/all-MiniLM-L6-v2",
    embed_dim=128,
)
encoder.eval()

sentence = ["Q3 2024 earnings report: revenue grew 12% YoY to $4.2B."]
tokenizer = encoder.st_model.tokenizer
encoded = tokenizer(
    sentence, padding=True, truncation=True, max_length=128, return_tensors="pt"
)
with torch.no_grad():
    layer_cls = encoder._get_all_layer_cls(
        encoded["input_ids"], encoded["attention_mask"]
    )
    scores = encoder._probe_scores(layer_cls)

print(f"Backbone: all-MiniLM-L6-v2 ({encoder.num_layers} encoder layers)")
print(f"{'Layer':>6} {'Alignment score':>18} {'Fusion weight (softmax)':>24}")
print("-" * 52)
weights = torch.softmax(encoder.log_layer_weights, dim=0)
for i, (s, w) in enumerate(zip(scores.tolist(), weights.tolist())):
    bar = "#" * int(s * 20)
    print(f"  L{i+1:>2}   {s:>14.4f}       {w:>14.4f}   {bar}")
print("layer_scores_printed")
```

> [!PULLQUOTE]
> Retrieval-relevant signal resides in internal representations, not just the final layer. MINER surfaces and fuses it without touching the backbone.

The alignment scores reflect how well each layer's probe output correlates with the final-layer representation. In a trained MINER setup [1], these probes would be optimised with a contrastive retrieval loss, making the scores reflect true retrieval quality rather than this proxy. The fusion weights start uniform (all `log_layer_weights` initialised to zero) and would shift toward high-scoring layers during training.

## Verify it works

Run the full comparison pipeline end-to-end:

```bash
cd /workspace && python run_comparison.py
```

You should see output ending with `report_complete` and a table comparing baseline vs MINER configurations. The exact nDCG@5 numbers will vary because the probes are randomly initialised (no training), but the structural output will be consistent.

For a quick smoke test that imports all modules cleanly:

```python
from miner_encoder import MINEREncoder, LayerProbe
from dataset import build_synthetic_dataset, RetrievalDataset
from evaluate import evaluate_retriever, ndcg_at_k, recall_at_k
import torch

# Verify ndcg_at_k logic
assert ndcg_at_k([0, 1, 2], relevant=[0], k=5) == 1.0, "top-1 hit should be perfect"
assert ndcg_at_k([1, 2, 3], relevant=[0], k=5) == 0.0, "miss should be zero"
assert 0 < ndcg_at_k([1, 0, 2], relevant=[0], k=5) < 1.0, "rank-2 hit should be partial"

# Verify encoder output shape
enc = MINEREncoder(embed_dim=64)
enc.eval()
with torch.no_grad():
    out = enc.forward(["hello world", "test sentence"])
assert out.shape == (2, 64), f"expected (2, 64), got {out.shape}"
assert abs(out.norm(dim=-1).mean().item() - 1.0) < 1e-5, "outputs should be L2-normalised"

print("all_assertions_passed")
```

## Troubleshooting

**`KeyError: 'hidden_states'` when calling `_get_all_layer_cls`.**
The underlying model was not called with `output_hidden_states=True`. Confirm that `self.transformer(...)` in `miner_encoder.py` passes that flag. Some `sentence-transformers` versions wrap the model in a way that intercepts kwargs; if so, access `self.st_model[0].auto_model` directly as shown in the tutorial.

**`AttributeError: 'BertModel' object has no attribute 'config'`.**
This happens if `self.st_model[0]` is not the `Transformer` wrapper. Print `type(self.st_model[0])` and `type(self.st_model[0].auto_model)` to confirm the layer structure. With `sentence-transformers>=2.0`, `st_model[0].auto_model` is always the raw HuggingFace model.

**All MINER configurations score the same as baseline.**
With randomly initialised probes and no training, the fusion weights start uniform and the probe projections are random orthogonal matrices. The scores will be close to baseline. This is expected: the tutorial demonstrates the architecture. To see real gains, fine-tune the probes with a contrastive loss on a labeled query-document dataset following the two-stage procedure in [1].

**`RuntimeError: Expected all tensors to be on the same device`.**
The `fused` tensor is initialised on CPU with `torch.zeros(...)` while probe outputs may be on a different device if you moved the encoder to GPU. Replace `torch.zeros(...)` with `torch.zeros(..., device=layer_cls[0].device)` in the `forward` method.

**`sentence-transformers` downloads the model on first run and times out.**
The model `all-MiniLM-L6-v2` is ~90 MB. If the download times out in your environment, set `SENTENCE_TRANSFORMERS_HOME=/tmp/st_cache` and retry. In air-gapped environments, pre-download with `huggingface-cli download sentence-transformers/all-MiniLM-L6-v2`.

**nDCG@5 is 0.0 for every query.**
Check that `relevance` indices in `dataset.py` are within `range(len(documents))`. If you extend the dataset and add documents without updating the relevance list, all queries will miss.

## Next steps

- **Train the probes with contrastive loss.** Replace the cosine-proxy scoring with a proper InfoNCE loss over query-document pairs. The MINER paper [1] uses a two-stage procedure: first train probes independently per layer, then jointly fine-tune the fusion weights. This is where the 4.5% nDCG@5 gain materialises.
- **Plug in a vision encoder.** Swap `all-MiniLM-L6-v2` for a CLIP or PaliGemma vision-language model. The `MINEREncoder` class only requires that `self.transformer` exposes `output_hidden_states=True`; the rest of the fusion logic is backbone-agnostic.
- **Benchmark on ViDoRe.** The ViDoRe benchmark suite (V1/V2/V3) used in [1] is publicly available on Hugging Face. Replace `build_synthetic_dataset()` with a ViDoRe loader and run the same harness to reproduce the paper's comparison numbers.
- **Add neuron-level masking.** The full MINER Adaptive Sparse Multi-Layer Fusion stage applies a learned binary mask at the neuron level within each selected layer, not just at the layer level. Implement this as a `nn.Parameter` sigmoid gate multiplied element-wise into each probe's output before fusion.

## FAQ

### What is the MINER layer-fusion approach and why does it improve retrieval?

MINER diagnoses that retrieval-relevant signal is spread across internal transformer layers, not concentrated at the output. It adds lightweight per-layer probes and fuses their outputs into a single vector, recovering up to 4.5% nDCG@5 improvement without retraining the backbone.

### How does the tutorial measure which layers carry retrieval signal?

The tutorial uses a cosine-similarity proxy: each layer's probe output is scored by its similarity to the final-layer embedding. In production, probes would be trained with contrastive loss on labeled query-document pairs to measure true retrieval alignment.

### Can MINER be applied to existing dense retrievers without retraining?

Yes. The probes are a lightweight plug-in module that wraps the frozen backbone encoder. Once trained on a labeled dataset, the fused embeddings replace single-vector outputs with no change to index size or ANN lookup speed.

### What happens if the probes are not trained, as in this tutorial?

With randomly initialised probes and uniform fusion weights, MINER scores close to the single-vector baseline. The tutorial demonstrates the architecture; real gains require fine-tuning the probes with contrastive loss on labeled data.

### How does adaptive sparse multi-layer fusion work?

The method selects the top-k layers by alignment score, applies softmax over their learnable fusion weights, and combines the weighted probe outputs into a single normalized vector. This concentrates capacity on the most retrieval-relevant layers.

## References

1. https://arxiv.org/abs/2605.06460v1