May 2025 arxiv Wave Advances Agentic RAG and LLM Agents

Agentic RAG: From Single-Shot to Iterative Retrieval

AgenticRAG, developed for enterprise knowledge bases, layers a lightweight harness on top of existing search infrastructure and equips a reasoning LLM with search, find, open, and summarize tools. The system iteratively retrieves information, navigates within documents, and analyzes evidence without requiring changes to the underlying search stack. Ablation studies identify the shift from single-shot retrieval to agentic tool use as the dominant factor, accounting for a 5.9x performance improvement. On open benchmarks, the system achieves 49.6% recall@1 on BRIGHT, a 21.8 percentage-point gain over the best embedding baseline, along with 0.96 factuality on WixQA and 92% answer correctness on FinanceBench [7].

LatentRAG takes a different approach to the latency problem that agentic RAG introduces. Rather than generating natural-language thoughts and subqueries token by token, LatentRAG shifts both reasoning and retrieval into continuous latent space, producing latent tokens from hidden states in a single forward pass. The system aligns LLMs with dense retrieval models in that latent space and incorporates a parallel latent decoding mechanism to maintain transparency. Across seven benchmark datasets, LatentRAG achieves performance comparable to explicit agentic RAG methods while reducing inference latency by approximately 90% [20].

TGS-RAG addresses the multi-hop reasoning problem through bidirectional text-graph synergy. A Graph-to-Text channel uses a global voting strategy from visited graph nodes to re-rank and filter textual evidence, while a Text-to-Graph channel applies a Memory-based Orphan Entity Bridging algorithm that uses textual cues to resurrect previously pruned reasoning paths from search history without additional database overhead. The framework targets what the authors call the “Information Island” problem, where asymmetric reasoning flows between unstructured text and structured graphs leave relevant evidence inaccessible [14].

Credit Assignment and Long-Horizon RL for Agents

Credit assignment across long interaction trajectories remains one of the central unsolved problems in training LLM agents with reinforcement learning. Three preprints from the May wave address it from distinct angles.

StraTA introduces an explicit trajectory-level strategy sampled from the initial task state, conditioning all subsequent actions on that strategy. The framework trains strategy generation and action execution jointly using a hierarchical GRPO-style rollout design, augmented by diverse strategy rollout and critical self-judgment. On ALFWorld, StraTA reaches a 93.1% success rate; on WebShop, 84.2%; and on SciWorld, a 63.5% overall score that the authors report outperforms frontier closed-source models [1].

BEACON takes a milestone-guided approach, partitioning trajectories at milestone boundaries, applying temporal reward shaping within segments, and estimating advantages at dual scales to prevent distant failures from corrupting the evaluation of local actions. On long-horizon ALFWorld tasks, BEACON achieves a 92.9% success rate, nearly doubling GRPO’s 53.5%, while improving effective sample utilization from 23.7% to 82.0% [23].

A2TGPO retains information gain as an intrinsic per-turn signal but redesigns normalization, accumulation, and consumption. Turn-group normalization compares each turn only against peers at the same interaction depth. Variance-rescaled discounted accumulation divides cumulative normalized information gain by the square root of accumulated terms to keep advantage magnitudes comparable across turn positions. Adaptive turn-level clipping widens the update region for informative turns and narrows it for uninformative ones [21].

Multi-Agent Coordination and Prompt Optimization

MASPO addresses the challenge of jointly optimizing prompts across interacting agents in multi-agent systems. Its core innovation is a joint evaluation mechanism that assesses prompts not by local validity alone but by their capacity to facilitate downstream success for successor agents. The framework uses a data-driven evolutionary beam search to navigate the high-dimensional prompt space. Across six diverse tasks, MASPO achieves an average accuracy improvement of 2.9 points over state-of-the-art prompt optimization methods without relying on ground-truth labels [2].

The STAT framework surfaces a complementary problem: standard aggregate metrics such as return, success rate, and completion time can mask coordination failures entirely. The STAT testbed systematically varies agents, tasks, and environment size while holding observation access and task rules fixed, and evaluates six representative value-based multi-agent reinforcement learning methods across varying levels of centralization. Results show that similar return trends can reflect distinct coordination mechanisms, including differences in redundant assignment, assignment diversity, and task-completion efficiency. The authors argue that performance under scale is shaped not only by nominal action-space size but also by assignment pressure, sparse decision opportunities, and redundant choices among interdependent agents [3].

Agent Safety, Monitoring, and Compliance

PrefixGuard addresses the problem of online failure warning for long, tool-using agent tasks where final outcome checks arrive too late for intervention. The framework uses an offline StepView induction step to derive deterministic typed-step adapters from raw trace samples, then trains a supervised monitor that learns an event abstraction and prefix-risk scorer from terminal outcomes. Across WebArena, tau2-Bench, SkillsBench, and TerminalBench, the strongest PrefixGuard monitors reach AUPRC scores of 0.900, 0.710, 0.533, and 0.557, respectively, improving over raw-text controls by an average of 0.137 AUPRC. The paper also derives an observability ceiling that separates monitor error from failures lacking evidence in the observed prefix [4].

MANTRA provides a framework for automatically synthesizing machine-checkable compliance benchmarks from natural-language procedural manuals and tool schemas. It independently generates a symbolic world model capturing procedural dependencies and a set of trace-level compliance checks, then validates their consistency using SMT solving. A structured repair loop resolves inconsistencies, requiring human intervention only as a fallback. Using MANTRA, the authors build a benchmark suite with 285 tasks across six domains, scaling to manuals exceeding 50 pages [19].

TurnGate targets a different safety surface: multi-turn dialogue attacks where malicious intent is distributed across multiple benign-looking turns. The system detects the earliest turn at which delivering a candidate response would make the accumulated interaction sufficient to enable harmful action. The authors construct the Multi-Turn Intent Dataset with branching attack rollouts, matched benign hard negatives, and annotations of earliest harm-enabling turns. TurnGate substantially outperforms existing baselines in harmful-intent detection while maintaining low over-refusal rates and generalizes across domains, attacker pipelines, and target models [26].

A separate study on backend code generation identifies a phenomenon the authors call constraint decay: as structural requirements accumulate across multi-file generation tasks, agent performance declines substantially. Capable configurations lose an average of 30 points in assertion pass rates from baseline to fully specified tasks. Framework sensitivity analysis shows significant disparities, with agents succeeding in minimal, explicit frameworks such as Flask but performing substantially worse in convention-heavy environments such as FastAPI and Django. Data-layer defects, including incorrect query composition and ORM runtime violations, are identified as the leading root causes [5].

Memory, Skill Retrieval, and Routing

STALE introduces a benchmark of 400 expert-validated conflict scenarios designed to test whether LLM agents can recognize when stored memories are no longer valid. The benchmark covers implicit conflicts, where a later observation invalidates an earlier memory without explicit negation, requiring contextual inference to detect. Across three probing dimensions and contexts up to 150K tokens, even the best evaluated model achieves only 55.2% overall accuracy. The authors also present CUPMem, a prototype that strengthens write-time revision through structured state consolidation and propagation-aware search [18].

SkillRet provides a large-scale benchmark for skill retrieval in LLM agents, containing 17,810 public agent skills organized with structured semantic tags across six major categories and 18 sub-categories. It provides 63,259 training samples and 4,997 evaluation queries with disjoint skill pools. Off-the-shelf retrievers struggle on realistic large-scale skill libraries, and task-specific fine-tuning on SkillRet improves NDCG@10 by 13.1 points over the strongest prior retriever and 16.9 points over the strongest off-the-shelf retriever [11].

MemReranker addresses calibration failures in generic reranking models used for agent memory retrieval. The model family, available at 0.6B and 4B parameter scales, is built on Qwen3-Reranker through multi-stage LLM knowledge distillation combining multi-teacher pairwise comparisons, BCE pointwise distillation, and InfoNCE contrastive learning. MemReranker-4B achieves 0.737 MAP while maintaining inference latency at 10 to 20 percent of large models [22].

BoundaryRouter provides a training-free approach to routing queries between lightweight LLM inference and full agent execution. The system builds a compact experience memory by executing both systems on a shared seed set and retrieves similar cases at inference time to guide routing decisions. On the RouteBench benchmark, BoundaryRouter reduces inference time by 60.6% compared to full agent execution while improving performance by 28.6% over direct LLM inference [38].

Key Patterns Across the Research Wave

Several themes recur across the May preprint wave. The latency-quality tradeoff in agentic inference is addressed from multiple directions simultaneously: LatentRAG compresses the retrieval loop into latent space, BoundaryRouter avoids agent invocation entirely for simpler queries, and inference-time budget control frameworks govern how retrieval budget is allocated across multi-hop question answering steps [12].

The shift from outcome-level to process-level credit assignment appears in StraTA, BEACON, and A2TGPO, each proposing distinct mechanisms for attributing reward to intermediate actions rather than terminal outcomes. The STAT framework extends this logic to multi-agent evaluation, arguing that aggregate return metrics are insufficient to characterize coordination quality [3].

A critical gap between citation validity and factual accuracy emerges from the deep research agent evaluation framework introduced in source [17]. Even the strongest frontier models maintain link validity above 94% and relevance above 80%, yet achieve only 39 to 77% factual accuracy. Fact check accuracy drops by approximately 42% on average as tool calls scale from 2 to 150, indicating that more retrieval does not produce more accurate citations. This finding applies directly to production deployments of research agents where citation volume is often treated as a proxy for thoroughness.

FAQ

Q. How does AgenticRAG integrate with existing enterprise search infrastructure without requiring a full replacement? AgenticRAG layers a lightweight harness on top of existing enterprise search infrastructure rather than replacing it, equipping a reasoning LLM with search, find, open, and summarize tools that interact with the existing stack. The design was informed by pre-production deployments, and the authors report that multi-query search and in-document navigation contribute to both quality and efficiency beyond the core tool-use shift [7].

Q. What are the known failure modes of PrefixGuard in production-like settings? First-alert diagnostics in the PrefixGuard evaluation show that strong ranking does not imply deployment utility. WebArena ranks well in AUPRC terms yet fails to support low-false-alarm alerts, while tau2-Bench and TerminalBench retain more actionable early alerts. The paper also notes that DFA extraction expands to 151 and 187 states on SkillsBench and TerminalBench, which may complicate finite-state audit in those domains [4].

Q. Does LatentRAG’s latency reduction come at a measurable accuracy cost? The authors report that LatentRAG achieves performance comparable to explicit agentic RAG methods across seven benchmark datasets while reducing inference latency by approximately 90%. The parallel latent decoding mechanism, which translates latent tokens back into natural language, is included specifically to maintain transparency and encourage semantically meaningful latent representations [20].

Q. What does the constraint decay finding mean for teams deploying coding agents on Django or FastAPI projects? The constraint decay study finds that capable agent configurations lose an average of 30 points in assertion pass rates when structural requirements accumulate, and that agents perform substantially worse in convention-heavy environments such as FastAPI and Django compared to minimal frameworks like Flask. Data-layer defects are identified as the leading root cause, suggesting that ORM-heavy projects carry elevated risk of structural non-compliance even when functional tests pass [5].

Q. Can the STALE benchmark be used to evaluate commercial memory frameworks, or is it limited to base LLMs? STALE evaluates both frontier LLMs and specialized memory frameworks, finding a pervasive gap between retrieving updated evidence and acting on it across both categories. The best evaluated model achieves only 55.2% overall accuracy, and the authors present CUPMem as a prototype baseline for state-aware memory rather than a production solution, positioning the benchmark as broadly applicable to any system claiming long-term personalized memory [18].

Key takeaways

• AgenticRAG’s ablations isolate iterative tool use as the single largest performance driver, producing a 5.9x gain over single-shot retrieval, while LatentRAG demonstrates that shifting retrieval to latent space can recover most of that quality gain at roughly one-tenth the latency [7][20].
• StraTA, BEACON, and A2TGPO each propose distinct mechanisms for process-level credit assignment in long-horizon RL, with StraTA reaching 93.1% on ALFWorld and BEACON nearly doubling GRPO’s success rate on the same benchmark [1][23].
• STAT shows that aggregate return metrics can mask distinct coordination mechanisms in multi-agent systems, motivating process-level diagnostics as a necessary complement to outcome-based benchmarking [3].
• The constraint decay phenomenon in backend code generation shows that structural compliance degrades sharply as requirements accumulate, with capable agents losing an average of 30 assertion-pass-rate points and performing worst in convention-heavy frameworks [5].
• Deep research agent evaluation reveals that factual accuracy in citations falls to 39-77% even when link validity exceeds 94%, and that scaling tool calls from 2 to 150 drops fact-check accuracy by approximately 42% on average [17].