COMPASS Tackles the Long-Horizon Problem That Trips Up AI Agents

A Framework Built for the Long Haul

Ask a large language model to answer a question, and it usually does fine. Ask it to run a 40-step research investigation that requires chaining together dozens of web searches, tool calls, and intermediate judgments, and something quietly falls apart. Small errors compound. The agent forgets what it already found, chases a dead-end path long after it should have pivoted, or declares victory before the job is done. This is the long-horizon problem, and a July 2026 wave of research has zeroed in on it as the single biggest obstacle standing between todays capable models and genuinely reliable AI agents.

A new paper, COMPASS: Enhancing Agent Long-Horizon Reasoning with Evolving Context, offers one of the more concrete attempts to fix it. Written by Guangya Wan, Mingyang Ling, Xiaoqi Ren, Rujun Han, Sheng Li, and Zizhao Zhang, a team spanning Google Cloud AI and the University of Virginia, the paper argues that the failures agents suffer on long tasks are not primarily a reasoning problem. They are a context problem.

Diagnosing the Bottleneck

The authors frame the core issue plainly: as an agent works through a long task, its history of actions and observations piles up into an unstructured, ever-growing transcript. That accumulation becomes a liability. According to the paper, extended histories cause agents to overlook critical evidence or become distracted by irrelevant information, leaving them unable to replan effectively or reflect on earlier mistakes. Even state-of-the-art models, the authors note, hallucinate or lose coherence once the context gets long and noisy enough.

To make the failure modes measurable rather than anecdotal, the paper introduces four diagnostic metrics for long-horizon behavior: Persist Appropriateness Rate, which tracks whether an agent correctly sticks with a valid plan; Pivot Recognition, whether it changes course when a plan is failing; Conclude Accuracy, whether it knows when to stop; and Error-Recovery Continuation, whether it retries productively after a mistake. Together, these are meant to expose exactly where agents drift off the rails during extended work.

How COMPASS Works

COMPASS, short for Context-Organized Multi-Agent Planning and Strategy System, is a lightweight hierarchical framework that splits an agents job into three specialized roles rather than piling everything onto a single monolithic loop.

The first is a Main Agent that handles tactical execution: reasoning and calling tools in a familiar ReAct-style loop. The second is a Meta-Thinker, which monitors the agents progress, watches for anomalies, and issues strategic interventions when the work appears to be going sideways. The third is a Context Manager, which maintains concise, strategically relevant progress briefs tailored to different stages of reasoning, so the Main Agent works from a clean, curated summary instead of drowning in the full raw transcript.

The design intent is a division of labor that mirrors how a well-run team operates: one worker executes, a supervisor keeps an eye on the strategy, and someone keeps the notes organized. By separating tactical execution from strategic oversight and from context organization, the authors aim to keep the agent coherent across a long horizon without simply throwing a bigger context window at the problem.

The Numbers

COMPASS was evaluated on three demanding agent benchmarks: GAIA, BrowseComp, and Humanitys Last Exam, all of which stress multi-step reasoning and tool use. Across those benchmarks, the authors report that COMPASS improves accuracy by more than 10 percent over both single-agent and multi-agent baselines. A test-time scaling extension pushes the gains further, boosting performance by up to 20 percent.

The BrowseComp results illustrate how the pieces contribute. A single-agent ReAct baseline reaches roughly 16.8 percent success, according to figures reported for the paper. Adding meta-thinking alone lifts that to about 26.4 percent, and context management alone to about 29.8 percent. Combining both in the full COMPASS system reaches about 35.4 percent, and layering on test-time scaling reaches about 43.7 percent. Ablation studies, the authors say, confirm that each designed component is doing real work rather than riding along for free.

Why This Matters

The significance here is less about any single benchmark score and more about what the long-horizon problem represents for the industry. Every major AI lab is now selling the promise of agents that can be handed a goal and left to work autonomously for minutes or hours. But autonomy only pays off if the agent stays reliable over the full length of the task, and reliability is precisely what degrades as context accumulates. An agent that is brilliant for ten steps and incoherent by step thirty is not something a business can trust with real work.

COMPASS is notable because it treats context as something to be actively managed and evolved, not passively accumulated. That reframing, echoed across the broader July 2026 research wave, suggests the field is converging on a view that context engineering, rather than raw model scale, may be the near-term lever for agent reliability. The metrics the paper introduces are arguably as valuable as the framework itself, giving researchers a shared vocabulary for the specific ways agents fail on long tasks.

What to Watch

The open questions are practical ones. COMPASS adds architectural overhead, with multiple coordinated components and, in its scaled form, extra test-time compute, so the cost-to-reliability tradeoff will matter as much as headline accuracy. It remains to be seen how well the approach generalizes beyond research and browsing benchmarks to messier real-world workflows, and whether the four proposed metrics catch on as standard tools. If they do, the long-horizon problem may finally have both a diagnosis and a growing menu of cures.