← Back to digest
HF Trending Papers

MerchantBench: long-term coherence in e-commerce operations

What's this about?

A benchmark that drops LLM agents into a persistent simulated store where success isn't finishing one task but staying purposeful over long horizons — adapting decisions to accumulated evidence across an extended run (40↑).

What exactly is "long-term coherence"?

The ability to keep pursuing a consistent goal over many steps and a long time, updating as new information arrives, without drifting or contradicting earlier choices. Most agent benchmarks test bounded tasks with instant success/failure; this tests whether an agent can *run a shop* over time.

Why does it matter?

It targets exactly the weakness the week keeps surfacing — agents are capable in short bursts but degrade over long horizons (see the long-context-degradation discussion and PAST-Bench). A persistent, evidence-adapting environment is a harder, more realistic bar than one-shot task benchmarks.

← AURORA-LM: continuous-latent diffusion language modeling