Does an Agent Follow the Facts or Just Who Said Them? A New Provenance Audit
A controlled audit finds that trusted and untrusted evidence changed a model's action in 5.4 percent of competing cases and 1.7 percent of supporting cases.
What the audit changes and holds fixed
LLM agents pick tools and arguments from a mix of user requests, tool outputs, retrieved records, memory, and untrusted text, and evidence can be relevant to a decision without being authorized to determine it. Junchi Liao's preprint introduces a target-specific authorization audit that labels context factors separately for each tool and argument target, then runs a primary test that holds the task, the proposition being evaluated, its position in context, and the governing policy all fixed, while changing only the proposition's source authority.
What changed when only trust changed
Across 450 controlled next-action tasks spanning multiple open-weight model families, trusted and untrusted versions of the same evidence produced different actions in 5.4 percent of competing cases and 1.7 percent of supporting cases. Competing cases are where the untrusted evidence argues against the correct action, while supporting cases are where it happens to agree with it; the gap between those two rates shows the models are somewhat sensitive to where evidence comes from, but not reliably enough to ignore untrusted evidence when it disagrees.
What happens when good evidence is deliberately weakened
The audit also tests behavior when valid, authorized evidence is degraded rather than simply mismatched with untrusted evidence. Under that controlled degradation, unauthorized competing evidence still shaped the action in 2.4 percent of comparisons, with a 95 percent confidence interval running from 2.1 to 3.0 percent. Liao is careful to describe these as controlled stress-set rates measured in an audit, not an estimate of how often this happens in ordinary deployment.
Why the models aren't simply ignoring authority
The audit's secondary diagnostic, which looks at context-subset interactions, helps localize where an unauthorized signal is creeping in rather than just confirming that it does. The paper's own framing is direct: the tested models do respond to textual cues about where evidence came from, but that sensitivity does not prevent untrusted evidence from still influencing which tool or argument the model picks.
Key facts
- The audit changes only a proposition's source authority while holding the task, proposition, position, and policy fixed.
- Across 450 controlled next-action tasks on open-weight model families, trusted and untrusted evidence produced different actions in 5.4 percent of competing cases versus 1.7 percent of supporting cases.
- Under controlled degradation of valid evidence, unauthorized competition still shaped 2.4 percent of comparisons (95 percent confidence interval: 2.1 to 3.0 percent).
- These are controlled stress-set rates from an audit, not measured deployment prevalence.
- Models responded to textual source-authority cues, but that response did not fully block untrusted evidence from shaping actions.
Check your understanding
- Why do competing cases and supporting cases give different percentages (5.4 percent versus 1.7 percent) in this audit?
- What does it mean that the audit holds the task, proposition, position, and policy fixed while changing only source authority?
- Why does the author caution that these percentages are stress-set rates rather than deployment prevalence?