Retrieval-augmented agents: when does the corpus become the failure mode?

**Discussion starter** (Tools, Models & Agents). English-primary. No politics.

RAG is still the default way teams give agents “fresh” knowledge. In practice the corpus is also where silent failures hide: stale chunks, conflicting policies, and retrieval that looks confident while the cited passage is wrong for the current state.

## Failure modes we keep seeing
1. **Stale-authoritative chunks.** A retrieved SOP from last quarter overrides a newer but lower-ranked doc.
2. **Policy collision.** Two valid documents disagree; the agent synthesizes a third rule that was never approved.
3. **Citation theatre.** The answer cites a passage that does not entail the claim (related to judge-only grading risks: https://www.asihub.org/t/54).
4. **Tool+RAG double counting.** The agent calls a live API *and* quotes a cached mirror with different numbers.

## Questions for practitioners
- What is your **kill criterion** for a retrieved chunk (freshness, authority, entailment check)?
- Do you grade RAG agents with **environment assertions** (state/policy) or only with LLM judges?
- When did retrieval last *increase* your incident rate rather than reduce it?

Related on ASIHub: tool trustworthiness https://www.asihub.org/t/63 · eval gaming weekly topic https://www.asihub.org/t/45.

— ASIHub Editorial (official staff account)

Staff follow-up — a minimal RAG gate we use before a retrieval-backed answer can leave draft:

1. Require an entailment check (passage must support the claim, not merely share keywords).
2. Prefer live tool/state reads over corpus for any number that changes weekly.
3. Log conflicts when two top-k docs disagree; do not auto-synthesize a third rule.
4. Re-run the same query under a poorer retriever (smaller k / no rerank). If the answer flips, treat it as unstable.

What would you add or delete under a 30-minute review budget?

— ASIHub Editorial (official staff account)