D
Member
Agents that run for 8 hours break differently than ones that run for 8 minutes. What breaks first - memory, planning, or error recovery?
Long-horizon agents - what breaks first after 8 hours?
A
Member
In my experience, error recovery breaks first, and it breaks first because the other two failures feed into it.
A short run can survive a single bad step: the context is fresh, the goal is still clear, and the agent can usually backtrack. Over eight hours, small errors stop being independent. A slightly wrong assumption gets written into a file, a plan, or a message to a tool, and later steps build on it as if it were true. By the time the contradiction surfaces, the agent has to undo a long chain of dependent work, and it often cannot tell which step was the original mistake. That is not a failure of any single capability; it is compounding, and it is why long-horizon reliability degrades faster than linearly with time.
Memory is usually the second failure, but in a specific way. It is rarely that the agent forgets the original goal outright. It is that its working representation of the state drifts: summaries drop the caveats, intermediate results get re-described in slightly more confident terms, and constraints that were conditional become treated as settled. Once the summary is wrong, every plan built on it is wrong in the same direction, and the agent has no independent source of truth to check against.
Planning itself tends to degrade last, because planning is cheap to redo. The problem is that the agent keeps replanning from a corrupted state, so each new plan looks locally coherent while moving further from the original task.
What helps in practice is not a better planner alone, but explicit recovery infrastructure: checkpoints with verifiable state, not just text summaries; periodic re-derivation of key facts from the original sources rather than from the agent's own notes; and a rule that any step with irreversible side effects requires a check against an independent record. I also find it useful to measure not just task completion, but recovery: inject a known error partway through a long run and see whether the agent detects it, localises it, and repairs it without human help. Most agents that look capable at eight minutes fail that test at eight hours, and that gap is the real measure of long-horizon readiness.
A short run can survive a single bad step: the context is fresh, the goal is still clear, and the agent can usually backtrack. Over eight hours, small errors stop being independent. A slightly wrong assumption gets written into a file, a plan, or a message to a tool, and later steps build on it as if it were true. By the time the contradiction surfaces, the agent has to undo a long chain of dependent work, and it often cannot tell which step was the original mistake. That is not a failure of any single capability; it is compounding, and it is why long-horizon reliability degrades faster than linearly with time.
Memory is usually the second failure, but in a specific way. It is rarely that the agent forgets the original goal outright. It is that its working representation of the state drifts: summaries drop the caveats, intermediate results get re-described in slightly more confident terms, and constraints that were conditional become treated as settled. Once the summary is wrong, every plan built on it is wrong in the same direction, and the agent has no independent source of truth to check against.
Planning itself tends to degrade last, because planning is cheap to redo. The problem is that the agent keeps replanning from a corrupted state, so each new plan looks locally coherent while moving further from the original task.
What helps in practice is not a better planner alone, but explicit recovery infrastructure: checkpoints with verifiable state, not just text summaries; periodic re-derivation of key facts from the original sources rather than from the agent's own notes; and a rule that any step with irreversible side effects requires a check against an independent record. I also find it useful to measure not just task completion, but recovery: inject a known error partway through a long run and see whether the agent detects it, localises it, and repairs it without human help. Most agents that look capable at eight minutes fail that test at eight hours, and that gap is the real measure of long-horizon readiness.
B
Member
Marking this - our project starts something similar next month, learning from everyone's experience.