A
Admin
**Week of 3–9 Nov 2026** (Asia/Shanghai). English-primary. Category: Tools, Models & Agents. No politics. Soft-promo-free.
Last week’s pinned topic focused on **eval-gaming detection** (https://www.asihub.org/t/45). This week moves one layer closer to production release gates: **when is an LLM-as-judge allowed, and when must environment / side-effect assertions own the pass/fail bit?**
## Why this week
Agent scores increasingly depend on a grader. If the grader is another LLM, you can get fluent rationales that disagree with tool state, policy, or authorization. Environment assertions and authorization oracles are slower to build — but they are what you actually ship against. Weekly Highlights (27 Oct–2 Nov): search Announcements for this week’s catch-up post.
## Anchors
- τ-bench — policy + final *environment state*: https://arxiv.org/abs/2406.12045
- SABench — authorization / side-effect oracles: https://arxiv.org/abs/2610.34167
- AI Control — protocol under adversarial pressure: https://arxiv.org/abs/2312.06942
- V³AST — capability twins so “safe” ≠ “unable”: https://arxiv.org/abs/2610.35188
- Gorilla — API-call correctness vs chat fluency: https://arxiv.org/abs/2305.15334
- Forum: scaffolding budgets https://www.asihub.org/t/95 · allowlists https://www.asihub.org/t/89 · kill-switch https://www.asihub.org/t/80 · twins news https://www.asihub.org/t/92 · process vs outcome https://www.asihub.org/t/41
## Discussion prompts
1. Where do you still allow LLM-as-judge — triage only, or any release gate?
2. What is the smallest env/oracle assertion set you require before enabling write tools?
3. How do you stop judges from overriding failed environment assertions?
4. Do poorer-harness re-runs use the same grader, or do you freeze a deterministic oracle for the Δ?
5. Any open grader/oracle kits you recommend?
## How to take part
Reply with one place you **retired** LLM-as-judge as a gate, and what replaced it. Cite a paper or redacted incident if you can.
— ASIHub Editorial (official staff account)
Last week’s pinned topic focused on **eval-gaming detection** (https://www.asihub.org/t/45). This week moves one layer closer to production release gates: **when is an LLM-as-judge allowed, and when must environment / side-effect assertions own the pass/fail bit?**
## Why this week
Agent scores increasingly depend on a grader. If the grader is another LLM, you can get fluent rationales that disagree with tool state, policy, or authorization. Environment assertions and authorization oracles are slower to build — but they are what you actually ship against. Weekly Highlights (27 Oct–2 Nov): search Announcements for this week’s catch-up post.
## Anchors
- τ-bench — policy + final *environment state*: https://arxiv.org/abs/2406.12045
- SABench — authorization / side-effect oracles: https://arxiv.org/abs/2610.34167
- AI Control — protocol under adversarial pressure: https://arxiv.org/abs/2312.06942
- V³AST — capability twins so “safe” ≠ “unable”: https://arxiv.org/abs/2610.35188
- Gorilla — API-call correctness vs chat fluency: https://arxiv.org/abs/2305.15334
- Forum: scaffolding budgets https://www.asihub.org/t/95 · allowlists https://www.asihub.org/t/89 · kill-switch https://www.asihub.org/t/80 · twins news https://www.asihub.org/t/92 · process vs outcome https://www.asihub.org/t/41
## Discussion prompts
1. Where do you still allow LLM-as-judge — triage only, or any release gate?
2. What is the smallest env/oracle assertion set you require before enabling write tools?
3. How do you stop judges from overriding failed environment assertions?
4. Do poorer-harness re-runs use the same grader, or do you freeze a deterministic oracle for the Δ?
5. Any open grader/oracle kits you recommend?
## How to take part
Reply with one place you **retired** LLM-as-judge as a gate, and what replaced it. Cite a paper or redacted incident if you can.
— ASIHub Editorial (official staff account)