T²PO Stabilizes Multi-Turn RL; MotionCache Cuts Video Steps 6x

Featured

01 Multi-Turn Agent RL Collapse Isn't About Credit Assignment

If you've trained a reasoning LLM with multi-turn RL, you know the collapse curve. Reward climbs, then suddenly tanks a few hundred steps in. The standard explanation blames credit assignment or weak trajectory filtering. T²PO offers a different diagnosis.

The policy keeps generating low-information actions. They neither reduce model uncertainty nor pull useful signal from the environment. The whole rollout spins in place. T²PO handles this in two layers. At the token level, it monitors marginal change in uncertainty and fires a "thinking" intervention if the change drops below threshold. At the turn level, it identifies interactions where exploration progress is negligible and resamples them outright.

Training stability and final performance both improve on WebShop, ALFWorld, and Search QA. The paper is ICML-accepted with open-source code. Threshold tuning and cross-task transfer need a closer read of the full paper. But the core idea — using a model's own uncertainty as an active training signal — runs orthogonal to mainstream credit-assignment plus trajectory filtering.

Key takeaways: - If multi-turn agent RL is unstable, check exploration efficiency first. Stop stacking tricks on top of credit assignment. - A model's own uncertainty can be an active training-time signal, not just an inference-time confidence value. - Teams running multi-turn agent training can compare this directly against their existing stabilization approach.

Source: T²PO: Uncertainty-Guided Exploration Control for Stable Multi-Turn Agentic Reinforcement Learning

02 Models Don't Know What They Don't Know

Factuality progress over the past few years has mostly come from making models remember more — pushing the knowledge frontier outward. This paper names a layer that gets skipped. Models still don't recognize their own ignorance, so even on the simplest factoid QA they hallucinate with confidence.

The authors argue past the binary "answer or refuse" framing. A third path: align linguistic uncertainty with internal uncertainty, also called metacognition. For tool builders, this is exactly the layer that gets jumped over when assembling a reliability stack. The default move is RAG plus a refusal threshold. The real question underneath — whether the model's actual uncertainty is being faithfully expressed — usually goes unaddressed.

One caveat. This is a position paper. The authors admit the problem is intrinsically hard and don't offer an implementation. Naming it clearly is the contribution.

Key takeaways: - Most factuality gains come from "remembering more," not "knowing what you don't know." The two paths get conflated in engineering practice. - For agent systems, calibrated uncertainty is the control layer that decides when to retrieve and when to trust the model directly. - This is a position paper, not a method. Implementation remains open.

Source: Hallucinations Undermine Trust; Metacognition is a Way Forward

03 Medical Agents Need an Execution-Grounded Scorecard

Earlier medical agent benchmarks mostly tested static knowledge or single-step decisions: memorize medical QA, simulate one prescription. The distance from real clinical workflow is wide. PhysicianBench moves 100 real consultation cases into an electronic health record (EHR) environment. The interactions use the same API as a commercial EHR vendor, and each task averages 27 tool calls — pulling data across encounters, making decisions, writing notes — with verifiable execution feedback at every step.

Scoring runs across 670 stage-level checkpoints. Of 13 agents tested, the best pass@1 is 46%; best open source hits 19%. The gap is wide, but the more important point is that the benchmark actually measures clinical execution, not exam scores.

Key takeaways: - The hard part of medical agents is long-horizon, compound, cross-encounter workflow. Single-step QA benchmarks no longer say anything useful. - Teams building vertical medical agents should treat execution-grounded evaluation as a pre-launch scorecard. - The 46% vs 19% proprietary–open-source gap shows open source still has obvious catching up to do on long-horizon tool use.

Source: PhysicianBench: Evaluating LLM Agents in Real-World EHR Environments

04 Where Video Caching Goes Blind

Within a single frame, fast-moving object edges and near-static background get treated identically by the cache. That's the blind spot in current AR video generation step-skipping. The root cause is chunk-level cache reuse: a whole time window decides reuse or recompute together. Static backgrounds tolerate this fine. Fast-moving objects accumulate visible error.

MotionCache pushes granularity to the pixel level. It uses frame-to-frame difference as a lightweight motion proxy. High-motion pixels get more denoising steps; static regions reuse aggressively. SkyReels-V2 sees 6.28x speedup, MAGI-1 only 1.64x, with VBench quality essentially preserved. The 4x gap between two models means speedup is highly architecture-dependent. Transferring to other AR video models needs per-model evaluation.

Key takeaways: - Chunk-level cache skipping is blind to high-motion pixels. Frame-difference-weighted pixel-level scoring is a direct fix. - This is deployment-side optimization with no weight changes. AR video inference services can try it directly. - Speedup varies 4x across models. Confirm the transfer effect per model before relying on the numbers.

Source: Motion-Aware Caching for Efficient Autoregressive Video Generation

05 What If Attention Is Just a Parameter-Prediction MLP?

Intuitively, attention's global modeling power comes from explicit token-to-token weighting. That intuition is exactly why attention has to be O(N²). WeightFormer offers a math rewrite that gives pause. Attention is equivalent to an MLP whose parameters are predicted from input. The "global capability" actually comes from those predicted parameters compressing global context — not from token-wise weighting itself.

From this angle, linear-complexity designs no longer have to circle around "how to approximate softmax attention." The question becomes "how to predict good parameters from input." The authors run several parameter-prediction strategies on vision models and show the path works at medium scale. Whether it scales to LLM size is an open question. The math rewrite itself already opens a new design space.

Key takeaways: - Attention can be mathematically rewritten as an MLP with input-predicted parameters. Global capability comes from parameter compression, not explicit token weighting. - The viewpoint shift moves linearization design from "approximate attention" to "predict good parameters." - Vision-scale validation works. LLM-scale generalization needs follow-up work.

Source: Linear-Time Global Visual Modeling without Explicit Attention