Left behind

Generative verification repurposes the language model’s native next‑token prediction head to produce a continuous Yes/No verification score, avoiding any added classification layer and preserving pretrained representations.

From

Beyond Solver Verdicts: Generative Reward Models for Autoformalization

Contradicted by

C353

The generative guard outputs an auditable analysis and a deterministic binary safety verdict that can be consumed by downstream controllers without an external judge at inference time.

From

HazardAuditor: From Executable Threats to Safer Computer-Use Agents

Confidence

0.75

Flagged

Wed Sep 30

DRACO generates rubrics dynamically during training to track the policy’s evolving capability.

From

DRACO: Fine-Grained Credit Assignment with Dynamic Rubrics for Long-Horizon Agent Training

Contradicted by

C393

In the outer recursion, the model is trained with rubric‑based rewards using GRPO while the harness and evaluation rubrics remain fixed throughout the reinforcement‑learning phase.

From

ScienceBuddy: Recursive-in-Recursive Self-Improvement for Interactive Scientific Agents

Confidence

0.75

Flagged

Wed Sep 30

Chat structure ranker selects more relevant explanations than any computational signal, achieving higher AUROC in 12 of 14 dataset‑model combinations without requiring a model forward pass for position selection.

From

Selecting The Most Informative Tokens in Natural Language Autoencoders

Contradicted by

C1147

Signals derived from activation vectors are the strongest individual predictors of on‑task explanations in 11 of 14 dataset‑model combinations.

From

Selecting The Most Informative Tokens in Natural Language Autoencoders

Confidence

0.85

Flagged

Wed Oct 14

Across both audit cases, zero recoveries were observed in 96 independent challenger episodes, yielding a finite‑sample upper bound on the recovery probability of 0.0468.

From

Scores Alone Do Not Prove Discovery: The Discovery Certification Protocol for Auditing AI Research Agents

Contradicted by

C289

In the paired feedback studies, 30 truthful recoveries and zero neutral recoveries were observed, and the null calibration passed with 60 paired null studies, establishing a statistically significant positive feedback effect.

From

Scores Alone Do Not Prove Discovery: The Discovery Certification Protocol for Auditing AI Research Agents

Confidence

0.75

Flagged

Tue Sep 29

Adding Desired Behavior and Motivation information to prompts significantly improves coding-agent performance, whereas including Environment Information or Reproduction Steps provides no measurable benefit beyond token overhead.

From

RealSWE: A Compositional Evaluation of Coding Agents under Realistic User Requests

Contradicted by

C136

Feedback-Enriched Environments (FEEs) consistently improve performance of self‑evolving agents on long‑horizon tasks across SciWorld and BFCL benchmarks.

From

Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks

Confidence

0.90

Flagged

Tue Sep 15

The strongest tested configuration, Claude Opus 5 under Claude Code, passes only 23.9% of the 53 evaluation simulations across four domains.

From

τ^τ-Bench: An Environment for End-To-End, Realistic Agent Construction

Contradicted by

C12

An expert‑authored reference implementation achieves an 82.2% success rate on the same benchmark, establishing a high ceiling.

From

τ^τ-Bench: An Environment for End-To-End, Realistic Agent Construction

Confidence

0.88

Flagged

Tue Sep 08

ZGCM‑1‑7B ranks first on average across 14 reasoning benchmarks at the 7B–8B scale, scoring 75.0% on AIME 2026, 97.1% on MATH‑500, and 70.4% on HMMT 2025.

From

ZGCM-1: A Fully Open and Extremely Efficient Foundation Model for Math and Agentic Search

Contradicted by

C897

Magenta achieves 100 % accuracy on the AIME 2025, AIME 2026, and HMMT February 2026 olympiad benchmarks.

From

Magenta: Closing the Loop Between Mathematical Reasoning and Lean Verification

Confidence

0.75

Flagged

Tue Oct 06

Rubric exploitability varies dramatically with the Oracle configuration: holding a generator’s rubrics, attack responses, and judge scores fixed while swapping only the Oracle yields exploitation rates of 33.3%, 75.6%, and 66.7% respectively.

From

ImpossibleRubrics: Stress-Testing Generated Rubrics as Reward Signals

Contradicted by

C478

All 15 attacks flagged as violations by the primary Oracle are also flagged by the two alternative Oracles, demonstrating cross‑Oracle consistency.

From

ImpossibleRubrics: Stress-Testing Generated Rubrics as Reward Signals

Confidence

0.75

Flagged

Thu Oct 01

The agent’s inference cost and token consumption are substantial, reaching up to $31.20 and 25 million tokens per successful trial, highlighting a barrier to practical deployment.

From

Agent as Policy for Robotic Manipulation

Contradicted by

C482

Zing-0.5 runs real‑time inference at 832×480 resolution and 24 FPS with an estimated server rental cost of $0.009 per stream‑minute on an RTX 5090.

From

Zing-0.5: Toward Playable Worlds with Real-Time Joint Action and Text Control

Confidence

0.75

Flagged

Thu Oct 01

Supervised fine‑tuning on code‑edit tasks overfits to the specific corruption patterns seen during training and fails to generalize to out‑of‑domain edits.

From

When Models Edit Too Much: On the Fidelity of Minimal Code Edits

Contradicted by

C469

Training the PhAI‑IDE model family (72B, 9B, 4B) on verified scientific interaction trajectories improves held‑out scientific‑code repair performance and yields gains on general‑purpose code, reasoning, and knowledge benchmarks, demonstrating positive transfer from scientific experience

From

ScienceIDE: Turning World's Scientific Codebase into Agent Learnable Environments

Confidence

0.75

Flagged

Thu Oct 01

Simple computations can be performed covertly without leaking information in chain‑of‑thought traces.

From

Hidden Reasoning Must Leak, but Need Not Be Readable: Fundamental Opportunities and Limits for Chain-of-Thought Monitoring

Contradicted by

C1367

Omitting the tenant‑salt provenance enables an observer to identify a victim’s prompt with 93% accuracy from timing side‑channels.

From

Preserving Provenance in Shared KV Caches for LLM Serving

Confidence

0.75

Flagged

Sun Oct 18

Altering token‑level supervision via different teacher profiles or matched loss interventions in Qwen leaves the student model’s behavior largely unchanged.

From

What Does Privileged Information Add to On-Policy Self-Distillation?

Contradicted by

C616

Teacher Self‑Deviation (TSD) affects roughly 30 % of tokens even with task‑agnostic interventions and is largely retained when richer privileged context is added.

From

Calibrating Teacher--Student Discrepancy for On-Policy Distillation

Confidence

0.75

Flagged

Sat Oct 03

C701

Distilling a teacher model that sees outcomes into a student model improves decision‑making on unseen tasks and raises end‑to‑end success on held‑out SWE‑bench Pro tasks.

From

The Tasteful Agent: Measuring and Improving Taste in Long-Horizon Tasks

Confidence

0.75

Flagged

Sun Oct 04

C750

ATD achieves capability transfer across multiple model families and scales, showing positive mean gains for Qwen generations, larger and smaller Qwen models, and Llama models, and works with both LoRA‑based and full‑parameter student fine‑tuning.

From

Post-Training Leaves Behavioral Shadows on Unrelated Decisions

Confidence

0.75

Flagged

Sun Oct 04

The anti‑hacking pipeline eliminates all observed reward‑hacking attempts (0 successful hacks) while preserving normal agent functionality during evaluation.

From

SWE-Bench Pro Verified: A Reliable Benchmark for Software Engineering Agents

Contradicted by

C705

Reward‑hacking rate of the discovered agents falls from 55 % to 32 % during the run, 7 percentage points below the human‑engineered baseline.

From

Recursive self-improvement of AI research agents

Confidence

0.75

Flagged

Sun Oct 04

On four memory‑dependent RMBench tasks, the same execution approach attains only 12.5% average success, highlighting challenges for memory‑intensive tasks.

From

EmbodiedSkills: A Unified Framework for Orchestrating, Training, and Deploying VLA Agents

Contradicted by

C265

MaP-WAM achieves state-of-the-art performance on RMBench with an 83.3% success rate.

From

Memory as Plans: World-Action Modeling with Memory-Grounded Planning

Confidence

0.78

Flagged

Sat Sep 26

Small open‑source VLMs can be adapted for low‑cost robot deployment with only a few GPU‑hours of fine‑tuning using Show‑Harness.

From

Show-Harness: Just a VLM Agent Can Play Robots

Contradicted by

C190

A well‑designed semantic interface alone can unlock substantial embodied capability from foundation VLMs, removing the need for additional model capacity or embodiment‑specific pretraining.

From

Show-Harness: Just a VLM Agent Can Play Robots

Confidence

0.77

Flagged

Sat Sep 19

Vosti produces bitwise‑identical logits across all tested execution variations.

From

Vosti: Specifying, Implementing, and Verifying Deterministic LLM Inference

Contradicted by

C1343

Production deterministic modes (vLLM batch‑invariant and SGLang deterministic) do not achieve full bitwise determinism across all execution variations.

From

Vosti: Specifying, Implementing, and Verifying Deterministic LLM Inference

Confidence

0.75

Flagged

Sat Oct 17

Using SyncWorld’s simulated rollouts at test time enables policy improvement without further training.

From

SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulators

Contradicted by

C676

World‑model grounding can be inherited without using the generative world model at test time, by aligning internal features during training only.

From

Think Like a World Model, Act Like a VLA: Distilling World-Model Representations into Compact Robot Policies

Confidence

0.75

Flagged

Sat Oct 03

Parallel sampling combined with simple selection outperforms sequential reflection for test‑time cost efficiency.

From

What Else Needs Fixing? Exploring Cost-Effective Test-Time Compute for Revision Propagation in Artifacts Generated Through Conversation

Contradicted by

C1119

Sequential sampling of rollouts (instead of parallel i.i.d. rollouts) lifts exploration, achieving solutions for 14–59% of tasks at various Pass@k thresholds.

From

RLTL;DR: Self-improvement by Internalizing Self-generated Feedback

Confidence

0.75

Flagged

Mon Oct 12

GraphSkillEvo improves average accuracy over the SkillOpt baseline by 4.01% on GPT‑5.4‑nano and 1.76% on GPT‑5.4 across five agent benchmarks.

From

GraphSkillEvo: Evolutionary Optimization of Graph-Structured Agent Skills

Contradicted by

C863

SkillOpt outperforms the strongest per‑cell baseline by an average of 5.4 points across all 52 evaluated (model, benchmark, harness) cells.

From

SkillOpt: Executive Strategy for Self-Evolving Agent Skills

Confidence

0.75

Flagged

Mon Oct 05

COBRA‑Skills remains robust to changes in the agent harness and works effectively when the target model itself is used for skill generation and refinement.

From

COBRA-Skills: Contextual Bandit-Guided Evolution for Agent Skill Optimization

Contradicted by

C849

The choice of agent harness materially changes how the same language model benefits from Skills during inference.

From

SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks

Confidence

0.75

Flagged

Mon Oct 05

OpenWAM‑α, pretrained on roughly 6,400 hours of egocentric human and robot data, achieves top‑tier performance across eight simulation benchmarks and real‑robot experiments, maintaining its ranking from simulation to the physical world.

From

OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining

Contradicted by

C837

InternW0-Δ outperforms prior methods on both simulation benchmarks and real‑robot platforms.

From

InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data

Confidence

0.75

Flagged

Mon Oct 05

An expert‑authored reference implementation achieves an 82.2% success rate on the same benchmark, establishing a high ceiling.

From

τ^τ-Bench: An Environment for End-To-End, Realistic Agent Construction

Contradicted by

C85

On four memory‑dependent RMBench tasks, the same execution approach attains only 12.5% average success, highlighting challenges for memory‑intensive tasks.

From

EmbodiedSkills: A Unified Framework for Orchestrating, Training, and Deploying VLA Agents

Confidence

0.78

Flagged

Fri Sep 11

A single ReAct agent without sub‑agents or test‑time verification can solve the hardest rollouts and achieve top performance when combined with the above training and context management.

From

Iris: Climbing to the Search Frontier

Contradicted by

C82

A shared executable‑skill interface lets high‑level skill selection, bounded low‑level VLA execution, and post‑action verification be combined in a single agent loop.

From

EmbodiedSkills: A Unified Framework for Orchestrating, Training, and Deploying VLA Agents

Confidence

0.78

Flagged

Fri Sep 11

Sufficiently complex hidden computation always leaves an information‑theoretic footprint in the CoT.

From

Hidden Reasoning Must Leak, but Need Not Be Readable: Fundamental Opportunities and Limits for Chain-of-Thought Monitoring

Contradicted by

C1244

The leakage need not be readable: under plausible cryptographic assumptions, even a one‑layer Transformer can encrypt its reasoning online so that no polynomial‑time monitor can extract information about the hidden computation.

From

Hidden Reasoning Must Leak, but Need Not Be Readable: Fundamental Opportunities and Limits for Chain-of-Thought Monitoring

Confidence

0.75

Flagged

Fri Oct 16

For tasks beyond a model‑size‑dependent threshold, solving the task necessarily leaks a near‑linear amount of information about the hidden input into the CoT.

From

Hidden Reasoning Must Leak, but Need Not Be Readable: Fundamental Opportunities and Limits for Chain-of-Thought Monitoring

Contradicted by

C1244

The leakage need not be readable: under plausible cryptographic assumptions, even a one‑layer Transformer can encrypt its reasoning online so that no polynomial‑time monitor can extract information about the hidden computation.

From

Hidden Reasoning Must Leak, but Need Not Be Readable: Fundamental Opportunities and Limits for Chain-of-Thought Monitoring

Confidence

0.80

Flagged

Fri Oct 16

Compact LLMs struggle to identify malicious behaviors hidden in complex Skill packages.

From

SKILLLITE: Evidence-Guided Malicious Skill Auditing with Compact LLMs

Contradicted by

C1184

SKILLLITE extracts security‑relevant behaviors and infers the intended functionality from complex Skill packages before assessing maliciousness with a compact LLM.

From

SKILLLITE: Evidence-Guided Malicious Skill Auditing with Compact LLMs

Confidence

0.75

Flagged

Fri Oct 16

Signature Hack (Choulea) can recover encrypted reasoning blocks with up to 98% success for traces under 4096 tokens and 91% success for longer traces in Generation 4.5.

From

Feyospace-v1: How the Cyber Mercury Seven Trained Frontier Cyber Models

Contradicted by

C1241

Simple computations can be performed covertly without leaking information in chain‑of‑thought traces.

From

Hidden Reasoning Must Leak, but Need Not Be Readable: Fundamental Opportunities and Limits for Chain-of-Thought Monitoring

Confidence

0.85

Flagged

Fri Oct 16

Fathom’s per‑query read‑depth key scan runs 1.67× faster in GPU time than the 136‑bit scans of Double Sparsity, Loki, and SparQ (r=32) on Qwen‑3‑8B at one‑million‑token sessions.

From

Fathom: Per-Query Read Depth for Sparse Decoding over Offloaded KV Caches

Contradicted by

C574

Fathom does not provide speedups when the index resides in GPU memory.

From

Fathom: Per-Query Read Depth for Sparse Decoding over Offloaded KV Caches

Confidence

0.75

Flagged

Fri Oct 02

Increasing the amount of domain QA training data continuously improves in‑task performance while maintaining robustness on out‑of‑domain benchmarks.

From

Training Specialist Models without Reasoning Trajectories for Domain Expert Distillation

Contradicted by

C587

In‑domain measurement overstates cross‑dataset transfer gains by approximately 5×, and improvements that look beneficial in‑domain can hurt out‑of‑domain performance.

From

RelateAnything: Real-Time Open-Vocabulary Relation Prediction From Any Inputs

Confidence

0.75

Flagged

Fri Oct 02