7 min read
TutorMoments: AI2's New Benchmark Shows LLM Tutors Over-Help by Default
TutorMoments, AI2's replay-based benchmark for LLM tutors, finds models over-help by default. How it scores scaffolding and rigor in real tutoring transcripts.
BENCHMARKS
32 articles - page 2 of 3
10 min read
The Plateau Was the Instrument
A benchmark plateau can be a broken yardstick: a SciCode audit found 263 defects, and corrected scores jump to 84-98 percent. How to read flat leaderboards.
10 min read
The Judge Is Leaving the Agent Loop
The LLM judge is leaving the agent loop: evidence gates, verifiable rewards and deploy-time certificates replace verdicts with structure. Here is our bet.
11 min read
The Fix for Broken Benchmarks Is Architecture, Not Smarter Models
Agent benchmark noise is being fixed by architecture - ledgers, counterfactuals, decomposition, personas, readout discipline - not by smarter judges.
10 min read
Your Benchmark Is Lying to You
Agent benchmarks are noisier than their scores admit: audits found misaligned ground truth, lenient model judges and scalars that hide failures. What to trust.
7 min read
AgentS4D: 66% of All Coding Agent Runs Were Unsafe Yet Still Completed
AgentS4D tests runtime safety for Claude Code, Codex, OpenClaw and Hermes across 6,560 runs: most unsafe runs still completed. Completion is not safety.
9 min read
DeepSeek V4 Flash 0731: The Budget Tier Just Overtook Pro Preview on Agent Benchmarks
DeepSeek V4 Flash 0731 was re-post-trained for agent work. Its benchmark numbers, native Responses API and Codex support, and how to wire it up today.
Read more
7 min read
Benchmarking Opus 5 on SlopCodeBench: AI Code Quality Under Iteration
Opus 5 on SlopCodeBench hit a 24% strict pass rate, beating Opus 4.8 and Sonnet 5, yet code quality still degraded. What it shows about coding agents over time.
Read more
11 min read
Terminal-Bench Shows Harness Scaling Is the Coding-Agent Benchmark Now
Terminal-Bench and StateM show harness scaling now drives coding-agent scores. Why runbooks, durable state and recovery loops matter as much as model choice.
5 min read
Apple SpeechAnalyzer vs Whisper: Independent Benchmark Shows Apple Winning on Accuracy
Apple SpeechAnalyzer vs Whisper: an independent benchmark of 5,559 utterances puts Apple at a 2.12% word error rate, ahead of every Whisper size tested.
Read more
8 min read
Cursor Composer 2.5 Developer Guide 2026
Cursor Composer 2.5 guide: Cursor's agentic coding model, setup, Standard vs Fast pricing, benchmarks against Opus 4.7 and GPT-5.5, and when to use it.
Read more
6 min read
GPT-5.5 Has a 3x Higher Hallucination Rate Than MIT-Licensed GLM-5.2
GPT-5.5 hallucinates 86% of the time on AA-Omniscience when it does not know, versus 28% for MIT-licensed GLM-5.2. What the metric measures and why it matters.
Read more
Showing 13-24 of 31 articles
Keep exploring Benchmarks
- - Benchmarks Topic Hub - tools and guides for Benchmarks from the Developers Digest directory
- - Glossary - dive deeper across the Developers Digest knowledge base
- - Developers Digest on YouTube - video tutorials covering Benchmarks and more
Get Smarter About AI Dev
New tutorials, open-source projects, and deep dives on coding agents - delivered weekly.
One email per weekReal code, not theoryFree forever
Explore 840 topics
Browse All Topics