57 articles - page 3 of 5
Copilot traces from 3.2M users and 761M LLM calls: 87% of calls are agent-initiated, KV cache hits drop at turn boundaries, tool failures multiply compute.
The LLM judge is leaving the agent loop: evidence gates, verifiable rewards and deploy-time certificates replace verdicts with structure. Here is our bet.
Agent benchmark noise is being fixed by architecture - ledgers, counterfactuals, decomposition, personas, readout discipline - not by smarter judges.
Agent benchmarks are noisier than their scores admit: audits found misaligned ground truth, lenient model judges and scalars that hide failures. What to trust.
Agent memory is moving into the model: in-backbone memory, parametric memory, reconstruction and transactional writes challenge external stores. What to do now.
Blind resampling beats self-repair in small code models: a placebo-controlled MBPP+ study finds retrying from scratch beats feeding failed code back.
CAPA benchmark: coding agents that read a user's past sessions resolve ambiguous requests with far fewer clarifying questions, tested across 12 models.
Change2Task turns merged pull requests into verified, executable coding agent tasks that stay current with the codebase, across five task families.
DeepSeek V4 Flash 0731 was re-post-trained for agent work. Its benchmark numbers, native Responses API and Codex support, and how to wire it up today.
Microsoft's study of Claude Code and GitHub Copilot CLI found about 24% more merged PRs among adopters. The bigger lesson is rollout design, not magic.
ORCA-bench puts five frontier coding agents on live oncall root cause analysis with real telemetry. The best scored 10% on hard tasks, far from oncall-ready.
OwlPath encodes code into an OWL2 ontology so coding agents can answer multi-hop structural queries with SPARQL, cutting tokens 28.8% on SWE-bench Pro.
Showing 25-36 of 56 articles
New tutorials, open-source projects, and deep dives on coding agents - delivered weekly.
Explore 840 topics
Browse All Topics