8 items
8 posts
Twelve frontier models sat at 60 percent on scientific coding, successors tying predecessors - a textbook saturation curve. A ground-truth audit found 263 defects in the benchmark and the corrected scores jump to 84 to 98 percent. The wall was the yardstick, and that changes how you should read every flat leaderboard.
The first production-scale trace of agentic coding says the context you keep paying for is already dead at every turn boundary. The fixes that moved numbers this week are not bigger models: kill the run, carry the state, start over. Here is the bet you can grade us on.
Evidence gates, verifiable reward games, deploy-time certificates: the fixes that moved agent quality this week did not make judges better, they removed the judge. We think the LLM verdict inside the agent loop is a transitional technology, and here is the bet you can grade us on.
Since we published your-benchmark-is-lying-to-you, roughly 25 new results have landed on the eval-integrity question. The surprise: every fix that works is structural - ledgers, counterfactuals, decompositions, personas, readout discipline - and none of them asks the model to be smarter.
A wave of audits in the last two days measured the noise floor of agent benchmarks: misaligned ground truth, lenient model judges, and aggregate scalars that hide real failures. Here is what the numbers actually mean, what to trust, and how to buy agents without being played.
A late-July research wave - native in-backbone memory, pretrained parametric memory at scale, memory reconstruction, and transactional memory writes - challenges the external-store paradigm every agent memory product is built on. Here is what changes by late 2027 and what developers should do now.
Model prices fell 80% in a single announcement this week. Run the trendline forward and the interesting question is not the price - it is what developers, teams, and the broader economy do when intelligence stops being the scarce input.
A thought experiment with the sci-fi removed: freeze the models at today's capability, drop the price to zero overnight, and work out what actually changes for a working developer. Less than you fear, more than you think, and not where you expect.

New tutorials, open-source projects, and deep dives on coding agents - delivered weekly.