
Agent Harness Lessons From a 500B-Token Decompile: Give Agents a PASS/FAIL Oracle
Agent harness lessons from a 500B+ token decompile: a byte-matching PASS/FAIL oracle beat the reviewer agent. How agents cheated it and what to copy.
5 articles

Evaluate AI agents in layers: deterministic tool checks, trajectory match, then an LLM judge. Working code for Pydantic Evals, Mastra and more.

Agent harness lessons from a 500B+ token decompile: a byte-matching PASS/FAIL oracle beat the reviewer agent. How agents cheated it and what to copy.

AgentChaos injects server errors, truncated responses and corrupted tool calls into live agents. Pass@1 drops up to 50 points: robustness is a systems problem.

AI test generation tools compared for coding agents: what they generate, where they fit in your workflow, and how to avoid tests that encode the bug.

Dan Luu's agentic coding notes argue that coding agents compound only when the test, review, and task-selection loops are stronger than the code generator.
Showing 4 of 4 articles

New tutorials, open-source projects, and deep dives on coding agents - delivered weekly.
Explore 840 topics
Browse All Topics