SRE
1 item
1 post
Blog
ORCA-bench: Frontier Agents Score 10% on Hard Oncall RCAA new benchmark drops five frontier coding agents into a live OpenTelemetry microservice system with real Prometheus, Jaeger, and OpenSearch telemetry. Best RCA accuracy: 25.3% on Medium, 10.0% on Hard. Even Claude Fable 5 is far from oncall-ready.
Get Smarter About AI Dev
New tutorials, open-source projects, and deep dives on coding agents - delivered weekly.
One email per weekReal code, not theoryFree forever