Operations | Monitoring | ITSM | DevOps | Cloud

On 28 September, Anthropic published a playbook for designing evals and hillclimbing against them, and shipped two commands that put it to work. In Claude Code, /claude-api build-eval interviews you and builds an eval inside your codebase. /claude-api hillclimb then improves your app against it, one change per round, with a held-out split to catch overfitting. It is a good piece of work, and the results are concrete.

Ask a team running an agent in production whether last week’s change made it better. Most can’t say. They swapped a model or rewrote a skill, eyeballed a few runs, and moved on. That works for a demo. It doesn’t work for an agent that touches production, where customers want two things that pull against each other: precision where a mistake is expensive, and low cost where the work is routine. A root cause someone acts on has to be right.