Agentic coding ≠ code generation
The question shifts from "does the code compile?" to "does the agent finish the task without drifting, looping or burning the budget?"
The four dimensions that matter
Tool reliability
A model that mis-calls tools derails a whole run. Check function-calling leaderboards and your own schemas.
Long-horizon coherence
SWE-bench Verified/Pro and terminal benchmarks are the best public proxies for multi-step reliability.
Context window
1M tokens on several 2026 models lets an agent hold a codebase — but retrieval quality beats raw size.
Cost per task
Cheap tokens aren't cheap if the agent needs 3× the attempts. Measure spend per merged change.
Reasoning effort is a cost lever
Many 2026 models expose an effort setting. High effort improves hard reasoning at a steep price — a single max-effort task can cost tens of dollars. Route simple edits to a fast model and reserve high effort for genuinely hard problems. A router plus two models often beats one flagship on both cost and quality.
Open weights for codebases that cannot leave
Teams with strict data policies increasingly pick open-weight models such as DeepSeek, Kimi, GLM or Qwen variants. Check the licence and the hardware footprint: frontier-scale open weights can require a large self-host cluster, so compare that against API pricing honestly.