The default is still an API
APIs win on speed, zero infrastructure and the strongest models. Go open-weight when data control, fine-tuning or steady high volume genuinely demands it.
Two columns, one decision
Hosted API
Fast to production, elastic scale, strongest models, no ops. You pay per token.
Open weights
Data stays yours, fine-tuning allowed, no lock-in — but you own the hardware and the on-call.
Data control
Critical for health, finance, government and defence: prompts never leave your environment.
Hidden costs
GPUs, networking, storage, inference engineering, monitoring, staffing and power all count. Idle clusters are expensive.
Licences are not all the same
"Open" spans a wide range. MIT and Apache-2.0 are permissive; many "open-weight" licences restrict commercial use, impose acceptable-use terms, or require attribution. Always read the model card for the exact licence before committing. Frontier open weights can be enormous — a 763B or multi-trillion-parameter model may need a serious cluster, while smaller 27B-class models are far more tractable.
A quick decision checklist
- Must keep data on-premise or in a specific region? → lean open weights / private deployment.
- Need the absolute strongest reasoning today? → lean hosted frontier API.
- Sustained high volume with predictable load? → model the TCO of both options.
- Small team, fast timeline? → hosted API, revisit later.