Strategy · 7 min read

Open Weights vs Hosted API

The open-source gap to frontier closed models has narrowed to a few months. That makes self-hosting a real option — but not always the right one. Here is how to decide.

The default is still an API

APIs win on speed, zero infrastructure and the strongest models. Go open-weight when data control, fine-tuning or steady high volume genuinely demands it.

Two columns, one decision

☁️

Hosted API

Fast to production, elastic scale, strongest models, no ops. You pay per token.

🖥️

Open weights

Data stays yours, fine-tuning allowed, no lock-in — but you own the hardware and the on-call.

🔐

Data control

Critical for health, finance, government and defence: prompts never leave your environment.

💸

Hidden costs

GPUs, networking, storage, inference engineering, monitoring, staffing and power all count. Idle clusters are expensive.

Licences are not all the same

"Open" spans a wide range. MIT and Apache-2.0 are permissive; many "open-weight" licences restrict commercial use, impose acceptable-use terms, or require attribution. Always read the model card for the exact licence before committing. Frontier open weights can be enormous — a 763B or multi-trillion-parameter model may need a serious cluster, while smaller 27B-class models are far more tractable.

A quick decision checklist

  1. Must keep data on-premise or in a specific region? → lean open weights / private deployment.
  2. Need the absolute strongest reasoning today? → lean hosted frontier API.
  3. Sustained high volume with predictable load? → model the TCO of both options.
  4. Small team, fast timeline? → hosted API, revisit later.
Explore self-hostable options in the open-source leaderboard, or weigh value rankings against API prices.