Contents · 11 sections↓
Open-weight vs API models is one of the most common AI decisions leaders face in 2026. An API model is rented: you send text to a vendor's servers and pay per use. An open-weight model is downloaded: you, or a provider you choose, run the model file yourself. This guide compares the two on cost, control, data, licences and staffing, with prices from each vendor's own pages, and ends with a scorecard you can use to decide what to run where.
Open-weight vs API models: which should your company use?
Most companies should start with API models and add open-weight models where a specific need pays for the extra work. Those needs are: data that must stay inside your own environment, a large and steady volume, a model you must keep unchanged for years, deep customization, or a policy about where models come from. Without one of these, self-hosting usually costs more in hardware and people than it saves.
Key takeaways:
It is not all or nothing. Many teams use API models for hard tasks and open-weight models for high-volume or sensitive ones.
Open-weight does not have to mean self-hosted. Cloud providers also host open-weight models behind an API, which gives you a way out later without running hardware now.
Hardware is a fixed cost; API use is a variable one. One rented high-end GPU costs about $5,100 a month around the clock at Azure's list price, before any staff time.
Licences differ more than the word "open" suggests. Read each model's actual licence.
Test on your own work. Public rankings change with every release; your tasks decide.
[02]
What is the difference between open-weight and API models?
A model's "weights" are the numbers learned during training. They are the model. With an API model, such as the models behind ChatGPT, Claude or Gemini, the vendor keeps the weights and you send requests over the internet. With an open-weight model, such as Llama, Qwen, Mistral's open models, DeepSeek or OpenAI's gpt-oss, the publisher lets you download the weights and run them wherever you like.
"Open-weight" is not the same as "open source". Many open-weight models come without the training data or the full training code, and some use licences with their own conditions.
In practice there are four ways to run a model, from least to most work:
API model. The vendor runs everything. You choose from the vendor's models and accept its terms and its update schedule.
Hosted open-weight model. A provider such as Cloudflare, AWS or Google Cloud runs an open-weight model for you and charges per use. You can move the same model elsewhere later.
Self-hosted in your cloud account. Your team rents GPUs, the specialized chips that run models, in your own cloud account and runs the model there.
On your own hardware. You buy the servers and run them in your own data center.
[03]
How do they compare?
Factor | API model | Open-weight, self-hosted |
|---|---|---|
Time to start | Minutes | Weeks, to set up hardware, serving and monitoring |
Cost shape | Pay per use; grows with volume | Mostly fixed; hardware costs the same busy or idle |
Where data goes | To the vendor, under its terms | Stays in your environment |
Model versions | The vendor retires old models on its schedule | You keep a version as long as you want |
Customization | Limited to what the vendor offers | Full, including fine-tuning (further training on your data) |
Choice of models | The vendor's catalog | Any open-weight model whose licence fits |
Staffing | Little | Engineers to run, secure, update and monitor it |
Dependence | On one vendor's prices and policies | On your own team and hardware supply |
A hosted open-weight model sits in between. It starts as fast as an API, costs per use, and keeps the option to move the same model in-house later.
[04]
What does each option cost?
API prices are per million tokens. A token is a small piece of a word; a page of text is roughly 600 to 800 tokens. Prices below are standard list prices for short prompts, read on each vendor's pricing page on October 22, 2026:
Model | Type | Input per 1M tokens | Output per 1M tokens |
|---|---|---|---|
Claude Opus 5.5 (Anthropic) | Large API model | $4.00 | $20.00 |
Gemini 3.1 Pro Preview (Google) | Large API model | $2.00 | $12.00 |
Gemini 2.5 Flash-Lite (Google) | Small API model | $0.10 | $0.40 |
Claude Haiku 5.5 (Anthropic) | Small API model | $0.10 | $0.50 |
gpt-oss-120b on Cloudflare Workers AI | Hosted open-weight | $0.35 | $0.75 |
Self-hosting is priced by the hour. Microsoft's retail price for an Azure virtual machine with one H100 GPU (NC40ads H100 v5, East US, Linux) is $6.98 an hour, about $5,100 a month if it runs all the time. OpenAI's model card says gpt-oss-120b runs on a single 80 GB GPU; larger models need several.
Here is one workload priced each way: one billion input tokens and 200 million output tokens a month, roughly a million requests with a page in and a paragraph out.
Three lessons come out of this:
The biggest saving is choosing a smaller model, not self-hosting. Moving routine work from a large model to a small one cuts the bill far more than buying hardware.
A GPU only pays when it is busy. At this volume, one GPU running all month costs about ten times the hosted price for the same open model. Self-hosting starts to make sense at much higher, steady volume, or for reasons other than cost.
People cost more than chips. The figures above leave out the engineers who keep a self-hosted model running, patched and monitored.
These are list prices for one example. Discounts, batch pricing, reserved hardware and real throughput all change the numbers, so ask your team to price your actual workload.
[05]
Data, licences and rules
Data. API vendors publish how they treat business data. OpenAI says API data is not used to train its models unless you opt in, and abuse-monitoring logs are kept for up to 30 days. Anthropic says it deletes API inputs and outputs within 30 days by default. Both offer stricter retention by agreement. If your data classification says certain data must never leave your environment, an API is out for that data, and an open-weight model you run yourself is the usual answer.
Licences. Open-weight licences fall into two groups: standard open-source licences, and terms written by the publisher with their own conditions and use policies. Examples, as named on each publisher's page:
Model family | Licence as published | Group |
|---|---|---|
OpenAI gpt-oss, Qwen3, Aleph Alpha Kolibri 1 | Apache 2.0 | Standard open-source |
DeepSeek-R1, DeepSeek-V4.1-Flash | MIT | Standard open-source |
Meta Llama 4 | Llama 4 Community License Agreement | Publisher's own terms |
Google Gemma | Gemma Terms of Use | Publisher's own terms |
Liquid AI LFM2.5 | LFM Open License v1.0 | Publisher's own terms |
NVIDIA Nemotron 3.5 | OpenMDW License Agreement 1.1 | Publisher's own terms |
Origin. Some organizations restrict models by where they come from. For example, on October 20, 2026 the US Department of Homeland Security published an order excluding DeepSeek products from federal civilian agency procurement and systems. Interest in "sovereign" models, built and run in a given country or region, is part of the same trend; Aleph Alpha's Kolibri 1, released October 3, 2026, is one recent example.
Ask your legal team to review the licence of any model you plan to build on, and whether any rules on model origin apply to you.
[06]
When to choose each
Choose an API model when | Choose an open-weight model when |
|---|---|
You are starting out or testing ideas | Data must stay inside your environment |
Volume is low or uneven | Volume is high and steady enough to keep hardware busy |
You need the strongest model available for hard tasks | A smaller model is good enough for a narrow, repeated task |
You have no team to run AI infrastructure | You have, or will hire, engineers to run it |
Speed to market matters most | You must keep one model version unchanged for years |
You need to fine-tune deeply or control every change |
[07]
A scorecard to decide
Score each question from 0 (no) to 2 (yes) for one use case, not for the whole company:
Question | Points toward open-weight |
|---|---|
Must this data stay inside our environment by our own classification? | 0–2 |
Will volume be high and steady for at least a year? | 0–2 |
Is a smaller model good enough for this task, on our own tests? | 0–2 |
Do we have engineers to run, patch and monitor models? | 0–2 |
Do we need to keep one model version unchanged for a long time? | 0–2 |
Do we need deep customization or fine-tuning? | 0–2 |
0 to 4: use an API model. 5 to 8: try a hosted open-weight model first, which keeps the option to self-host. 9 to 12: plan a self-hosted pilot, with the API as a fallback.
[08]
How to run the decision
Pick one use case. Decide per task, not for the company.
Write down the constraint. Data classification, volume, version needs, customization.
Test two or three models on your own examples. Include at least one API model and one open-weight model. Best LLM for AI agents and best local LLM for AI agents explain how to compare them.
Price the real workload. Tokens per month for APIs, hours and people for self-hosting.
Keep switching cheap. Ask your developers to keep the model choice in configuration, so a change of model is a setting, not a rewrite. Agent frameworks such as Promptise Foundry, which run both API models and local models through Ollama with the same agent code, are one way to do that.
Review every six months. Prices and models change fast.
[09]
Common mistakes
Mistake | Why it fails | Fix |
|---|---|---|
Self-hosting to save money at low volume | Idle hardware still costs the full price | Use an API or a hosted open model until volume is steady |
Treating "open" as one kind of licence | Licences carry different conditions | Read each licence before you build on it |
Comparing only the model bill | Staff and operations dominate self-hosting | Count people, monitoring and security work |
Picking from a public ranking | Rankings measure other people's tasks | Test on your own examples |
Hard-wiring one vendor into the code | Switching later becomes a project | Keep the model choice in configuration |
Deciding once for the whole company | Needs differ by use case | Decide per use case and review twice a year |
[10]
FAQ
What is the difference between open-weight and API models?
An API model runs on the vendor's servers and you pay per use. An open-weight model's files are published, so you or a provider you choose can run it anywhere, including on your own hardware. The trade-off is control and data location against setup work and staff.
Is self-hosting an open-weight model cheaper than using an API?
Only at high, steady volume. Hardware costs the same whether it is busy or idle, and it needs engineers. At moderate volume, a small API model or a hosted open-weight model is usually much cheaper than running your own GPU.
Is open-weight the same as open source?
No. Open-weight means the model files are available. Open source usually also means the code and, ideally, the training data are available under an open-source licence. Many open-weight models publish only the weights, and some use the publisher's own licence terms.
What is a sovereign AI model?
It is a model that an organization can run under its own control, often built or hosted in a specific country or region. Companies look at sovereign models when they want control over where data is processed and who supplies the model.
Can we use open-weight and API models together?
Yes, and many companies do. A common pattern is an API model for hard, low-volume tasks and an open-weight model for routine, high-volume or sensitive ones, with the model choice kept in configuration so either can change.
Is our data safer with a self-hosted model?
It stays in your environment, which removes one party from the picture. It is only as safe as your own setup, though: access control, patching, logging and monitoring become your job instead of the vendor's.
[11]
Next steps
Compare models for agent work: best LLM for AI agents and best local LLM for AI agents.
Decide what to build yourself: build vs buy AI agents.
Pick the right first use case: how to choose AI use cases.
Make the business case: how to measure the ROI of AI.
Check vendors that will hold your keys: AI vendor security questionnaire.