Open Source ModelsDeepSeekQwenKimiGLMSelf-Hosting

The Open-Source LLM Guide: Weights, Licenses, and Deployment

A practical guide to open-weight LLM licenses, benchmark comparisons, local hardware planning, and the real costs of hosted APIs versus self-hosting.

Published: Last checked:

Guide to evaluating open-weight language model licenses, deployment, and cost
Analyzed Foundation Models (3)Full LLM Leaderboard→
Kimi K3
Moonshot AI
56.0SCORE
DeepSeek-V4-Pro-0813
DeepSeek
54.4SCORE
GLM-5.3
Zhipu AI
56.2SCORE

“Open-source LLM” is often used as a shortcut for several different things: downloadable weights, an open training recipe, public data and code, or a license that permits some kinds of reuse. These are not interchangeable. The Open Source Initiative’s Open Source AI Definition sets out its criteria for an open-source AI system; many commercial releases describe themselves more narrowly as open-weight. For deployment decisions, check the exact artifact and its license rather than relying on a model’s label.

Start with the model’s release terms

An open-weight release makes model parameters available for download. It does not automatically publish the training data, complete training code, or every detail needed to reproduce the model. A model card can describe intended use and known limitations, but it does not replace the license or a complete account of the training and serving materials.

Before using a checkpoint, read the license shipped with that exact revision and verify:

  • whether commercial use, redistribution, and modification are allowed;
  • whether there are use restrictions, attribution requirements, or acceptable-use terms;
  • whether the license covers the weights, code, tokenizer, and related assets;
  • whether downstream services must meet additional obligations.

Do not infer that a license is permissive because another model from the same provider uses MIT or Apache 2.0. Keep a copy of the terms and checkpoint identifier used for your review.

Compare models on the task you need

For coding, issue-resolution benchmarks and terminal-agent benchmarks measure different work. SWE-bench Verified evaluates fixes for selected GitHub issues within its benchmark harness. Terminal-Bench evaluates tasks performed through a command-line environment. DeepSWE is a separate evaluation; its scores should not be reported as SWE-bench Verified results.

Compare exact benchmark versions and record the model snapshot, scaffold, tools, attempt budget, and scoring rules. Results from different harnesses are not directly interchangeable, and a benchmark score does not guarantee performance on your repository. Our benchmark guide explains common evaluation categories, while the leaderboard shows the dataset currently maintained by RankLLMs.

Hosted API or self-hosting?

A downloadable checkpoint can be run locally or on infrastructure you control, subject to the license and the serving software. That offers more control over where inference occurs, but the whole application still matters: logs, telemetry, network-connected tools, monitoring, and backups can all move or retain data.

A hosted API avoids acquiring and operating accelerators, but adds provider, network, contract, and usage considerations. Compare expected costs using your real request pattern:

Calculate token charges by category using the provider’s published rate units, then add infrastructure and operations:

monthly token cost = (uncached_input_tokens / 1,000,000 * uncached_rate_per_1M)
+ (cached_input_tokens / 1,000,000 * cached_rate_per_1M)
+ (output_tokens / 1,000,000 * output_rate_per_1M)
monthly total = monthly token cost + hosting/service charges + operating costs

For a self-hosted deployment, token rates may not apply; estimate accelerator capacity and utilization instead. Do not add raw token counts to dollar charges.

For self-hosting, include accelerator rental or purchase, utilization, redundancy, storage, power, engineering time, monitoring, and serving overhead. There is no universal token-volume threshold at which self-hosting becomes cheaper. The result depends on model size, quantization, throughput, uptime, context length, and actual utilization.

Plan hardware from the checkpoint

Parameter count alone is not a hardware estimate. Quantized weight files, runtime buffers, KV cache, context length, batch size, and any multimodal encoders all consume memory. A large MoE model still needs access to all of its weights even if only a subset is activated per token. Conversely, a quantized smaller checkpoint may fit on a local device but run too slowly or with too little context for the intended application.

Check the release’s published checkpoint sizes and the inference engine’s compatibility guide. Then measure peak memory and throughput with the exact quantization and context you expect to use. For example, Meta says its Muse Glimmer quantized language model is under 20 GB and describes a 24 GB or 32 GB deployment envelope; that is a vendor configuration target, not a universal guarantee for every device and workload.

A practical evaluation checklist

  1. Confirm the exact model revision, license, and source of its weights.
  2. Choose a benchmark that resembles the real task, then check version and harness.
  3. Run a small set of representative prompts and measure quality, latency, and cost.
  4. Test failure handling, tool permissions, and privacy controls in the full application.
  5. Revisit the choice when the model, serving stack, or provider terms change.

Open weights can be useful when control, customization, or local inference matters. They do not automatically provide lower total cost, stronger privacy, or better results. Compare model records in our AI Model Leaderboard and see our Best LLMs for Coding guide.

Frequently Asked Questions

What is the difference between open source and open weights?

Open weights make model parameters available to download. They do not necessarily include training data, full training code, or a license that permits every kind of reuse. Check the exact release terms and artifacts.

Can open-weight models match proprietary models?

Some can be competitive on particular evaluations, but results depend on the model, task, benchmark version, harness, and budget. A score on one benchmark does not establish general equivalence.

When does self-hosting an open-weight model become cheaper than API usage?

There is no universal token-volume break-even point. Compare provider charges with accelerator cost, utilization, storage, redundancy, power, monitoring, and engineering time for your workload.

Are open-weight models permissible for commercial deployment?

It depends on the exact model revision and its license. Review the terms for the weights, code, tokenizer, and related assets, including any restrictions or attribution requirements, before commercial deployment.

Sources & disclosures

Source links for data and claims discussed in this article are listed below. Refer to each source for its current figures and methodology.

Share Article

Recommended Reading

Related by topic and recency.

All articles→
GuidesCoding Agents

Best LLMs for Coding in 2026: How to Compare Them

Choose a coding LLM by matching the benchmark to your workflow, checking the agent harness, and measuring task quality and cost on your own repository.

Sep 4, 2026 · 3 min read
ComparisonsQwen

Qwen 2.5 7B vs Llama 3.1 8B Benchmarks (2026)

Side-by-side Qwen 2.5 7B vs Llama 3.1 8B for local use: license checks, benchmark protocol cautions, coding runs, memory needs, and tokens/sec planning.

Sep 4, 2026 · 4 min read
ComparisonsLlama

Llama 3.1 8B vs Llama 3.2 3B: Local VRAM, Speed & Benchmarks

Llama 3.1 8B vs Llama 3.2 3B compared for local deployment: published Meta benchmarks, quantization VRAM requirements, edge speed, and coding accuracy.

Oct 17, 2026 · 7 min read