A short tour of the 2026 open-weights landscape
The open-weights ecosystem has moved fast. Two years ago, running anything stronger than GPT-3.5 locally meant a server with two A100s. Today the same quality runs on a 16 GB laptop because five labs — Meta, Mistral, Google, Microsoft, and Alibaba — keep releasing competitive models under permissive licences. Most are available in a range of parameter counts (1B → 70B+), pre-quantised to GGUF Q4 for CPU/GPU inference, and ready to pull through Ollama in a single command.
This page is a reference, not a leaderboard. Benchmark numbers shift every few weeks; the right model for you depends on three concrete things — the hardware you have (CPU only? 8 / 16 / 32 / 64 GB RAM? GPU VRAM?), the kind of task (chat, summarisation, retrieval, code, reasoning), and the licence you can ship (Llama Community vs Apache 2.0 vs research-only). Each family below is summarised with that lens. If you only have time to skim one section, jump straight to the Quick comparison by use case table further down.
Llama 3 — Meta
The most widely used open-source family and the de-facto baseline that every other model is compared against. Meta released Llama 3 in April 2024, followed by Llama 3.1 (July 2024, 128k context) and the small-footprint Llama 3.2 1B/3B for edge devices later that year. The community ecosystem around Llama — fine-tunes, GGUF conversions, tool adapters, evaluation harnesses — dwarfs every other family, which is the main reason it is the safe default for production.
Licensed under the Llama Community License: free to use commercially up to 700 million monthly active users (effectively unlimited for most apps), but it is not OSI-approved open source. Outputs cannot be used to train competing foundation models. If your legal team needs strict Apache 2.0, Mistral or Qwen are the cleaner choices.
| Model | Size | Best for |
|---|---|---|
| llama3.2:1b | ~1 GB | Edge devices, simple extraction |
| llama3.2:3b | ~2 GB | Everyday tasks on 8 GB RAM |
| llama3.1:8b | ~5 GB | Strong all-rounder, 128k context |
| llama3.1:70b | ~40 GB | GPT-4 class, needs 64 GB RAM |
Llama 3.1 8B is the best starting point for most developers — it balances quality, speed, and RAM requirements.
Mistral — Mistral AI
The Paris-based lab that put a small open model on the map. Mistral 7B (September 2023) was the first sub-10B model whose quality genuinely matched Llama 13B, thanks to grouped-query attention and sliding window attention. Their pure open-weight releases are Apache 2.0, which is the most permissive licence in this group — no usage cap, no MAU thresholds, no fine-tuning restrictions. That makes Mistral the right default if you are shipping into a customer-managed environment where the legal team scrutinises licences.
Mixtral 8x7B (December 2023) was the first usable open mixture-of-experts model: 47B total parameters but only ~13B active per token, so quality lands near GPT-3.5 while inference cost looks like a 13B dense model. Mistral Nemo (12B, 128k context) is a 2024 collaboration with NVIDIA and the best open model for long-context summarisation under 16 GB RAM.
| Model | Notes |
|---|---|
| mistral:7b | Fast, efficient. Best 7B for instruction following |
| mistral-nemo:12b | 12B, 128k context window — great for long documents |
| mixtral:8x7b | Mixture-of-experts, ~26 GB, near GPT-3.5 quality |
Gemma — Google
Google's open-weight family, derived from the same research that produced Gemini. Gemma 1 launched in February 2024; Gemma 2 (June 2024) and Gemma 3 (early 2025) brought meaningful jumps in reasoning, factual recall, and instruction following. Gemma 3 is the first in the family with native multimodal input — the 12B and 27B variants accept images alongside text, which is rare in the open-weight world without resorting to a separate vision encoder.
Licensed under the Gemma Terms of Use, which is more restrictive than Apache 2.0 but generally permissive for commercial use. Gemma punches noticeably above its weight on factual benchmarks (MMLU, ARC) and is often the right pick when you want grounded answers rather than verbose chat. The 4B variant is a strong fit for an 8 GB laptop where Llama 3.2 3B feels too small but Llama 3.1 8B is too slow.
| Model | Notes |
|---|---|
| gemma3:4b | Lightweight, surprisingly capable at 4B |
| gemma3:12b | Best balance of quality and speed |
| gemma3:27b | Top open-weight model at 27B, needs 20 GB RAM |
Phi — Microsoft
The "textbooks are all you need" experiment that turned into a product line. Microsoft's Phi team trains on synthetic, curriculum-style data instead of broad web scrapes, which produces models with unusually high reasoning density per parameter. Phi-3 (April 2024) was the first compelling sub-4B model; Phi-4 (December 2024, 14B) often beats Llama 3.1 70B on maths and coding benchmarks despite being a fifth of the size.
The trade-off is breadth: Phi models are tuned for instruction-following, code, and reasoning, and they can feel terse or under-trained on casual conversation, creative writing, and obscure factual recall. If your workload is a coding copilot, a maths tutor, or a logic-heavy agent, Phi-4 14B is the strongest local choice under 16 GB RAM. For open-ended chat, pair it with a more conversational model or fall back to Llama 3.1 8B. Phi-4 is MIT-licensed.
| Model | Notes |
|---|---|
| phi4:14b | Exceptional reasoning at 14B — beats many 70B models on maths/code |
| phi4-mini:3.8b | 3.8B with strong reasoning — great for 8 GB machines |
If coding or maths reasoning is your primary use case and you have 16 GB RAM, phi4:14b is often the best local choice.
Qwen — Alibaba
The strongest non-Western open-weight family and, on several benchmarks, the strongest open-weight family overall. Qwen 2.5 (September 2024) is the workhorse generation: Apache 2.0 licensed, available in 0.5B → 72B, with dedicated Coder and Math variants. Qwen 2.5 Coder 7B is often a better local Copilot than DeepSeek Coder at the same size, and Qwen 2.5 14B competes directly with Llama 3.1 70B on reasoning while needing a quarter of the RAM.
QwQ 32B is the family's reasoning-first variant, modelled after OpenAI's o1: it produces a long "thinking" trace before answering, which dramatically improves results on maths, planning, and multi-step problems at the cost of latency. Qwen's multilingual coverage — Chinese, Japanese, Korean, Arabic, plus most European languages — is the best in the open-weight world; if you ship outside English-only markets, start here.
| Model | Notes |
|---|---|
| qwen2.5:7b | Strong multilingual, competitive with Llama 3.1 8B |
| qwen2.5:14b | Excellent code generation, 128k context |
| qwen2.5-coder:7b | Code-specialised — competitive with DeepSeek Coder |
| qwq:32b | Reasoning model, think-before-answer like o1 |
Quick comparison by use case
| Use case | 8 GB RAM | 16 GB RAM | 32 GB+ RAM |
|---|---|---|---|
| General chat | llama3.2:3b | llama3.1:8b | gemma3:27b |
| Code generation | phi4-mini | phi4:14b | qwen2.5:14b |
| Reasoning / maths | phi4-mini | phi4:14b | qwq:32b |
| Long documents | mistral:7b | mistral-nemo:12b | llama3.1:70b |
| Multilingual | qwen2.5:7b | qwen2.5:14b | qwen2.5:72b |
How to choose your first model
If you are setting this up for the first time, do not download five models and benchmark them. Pick one based on the constraint that matters most, ship something with it, and only swap when you can name a specific failure mode you want to fix.
- Start from your RAM ceiling. A model larger than your free RAM at Q4 will either refuse to load or spill to disk and run at one token per second. Subtract 4–6 GB for the OS and your other apps before sizing — a "16 GB laptop" realistically has 10–12 GB free for the model.
- Then narrow by task. General chat → Llama 3.1 8B. Code → Phi-4 or Qwen 2.5 Coder. Long documents → Mistral Nemo. Multilingual → Qwen 2.5. Reasoning-heavy agents → Phi-4 or QwQ 32B.
- Check the licence early. If you ship to enterprise customers who require Apache 2.0, your shortlist is Mistral + Qwen. Otherwise Llama is fine for almost any commercial use up to 700M MAUs.
- Ignore the leaderboard delta under ~3 points. Public benchmarks are contaminated, noisy, and don't reflect your prompts. A model that scores 2 MMLU points lower but suits your hardware will ship faster and iterate faster.
Notes
Ollama tags default to Q4_K_M, not the full-precision weights
When you run ollama pull llama3.1:8b you are getting a ~4.7 GB Q4_K_M quantisation, not the 16 GB FP16 release. This is almost always what you want — quality loss is small and RAM savings are huge — but if you are benchmarking against published numbers, the published numbers are usually FP16. Use :fp16 or :q8_0 tags when comparing.
Context window is advertised, not free
Llama 3.1's 128k context is real, but every 1k tokens of context costs roughly 50–100 MB of additional RAM for the KV cache. A "16 GB laptop running Llama 3.1 8B" runs out of memory long before 128k tokens. Realistic local context limits are 8k–32k unless you have GPU VRAM to spare.
Model freshness matters more than parameter count
A 9-month-old 70B model is often worse than a 1-month-old 14B from a different family — training data quality and post-training (RLHF, DPO) have improved much faster than raw scale. Always check the release date on the model card before committing.
"Uncensored" forks are not magically better
Community fine-tunes labelled uncensored, dolphin, or abliterated remove refusals but typically regress instruction-following and factual accuracy. Use them only if you have a specific use case that the base model refuses; for general work, the official release is stronger.
Small models hallucinate confidently
Sub-7B models will fabricate citations, code, and API signatures with high confidence. They are still useful — for extraction, classification, templated rewrites, edge inference — but never use them for open-ended factual answers without a retrieval layer. See the limitations chapter for concrete failure modes.
Quick summary
- Llama 3.1 8B — best all-rounder default, 128k context, 5 GB
- Mistral 7B — fast and efficient, great for instruction following
- Gemma 3 12B/27B — strong reasoning, multimodal on larger sizes
- Phi-4 14B — best coding and maths reasoning at 16 GB RAM
- Qwen 2.5 — best multilingual and code generation across sizes
Ready to install one of these models? Head to the Ollama tutorial to get up and running in minutes. Then learn how to choose the right model for your task and understand quantization tradeoffs.
Open-Source LLM Landscape FAQ
Which open-source LLM is the best all-rounder for local use?
Llama 3.1 8B is the best starting point for most developers. It balances quality, speed, and RAM requirements at around 5 GB, with a 128k context window.
Can I run a 70B parameter model on a laptop?
A 70B model needs approximately 40 GB of RAM at Q4 quantization. Consumer laptops with 64 GB RAM can run it, but inference will be slow on CPU-only machines. Most users are better served by 7B-14B models.
What is the difference between Llama, Mistral, and Gemma?
Llama 3 (Meta) is the most widely used general-purpose family. Mistral (Mistral AI) is known for efficiency — its 7B rivals Llama 13B on many benchmarks. Gemma 3 (Google) excels at reasoning and adds multimodal support at 12B and 27B sizes.
Which local LLM is best for code generation?
Qwen 2.5 Coder and Phi-4 are the strongest code-focused models for local use. Phi-4 14B is the best choice if you have 16 GB RAM. For 8 GB machines, Phi-4 Mini works well.
How much RAM do I need to run LLMs locally?
8 GB RAM can run 1B-7B models comfortably at Q4 quantization. 16 GB handles up to 14B models. 32 GB+ opens up 27B-33B models. 64 GB is needed for 70B models.