Popular open-weights models
Llama 3.1 (Meta)
Sizes: 8B, 70B, 405B. Strong reasoning, good instruction-following. Best for general use and code. Most fine-tuning guides use Llama.
Mistral 7B (Mistral AI)
Size: 7B. Efficient, good performance/size ratio. Fast inference. Good for cost-sensitive projects.
Phi-3 (Microsoft)
Sizes: 3.8B, 14B. Small but capable. Excellent for mobile and on-device. Training data is high-quality, not internet-scale.
Gemma 2 (Google)
Sizes: 2B, 9B, 27B. Instruction-tuned, safe. Good for beginners. Runs easily on consumer hardware.
Qwen 2.5 (Alibaba)
Sizes: 0.5B–72B. Multilingual, strong on math and coding. Good for non-English tasks.
Model size guide
Model size (parameter count) determines memory, speed, and capability. Match it to your hardware.
1B – 3.8B (Mobile)
Runs on phones, edge devices. Limited capability. For simple tasks only. Min 2GB VRAM.
7B (Consumer GPU)
NVIDIA RTX 4070 / RTX 4090. Runs on 8GB+ VRAM. Good quality/speed tradeoff. Most accessible for hobbyists.
70B (Server GPU)
NVIDIA H100 / A100. Excellent quality. Expensive. Used by companies at scale.
405B (Multi-GPU)
Requires 8+ H100s. Frontier model. For research and large companies only.
API-based fine-tuning
Don't want to manage GPU infra? Use a provider's fine-tuning API.
OpenAI Fine-tuning API
Fine-tune GPT-3.5, GPT-4. Simple one-line API. Prices range from $25–$300/run. Best for enterprise.
Together AI
Fine-tune open models (Llama, Mistral). Simple API, competitive pricing ($0.15–$2 per 1M tokens trained).
Anyscale
Distributed fine-tuning for large models. Good for 70B+ models. Pay for compute per hour.
HuggingFace Inference API
Use your own infrastructure or their cloud. Flexible pricing. Good for open models.
Quick comparison table
| Model | Size | Min GPU VRAM | Best For |
|---|---|---|---|
| Phi-3 Mini | 3.8B | 2GB | Mobile, edge |
| Mistral 7B | 7B | 8GB | Cost-sensitive projects |
| Llama 3.1 8B | 8B | 8GB | General use, code |
| Llama 3.1 70B | 70B | 80GB | High-quality specialized tasks |
| Qwen 2.5 Turbo | 72B | 88GB | Multilingual, math, coding |
Recommendation matrix
I want: Code generation
→ Llama 3.1 8B (good code quality, affordable)
I want: Chat / customer support
→ Mistral 7B or Llama 3.1 8B (instruction-tuned, responsive)
I want: Run on small device / mobile
→ Phi-3 Mini (3.8B, optimized for edge)
I want: Multilingual / math / scientific
→ Qwen 2.5 (strong on these domains)
I want: Highest quality, cost is not an issue
→ Llama 3.1 70B or OpenAI GPT-4o mini (via API)
Quick decision rule
If you have a GPU with 8GB+ VRAM, start with Llama 3.1 8B or Mistral 7B. If you don't have a GPU or want simplicity, use an API like Together AI or OpenAI. If you need to run on a phone or tiny device, use Phi-3 Mini.
Choosing in practice
Model selection often comes down to a single constraint: what hardware do you have? If you own an RTX 4090 with 24GB VRAM, a 7B model with QLoRA is your sweet spot. If you are using Google Colab's free tier (T4 with 16GB), the same 7B models work with aggressive quantization. If you have access to an A100 (80GB), you can comfortably fine-tune 13B models or run 70B with QLoRA.
The model ecosystem evolves quickly. At the time of writing, Llama 3.1 8B and Mistral 7B remain the most battle-tested choices for fine-tuning because they have the widest community support, the most tutorials, and the best-understood failure modes. Newer models like Qwen 2.5 and Gemma 2 are strong contenders, especially for multilingual tasks or on-device deployment, but their fine-tuning tooling is less mature. When building training datasets, you may need to scrape documentation or technical content from the web — our Firecrawl basic scraping guide covers how to extract clean text programmatically.
Base Model Selection FAQ
Which model should I fine-tune first?
Start with Llama 3.1 8B or Mistral 7B. Both have strong community support, extensive documentation, and work well with LoRA on a single consumer GPU.
Does model size matter for fine-tuning?
Yes. Larger models are more capable but need more memory and compute. A 7B model fits on a 16GB GPU with LoRA, while 70B models need multiple A100s or quantization.
Can I fine-tune closed models like GPT-4?
OpenAI offers fine-tuning for GPT-4o-mini and GPT-3.5 Turbo through their API. You upload training data and they handle the infrastructure, but you get less control over hyperparameters.
What is the difference between Llama and Mistral?
Llama 3.1 offers more size options (8B to 405B) and excels at reasoning. Mistral 7B is more efficient and faster at inference, making it ideal for cost-sensitive deployments.
Should I use a chat model or a base model?
Use the chat or instruct variant for conversational tasks. Use the base model when you want to teach entirely new behavior patterns without inherited chat formatting.
Related tutorials
Continue learning with our PEFT and LoRA guide and data preparation tutorial.