Why evaluation matters
Training loss going down is a good sign, but it's not enough. A model with low training loss can:
- Overfit: Memorize training data but fail on new inputs
- Suffer catastrophic forgetting: Forget general knowledge to specialize
- Miss edge cases: Work well on typical inputs but fail on unusual ones
Evaluation reveals these problems. Always evaluate before deployment.
Automatic metrics
Perplexity
Probability assigned to held-out text. Lower is better. Standard during training but doesn't always correlate with task performance.
BLEU / ROUGE
Compare generated text to reference. BLEU: word overlap. ROUGE: recall of reference. Simple but limited (penalize paraphrasing).
Task accuracy
If you have a specific task (classification, extraction), measure accuracy directly. Most relevant for your use case.
Semantic similarity
Embed generated and reference text, compare cosine similarity. Better than word-level overlap.
LLM-as-judge
Use Claude or GPT-4 to score your model's outputs. Surprisingly effective and aligns with human judgment.
import anthropic
client = anthropic.Anthropic()
# Score your model's output
prompt_text = "Rate this response from 1-10. Does it answer the question correctly?"
response = client.messages.create(
model="claude-opus-4-7",
max_tokens=100,
messages=[{"role": "user", "content": prompt_text}],
)
print(response.content[0].text)Advantages: Flexible evaluation criteria, understands nuance, correlates well with human judgment.
Human evaluation
Gold standard but expensive. For critical systems, have a human review a sample.
Evaluation checklist:
- Better than base model? (Side-by-side comparison)
- Consistent behavior? (Same instruction should produce similar outputs)
- Follows instructions? (If you ask for JSON, does it return JSON?)
- Handles edge cases? (Test with tricky or unusual inputs)
- Safe and appropriate? (No inappropriate outputs?)
Side-by-side comparison
Best practical approach: run 50–100 test prompts on base model and fine-tuned model, compare outputs (blind if possible).
from transformers import pipeline
ft_pipe = pipeline("text-generation", model="./fine_tuned_model/adapter")
base_pipe = pipeline("text-generation", model="mistralai/Mistral-7B-Instruct-v0.2")
test_prompts = [
"How do I cancel my subscription?",
"What payment methods do you accept?",
"My order hasn't arrived after 10 days.",
]
for prompt in test_prompts:
base_out = base_pipe(prompt, max_new_tokens=100)[0]["generated_text"]
ft_out = ft_pipe(prompt, max_new_tokens=100)[0]["generated_text"]
print(f"\nPrompt: {prompt}")
print(f"Base: {base_out}")
print(f"Fine-tuned: {ft_out}")
print("-" * 60)This script generates side-by-side outputs. Copy to a spreadsheet, have humans score. Score higher output. Tally wins: if fine-tuned wins >60% of the time, it's better.
Red flags to watch for
Overfitting
Signal: Training loss keeps dropping but eval loss increases. Solution: Use early stopping, reduce max_steps, or increase learning_rate (confusingly, higher LR can reduce overfitting by preventing convergence to noisy local minima).
Catastrophic forgetting
Signal: Fine-tuned model fails on basic tasks the base model handles (math, general knowledge). Solution: Mix training data: 80% your task + 20% general instruction data.
Prompt mismatch
Signal: Model was trained with a system prompt but inference doesn't use it (or uses different one). Solution: Use the EXACT same chat template in training and inference.
Mode collapse
Signal: Model produces the same output regardless of input. Solution: Check training data diversity. Add more varied examples.
Evaluation checklist
- □ Run side-by-side comparison (50+ prompts)
- □ Check eval loss (should decrease, not increase)
- □ Test edge cases and unusual inputs
- □ Verify no catastrophic forgetting (test base knowledge)
- □ Have a human spot-check outputs
Building an evaluation pipeline
The most reliable evaluation approach combines all three methods: automated metrics for quick iteration, LLM-as-judge for nuanced quality scoring, and human review for final validation. In practice, you should automate the first two and reserve human evaluation for milestone checkpoints. A typical workflow runs automated metrics after every training run, triggers LLM-as-judge scoring when metrics improve, and schedules human review before any production deployment.
LLM-as-judge deserves special attention because it scales well and correlates strongly with human preferences. The key to making it work is a detailed scoring rubric. Instead of asking "rate this response 1-10," specify exact criteria: "Score 1 if the response is factually incorrect, 5 if correct but verbose, 10 if correct, concise, and well-formatted." Include 2-3 reference examples in the judge prompt so the scoring model understands your quality bar. For teams building evaluation datasets from web content, Firecrawl's structured extraction can pull gold-standard answers from authoritative sources for comparison.
Model Evaluation FAQ
How do I evaluate a fine-tuned LLM?
Use a held-out test set the model has never seen. Measure task-specific metrics like accuracy or F1, run LLM-as-judge evaluations, and conduct human reviews on a sample of outputs.
What is LLM-as-judge evaluation?
LLM-as-judge uses a stronger model like GPT-4 to score your fine-tuned model's outputs on criteria like relevance, accuracy, and helpfulness. It scales better than human evaluation for large test sets.
What does lower loss mean in fine-tuning?
Lower training loss means the model fits your data better, but it does not always mean better real-world performance. A model can overfit to training data while performing worse on new inputs.
How do I detect overfitting in fine-tuning?
Watch for a gap between training loss and validation loss. If training loss keeps dropping but validation loss increases, the model is memorizing examples rather than learning generalizable patterns.
When should I use human evaluation?
Use human evaluation for subjective tasks like creative writing, tone matching, or safety checking. Automated metrics work well for structured outputs like classification or extraction.
Related tutorials
Continue learning with our deployment guide and data preparation tutorial.