Fine-tuning with LoRA: Practical Guide

Installation

Install the required libraries. We'll use HuggingFace transformers, PEFT (for LoRA), TRL (for training), and bitsandbytes (for 4-bit quantization):

pip3 install transformers peft trl bitsandbytes datasets torch

Step 1: Load model in 4-bit + configure LoRA

Load your base model with QLoRA (4-bit quantization) and configure LoRA adapters:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import LoraConfig, get_peft_model

model_id = "mistralai/Mistral-7B-Instruct-v0.2"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_use_double_quant=True,
)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=bnb_config,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.pad_token = tokenizer.eos_token

lora_config = LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
)
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()
# trainable params: 4,194,304 || all params: 3,756,154,880 || trainable%: 0.1117

What this does: Loads Mistral 7B in 4-bit (uses ~6GB VRAM), applies LoRA to the query and value projection layers with rank 16. The output shows how many parameters are trainable (0.1117% = ~4M params instead of 3.7B).

Step 2: Prepare dataset

Load your JSONL training data using the datasets library:

from datasets import load_dataset

dataset = load_dataset("json", data_files={"train": "dataset.jsonl"})
print(dataset)

The dataset should have a "messages" column with chat format (system/user/assistant roles).

Step 3: Configure training & train

Set up SFTTrainer with training arguments and start training:

from datasets import load_dataset
from trl import SFTTrainer, SFTConfig

dataset = load_dataset("json", data_files={"train": "dataset.jsonl"})

training_args = SFTConfig(
    output_dir="./fine_tuned_model",
    max_steps=200,
    per_device_train_batch_size=2,
    gradient_accumulation_steps=4,
    learning_rate=2e-4,
    warmup_steps=20,
    logging_steps=10,
    save_steps=100,
    fp16=True,
    max_seq_length=1024,
    dataset_text_field="messages",
)

trainer = SFTTrainer(
    model=model,
    args=training_args,
    train_dataset=dataset["train"],
    tokenizer=tokenizer,
)

trainer.train()
trainer.save_model("./fine_tuned_model/adapter")
print("Training complete. Adapter saved.")

Key parameters:

• max_steps=200: Train for 200 gradient steps (~15 minutes on RTX 4090)
• per_device_train_batch_size=2: Batch size 2 per GPU (memory efficient)
• gradient_accumulation_steps=4: Effective batch size = 2 × 4 = 8
• learning_rate=2e-4: Standard for fine-tuning
• fp16=True: Use half precision (faster, less VRAM)
• max_seq_length=1024: Max tokens per example

Step 4: Save adapter weights

The SFTTrainer automatically saves checkpoints. The adapter weights (tiny files, ~20MB each) are stored separately from the base model. On training completion, load and use:

from peft import AutoPeftModelForCausalLM

# Load the checkpoint
model = AutoPeftModelForCausalLM.from_pretrained(
    "./fine_tuned_model",  # this directory contains the adapter weights
    device_map="auto",
)

# Merge if needed (for deployment)
model = model.merge_and_unload()
model.save_pretrained("./merged_model")

Understanding training behavior

Cloud GPU options

Training tips from practitioners

Experienced practitioners follow a consistent pattern: start with a small subset of data (50-100 examples), train for a short run (50 steps), and manually inspect outputs. This quick iteration loop catches data formatting issues, chat template mismatches, and obvious quality problems before you commit to a full training run. A 50-step run on an RTX 4090 takes under two minutes and can save hours of debugging later.

Learning rate is the most sensitive hyperparameter. The default 2e-4 works for most LoRA fine-tunes, but if you see training loss oscillating wildly, drop to 1e-4. If loss barely moves after 50 steps, increase to 5e-4. Gradient accumulation is your friend when GPU memory is tight — instead of increasing batch size (which requires more VRAM), accumulate gradients over multiple smaller batches. An effective batch size of 8-16 is typically optimal for instruction fine-tuning. For projects that require gathering training data from web sources, combining this training pipeline with Firecrawl's RAG integration creates an end-to-end system from data collection to model deployment.

LoRA Training FAQ

Related tutorials

Continue learning with our evaluation tutorial and deployment guide.