Fine-Tuning vs. Prompting: When to Train
You'll learn to
- -Compare fine-tuning, RAG, and prompting as levers
- -Recognize when fine-tuning is actually the right call
- -Understand LoRA as an efficient alternative to full fine-tuning
A law firm needs outputs in an exact legal citation format, with jurisdiction-specific language and court-approved templates. Careful prompting alone gets them to roughly 60% accuracy on this format. They need 95%. This is the exact scenario where the question this chapter answers actually matters: when does prompting stop being enough, and fine-tuning become the right lever to reach for instead?
Three Levers, Not One
By this point in the course you have three distinct tools for shaping model behavior, and it is worth being precise about what each one is actually good at. Prompting shapes behavior through instructions given at request time, cheap and instant to iterate on, but limited by how much can be reliably communicated in a prompt. RAG (Tier 8) grounds answers in specific, retrievable facts the model was never trained on. Fine-tuning changes the model's weights themselves, teaching it a consistent format, tone, or specialized skill directly, at the cost of an actual training run.
Fine-tuning is not a fix for the model not knowing something. That is what RAG is for. Fine-tuning is a fix for the model not consistently behaving a certain way. Reaching for fine-tuning to inject new facts is a common, expensive mistake: the facts will drift out of date instantly, and every update requires a full re-training run rather than just re-indexing documents.
LoRA: Fine-Tuning Without Retraining Everything
Full fine-tuning updates every single weight in a model, which is expensive and slow. LoRA, Low-Rank Adaptation, instead freezes the original model entirely and trains a small set of additional parameters, commonly around 1% of the original model's size, that get combined with the frozen weights at inference time. This is dramatically cheaper: a LoRA fine-tune of an 8-billion-parameter model can run in a few hours on a single GPU rather than requiring a cluster for days.
Data Quality Is the Real Bottleneck
The single most common reason a fine-tune underperforms has nothing to do with the training technique. It is the training data. Five thousand genuinely excellent, carefully curated instruction and response pairs consistently outperform fifty thousand noisy, inconsistent ones, since the model is learning the pattern in the data as faithfully as the data actually represents it. Invest the time in curating a small, gold-standard dataset before reaching for more volume.
The Workflow
- -Prepare data: clean examples, formatted as consistent instruction and response pairs, ideally reviewed by a human for quality, not just quantity.
- -Train with LoRA on an open model when data sovereignty or cost is a real constraint, which the level 34 case brief specifically calls out for a law firm handling sensitive documents.
- -Evaluate on a genuinely held-out set the model never saw during training, the exact train, validation, test discipline from Phase 1.
- -Deploy only once the evaluation clears a real accuracy bar, through the same production serving framework covered earlier in this tier, not an ad hoc script.
Interview Signal is part of Pro
See a real weak answer next to a real strong one for this exact topic.
Quiz is part of Pro
Test what you just read with a short quiz, and bank the XP.