In brief
- Fine-tuning changes model parameters; a prompt only changes the context of one request.
- It is more suitable for repeatable behaviour than for frequently changing facts.
- The quality and representativeness of training examples matter more than volume alone.
- Separate evals are needed before and after training, together with a plan for versions and operation.
What happens during fine-tuning?
A base model is further optimised using examples of inputs and desired outputs. This increases the likelihood that it handles similar tasks in the demonstrated style or format. Training data is not stored like rows in a database and cannot be queried as an authoritative source.
- Select a base model and supported training method
- Clean examples and convert them to the expected format
- A training run produces a new model version
- Independent cases test benefit and unintended effects
When can fine-tuning be useful?
Fine-tuning may be worthwhile when a measurable task remains systematically inconsistent despite a good prompt and suitable context. It can be especially useful for stable classifications, specialised output formats or a large volume of similar requests.
- Apply domain terminology or recurring categories more consistently
- Produce a complex output format more reliably
- Shorten long example instructions and reduce runtime cost
- Optimise a smaller model for a narrowly bounded task
Which approach solves which problem?
Current product data, policies and customer details should come from controlled sources. The following decision aid separates four approaches that are often confused. They can be combined, but they change different parts of a solution.
| Approach | Suitable when | Not the primary solution for | How benefit is demonstrated |
|---|---|---|---|
| Prompting | The task, tone or output format must change quickly | New facts, technical permissions or exact calculations | Better results on the same test set without model training |
| RAG | Changing knowledge with citations and user permissions is required | Learning persistent behaviour or executing live transactions | The correct authorised passage is retrieved and cited accurately |
| API or tool | Exact live values, calculations or controlled actions are needed | Open-ended interpretation of unstructured content | Parameters, result and failure behaviour are technically verifiable |
| Fine-tuning | Stable behaviour remains inconsistent despite a strong prompt | Frequently changing knowledge, citations or permissions | A new model version measurably and consistently reduces defined errors |
How do you make an evidence-based decision?
A training project needs a measurable baseline. The team compares the base model, improved prompting, RAG or tools and fine-tuning on the same test set. Only a demonstrated advantage justifies data preparation and additional model operations.
- Step 1
Demonstrate failure patterns
Show which recurring errors occur in the current solution.
- Step 2
Test simpler options
Try prompt, context, structured output and a suitable model first.
- Step 3
Separate data
Build non-overlapping training, validation and test sets.
- Step 4
Assess total cost
Include training, evaluation, versioning, privacy and operation.
Example: classifying technical support cases
A company has thousands of reviewed tickets with stable issue categories. Prompting produces too many inconsistent labels for rare domain terms. Fine-tuning on cleaned historical examples improves classification by a smaller model. Current product versions and resolution steps still come from RAG; the trained model does not replace those sources.
What to remember
Fine-tuning is targeted optimisation after a measurable need has been shown. Use it for stable behaviour, not as a shortcut for data integration, permissions or current knowledge.
Sources and further reading
These primary sources provide further detail on definitions, technical foundations or responsible use.
Content reviewed