A logistics client fine-tuned an open model on eighteen months of dispatcher notes so it would write in their voice: short, clipped, no pleasantries. It worked. Six weeks later the base model's provider shipped a new version and quietly deprecated the old one. Nobody had kept the training set in a state anyone could rerun. The tone the team had paid for was gone, and rebuilding it meant redoing the whole job from a folder of half-labelled exports, because the person who built it the first time had since left.
Fine-tuning is the process of taking a model that already knows language and further training it on your own examples, so its weights shift towards your task, your vocabulary, your way of phrasing an answer. It is not the only way to adapt a model, and it is very often not the first one worth trying. Prompting reshapes behaviour with instructions and examples at the moment of asking. Retrieval augmentation reshapes behaviour by handing the model facts it did not have. Fine-tuning reshapes the model itself.
We build data and AI systems at Shipshape Data, and fine-tuning is one of the requests we get asked about most and recommend least, not because it does not work but because most of what people want from it is available more cheaply somewhere else. This guide covers what fine-tuning actually changes, how it differs from prompting and retrieval, what the data needs to look like, what it costs beyond the training run, and the maintenance work that quietly outlasts the project that commissioned it.
What changes when you fine-tune, and what stays the same
A pre-trained model arrives with a huge amount of general capability baked into its weights: grammar, world knowledge, reasoning patterns, a rough sense of how humans phrase things. Fine-tuning takes that model and continues training it on a smaller, focused set of examples specific to your task, adjusting the weights so the model's default behaviour moves towards what those examples show. Feed it a few thousand support tickets answered the way your best agent answers them, and the model starts producing answers with that shape, unprompted.
What stays the same is everything the base model already knew that your examples did not touch. Fine-tuning does not wipe the model and start over. It nudges. That is exactly why it is powerful for narrow, repeatable behaviour and unreliable as a way of teaching a model new facts, because a few thousand examples of a phrasing pattern is a strong, consistent signal, while a few thousand mentions of a fact scattered through varied text is a weak one that the model may or may not retain.
The distinction that trips people up
The confusion we see most is between changing how a model responds and changing what it knows. Fine-tuning is genuinely good at the first. A model fine-tuned on your compliance team's approved phrasing will reliably use that phrasing. It is a poor tool for the second. A model fine-tuned on your product catalogue does not reliably know the catalogue; it has absorbed a style of talking about products that resembles yours, and it will still invent a SKU number with total confidence when asked about one it never saw enough times to memorise. If the goal is facts, not style, fine-tuning is solving a different problem than the one on the table.
Fine-tuning versus prompting versus retrieval
These three approaches get treated as competing options in most conversations we have, and they are not quite that. They sit at different points in a spectrum of how much work you are willing to do versus how much control you need, and the honest answer for most tasks is to work through them in order rather than reach for the heaviest tool first.
When prompting is enough
Prompting is instructions given at the time of the request: a system message, a few worked examples, a clear description of the output format. It costs nothing to change, nothing to deploy, and nothing to maintain beyond the prompt text itself. For a surprising number of tasks it gets you most of the way there. If a model is producing roughly the right answer but in the wrong tone, or missing a formatting rule, or needs a couple of examples to understand an edge case, that is a prompting problem, not a training problem, and treating it as a training problem is a lot of infrastructure to solve something a paragraph of instructions would have fixed. Our guide to prompt engineering covers the techniques worth exhausting before you reach for anything heavier.
When retrieval augmentation is enough
Retrieval augmentation, RAG, solves a different problem: the model does not have the facts, so you fetch them and put them in the context window at query time. If your issue is that the model does not know your return policy, your current pricing, or what happened in a document uploaded this morning, fine-tuning will not fix that reliably, because facts baked into weights during training are frozen at the training date and diluted across everything else the model learned. Retrieval keeps the facts external, current, and traceable back to a source, which also means you can update the policy tomorrow without retraining anything. Our piece on retrieval-augmented generation goes into how that pipeline actually gets built.
When neither will do
Fine-tuning earns its place when the thing you need is not information and not a one-off instruction, but a consistent behaviour that would otherwise need to be re-explained in every single prompt: a house style applied across thousands of documents a day, a rigid output format an application parses downstream, domain jargon dense enough that spelling it out each time burns through the context window, or a latency and cost budget so tight that a long system prompt with several examples is itself the problem. If none of that describes your situation, you are very likely looking at a prompting or retrieval task wearing a fine-tuning costume.
The data question nobody likes answering honestly
Every fine-tuning conversation starts the same way. Someone is confident there is plenty of data. There usually is data. Whether it is fine-tuning data is a separate question, and the honest answer is usually no, not yet.
What good training data looks like
You need examples in the exact shape you want the model to produce: input paired with the output you would have wanted, at the volume and variety that covers the cases you actually see. A few hundred examples can shift tone for a narrow task. Genuinely useful behavioural change across a varied set of inputs tends to need low thousands, sometimes tens of thousands, and the number moves a lot depending on how different the target behaviour is from what the base model already does by default.
Where it usually falls apart
The data that exists is rarely the data you need. Support transcripts include the bad answers your best agents overrode. Historical documents were written to different standards at different points, so the model gets a blended, inconsistent signal about what "good" looks like. Labels, where they exist, were applied by different people with different judgement calls, which means the model is learning your organisation's disagreements as if they were a single, coherent rule. Cleaning this properly is not a side task, it is most of the job, and it looks a great deal like ordinary data quality work applied to a training set rather than a warehouse. Expect the fine-tuning project to surface every gap in your data you had been quietly living with.
Where real examples are thin, synthetic data can fill some of the gap, generating plausible variations of the cases you have too few of. It is a genuine option and we use it. It is not a substitute for having looked hard at the real data first, because a model trained mostly on generated examples of what you assume the edge cases look like will confidently reproduce your assumptions rather than reality.
The cost that shows up on the invoice, and the one that doesn't
The training run itself is usually the smallest number in the whole exercise, and it is also the only one anybody puts in the initial estimate.
What actually gets budgeted
Compute for the run, whatever platform fee sits on top of it if you are using a managed fine-tuning service, and the engineering time to prepare data and kick it off. For a lot of tasks this is genuinely modest, which is part of why fine-tuning gets greenlit quickly and why the surprise arrives later.
What doesn't
Hosting a fine-tuned model, if it is not served through the same managed endpoint you trained it on, is a running cost that does not go away. Evaluation is its own project: building a test set that actually measures whether the fine-tune improved things, rather than eyeballing ten examples and calling it good, takes real time and real judgement about what "better" means for your task. And every one of these has to be repeated whenever the underlying base model changes, which brings us to the part that catches almost everyone out.
The maintenance bill you signed up for without noticing
This is the section most fine-tuning conversations skip, and it is the one that decides whether the project was worth doing a year later.
Base models move on without you
The model you fine-tuned is a snapshot. Providers release new base models regularly, and the old ones eventually get deprecated on a timeline you do not control. When that happens, your fine-tune either has to be redone against the new base or it keeps running on a version that is quietly falling behind on everything the newer model improved. Redoing it is not a formality. Different base models respond differently to the same training data, so a fine-tune that worked well on one version can behave oddly on the next, and you find that out by testing, not by assuming.
Your inputs keep moving too
Your own inputs change too. The products change, the policy wording changes, the way customers phrase complaints changes as new channels open. A fine-tune trained on last year's tickets slowly drifts out of step with this year's, in the same quiet way a forecasting model drifts once the world it was trained on moves on. Catching that needs the same discipline as monitoring any model in production: tracking real outputs against expectations over time, not trusting that a model which was good at launch stays good.
A fine-tuned model is not a thing you ship once. It is a thing you now own, in the same way you own a piece of software with a dependency you cannot control the release schedule of.
Who actually owns it afterwards
The uncomfortable question is who is responsible for noticing when a fine-tune has gone stale, retraining it, evaluating the new version, and rolling it out without breaking whatever depends on the old one's exact behaviour. On most of the projects we see fail quietly, the answer was nobody, because the project plan ended at "deploy" and the retraining cadence was never written down anywhere a calendar could remind someone about.
Catastrophic forgetting and the other ways it goes wrong
Fine-tuning adds behaviour, but it can just as easily remove it, and the removal is rarely on purpose. Push a model hard enough towards a narrow set of examples and it can lose general capability it had before, a failure researchers call catastrophic forgetting: the model gets very good at the thing you trained it on and noticeably worse at everything adjacent that you did not think to test.
A narrower version of the same problem shows up as overfitting to the training set's quirks rather than the underlying pattern you actually wanted. If every example in your training data happened to end with a particular sign-off phrase, the model learns the phrase as a rule rather than a coincidence, and it starts appending it to answers where it makes no sense. Small, unrepresentative training sets are especially prone to this, because there is nothing in the data to show the model which patterns were actually meaningful and which were just how the last export happened to look. Getting the balance right leans on the same instincts as tuning any model's settings: a training run left going too long overfits, one cut short underfits, and the sweet spot is found by testing, not guessed at.
When fine-tuning is actually the right call
Despite all of the above, there are situations where fine-tuning is the tool that fits and nothing else does the job as cleanly. A classifier that needs to sort inbound documents into forty categories your business invented, none of which the base model has ever heard of, benefits from direct training far more than from a prompt trying to describe forty categories from scratch on every call. A model that needs to produce a rigid structured format at high volume, where a downstream system parses the output and any deviation breaks something, is a good fit, because fine-tuning bakes the format in rather than hoping the instructions hold under pressure. And a task running at genuine scale, where trimming a long, example-heavy prompt down to a fine-tune's shorter one saves real money on every single call, is a legitimate reason on its own.
What these have in common is that the behaviour is narrow, it is stable, and somebody has already tried to get there with a prompt and hit a wall that more instructions did not fix. That is the point at which the extra weight of fine-tuning, and everything it commits you to afterwards, starts to be worth carrying.
Where to start
Do not start with the training run. Start by writing down, specifically, what the model is doing wrong today and whether that is a knowledge gap, an instruction gap, or a behavioural pattern. Most of what looks like the third turns out to be the first or second once someone actually tries a better prompt or a retrieval layer before reaching for anything heavier.
If you get to the end of that and the gap is genuinely behavioural, narrow, and stable, gather the real examples first and be honest about how messy they are before committing to a build. Decide who owns retraining before you deploy anything, not after the first support ticket asks why the model's tone has changed since a provider's update landed. Fine-tuning is a good tool for the right narrow job, and a maintenance commitment for as long as you keep using it. If you are trying to work out which side of that line your task sits on, or you have a fine-tuned model already drifting and nobody quite owns it, talk to us.