Author(s): Saurabh Singh

Originally published on Towards AI.

Fine-Tuning Is Super, Super Hard. Even OpenAI Is Backing Away From It.

Everyone pictures a CSV and a Train button. The real job is data, evals, GPU memory, a model you now have to babysit, and a base model with a shelf life of about three months.

On Friday, 23 October 2026, OpenAI will switch off every fine-tuned copy of gpt-4.1-nano. Somewhere a team spent a quarter curating examples, tuning a model and wiring it into production. Its deprecation notice is one row in a table, and the suggested replacement is a different model, which they'll have to tune all over again.

That isn’t even the biggest line on the page. Since 7 May 2026, OpenAI’s self-serve fine-tuning has been closed to organisations that have never used it. On 6 January 2027, in OpenAI’s own words, “Active existing customers will no longer be able to create new fine-tuning jobs on this date.” According to an email to developers reported by Tessl, the company’s reasoning was that “Newer base models like GPT-5.5 are much better at following instructions and formats than prior models.”

So the company that made fine-tuning a button is taking the button away.

That surprises people, because fine-tuning sounds like the obvious move. You have a general model and a specific problem, so you show the model your problem a few thousand times and it learns it. Upload a CSV, press Train, go to lunch.

That isn’t how it goes. Here’s what actually happens, step by step, and the much smaller set of cases where it’s still worth doing.

1. The CSV is the hard part

The first surprise is that the training run is the easy step. It’s also the shortest. What fills the weeks is everything around it: collecting examples, throwing most of them out, writing the ones you can’t find, formatting them exactly as the model will see them in production, and holding some back so you can test.

The best-known demonstration of how much the data matters is Meta’s 2023 LIMA paper. The researchers took a 65-billion-parameter LLaMa and fine-tuned it on just 1,000 examples, but they picked each one carefully: mostly answers drawn from sites like Stack Exchange and wikiHow, plus a couple of hundred written by hand. Their conclusion, which they called the superficial alignment hypothesis, was that a model learns nearly everything it knows during pretraining, and fine-tuning mostly teaches it which style and format to answer in.

That sounds like good news. You only need a thousand examples! But read it the other way round. If a thousand curated examples can reshape how a model talks, a thousand sloppy ones can reshape it just as much. Every inconsistency in your data becomes a habit. If half your support transcripts end with “Let me know if you need anything else!”, your model will say it too, every time, forever.

And if the examples don’t exist, you have to make them. Most teams end up generating training data with a bigger model, then filtering it, which brings its own problems. (There’s a whole handbook on synthetic data: how to generate it, and how not to train your model on its own mistakes.) The full walk-through of the process, from picking a base model to shipping, is in the fine-tuning handbook.

2. You can’t tell if it got better

Suppose the run finishes. The loss went down. Is the model better?

You don’t know yet, and finding out is harder than it sounds. A falling training loss only proves the model got better at reproducing your examples, and overfitting is exactly that. To know whether it got better at the job, you need an evaluation set the model never trained on, scored in a way you trust, compared against the model you started with.

The QLoRA paper shows how slippery this gets. Its authors reported that their Guanaco model reached 99.3% of ChatGPT’s performance on the Vicuna benchmark after 24 hours of fine-tuning on one GPU. That headline went everywhere. The same paper also says that current chatbot benchmarks are not trustworthy for judging how good chatbots really are. The researchers who produced the impressive number were warning you not to rely on numbers like it.

Then there’s the part your evaluation set probably doesn’t cover: everything else the model used to be good at.

Neural networks forget. Train one on task B after task A and it can lose task A, sometimes completely. McCloskey and Cohen named this catastrophic interference back in 1989, and it never went away. A 2024 study, aptly titled LoRA Learns Less and Forgets Less, found that the lighter fine-tuning method most teams now use does preserve more of the base model’s other skills than full fine-tuning, and it pays for that by learning less of the new task. Either way you’re making a trade, and you only see the cost if you test for it.

Safety is the scariest thing to lose. In a 2023 paper, Qi and colleagues fine-tuned GPT-3.5 Turbo on just 10 adversarial examples, for under $0.20 through OpenAI’s API, and the model’s safety guardrails were gone. More worryingly, they found that fine-tuning on ordinary, harmless datasets also weakened the guardrails, just less. Nobody on your team needs to mean any harm for this to happen.

So a serious fine-tune ships with three test suites: one for the new task, one for the old skills, and one for the refusals. (How to build evals you can actually trust.)

3. The memory math

If you train an open model yourself, the first wall is GPU memory, and the arithmetic is unforgiving.

Full fine-tuning with the standard recipe (mixed precision and the Adam optimizer) keeps about 16 bytes per parameter in memory: the weights, their gradients, and a full-precision copy of the weights plus two running averages for the optimizer. That figure comes from Microsoft’s ZeRO paper. Multiply it out:

• 7B model, full fine-tune: about 112 GB, more than an 80 GB H100 holds.
• 70B model, full fine-tune: about 1,120 GB, roughly fourteen H100s’ worth of memory before you’ve stored a single activation.

This is why almost nobody fine-tunes all the weights any more. LoRA freezes the original weights and trains small low-rank matrices beside them; on GPT-3 175B its authors cut the trainable parameters by 10,000× and the GPU memory by 3×. With the base held in 16-bit, a 7B model needs roughly 14 GB for weights and a 70B roughly 140 GB. QLoRA goes further by storing the frozen base in 4 bits: roughly 4 GB for a 7B and around 40 GB for a 70B. Its authors fine-tuned a 65B model on a single 48 GB GPU.

Those are my estimates for the model state alone. Activations come on top and grow with batch size and sequence length, so plug your real numbers into a GPU VRAM calculator before you rent anything. (And if you’re unsure which method fits your case, here’s LoRA versus full fine-tuning side by side.)

Once it fits, you still have to choose the settings, and they don’t carry over between methods. Thinking Machines, the lab founded by former OpenAI CTO Mira Murati, published experiments in 2025 finding that the best learning rate for LoRA is about 10× the best rate for full fine-tuning. If you copy a full fine-tuning recipe into a LoRA run, you’ll get a model that seems to have learned almost nothing, and you might blame the data. Rank, learning rate, epochs, batch size and warm-up all interact, and every combination you try is another GPU bill.

4. The training run is the cheap part

Here’s the number that misleads people. On OpenAI’s current price list, training gpt-4.1-mini costs $5 per million tokens. A thousand examples of about a thousand tokens each, run for three epochs, is three million tokens: about $15. That sounds like a bargain.

Now look at the rest of the bill.

Serving costs more. The same price list charges $0.80 in / $3.20 out per million tokens to run your fine-tuned gpt-4.1-mini, against $0.40 / $1.60 for the base model. That’s double, on every request, for as long as the model is live. Self-hosting doesn’t escape this either: you’re now paying for a GPU that serves a model nobody else shares.

People cost the most. Most of the time goes into collecting and labelling data, building evals, and running a sweep of settings. One training run is $15. A month of an engineer is not.

You now own a model. It needs a version number, a record of exactly which data and settings produced it, a rollback path, monitoring for drift, and a plan for when the base model it was built on is retired. That’s a pipeline, not a file. (Here’s the whole fine-tuning pipeline as a system design, drawn step by step: data store, training jobs, eval gates, model registry, serving.)

If you want to put real numbers on your own plan, the fine-tuning cost estimator does the arithmetic for you.

5. Your base model has a shelf life

This is the cost nobody puts in the plan.

Meta released Llama 3 on 18 April 2024, Llama 3.1 on 23 July, 3.2 on 25 September and 3.3 on 6 December. That’s four versions in under eight months, so a new base model roughly every two to three months. Every adapter you trained is tied to the exact weights it was trained on. A LoRA built for Llama 3 doesn’t carry over to 3.1, so to benefit from the newer model you have to train again.

Hosted models expire on a fixed date. On 4 January 2024, OpenAI shut down fine-tuning on its original GPT-3 models, and its deprecations page put it plainly: “models fine-tuned using the /v1/fine-tunes API will no longer be accessible.” Its October 2026 notice for the GPT-4 series says fine-tuned versions “follow the same shutdown schedule as their underlying base models.” Your model lives only as long as its base does.

And sometimes the next general model simply beats you. In March 2023, Bloomberg published BloombergGPT, a 50-billion-parameter model trained on a 363-billion-token financial dataset. Strictly speaking that was pretraining, not fine-tuning, but the lesson is the same. Within weeks, a study from Queen’s University and J.P. Morgan AI Research found that ChatGPT and GPT-4 outperformed it, along with fine-tuned finance models, on many of the finance tasks they tested. The general model caught up faster than the specialist could pay for itself.

That’s the real reason OpenAI is winding the button down. Each new base model is better at following instructions and formats from the prompt alone, so a fine-tune only stays worth it while it’s ahead of the next release, and that window keeps getting shorter.

6. When it’s actually worth it

None of this means fine-tuning is dead. It means it’s a specialist’s tool that got sold as the default. Ask two questions first.

Is the problem knowledge? Facts about your product, your documents, last week’s prices: that’s retrieval, not training. A fine-tune bakes in facts as of the day you trained, can’t cite where they came from, and can’t forget something when it’s wrong. LIMA’s own finding points the same way: fine-tuning mostly teaches format, not new knowledge. Put the facts in the context window with RAG and update them whenever you like. (RAG vs fine-tuning, in full.)

Does a good prompt already work? Try a careful system prompt with five worked examples before you collect five thousand. If that gets you there, you’re done, and you can switch base models whenever you like.

When the answers are no and no, fine-tuning earns its keep in three places:

1. Fixed format or voice, at volume. You need exactly the same JSON shape, house style or classification labels millions of times, and a long prompt full of examples is costing you tokens and latency on every call. Training the behaviour into the weights removes the prompt.
2. Distillation: making a small model do a big model’s job. This is the strongest case. You use a large model to produce good answers for your narrow task, then train a small, cheap, fast model to copy them. It’s an old idea (Hinton’s 2015 knowledge distillation paper) that works very well in practice. In January 2025, DeepSeek reported that a 1.5B model distilled from its R1 reasoning model scored 28.9% on the AIME 2024 maths benchmark, against 9.3% for a GPT-4o snapshot from May 2024. That’s DeepSeek’s own number, on one narrow benchmark, but that’s exactly the kind of narrow task distillation is for. (Quantization versus distillation covers the two main ways to shrink a model.)
3. A language the base model barely speaks. Specialised codes, rare programming languages, clinical shorthand, a low-resource human language: vocabulary that’s so far from the pretraining data that no prompt can bridge the gap.

Notice that all three are about behaviour, not knowledge. And in all three you’re still signing up for everything above: the data, the evals, the forgetting, the pipeline and the retraining.

So before you upload that CSV, ask whether you’re trying to teach the model something new, or just tell it something. If it’s the second, put it in the prompt.

Fine-tuning changes how a model behaves. It’s the wrong tool for telling it what it should know.

I write about how AI actually works — the machinery under the headlines. Follow so you don’t miss the next one.

Everything linked here lives on Vibe Engines, where AI and system design are taught with interactive builds instead of walls of text.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI