Usually start with prompting and retrieval (RAG) — they're cheaper, faster to iterate, and handle most needs; fine-tune only when you need consistent style, format, or task behavior that prompting can't reliably deliver, and use RAG (not fine-tuning) for facts.
"Should I fine-tune, or just prompt?" is one of the most useful questions to get right, because fine-tuning is often reached for when a cheaper, faster tool would do the job better. There are three ways to steer a model's behavior — prompting, retrieval, and fine-tuning — and the honest advice is to climb that ladder in order, only fine-tuning when the simpler tools genuinely fall short.
Start with prompting, because it's free to iterate and instant. A well-crafted prompt — clear instructions, a good system message, and a few examples of the input-output pattern you want (few-shot prompting) — solves a surprising fraction of problems without touching the model at all. You can change it in seconds, it costs nothing extra, and it works on any model. Before considering anything heavier, push prompting as far as it goes: if careful prompting gets you the behavior you need, you're done, and you've spent no money or training time.
Next, if your real need is "the model should answer using my documents / current data / knowledge base," the right tool is retrieval — Retrieval-Augmented Generation (RAG) — not fine-tuning. You store your documents as embeddings, look up the relevant pieces at query time, and feed them into the model's context. This is cheaper than fine-tuning, updates instantly when your documents change, and keeps the model general. Crucially, fine-tuning is the wrong tool for injecting facts: the facts go stale, re-tuning is costly, and models don't reliably absorb new knowledge from fine-tuning the way they absorb style and format. If your problem is about knowledge, reach for RAG.
Fine-tuning earns its place for a specific job: teaching consistent style, format, or task behavior that prompting can't reliably deliver. If you need the model to always produce a specific JSON structure, adopt a precise brand voice, follow a fixed workflow, classify into your exact categories, or handle the quirks of a narrow domain — and prompting keeps drifting or being inconsistent — fine-tuning bakes that behavior in more reliably. It can also make a smaller model punch above its weight on a narrow task, sometimes letting a fine-tuned small model replace a larger general one, which saves on running cost. That's the honest sweet spot: behavior and format, at scale, where consistency matters.
The cost and effort comparison is stark, which is why order matters. Prompting is free and instant. RAG requires building a retrieval pipeline but no model training. Fine-tuning requires curating a good dataset (the hard part — a few hundred to a few thousand clean, consistent examples beats a large messy set), running a training job (though parameter-efficient methods like LoRA and its 4-bit variant QLoRA make this feasible on a single consumer or rented GPU rather than a data center), and then serving the result. So fine-tuning is the most expensive and slowest to iterate of the three — worth it only when the payoff (reliable, consistent behavior) justifies the investment.
Set expectations honestly about what fine-tuning does and doesn't do. It reliably improves format adherence, tone, and task-specific behavior. It does not turn a weak base model into a strong general reasoner, and it does not reliably add facts the base didn't learn (that's RAG's job). Match the tool to the goal: prompt for quick adjustments, retrieve for what the model should know, fine-tune for how it should consistently behave. A common, effective production pattern combines them — fine-tune for behavior and format, use RAG for facts, and prompt on top.
The practical rule of thumb: don't fine-tune first. Exhaust prompting, add RAG if you need facts, and fine-tune only when you've hit a real, repeatable wall on style, format, or task consistency at a scale that justifies the cost. Many projects never need to fine-tune at all.
If you do decide to fine-tune and then run the result yourself, the cost math is the same as running any model. Spanvero shows the honest, $0-markup cost of running a base or fine-tuned model locally, on a rented GPU (where you'd also do the training), or via your own API key. Browse candidate base models to adapt at /models/, and use /calculator/ to compare the running costs. For the mechanics, see the explainers at /learn/fine-tuning/, /learn/lora/, and /learn/embeddings/.
Fine-tuning · LoRA · Embeddings · Base vs instruct model · How do I choose which AI model to run? · Local vs API vs renting a GPU · Inference · Safetensors
All explainers → · Browse models →
Open the free Spanvero advisor → · Honest, $0-markup. © 2026 Cynosure LLC.
A short email of real AI price moves, straight from the daily log — no hype. We're collecting the list now; the first issue goes out when it opens. Unsubscribe with one click.