Fine-tuning

Continuing to train an existing model on your own data so it adapts to a specific task, domain, style, or format.

Fine-tuning takes a pretrained model and trains it further on a focused dataset so it specializes — for example, to follow a particular output format, speak in a brand voice, or handle a niche domain. It changes the model's weights, unlike prompting, which only changes the input you give an unchanged model.

Full fine-tuning updates all weights and is expensive (it needs roughly the resources of training plus optimizer memory). In practice most people use parameter-efficient methods like LoRA/QLoRA that train a tiny fraction of the weights, getting most of the benefit at a fraction of the cost.

Fine-tuning is great for teaching style, format, and task behavior, but it is not the best tool for injecting fresh facts — for that, retrieval (feeding documents into the context) usually works better and cheaper. Instruct models you download are themselves the product of fine-tuning a base model.

LoRA · Base vs instruct model · Inference · Safetensors

All explainers → · Browse models →

Fine-tuning

Related