TLDR:
Fine-tuning is the process of further training a pre-trained foundation model on a curated dataset specific to a task, domain, or style, producing a model that performs better on that target than the base model. Modern fine-tuning is dominated by parameter-efficient methods like LoRA that adapt models with minimal compute.
Fine-Tuning Methods
Several fine-tuning approaches exist: full fine-tuning (updates all model parameters—expensive for large models), LoRA (Low-Rank Adaptation, adds small trainable matrices—typical method for modern fine-tuning), QLoRA (LoRA with 4-bit quantization—runs on consumer hardware), prompt tuning (learns soft prompts rather than modifying weights), and instruction tuning (fine-tuning on instruction-following data). Direct preference optimization (DPO) and similar methods further refine model behavior toward human preferences.
When to Fine-Tune
Fine-tuning makes sense when: the base model struggles with your specific task even with strong prompting, you have at least 100-1,000+ high-quality training examples, you need consistent output formatting or style, you want to reduce token costs by encoding behavior into model weights rather than long prompts, or you need to deploy a smaller, faster model that matches a larger model’s task performance. For most generic tasks, RAG and prompt engineering should be tried first.
Data and Cost Considerations
Fine-tuning data quality matters far more than quantity—curated, diverse, error-free examples produce better models than larger noisy datasets. Costs vary widely: API fine-tuning (OpenAI, Anthropic) ranges from cents to dollars per million tokens; self-hosted LoRA fine-tuning of open models can be done on a single GPU for under $100. Legal considerations include training data licensing, output ownership, and compliance with platform terms of service.