LocalLLaMA debates fine-tune dataset size

The useful answer is not a raw record count. For fine-tuning, trust comes from a held-out evaluation that still improves as you scale data, not from hitting some magic number.

• –Commenters cite wildly different starting points, from about 2,000 examples to 10,000-16,000 entries, which underscores how model size and task complexity drive the requirement.
• –The strongest advice in the thread is to define evaluation first, then increase dataset size incrementally and watch for regression on general behavior.
• –Small-model LoRA runs can overfit quickly, including weird output-length behavior and loss of broad capability if you train too aggressively.
• –For dataset sellers, the real differentiator is not just volume; it is whether the dataset comes with clear labels, metrics, and a way for buyers to validate gains.
• –The thread reflects the broader fine-tuning reality: data quantity matters, but data quality and benchmark discipline matter more.