Qwen3.5 Small Hits Synthetic Data Limits
A Reddit user on r/LocalLLaMA is using Qwen3.5 Small 9B to generate training data for a private Lua/AutoHotkey-like language, but the model keeps inventing bad syntax such as .msg instead of the documented .box. They want a fully local alternative that fits on a 16GB GPU and know RAG would be easier, but want to see how far fine-tuning alone can go.
This is the classic student-writing-its-own-textbook trap: once a model writes labels for a niche DSL, hallucinations get promoted into training data. Replies on the thread suggest stepping up to Phi-4 14B or Qwen3 32B at 4-bit, but the real gate is still manual review and validation. For a private coding language, schema checks, rule-based post-processing, or a verifier pass will probably improve the dataset more than another small-model swap.
DISCOVERED
59d ago
2026-03-29
PUBLISHED
59d ago
2026-03-29
RELEVANCE
AUTHOR
Revolutionary_Mine29