Qwen3.5-4B fine-tuning recipe hits Reddit

Fine-tuning Qwen3.5-4B is a precision game where BF16 and loss masking are the difference between an expert assistant and a repetition loop. BF16 is mandatory for stability because Qwen's dense architecture is prone to loss spikes in standard FP16, while masking user tokens prevents the model from over-learning prompt structures at the expense of reasoning. High weight decay of 0.1 and low learning rates of 1e-5 are essential to prevent overfitting, and optimization kernels like Unsloth are nearly necessary to manage the 40GB+ VRAM requirements for full FFT on non-enterprise hardware.