Gemma 4 WinoGrande Score Raises Pipeline Doubts

Hot take: this reads like an eval harness problem first, a model-quality problem second. WinoGrande is brittle, so small changes in prompt template or scoring setup can move the result a lot. Quantized GGUF runs through llama.cpp can be especially sensitive to tokenizer and cache behavior, so a near-50% score may reflect setup drift rather than genuine incompetence. Comparing Gemma 4 against Qwen in this one pipeline says more about the benchmark configuration than the models themselves.