Mistral Medium 3.5 Q3 disappoints on 3x3090

// 96d agoBENCHMARK RESULT

Mistral Medium 3.5 Q3 disappoints on 3x3090

A Reddit user says Mistral Medium 3.5 runs fast on three 3090s, but the Q3_K_M quantization hurts output quality, especially for code and structured markup. The base model is pitched as a 128B flagship for coding, reasoning, and long-context work, so this reads more like a quantization warning than a verdict on the model itself.

// ANALYSIS

The hot take: speed is not the story here. Q3_K_M is a brutally aggressive local quant, and the failure mode shows up exactly where you'd expect it to show up first: syntax-sensitive generation and formatting discipline.

–The GGUF maintainer labels Q3_K_M as low quality, while recommending Q4/Q5 tiers for better results
–The user's report that Python was acceptable but SVG/HTML broke fits the usual pattern of low-bit quantization harming structured output
–Mistral's own positioning for Medium 3.5 is coding, reasoning, and agentic work, so this is a deployment-quality issue, not necessarily a model-capability issue
–For local inference on consumer GPUs, the practical takeaway is to test a higher quant before concluding the base model is weak at code generation
–The three-3090 setup looks viable for throughput, but not enough to rescue a weak quant when correctness matters more than raw speed

// TAGS

llmopen-weightsquantizationcode-generationai-codinggpumistral-medium-3-5

DISCOVERED

96d ago

2026-05-02

PUBLISHED

96d ago

2026-05-01

RELEVANCE

8/ 10

AUTHOR

jacek2023

// KEEP READING

More AI developer news from the feed

EXPLORE FULL FEED

UPDATE30m ago

Netlify adds open-source model support via OpenRouter

Netlify has announced native support for open-source AI models across its developer platform. Through a new integration with OpenRouter, developers can call hundreds of open models directly from Netlify functions and edge functions with a zero-config setup that eliminates manual API key management and account creation. Furthermore, Netlify's Agent Runners now feature OpenCode, an open-source coding agent designed specifically for open models such as GLM 5.2 and Kimi 3, enabling developers to seamlessly mix and match frontier proprietary models with open-source alternatives.

MODEL48m ago

OpenAI rumored model Astra could launch next week

Reports from AI leakers suggest that OpenAI's next flagship model, codenamed Astra, may launch as early as next week. Described as OpenAI's largest pre-training effort since GPT-4.5, the model is currently being tested internally under the checkpoint codename "mewfour".

RESEARCH51m ago

Skill Entropy quantifies cross-skill LLM transitions

A novel research paper introduces "Skill Entropy," a metric designed to quantify the complexity of multi-step tasks requiring LLMs to dynamically switch between different cognitive skills. To evaluate this challenge, the authors created the Skill^2-Bench benchmark covering 558 skills across nine domains, and introduced Skill-Entropy Reinforcement Learning (RL), a training approach that incorporates skill prediction and sequence alignment into model rewards to significantly boost long-horizon reasoning performance.