OPEN LINK ↗
// 45d agoTUTORIAL
Llama.cpp token budget fixes Gemma 4 Vision
Google's Gemma 4 multimodal models often fail at OCR due to restrictive default vision budgets, but performance can be drastically improved via specific llama.cpp configurations. By overriding the default token limits and adjusting batch sizes, developers can unlock state-of-the-art vision capabilities on local hardware.
// ANALYSIS
Gemma 4’s "blindness" in local deployments is a configuration trap, not a model limitation—high-fidelity OCR is locked behind a visual token budget.
- –Increasing `--image-max-tokens` to 1120 or 2240 enables the model to resolve minute details that are lost at the default 280-token limit.
- –Critical: `--ubatch-size` and `--batch-size` must be set higher than the max token budget to prevent immediate GGML engine crashes.
- –While effective, high-res vision is VRAM-intensive, pushing a Q8 31B model from 63GB to nearly 77GB of required memory.
- –Properly configured Gemma 4 destroys competitors like Qwen 3.5 and GLM in OCR tasks, making it a SOTA choice for document parsing.
// TAGS
gemma-4multimodalllmocrllama-cppopen-weights
DISCOVERED
45d ago
2026-04-21
PUBLISHED
45d ago
2026-04-21
RELEVANCE
8/ 10
AUTHOR
seamonn