Gemma 4 flexes local Haystack agents

Gemma 4 looks strongest here as a general-purpose local agent model: it can reason, call tools, and handle multimodal inputs in one notebook. The GitHub MCP demo is the most interesting part because it shows both the promise and the friction of large tool catalogs in small local contexts.

• –The RAG example uses a quantized Gemma 4 E4B through Ollama, which makes the local-first angle concrete rather than theoretical
• –The image QA and weather demos show the model can connect vision to action, not just caption images
• –The MCP section is the real signal: tool discovery matters when the schema set is too large to stuff into context
• –Haystack’s `SearchableToolset` is a useful pattern for making large tool ecosystems workable on constrained hardware
• –The notebook also hints at the tradeoff for local agents: great flexibility, but you still need careful context management and model/runtime tuning