Gemma 3 4B fits offline chatbots

The real constraint here is not just model quality, but the whole offline stack: latency, quantization, speech, and tool routing matter as much as raw parameter count.

• –Gemma 3 4B is a credible choice because it is a lightweight open model with a large context window and strong general chat ability
• –If the machine is modest, 3B-class models may feel better in practice than a heavier 4B model with a sluggish runtime
• –Llama 3.2 3B is a practical fallback for assistant-style chat and mobile-ish deployments
• –Phi-4-mini is another strong small-model candidate if you care more about efficiency than peak reasoning
• –For a “TARS-like” experience, prompt style and tool use will matter more than trying to force one model to do everything