Ollama on Radeon 780M hits limits

The hot take: this is a sensible homelab experiment, but the ceiling is set less by peak benchmark numbers than by memory pressure, ROCm maturity, and how much contention the rest of the stack creates.

• –The post is useful because it frames the real decision: not “can it run?” but “what model sizes stay pleasant once the machine is multitasking?”
• –The 7B throughput claim is the least stable part of the discussion; actual tok/s will vary heavily with ROCm version, kernel, quantization, context length, and whether the iGPU is also serving other workloads.
• –The most actionable part is the memory question: 32GB shared RAM sounds large, but Proxmox, containers, frame buffers, and device overhead can make the practical model ceiling noticeably lower than the theoretical maximum.
• –For this class of hardware, sub-14B models are the likely sweet spot for mixed coding and writing use, with 7B-9B models probably giving the best balance of responsiveness and usability.
• –The Proxmox LXC/ROCm angle is important because passing AMD graphics through cleanly is often more about driver compatibility and container plumbing than raw hardware capability.
• –As a product story, this is really about Ollama as the local inference layer for a privacy-first, no-API-cost assistant setup.