Ollama users want smarter AMD offloading

This is a real local-inference infrastructure gap: most "easy" model runners still behave like single-model launchers, not schedulers that can intelligently tier workloads across GPU and system RAM.

• –The post describes a concrete limitation in Ollama today: once the next model no longer fits in VRAM, it unloads other models instead of partially offloading layers to CPU.
• –That behavior is especially painful for agent-style setups where one heavyweight model handles reasoning while smaller models do extraction, summarization, or utility work in parallel.
• –The AMD/ROCm angle matters because users often prioritize llama.cpp stability on Radeon hardware, even when other servers may look stronger on paper.
• –The only reply points to `llama.cpp` plus `llama-swap` over Vulkan as the most practical workaround, which suggests advanced users still need lower-level tooling for smarter residency control.