SRE Kernel tames multi-model VRAM on 8GB

Running a full agentic stack on 8GB VRAM requires OS-level scheduling discipline — this design takes the nuclear option and it works, at a cost.

• –The "Traffic Cop" pattern (blocking concurrent model execution + hard locking of audio handles) solves a real driver-level collision problem between CUDA contexts that frameworks typically ignore
• –The "Nuclear Flush" (forced CUDA cache purge vs. relying on PyTorch/framework garbage collection) addresses a legitimate pain point: lazy GC leaves fragmented VRAM that can block loading a model that should fit
• –The "Odometer" heuristic (tracking cumulative PCIe data as a fragmentation proxy) is a simple but clever signal — no expensive memory inspection needed
• –Serial execution with several-second PCIe transfer pauses per handoff is a steep latency tax; this is a personal-use pattern, not a blueprint for low-latency inference
• –The design essentially reinvents what hypervisors do for CPU scheduling, but applied to GPU VRAM — interesting convergence as consumer multi-model workflows become more common