FoveatedKV hits 2x KV compression for Mac

FoveatedKV is a masterclass in exploiting Apple Silicon's shared memory architecture and Metal performance for LLM optimization. It employs a mixed-precision strategy using fp16 for near tokens and fp8/INT4 for far tokens to preserve fidelity while slashing memory footprint. Custom fused Metal kernels perform dequantization in registers, bypassing global memory bottlenecks and maximizing throughput. A novel spike-driven promotion mechanism from NVMe allows the system to recover precision for suddenly-important tokens dynamically. This enables 7B models on 8GB machines where memory pressure usually prevents effective local inference. Finally, the use of precision asymmetry—fp8 for keys and INT4 for values—correctly identifies that keys have a greater impact on softmax stability than values.