OPEN LINK ↗
// 1h agoINFRASTRUCTURE
Apple MLX Runs Muse Spark Open Locally
Apple’s latest MLX release uses RDMA over Thunderbolt 5 and M5 Ultra Mac Studios to run Meta’s 816B-parameter Muse Spark Open model locally. A four-Mac cluster reaches 36.73 tokens per second while preserving private, OpenAI-compatible inference.
// ANALYSIS
MLX is becoming serious local-inference infrastructure, turning high-memory Macs into a practical distributed AI cluster. The tradeoff is steep hardware cost and Apple-specific networking, but the privacy and deployment simplicity are compelling.
- –Muse Spark Open uses 42B active parameters but still needs roughly 493GB in NVFP4, requiring multiple Mac Studios.
- –MLX’s `jaccl` collectives and `mlx.launch` make pipeline and tensor parallelism accessible across Thunderbolt-connected Macs.
- –Four M5 Ultra systems reach 36.73 tokens per second at an 8,192-token context, making local agentic workloads plausible.
- –The OpenAI-compatible server can plug into coding agents such as OpenCode without cloud APIs or external inference providers.
- –The setup requires macOS 26.2, MLX 0.30.1+, Thunderbolt 5 cabling, and expensive unified-memory hardware.
// TAGS
mlxmuse-spark-openinferenceedge-aiopen-sourcelocal-first
DISCOVERED
1h ago
2026-10-16
PUBLISHED
4h ago
2026-10-16
RELEVANCE
9/ 10
AUTHOR
antipaul
