Apple Machine Learning Research released MLX version 0.30.1 on 14 November, adding support for distributed inference across Macs via RDMA over Thunderbolt 5. The update enables a cluster of four M5 Ultra Mac Studios to run the 816-billion-parameter Muse Spark Open mixture-of-experts model at over 36 tokens per second.

The distributed inference capability builds on a low-latency RDMA driver introduced in macOS 26.2. MLX, Apple’s open-source array framework for machine learning on Apple silicon, now provides distributed collectives (jaccl) for parallelisation schemes such as pipeline parallelism. Apple Machine Learning Research published the benchmarks on 14 November.

In single-Mac comparisons, the M5 Ultra Mac Studio (the latest generation, succeeding the M3 Ultra) achieves up to 4.6 times faster time-to-first-token and 1.6 times faster generation speed via its GPU Neural Accelerator. Unified memory bandwidth reaches 1.2 TB/s, up from 819 GB/s. Peak power consumption for LLM inference is about 300 W for the M5 Ultra and 200 W for the M3 Ultra.

The Muse Spark Open model in NVFP4 quantisation requires about 493 GB of memory, preventing it from fitting on a single Mac Studio. The post also benchmarks Mistral Medium 3.5 (128 B, dense, FP8) and Qwen 3.8 (27 B, dense, BF16).

Apple previously demonstrated a similar cluster with an earlier version of MLX.

Why it matters

In our view, MLX distributed inference positions a cluster of Mac Studios as a viable alternative to cloud GPU servers for running large open-weight models locally. The power efficiency and unified memory advantage are significant, though the requirement for four high-end machines (each with up to 512 GB of unified memory) makes it a specialist setup. Nonetheless, it shows Apple’s growing commitment to serious on-device AI.