Apple’s Machine Learning Research team on 15 October published benchmarks showing how the latest version of its open-source MLX framework runs very large language models across several Mac Studio computers linked with Thunderbolt 5.
On a cluster of four Mac Studios with M5 Ultra, using tensor parallelisation, the 816-billion-parameter Muse Spark Open model (a mixture-of-experts model with 42 billion active parameters, run in 4-bit NVFP4 precision) generated over 36 tokens per second. In 4-bit form the model takes up about 493GB, and once the memory needed for intermediate computations is added it no longer fits on a single machine, even with 512GB of unified memory.
The distributed set-up builds on the low-latency RDMA over Thunderbolt 5 driver introduced in macOS 26.2. MLX v0.30.1 and later add communication collectives called jaccl on top of it, supporting both pipeline parallelisation (splitting a model’s layers between machines) and tensor parallelisation (splitting each layer across machines). In Apple’s set-up, each pair of Mac Studios is linked by two Thunderbolt cables to double the bandwidth.
On a single Mac Studio with 512GB, time to first token for the dense Qwen 3.8 (27B) and Mistral Medium 3.5 (128B) models was up to 4.6 times faster on M5 Ultra than on M3 Ultra, while generation was 1.6 times faster, which Apple attributes to the M5 Ultra’s 1.2TB/s memory bandwidth versus 819GB/s on M3 Ultra. Peak power during inference was about 300W for M5 Ultra and 200W for M3 Ultra.
Why it matters
Our take: Apple is positioning the Mac Studio as a desk-side alternative to cloud GPUs for frontier-class open models. Running a model of this size privately on four machines drawing roughly 300W each at peak is a strong pitch to researchers and developers who want to keep data local.