---
title: "Apple shows four Mac Studios running 816B Muse Spark Open model with MLX"
description: "Running the 816B Muse Spark Open model at over 36 tokens per second on a cluster of four Mac Studios"
url: https://mactoreality.com/apple-mlx-framework-gains-distributed-inference-across-thunderbolt-linked-macs
published: 2026-10-16T05:15:52Z
updated: 2026-10-16T05:15:52Z
author: "Mac to Reality Newsdesk"
category: News
tags: ["Apple Intelligence & Siri","Mac","Software"]
publisher: Mac to Reality
ai_assisted: true
apple_heat_impact: 1
---

# Apple shows four Mac Studios running 816B Muse Spark Open model with MLX

*Running the 816B Muse Spark Open model at over 36 tokens per second on a cluster of four Mac Studios*

**Key points**

- Apple ML Research published benchmarks on 15 October showing four Mac Studios with M5 Ultra running the 816B Muse Spark Open model at over 36 tokens per second.
- The set-up uses MLX's jaccl communication collectives (MLX v0.30.1 and later) on top of the RDMA over Thunderbolt 5 driver introduced in macOS 26.2.
- On a single Mac Studio, M5 Ultra delivered up to 4.6 times faster time to first token and 1.6 times faster generation than M3 Ultra.

Apple’s [Machine Learning Research team](https://machinelearning.apple.com/research/mlx-distributed-inference) on 15 October published benchmarks showing how the latest version of its open-source MLX framework runs very large language models across several Mac Studio computers linked with Thunderbolt 5.

On a cluster of four Mac Studios with M5 Ultra, using tensor parallelisation, the 816-billion-parameter Muse Spark Open model (a mixture-of-experts model with 42 billion active parameters, run in 4-bit NVFP4 precision) generated over 36 tokens per second. In 4-bit form the model takes up about 493GB, and once the memory needed for intermediate computations is added it no longer fits on a single machine, even with 512GB of unified memory.

The distributed set-up builds on the low-latency RDMA over Thunderbolt 5 driver introduced in macOS 26.2. MLX v0.30.1 and later add communication collectives called jaccl on top of it, supporting both pipeline parallelisation (splitting a model’s layers between machines) and tensor parallelisation (splitting each layer across machines). In Apple’s set-up, each pair of Mac Studios is linked by two Thunderbolt cables to double the bandwidth.

On a single Mac Studio with 512GB, time to first token for the dense Qwen 3.8 (27B) and Mistral Medium 3.5 (128B) models was up to 4.6 times faster on M5 Ultra than on M3 Ultra, while generation was 1.6 times faster, which Apple attributes to the M5 Ultra’s 1.2TB/s memory bandwidth versus 819GB/s on M3 Ultra. Peak power during inference was about 300W for M5 Ultra and 200W for M3 Ultra.

## Why it matters

Our take: Apple is positioning the Mac Studio as a desk-side alternative to cloud GPUs for frontier-class open models. Running a model of this size privately on four machines drawing roughly 300W each at peak is a strong pitch to researchers and developers who want to keep data local.

## Sources

- [Apple Machine Learning Research: Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio](https://machinelearning.apple.com/research/mlx-distributed-inference)

*This report was written by the Mac to Reality newsdesk with AI assistance from the sources listed above, under the editorial policy at https://mactoreality.com/editorial-policy.*