Apple announced today an online reinforcement learning framework for training UI navigation agents on live iOS applications. The best-performing 8B parameter model outperformed Google’s Gemini-3.5-Flash and matched Gemini-3.1-Pro on benchmarks with real iOS apps.
The system decouples the agent into a Planning agent, which chooses high-level actions, and a UI Grounding model, which translates them into interface interactions. An LLM-as-a-Judge supplies dense step-level feedback and a terminal reward for task completion. The agent is optimised using Proximal Policy Optimisation (PPO) alongside a novel neighbour-conditioned value estimation method called GiGPPO, which stabilises the critic using reward signals from visually similar states across concurrent trajectories.
To gather experience at scale, the training system coordinates hundreds of virtual iOS devices running in parallel. Tests against the UINavBench, iOSWorld and AndroidWorld benchmarks show agents trained with this online RL method substantially outperform supervised fine-tuning (SFT) baselines and agents trained with terminal-only rewards. The 8B RL agents achieved nearly a sevenfold improvement over the SFT baseline when generalising out-of-distribution to Android environments.
It is not yet known whether the framework will be integrated into future Apple products or Siri, and Apple has not disclosed which language model serves as the LLM-as-a-Judge.
Why it matters
This work adds to Apple’s research into efficient on-device multimodal agents. Earlier this year, Apple researchers cut agent latency by 90 per cent using speculative decoding. In our view, the online RL framework complements that effort by enabling agents trained directly on live apps to recover from real-world errors. Together, these advances suggest Apple is building the foundations for a more capable Siri and new iOS automation features without relying on expensive cloud models.