ARO: A new lens on matrix optimization for LLMs

We present Adaptively Rotated Optimization (ARO), a matrix optimizer that speeds up LLM training by applying updates in a rotated, geometry-aware coordinate system. Guided by new insights on global structures on LLM loss landscapes, ARO treats rotation as a unifying principle for sample efficiency, and proposed a new update policy that is applicable to all model weight matrices. In large scale controlled experiments, ARO consistently outperforms AdamW and orthogonalization-based method, maintaining its gains as models and training budgets scale. Paper: https://arxiv.org/abs/2602.09006 This session aired on March 3, 2026, at Microsoft Research Forum, Season 2 Episode 3. Register for the series to learn about future episodes: https://events.microsoft.com/flow/ms/researchforum/register/page/microsoftresearchforumreg/?OCID=msr_researchforum_YTDescription Explore all previous episodes: https://aka.ms/researchforumYTplaylist