FP8 Training on AMD GPUs with TorchTitan and TorchAO: Upstreaming Performance Improvements

At the PyTorch Conference 2025, we demonstrated linear scaling beyond 1,000 GPUs on AMD Instinct clusters using Primus-Turbo, an AMD optimization library for training frameworks such as TorchTitan.