Scaling MoE reinforcement learning on Amazon EKS with EFA and DeepEP with 40% more throughput

What happened

When you post-train a Mixture-of-Experts (MoE) model with Reinforcement Learning from Human Feedback (RLHF) or Group Relative Policy Optimization (GRPO) at scale, three simultaneous challenges emerge. The first requires coordinating heterogeneous compute for rollout generation and policy training.

Second, sustaining high-throughput (how much work a system gets through in a given time) communication across hundreds of accelerators. On AWS, you can address these challenges using Amazon Elastic Kubernetes (software that runs and manages applications across many servers) Service (Amazon EKS), Elastic Fabric Adapter (EFA), and DeepEP.

Mixture-of-Experts (MoE) has become a standard architecture for scaling large language model (the kind of AI system trained on text to produce text)s (LLMs) to hundreds of billions or even trillions of parameters, while maintaining efficient inference through sparsity. As part of the standard training pipeline, these models must undergo pre-training, mid-training, supervised fine-tuning (SFT), and reinforcement learning (RL).

Key facts

• The first — requires: coordinating heterogeneous compute for rollout generation and policy training