Huawei Ascend Achieves Training-Inference Consistency in Reinforcement Learning with 60% Performance Gain
Huawei Ascend has achieved complete training-inference consistency (logdiff of 0) in reinforcement learning for LLMs using the Ascend C programming language. Tested on the Qwen3-30B MoE model, this alignment also delivers a 20% to 60% performance gain in Eager mode through optimized FlashAttention operators. Numerical discrepancies between rollout (inference) and training engines often destabilize reinforcement learning for large language models. Resolving this system-level challenge without sacrificing performance strengthens the viability of the Ascend ecosystem for advanced LLM post-training. The inconsistency was resolved by aligning operator numerical paths, which included unifying attention semantics, locking reduce accumulation order, and matching online Softmax tiling strategies. Huawei has open-sourced these recipes and plans to release a tool to help developers identify remaining logdiff issues.
## BACKGROUND
In reinforcement learning for LLMs, the rollout engine generates text sequences (inference) while the training engine updates the model's parameters. Because floating-point operations are non-associative, minor differences in hardware execution paths, parallelization strategies, or operator tiling between these two engines lead to divergent outputs (measured by logdiff), harming training stability.