Open source Policy & regulation Infrastructure & hardware

Mixture-of-Experts kernel engineering on Amazon SageMaker AI

AWS Machine Learning Mona Mona

Publication summary. The terms of AWS Machine Learning do not allow reproduction of the full article: read the complete text on the original website.

Source summary (English)

AWS engineers share kernel optimizations for fused Mixture-of-Experts (MoE) models on Amazon SageMaker AI: Swap-AB for WGMMA utilization, Triton backend routing for FP8 quantization, and eliminating Triton JIT recompilation. Together they deliver up to 10% higher throughput and up to 9% faster Time to First Token on H200 GPUs.

Read the full article

aws.amazon.com

View on AWS Machine Learning (new tab)

Topics

Share

X (new tab) LinkedIn (new tab) Facebook (new tab) Email

Related articles