Research Developer tools Infrastructure & hardware

Beyond the KV cache: Training and serving i14’s Laplace 3B linear attention model on AWS Trainium2 with native PyTorch

AWS Machine Learning Kevin Shabahang

Publication summary. The terms of AWS Machine Learning do not allow reproduction of the full article: read the complete text on the original website.

Source summary (English)

Learn how i14 trained and served Laplace 3B, a linear attention model, on AWS Trainium2 using native PyTorch. Its recurrent state stays fixed at 29.5 MB per sequence regardless of context length, avoiding the growing key-value cache of transformer models and keeping serving memory predictable.

Read the full article

aws.amazon.com

View on AWS Machine Learning (new tab)

Topics

Share

X (new tab) LinkedIn (new tab) Facebook (new tab) Email

Related articles