Research Developer tools

Optimizing cost and latency with Amazon Bedrock prompt caching

AWS Machine Learning Daniel Abib

Publication summary. The terms of AWS Machine Learning do not allow reproduction of the full article: read the complete text on the original website.

Source summary (English)

Prompt caching in Amazon Bedrock can cut input token costs by up to 90% when you repeatedly send the same context to foundation models. This post walks through six practical prompt caching scenarios using the Converse API: message content, system prompt, tool definition, mixed TTL, tenant isolation, and LangChain integration.

Read the full article

aws.amazon.com

View on AWS Machine Learning (new tab)

Topics

Share

X (new tab) LinkedIn (new tab) Facebook (new tab) Email

Related articles