How to optimize Amazon Bedrock | The Keys to AWS Optimization | S13 E3

The episode focuses on optimizing Amazon Bedrock, AWS’s managed AI service, with insights from FinOps and generative AI experts. The hosts clarify key AI terminology, distinguishing between traditional machine learning (ML), which relies on statistical modeling, and generative AI (GenAI), which can create new content and reason beyond its training data. They explain foundational models (FMs), large language models (LLMs), and tokens-the billing unit for LLM usage. Bedrock simplifies AI deployment by managing infrastructure and billing by tokens, offering flexibility through on-demand, provisioned, and batch pricing models. The discussion covers strategies for selecting the right model based on use case, cost, and performance, emphasizing the importance of understanding Bedrock’s native and marketplace model billing in AWS Cost Explorer. Optimization techniques include model distillation (creating smaller, faster models for specific tasks), fine-tuning (improving model performance for particular domains), latency optimization (paying a premium for faster responses), and prompt caching (reducing costs for repeated queries). The episode also introduces retrieval-augmented generation (RAG), which enhances model outputs with external data via Bedrock knowledge bases, and highlights the need for careful cost management of related AWS resources. Finally, the guests share tools and best practices for evaluating model performance, ensuring responsible AI use, and maximizing ROI from GenAI investments.

https://docs.aws.amazon.com/bedrock/l...
https://aws.amazon.com/blogs/machine-...
https://docs.aws.amazon.com/sagemaker...
https://aws.amazon.com/blogs/aws/redu...

  / david-tepper