AWS Trainium and Inferentia // Kamran Khan and Matthew McClean // MLOps Podcast

Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/

AWS Trainium and Inferentia // MLOps podcast #238 with Kamran Khan, BD, Annapurna ML and Matthew McClean, Annapurna Labs Lead Solution Architecture at AWS.

Huge thank you to ‪@amazonwebservices‬ for sponsoring this episode. AWS - https://aws.amazon.com/

// Abstract
Unlock unparalleled performance and cost savings with AWS Trainium and Inferentia! These powerful AI accelerators offer MLOps community members enhanced availability, compute elasticity, and energy efficiency. Seamlessly integrate with PyTorch, JAX, and Hugging Face, and enjoy robust support from industry leaders like W&B, Anyscale, and Outerbounds. Perfectly compatible with AWS services like Amazon SageMaker, getting started has never been easier. Elevate your AI game with AWS Trainium and Inferentia!

// Bio
Kamran Khan
Helping developers and users achieve their AI performance and cost goals for almost 2 decades.

Matthew McClean
Leads the Annapurna Labs Solution Architecture and Prototyping teams helping customers train and deploy their Generative AI models with AWS Trainium and AWS Inferentia

// MLOps Jobs board
https://mlops.pallet.xyz/jobs

// MLOps Swag/Merch
https://mlops-community.myshopify.com/

// Related Links
AWS Trainium: https://aws.amazon.com/machine-learni...
AWS Inferentia: https://aws.amazon.com/machine-learni...

-------------- ✌️Connect With Us ✌️ ------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn:   / dpbrinkm  
Connect with Kamran on LinkedIn:   / kamranjk  
Connect with Matt on LinkedIn:   / matthewmcclean  

Timestamps:
[00:00] Matt's & Kamran's preferred coffee
[00:53] Takeaways
[01:57] Please like, share, leave a review, and subscribe to our MLOps channels!
[02:22] AWS Trainium and Inferentia rundown
[06:04] Inferentia vs GPUs: Comparison
[11:20] Using Neuron for ML
[15:54] Should Trainium and Inferentia go together?
[18:15] ML Workflow Integration Overview
[23:10] The Ec2 instance
[24:55] Bedrock vs SageMaker
[31:16] Shifting mindset toward open source in enterprise
[35:50] Fine-tuning open-source models, reducing costs significantly
[39:43] Model deployment cost can be reduced innovatively
[43:49] Benefits of using Inferentia and Trainium
[45:03] Wrap up

#amazonwebservices