Curious About What's Inside an LLM? A Python Walkthrough for Beginners

What’s actually hiding inside an LLM? Using Python, we're diving deep into an OpenWeights LLM to reveal its inner workings. This is your chance to go beyond the hype and see the raw data—the weights and biases—that define how these models think. I'll guide you step-by-step through a practical, hands-on demonstration. By the end, you'll have a new appreciation for these LLMs

Timestamps & Chapters
00:00 – Intro: My Curiosity About LLMs
00:32 - Open Weights LLM Models
00:44 - Not meant for Experts
01:11 – BERT in Everyday Use
01:54 - DistilBERT
02:22 – Loading DistilBERT in Colab
04:00 - Live Colab Demo
05:56 – What Do I Even Look For?
06:54 - Do the number of parameters equal 66M?
08:50 – Peeking at Layers & Transformer Architecture
09:48 – Looking at the Numbers (Weights & Biases)
11:04 - Matplotlib Histogram of an LLM Layer values
11:43 - Are the weights mostly infinitesimal?
12:36 - How different are 2 of the Transformer layer values?
13:28 - So what did we learn?
15:07 – Outro: A Brief Peek, More to Come

Links:
DistilBERT on HuggingFace: https://huggingface.co/distilbert/dis...
Arxiv: https://arxiv.org/abs/1910.01108
Medium Article by Victor Sanh -   / distilbert  

#BERTLLM #HowdoLLMsWork #smallLLMs