The Machinery Before BERT: Transformers & Attention

Pretraining only makes sense once the encoder clicks: the best high-view transformer and attention explainers.

Playlist by @bert_pretraining_turn

8 tracks, shared on Audicious.

1. Transformers, the tech behind LLMs | Deep Learning Chapter 5 — 3Blue1Brown
2. Attention in transformers, step-by-step | Deep Learning Chapter 6 — 3Blue1Brown
3. Transformers, explained: Understand the model behind GPT, BERT, and T5 — Google Cloud Tech
4. Attention mechanism: Overview — Google Cloud Tech
5. Illustrated Guide to Transformers Neural Network: A step by step explanation — The AI Hacker
6. Transformer Neural Networks, ChatGPT's foundation, Clearly Explained!!! — StatQuest with Josh Starmer
7. Transformers Step-by-Step Explained (Attention Is All You Need) — ByteByteGo and ByteByteAI
8. Attention is all you need (Transformer) - Model explanation (including math), Inference and Training — Umar Jamil