Curated watch path · 10 videos
AI Foundations: From Neural Networks to LLMs
This AI foundations route builds a working mental model of machine learning, neural networks and backpropagation, then follows the same ideas into Transformers and modern language models.
Who this is for
Curious programmers and technical learners who want a structured route from first principles to an implemented GPT rather than a list of disconnected AI explainers.
Before you start
High-school algebra is enough for the visual opening. Basic Python helps from micrograd onward; the tokenizer and GPT builds move into intermediate implementation detail.
How this AI foundations route was selected
The order begins with the learning problem, uses visual explanations for neural-network training, grounds those ideas in Andrej Karpathy’s code, and then advances through Transformers, tokenization and the LLM training lifecycle.
The independent site operator reviewed the October 31, 2026 source records for direct topic fit, learning order, evergreen status and positive Hacker News recommendation history. Titles, summaries, levels and labels come from the public dataset and may contain model-generated errors. This is an editorial ordering of source records, not a claim that every video was watched end to end. Read the source methodology and limits, or report a mismatch or broken video.
What the route connects
The sequence is designed to help you explain how a neural network learns and connect automatic differentiation, attention, tokenization and GPT training as parts of one system.
How to study it
- Use the first four videos to build intuition before taking detailed notes; from micrograd onward, pause and run the code when possible.
- Keep separate notes for Transformers, attention and tokenization, then revisit them together during the GPT build.
AI foundations in 10 ordered videos
-
↗course · intro
Lecture 01 - The Learning Problem
Start with the broader learning problem so supervised, unsupervised and reinforcement learning have a shared frame.
5 positive HN recommenders across 5 recommendation months
-
↗explainer · intro
But what is a neural network? | Deep learning chapter 1
Build a visual mental model of neurons and layers before touching training equations.
16 positive HN recommenders across 14 recommendation months
-
↗explainer · intro
Gradient descent, how neural networks learn | Deep Learning Chapter 2
See how a cost function and gradient descent turn prediction errors into learning.
4 positive HN recommenders across 3 recommendation months
-
↗explainer · intro
Backpropagation, intuitively | Deep Learning Chapter 3
Follow the error signal backward through a network without beginning from symbolic calculus.
4 positive HN recommenders across 3 recommendation months
-
↗lecture · intro
The spelled-out intro to neural networks and backpropagation: building micrograd
Implement the mechanism in Python and connect derivatives, autograd and neural-network training.
17 positive HN recommenders across 16 recommendation months
-
↗explainer · intro
Transformers, the tech behind LLMs | Deep Learning Chapter 5
Move from ordinary neural networks to tokens, embeddings, attention and Transformer architecture.
13 positive HN recommenders across 10 recommendation months
-
↗explainer · intro
Attention in transformers, step-by-step | Deep Learning Chapter 6
Slow down on self-attention and multi-head attention before implementing a language model.
5 positive HN recommenders across 4 recommendation months
-
↗lecture · intermediate
Let's build the GPT Tokenizer
Understand tokenization, Unicode and BPE—the text layer behind many surprising LLM behaviors.
5 positive HN recommenders across 5 recommendation months
-
↗tutorial · intermediate
Let's build GPT: from scratch, in code, spelled out.
Assemble a working GPT in code and connect Transformer components to generation.
14 positive HN recommenders across 8 recommendation months
-
↗lecture · intro
Deep Dive into LLMs like ChatGPT
Finish with the modern LLM lifecycle, from pretraining and post-training to limitations.
10 positive HN recommenders across 8 recommendation months
A checkpoint for your study notes
Try it: one variable, two gradient paths
Use this checkpoint alongside micrograd. You need basic Python and the derivative of a one-variable polynomial.
- Set x = 3 and y = x × x + 2 × x. Calculate y and dy/dx on paper, then build the expression with micrograd’s Value and call y.backward(). Compare the gradient with the central difference (f(x+h) − f(x−h)) / (2h), using h = 0.0001.
- Write out your reasoning before opening the answer below.
Show the answer and checks
Expect y = 15 and dy/dx = 8. The square contributes 6 and the linear term contributes 2. A result of only 6 or 2 suggests an overwritten contribution. With x = −2, expect y = 0 and gradient = −2.
Open the micrograd video · Check Karpathy’s micrograd source and usage ↗
An original site exercise, checked against the linked reference; not the speaker’s assignment or a viewing transcript. Checked .