Machine Learning
Topic 29 · Deep learning · ⏱ about 6 h

CNNs, RNNs & Transformers

Specialised architectures for images, sequences and language.

Show:
In this topic
Beginner

Convolutional Neural Networks (CNNs)

A plain multi-layer network treats its inputs as an unordered list of numbers, so it has no idea that in an image, neighbouring pixels belong together. Convolutional neural networks (CNNs) are designed for images and other grid-like data, and they build in two sensible assumptions: that nearby pixels are related, and that a pattern means the same thing wherever it appears in the image.

imageconv: edgespool → convshapesdeeper: partsdensecat 0.93
A CNN builds a hierarchy: early layers detect edges, later layers shapes and parts, and the final layers classify.

The key operation is the convolution. A small grid of learnable weights, called a filter or kernel (typically 3 by 3 pixels), is slid across the whole image. At each position, the filter's weights are multiplied by the pixel values beneath them and added up, giving one number. Doing this at every position produces a new grid called a feature map, which is large wherever the image locally matches the filter's pattern. A filter that has learned to detect vertical edges, for example, produces a feature map that lights up along every vertical edge in the image. A convolutional layer contains many filters, each detecting a different pattern.

Because the same small filter is reused at every position, a technique called parameter sharing, a convolutional layer needs very few weights compared with a fully connected layer, and it detects a pattern wherever it occurs: a cat in the top-left corner and a cat in the bottom-right are recognised by the same filters. Pooling layers, most often max-pooling, then shrink each feature map by keeping only the largest value in each small block. This reduces the amount of computation and makes the network less sensitive to the exact position of features.

Stacking convolutional layers builds a hierarchy of features. The first layers detect simple things such as edges and colour patches; the next combine those into textures and simple shapes; deeper layers combine shapes into parts such as eyes or wheels; and the final layers recognise whole objects. Landmark CNNs trace the history of the field: LeNet read handwritten digits in the 1990s; AlexNet's victory in the 2012 ImageNet competition launched the deep-learning boom; VGG showed the value of many small filters; and ResNet used residual connections to train networks over a hundred layers deep.

In practice you rarely train a CNN from scratch. With transfer learning, you start from a network already trained on millions of images and fine-tune it on your own, much smaller dataset. Because the pretrained network already knows how to detect edges, textures and shapes, it can learn your new task from just hundreds or thousands of examples.

Beginner

Watch it: Convolution: a small filter sliding over an image

A short narrated animation. Press ▶ Play, or step through it with ⏮ ⏭. Tick 🔊 narrate to hear each step.

Interactive: this part runs in your browser.

Try it

Try it: convolution, pixel by pixel

The 3×3 kernel slides over the image; each output value is the sum of pixel × weight under the window. Click output cells to see the calculation, click input pixels to draw, and switch kernels.

Interactive: this part runs in your browser.

Beginner

Recurrent Neural Networks (RNNs)

Many kinds of data come as sequences, where order matters: the words in a sentence, the readings from a sensor over time, the samples in an audio recording. Recurrent neural networks (RNNs) were the first architecture designed for sequences.

RNN cellx₁ "the"h₁RNN cellx₂ "cat"h₂RNN cellx₃ "sat"h₃RNN cellx₄ "down"h₄hidden state carries memory forward; the SAME weights are used at every step
An RNN unrolled through time: each step combines the new input with the previous hidden state.

An RNN reads a sequence one element at a time, and as it goes it maintains a hidden state, a vector that acts as its memory of everything it has read so far. At each step, it combines the current input with its previous hidden state to produce a new hidden state:

ht=tanh⁡ ⁣(Wxt+Uht−1+b). h_t = \tanh\!\big(W x_t + U h_{t-1} + b\big).

The same weights WW and UU are used at every step, which is the sequence equivalent of a CNN's parameter sharing, and it means the network can handle sequences of any length.

RNNs are trained with backpropagation through time, which unrolls the network across all the time steps and applies ordinary backpropagation. Unfortunately, the gradient must pass through the same weight matrix once for every step, so over long sequences it tends to vanish or explode, exactly the problem described in the training topic. In practice, a plain RNN forgets information from more than a few dozen steps back.

The LSTM (long short-term memory) network, and its simpler cousin the GRU, solve this with an extra memory track, the cell state, and with gates. A gate is a set of numbers between 0 and 1, computed from the current input and the previous hidden state, that controls how much information flows. The forget gate decides how much of the old memory to keep, the input gate decides how much new information to write, and the output gate decides how much of the memory to reveal at this step. Because the cell state is updated mainly by adding new information rather than repeatedly multiplying by a matrix, information and gradients can survive across hundreds of steps.

The main limitation of all recurrent networks is that they must process a sequence step by step, since each step depends on the previous one. This makes them slow to train on long sequences, because the work can't be spread across the thousands of parallel cores of a GPU. That limitation is what transformers were invented to remove.

Interactive: this part runs in your browser.

Interactive. Why plain RNNs forget: the gradient is multiplied by the same factor at every step back in time.
Beginner

Transformers & attention

The transformer, introduced in the 2017 paper Attention Is All You Need by Vaswani and colleagues, is the architecture behind today's large language models and much of modern AI. It does away with recurrence entirely and processes all the elements of a sequence at the same time, using a mechanism called self-attention.

Thecatsatbecauseitwastired0.62query "it" compares with every key; softmax turns scores into weights; output = weighted mix of values
Self-attention: the word "it" spreads its attention over the sentence, putting most weight on "cat".
input tokens + positionsLayerNormmulti-head attention+residualLayerNorm → FFN → +tokens exchange informationeach token processed separately× N blocks stacked
One transformer block: attention lets tokens share information, the feed-forward network processes each token, and residual connections with layer norm keep training stable.

The idea of self-attention is that, to understand any one word in a sentence, you should look at all the other words and decide which of them matter. In "The animal didn't cross the street because it was too tired", understanding "it" requires looking back at "animal". Self-attention lets every word gather information from every other word, weighted by relevance.

To decide relevance, each word is turned into three vectors. Its query represents what it is looking for. Its key represents what it offers to others looking for information. Its value is the information it will pass on. For each word, we compare its query with the key of every word using a dot product; a high dot product means a good match. These scores are scaled and passed through a softmax, which turns them into weights that are positive and add up to 1. The word's new representation is then the weighted average of all the words' values. In matrix form, for all words at once:

Attention⁡(Q,K,V)=softmax⁡ ⁣(QK⊤dk)V. \operatorname{Attention}(Q, K, V) = \operatorname{softmax}\!\Big(\frac{QK^\top}{\sqrt{d_k}}\Big)V.

Two refinements complete the picture. Multi-head attention runs several attention operations in parallel, each with its own queries, keys and values, so that different heads can learn different kinds of relationship, for example one tracking grammar and another tracking which noun a pronoun refers to. And because attention itself ignores order (it treats the words as a set), positional encodings are added to each word's representation to tell the model where in the sequence each word is.

Because every word is processed at the same time, transformers make full use of GPUs and can be trained on enormous datasets. That scalability is a large part of their success. Their main cost is that every word attends to every other word, so the computation grows with the square of the sequence length.

Transformers come in several forms. Encoder models such as BERT read a whole text at once and are good at understanding tasks such as classification. Decoder models such as the GPT family generate text one word at a time, each word attending only to the words before it. Vision transformers apply the same idea to images by splitting them into patches. Large language models are first pre-trained on vast amounts of text with a self-supervised objective, predicting the next word, and are then fine-tuned to follow instructions and to be helpful, for example with instruction tuning and reinforcement learning from human feedback (RLHF).

Interactive: this part runs in your browser.

Breakdown. Self-attention computed step by step on a three-word sentence.
Try it

Try it: self-attention and the √dₖ scaling

Each row shows how much one word attends to every other word (rows sum to 1). Click a word to see its weights as bars. Untick divide by √dₖ and raise dₖ to see why the scaling matters.

Interactive: this part runs in your browser.

Intermediate · Maths

Convolutions and recurrences, precisely

📐 Maths you'll need: m-matrices · m-calculus
💡 The idea in plain words

A convolution re-uses one small set of weights at every position of an image, so it needs few parameters and detects a pattern wherever it appears. An RNN re-uses one set of weights at every time step, carrying a memory forward. The LSTM's trick is to update that memory by adding, which lets gradients survive across long sequences.

🔤 Symbols in this section
I, KI,\ Kinput image and kernel (filter)
k, P, sk,\ P,\ skernel size, padding, stride
Cin, CoutC_{in},\ C_{out}input / output channels
hth_thidden state at time t
ctc_tLSTM cell state (memory)
ft, it, otf_t,\ i_t,\ o_tforget, input and output gates (0–1)

This section gives the precise equations for convolutional and recurrent layers, including how to compute output sizes and parameter counts, and shows mathematically why LSTMs fix the vanishing-gradient problem.

Convolution. In deep learning, a 2-D convolution of an input II with CC channels (for example, the red, green and blue channels of an image) by a k×kk\times k kernel KK is computed as

S(i,j)=∑c∑m=0k−1∑n=0k−1Ic(i+m, j+n) Kc(m,n)+b. S(i,j) = \sum_{c}\sum_{m=0}^{k-1}\sum_{n=0}^{k-1} I_c(i+m,\ j+n)\,K_c(m,n) + b.

In words: place the kernel with its top-left corner at position (i,j)(i, j), multiply each kernel weight by the input value beneath it across all channels, add everything up, and add a bias. (Mathematicians would call this cross-correlation, because the kernel isn't flipped, but deep learning calls it convolution.) Three numbers control the size of the output. The padding PP adds a border of zeros around the input so the kernel can be centred on edge pixels, and the stride ss is how many pixels the kernel moves between positions. For an input of width WW, the output width is

⌊W−k+2Ps⌋+1, \Big\lfloor\frac{W - k + 2P}{s}\Big\rfloor + 1,

where ⌊⋅⌋\lfloor\cdot\rfloor means round down. For example, a 3-by-3 kernel with padding 1 and stride 1 on a 32-pixel-wide input gives ⌊(32−3+2)/1⌋+1=32\lfloor(32 - 3 + 2)/1\rfloor + 1 = 32, the same width, which is why that combination is so common.

A convolutional layer with CinC_{in} input channels and CoutC_{out} filters has k⋅k⋅Cin⋅Cout+Coutk\cdot k\cdot C_{in}\cdot C_{out} + C_{out} parameters, and crucially this number does not depend on the size of the image. A 3-by-3 layer from 64 to 128 channels has 3⋅3⋅64⋅128+128=73,8563\cdot3\cdot64\cdot128 + 128 = 73{,}856 parameters, whereas a fully connected layer from a 32×32×64 input to just 128 outputs would need about 8.4 million. The receptive field, the region of the original image that one output value can "see", grows as layers are stacked: LL layers of 3-by-3 convolutions with stride 1 see a region 2L+12L + 1 pixels wide.

Recurrent networks. A simple RNN updates its hidden state and makes a prediction at each step with

ht=tanh⁡(Wxhxt+Whhht−1+b),y^t=Whyht. h_t = \tanh(W_{xh}x_t + W_{hh}h_{t-1} + b), \qquad \hat y_t = W_{hy}h_t.

Backpropagation through time requires the gradient of a late hidden state with respect to an early one, which by the chain rule is a product of one factor per step:

∂hT∂ht=∏k=t+1Tdiag⁡(1−hk2) Whh. \frac{\partial h_T}{\partial h_t} = \prod_{k=t+1}^{T}\operatorname{diag}(1 - h_k^2)\,W_{hh}.

(The factor 1−hk21 - h_k^2 is the slope of tanh.) Each factor contains the same matrix WhhW_{hh}, so the product behaves roughly like a power of it. If WhhW_{hh}'s largest singular value is below 1, the product shrinks exponentially and long-range gradients vanish; if it is above 1, they can explode.

LSTMs. An LSTM adds a cell state ctc_t and three gates, each a sigmoid (so each entry is between 0 and 1) computed from the previous hidden state and the current input, written together as [ht−1,xt][h_{t-1}, x_t]:

ft=σ(Wf[ht−1,xt]+bf),it=σ(Wi[ht−1,xt]+bi) f_t = \sigma(W_f[h_{t-1}, x_t] + b_f),\qquad i_t = \sigma(W_i[h_{t-1}, x_t] + b_i)
ot=σ(Wo[ht−1,xt]+bo),c~t=tanh⁡(Wc[ht−1,xt]+bc) o_t = \sigma(W_o[h_{t-1}, x_t] + b_o),\qquad \tilde c_t = \tanh(W_c[h_{t-1}, x_t] + b_c)

Here ftf_t is the forget gate, iti_t the input gate, oto_t the output gate, and c~t\tilde c_t a candidate new memory. The cell state and hidden state are then updated as

ct=ft⊙ct−1+it⊙c~t,ht=ot⊙tanh⁡(ct). c_t = f_t\odot c_{t-1} + i_t\odot\tilde c_t, \qquad h_t = o_t\odot\tanh(c_t).

The crucial feature is the additive update of the cell state: the old memory is scaled by the forget gate, and new information is added to it. As a result, ∂ct/∂ct−1=ft\partial c_t/\partial c_{t-1} = f_t, with no repeated weight matrix in the way. When the forget gate is close to 1, the gradient passes almost unchanged from step to step, so information can be carried across hundreds of steps. This is the same principle that makes residual connections work in very deep networks.

Key takeaway. Conv output size = ⌊(W − k + 2P)/s⌋ + 1 and parameters don't depend on image size; LSTMs avoid vanishing gradients via the additive cell update.
Advanced

The Transformer, equation by equation

📐 Maths you'll need: m-vectors · m-matrices · m-prob
💡 The idea in plain words

Attention lets every word look at every other word and decide which ones matter for understanding it. Each word asks a question (query), each word advertises what it contains (key), and the dot product of query and key, turned into percentages by softmax, decides how much of each word's information (value) to mix in.

🔤 Symbols in this section
X∈Rn×dX\in\mathbb R^{n\times d}n token vectors of size d
Q, K, VQ,\ K,\ Vqueries, keys, values
dkd_ksize of each query/key vector
MMmask (−∞ blocks a position)
hhnumber of heads
LN, FFN\text{LN},\ \text{FFN}layer norm, feed-forward network

This section writes out a transformer precisely, equation by equation, and explains the reasoning behind each design choice.

Queries, keys and values. Suppose the input is a sequence of nn tokens, each represented by a vector of length dd (its embedding plus positional information), stacked into a matrix X∈Rn×dX\in\mathbb R^{n\times d}. One attention head has three learned weight matrices, which turn every token into a query, a key and a value:

Q=XWQ,K=XWK,V=XWV. Q = XW_Q,\qquad K = XW_K,\qquad V = XW_V.

Each row of QQ is a token's query, and similarly for KK and VV. The attention output is then

Attention⁡(Q,K,V)=softmax⁡ ⁣(QK⊤dk+M)V. \operatorname{Attention}(Q,K,V) = \operatorname{softmax}\!\Big(\frac{QK^\top}{\sqrt{d_k}} + M\Big)V.

Reading from the inside out: QK⊤QK^\top is an n×nn\times n matrix whose entry (i,j)(i, j) is the dot product of token ii's query with token jj's key, a raw relevance score. The softmax is applied to each row, so row ii becomes a set of weights that are positive and add up to 1, describing how much token ii attends to each token jj. Multiplying by VV then replaces each token with the corresponding weighted average of all the value vectors.

Why divide by dk\sqrt{d_k}? Suppose the entries of a query qq and a key kk are independent with mean 0 and variance 1. Their dot product q⊤k=∑j=1dkqjkjq^\top k = \sum_{j=1}^{d_k}q_jk_j is a sum of dkd_k terms, each with variance 1, so the dot product has variance dkd_k. With dk=64d_k = 64, typical scores would be around ±8, and the softmax of such large numbers is almost one-hot: one weight close to 1 and the rest close to 0. In that regime the softmax's gradients are tiny and learning stalls. Dividing by dk\sqrt{d_k} brings the scores back to variance 1, keeping the softmax in a well-behaved range.

The causal mask. In a model that generates text left to right, a token must not be allowed to look at tokens that come after it, because at generation time those don't exist yet. The mask MM sets Mij=−∞M_{ij} = -\infty whenever j>ij > i, and e−∞=0e^{-\infty} = 0, so those positions receive zero attention weight.

Multiple heads. Rather than one attention operation with large vectors, transformers run hh smaller heads in parallel, each with its own WQW_Q, WKW_K and WVW_V and with dk=d/hd_k = d/h. Their outputs are joined side by side and mixed by a final matrix WOW_O:

MHA⁡(X)=Concat⁡(head1,…,headh) WO. \operatorname{MHA}(X) = \operatorname{Concat}(\text{head}_1,\dots,\text{head}_h)\,W_O.

Different heads are free to specialise in different relationships, such as syntax, coreference or position.

Position. Attention on its own treats the tokens as an unordered set, so position information must be added to the input. The original paper used fixed sinusoidal encodings,

PE(pos, 2i)=sin⁡ ⁣(pos/100002i/d),PE(pos, 2i+1)=cos⁡ ⁣(pos/100002i/d), PE_{(pos,\,2i)} = \sin\!\big(pos/10000^{2i/d}\big), \qquad PE_{(pos,\,2i+1)} = \cos\!\big(pos/10000^{2i/d}\big),

which give each position a unique pattern of waves at many different frequencies. Many modern models instead use learned position embeddings or rotary position embeddings (RoPE).

A transformer block. Each block combines attention with a small two-layer network applied to each token separately (the feed-forward network, FFN), and wraps each part with a residual connection and layer normalisation. In the "pre-norm" arrangement used by most modern models:

X′=X+MHA⁡(LN⁡(X)),X′′=X′+FFN⁡(LN⁡(X′)) X' = X + \operatorname{MHA}(\operatorname{LN}(X)), \qquad X'' = X' + \operatorname{FFN}(\operatorname{LN}(X'))
FFN⁡(x)=W2 GELU⁡(W1x+b1)+b2. \operatorname{FFN}(x) = W_2\,\operatorname{GELU}(W_1x + b_1) + b_2.

The FFN's hidden layer is usually about four times wider than dd. Attention lets tokens exchange information; the FFN then processes each token's gathered information individually. A full model stacks dozens of these blocks.

Cost. Because the score matrix is n×nn\times n, attention takes time proportional to n2dn^2d and memory proportional to n2n^2 in the sequence length nn, which becomes the bottleneck for long documents. This motivated FlashAttention (which computes exactly the same result but organises the computation to use fast GPU memory efficiently), sparse and linear approximations to attention, and KV-caching, which stores the keys and values of earlier tokens during generation so they aren't recomputed for every new token.

Training objectives. GPT-style decoder models are trained to predict each next token, minimising the cross-entropy −∑tlog⁡p(xt∣x<t)-\sum_t\log p(x_t\mid x_{<t}) over enormous text corpora. BERT-style encoders are trained to fill in randomly masked tokens. A striking empirical finding is that the loss of these models falls smoothly as a power law in the number of parameters, the amount of training data and the amount of compute. These scaling laws have guided the development of ever-larger models.

Key takeaway. Attention(Q,K,V) = softmax(QKᵀ/√d_k)V; the √d_k keeps scores at unit variance; transformer blocks = attention + FFN, each with a residual connection and layer norm.
Code

Code: convolution and attention from scratch, plus PyTorch models

NumPy versions of the two key operations, then a small CNN and a Transformer encoder in PyTorch.

Python · NumPy: conv2d and scaled dot-product attention
import numpy as np

def conv2d(img, kernel, stride=1, pad=0):
    img = np.pad(img, pad)
    k = kernel.shape[0]
    H = (img.shape[0] - k) // stride + 1
    W = (img.shape[1] - k) // stride + 1
    out = np.zeros((H, W))
    for i in range(H):
        for j in range(W):
            patch = img[i * stride:i * stride + k, j * stride:j * stride + k]
            out[i, j] = np.sum(patch * kernel)
    return out

img = np.zeros((8, 8)); img[:, 4:] = 1.0                # dark left, bright right
sobel_x = np.array([[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]])
print(conv2d(img, sobel_x))                             # lights up at the vertical edge

def softmax(z, axis=-1):
    z = z - z.max(axis, keepdims=True)
    e = np.exp(z)
    return e / e.sum(axis, keepdims=True)

def attention(X, Wq, Wk, Wv, causal=False):
    Q, K, V = X @ Wq, X @ Wk, X @ Wv
    scores = Q @ K.T / np.sqrt(K.shape[1])
    if causal:
        scores = scores + np.triu(np.full(scores.shape, -np.inf), k=1)
    A = softmax(scores)                                 # (n, n) attention weights
    return A @ V, A

rng = np.random.default_rng(0)
n, d, dk = 5, 16, 8
X = rng.normal(size=(n, d))
out, A = attention(X, *(rng.normal(size=(d, dk)) / np.sqrt(d) for _ in range(3)), causal=True)
print(A.round(2))                                       # lower-triangular, rows sum to 1
Open in playground ↗
Python · PyTorch: a small CNN and a Transformer encoder
import torch
from torch import nn

class SmallCNN(nn.Module):                 # e.g. for 28x28 greyscale digits
    def __init__(self, n_classes=10):
        super().__init__()
        self.features = nn.Sequential(
            nn.Conv2d(1, 32, 3, padding=1), nn.BatchNorm2d(32), nn.ReLU(), nn.MaxPool2d(2),   # 14x14
            nn.Conv2d(32, 64, 3, padding=1), nn.BatchNorm2d(64), nn.ReLU(), nn.MaxPool2d(2),  # 7x7
        )
        self.head = nn.Sequential(nn.Flatten(), nn.Dropout(0.3), nn.Linear(64 * 7 * 7, n_classes))
    def forward(self, x):
        return self.head(self.features(x))

print(SmallCNN()(torch.randn(8, 1, 28, 28)).shape)          # torch.Size([8, 10])

# Transformer encoder for sequence classification
class TinyTransformer(nn.Module):
    def __init__(self, vocab=10000, d=128, heads=4, layers=2, max_len=256, n_classes=2):
        super().__init__()
        self.tok = nn.Embedding(vocab, d)
        self.pos = nn.Embedding(max_len, d)                  # learned positions
        block = nn.TransformerEncoderLayer(d, heads, dim_feedforward=4 * d,
                                           activation="gelu", batch_first=True, norm_first=True)
        self.encoder = nn.TransformerEncoder(block, layers)
        self.cls = nn.Linear(d, n_classes)
    def forward(self, ids):
        pos = torch.arange(ids.shape[1], device=ids.device)
        h = self.encoder(self.tok(ids) + self.pos(pos))
        return self.cls(h.mean(dim=1))                       # mean-pool over tokens

print(TinyTransformer()(torch.randint(0, 10000, (4, 50))).shape)   # torch.Size([4, 2])
# Fused attention kernel: torch.nn.functional.scaled_dot_product_attention(q, k, v, is_causal=True)
This code needs PyTorch, which cannot run inside a web page. To run it, press Copy, then paste it into a new notebook in Google Colab ↗ (free; Runtime → Change runtime type → GPU for deep learning).
Check yourself

Quiz

1. Which architecture uses sliding learnable filters?

2. LSTMs were designed mainly to fix…

3. In self-attention, attention weights come from…

4. Output width of a conv with W=32, k=3, P=1, s=1 is…

5. Attention scores are divided by √dₖ because…

6. Self-attention's cost in sequence length n is…

Go deeper

📚 Further reading

Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.

Ready