Deep Learning
Topic 1 ยท Deep Learning ยท โฑ about 1 h

Deep Learning: Start Here

What this track adds to the ML course, what deep-learning roles expect, and how to set up a GPU in Colab or on your own machine.

Show:
In this topic
Beginner

From "I know what a neural network is" to "I can build and fix one"

The core ML course already introduced neural networks, how they are trained, CNNs, RNNs and transformers and PyTorch. That is enough to use deep learning. This track goes one level deeper, so that you can build every piece yourself, understand why training fails, and fix it. That is what separates someone who can run a notebook from someone a team trusts with a model.

The plan is the same for every topic: understand the idea, see it animated, play with an interactive demo, write it from scratch in NumPy (in your browser, checked by tests), and then see the two or three lines that do the same thing in PyTorch. Writing the from-scratch version once is the fastest way to stop treating the framework as magic.

Prerequisites: the Deep learning phase of the core course, plus matrices, gradients and the chain rule and NumPy. If the Deep learning checkpoint felt hard, revisit those first.

Beginner

What deep-learning roles expect from a junior

Job adverts for junior ML or deep-learning engineers vary, but the same expectations come up again and again:

  • Write and debug a training loop in PyTorch: data loading, batching, loss, optimiser, evaluation, checkpoints.
  • Reason about shapes: know what shape every tensor has at every line, and fix shape errors quickly.
  • Diagnose training from loss curves and gradient statistics: learning rate too high, overfitting, a data bug.
  • Use pretrained models well: choose one, fine-tune it with the right learning rate, and evaluate it honestly.
  • Know the cost: estimate GPU memory, use mixed precision, and make a model fast enough to serve.
  • Read a paper's architecture diagram and turn it into code; explain your choices to non-experts.

Every item has a topic in this track and a task in its game season, The Applied AI team.

Beginner

How this track is organised

PartTopicsGame milestone
Foundations, from scratchTensors ยท Autograd ยท Building networksD1: The reference implementation
Making training workOptimisers ยท Normalisation & regularisation ยท Debugging trainingD2: The run that wouldn't converge
ArchitecturesCNNs in depth ยท Sequence modelsD3: Referral forms and vital signs
Modern practiceTransformers ยท Transfer learning ยท Generative models ยท Efficiency & scaleD4: Small, fast and safe

The browser exercises use NumPy, because PyTorch cannot run inside a web page. For real training on a GPU, three Colab notebooks (below) repeat the main exercises in PyTorch. The Computer Vision and NLP tracks build on this one.

Beginner

Set up a GPU

Google Colab (free): open a notebook, choose Runtime โ†’ Change runtime type โ†’ T4 GPU. Free sessions time out, so save checkpoints to Google Drive. Kaggle Notebooks give a similar free weekly GPU allowance.

Your own machine: create a virtual environment, then install PyTorch with the command generated on pytorch.org for your operating system and CUDA version (NVIDIA GPUs). On Apple-silicon Macs PyTorch uses the mps device; without a GPU everything still runs on the CPU, just slower. Small models in this track train in minutes on a CPU.

Try it

Hands-on: three Colab notebooks for this track

Use these alongside the topics: 06 after Building Networks and Optimisers, 07 after Debugging Training and CNNs, 08 after Transformers. Copy a notebook's code and paste it into a new Colab notebook, or upload the .ipynb file.

Interactive: this part runs in your browser.

Code

Code: check your environment

Run this first in any new notebook. It picks the best available device.

Python ยท PyTorch
import torch, platform
print("PyTorch", torch.__version__, "| Python", platform.python_version())
device = ("cuda" if torch.cuda.is_available()
          else "mps" if torch.backends.mps.is_available() else "cpu")
print("device:", device)
if device == "cuda":
    print(torch.cuda.get_device_name(0),
          f"{torch.cuda.get_device_properties(0).total_memory / 1e9:.1f} GB")
x = torch.randn(1024, 1024, device=device)
print((x @ x).shape)          # a quick matrix multiply on the device
This code needs PyTorch, which cannot run inside a web page. To run it, press Copy, then paste it into a new notebook in Google Colab โ†— (free; Runtime โ†’ Change runtime type โ†’ GPU for deep learning).
Check yourself

Quiz

1. Why does this track have you write layers and backpropagation in NumPy before using PyTorch?

2. Which skill do junior deep-learning roles almost always expect?

3. In Colab, how do you get a GPU?

4. Why can't the in-browser exercises use PyTorch?

Go deeper

๐Ÿ“š Further reading

Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.

Ready