Interactive lessons
Watch machine learning move
Four short lessons you can poke at. Drag a slider and the maths redraws. Nothing to install, no account, no API key.
Activation functions and their slopes
A neuron adds up its inputs into z, then bends it with an activation function. Backpropagation multiplies by that function’s slope at every layer, so the slope decides whether a deep network can learn.
Cheat sheet: all 13 side by side
| Function | Range | Max slope | Slope at z=-4 | Zero-centred | Smooth |
|---|
Optimisers on a 2-D loss surface
Training means walking downhill on a loss surface. Here the surface has two weights, so you can see it. Each optimiser starts at the same point and takes the same number of steps. Brighter is higher loss.
Click or tap the map to move the start point.
Loss after each step
Where each one ended
| Optimiser | Final loss | Reached minimum | Ended at |
|---|
“Reached” means the loss came within 0.0001 of the best possible value.
Fitting a line, one batch at a time
There is a formula for the best straight line through some points, but neural networks have no such formula. They start with a guess, y = 4, and nudge the intercept b and slope m against the gradient of the mean squared error, using a few points (a batch) per step.
The data
Error after each update
Estimating π by throwing darts
Throw darts uniformly at a 2×2 square with a circle of radius 1 inside it. The circle covers π/4 of the square, so 4 × (darts inside ÷ darts thrown) estimates π. Randomness turns into a number, and the error shrinks like 1/√n.
Repeat the whole experiment
One run is one estimate. Run it hundreds of times and the estimates pile up into a bell curve centred on π. Quadruple the darts and the curve gets half as wide.