Backpropagation

In one line

How a network figures out which of its knobs to blame for a mistake, so it can fix them.

Picture it

After a group project goes wrong, tracing back who contributed what to the error and giving each person specific feedback.

Under the hood

Applies the chain rule to compute the gradient of the loss with respect to every parameter, from the output backwards.