Transformer

In one line

The 2017 design that nearly all modern language AI is built on.

Picture it

A room where every word can look at every other word at once, instead of reading left to right one at a time.

Under the hood

Stacks of attention and feed-forward layers with residual connections, introduced in the paper 'Attention Is All You Need'. It scales very well.