Applied AI·How models work
every mainstream model you call is one of these, which is why they all share the same context-window and attention economics.
Transformer
Draft summary, pending review
The neural network architecture behind every mainstream LLM, introduced in the 2017 paper 'Attention is all you need'. Its key property is that it processes all tokens in a sequence in parallel and lets every token look at every other token via attention.