Applied AI·How models work
a learned relevance score between every pair of tokens in the context.
Attention
The mechanism inside a transformer that lets the model weigh which earlier tokens matter most when predicting the next one. When people say a model 'attends to' part of the input, they mean attention assigned it high weight.