Applied AI·Tokens
the model never sees your text at all, only the list of integers the tokeniser handed over.
Token ID
Draft summary, pending review
The integer index of a token in the vocabulary. The model never sees text, only sequences of these integers.