Applied AI·Tokens
the model cannot see individual characters unless the tokeniser happens to split them out.
Tokeniser quirks
A family of odd behaviours explained by tokenisation: models miscounting letters in 'strawberry' (they see word pieces, not characters), fragile arithmetic on long numbers (digits split inconsistently), and weaker performance in languages that tokenise inefficiently.