Some tokenizers
Powered by
Tokenizers.js
.
Input
pika 5
pika 4
pika 3.1
pika 3
pika 2
Small German Tokenizer
pika
GPT-NeoX
Strawberry (approximate)
MicroSupra
gpt-oss
GPT-1
Gemma 4
Qwen3.5
Qwen3
Qwen2.5
Qwen1.5
Tokenize
Clear cache
Output