Paste any text and see every token as a colored chip — the same o200k vocabulary current GPT models use, running privately in your browser.
Tokenizer loading — counts below use the 4-characters-per-token estimate…
The browser runs the real o200k tokenizer family, so counts match what GPT models see. Claude, Gemini, Grok, Kimi, and DeepSeek don't publish their tokenizers — counts for them are close, clearly-labeled approximations.
Common words are one cheap token; rare names, numbers, URLs, and code shatter into many pieces. The chip view makes tokenizer behavior visible instead of theoretical.
Each chip shows its position and token ID on click — useful for debugging chunking, overlap windows in RAG, or why a string is more expensive than it looks.
Anthropic does not publish its tokenizer as a public library. This playground's o200k counts are a close approximation for Claude; for exact pre-flight counts, Anthropic exposes a server-side count-tokens endpoint.
o200k_base — the vocabulary used by OpenAI's current GPT models. Counts are exact for GPT; Claude, Gemini, Grok, Kimi, and DeepSeek use their own unpublished tokenizers, so treat those counts as close estimates (typically within a few percent for English prose).
No. The tokenizer is a static script downloaded once and runs entirely on your device — nothing is sent to any server.
Tokenizers reward frequency. Emoji are multi-byte Unicode sequences, and code is full of rare identifiers and symbols, so they split into more pieces than common English words.
