gpt-tokenizer is an open-source command-line tool for visualizing GPT tokenization using tiktoken. It helps developers and NLP practitioners understand how text is tokenized for large language models directly in the terminal.
gpt-tokenizer sits in PulseGate's CLI tools & terminal category. It focuses on visualizing GPT tokenization in the terminal for developers and NLP practitioners. It is built as an open-source project for developers and NLP practitioners. gpt-tokenizer is open source under the MIT license. gpt-tokenizer is available on the command line.
gpt-tokenizer first shipped in 2026. Development happens publicly on GitHub with 8 commits in the last 90 days. PulseGate's similarity index finds few close equivalents — gpt-tokenizer occupies a relatively distinct niche. Key capabilities include token visualization, terminal interface, and GPT support.
Latest indexed changes and source events
gpt-tokenizer discovered by the PulseGate indexer
Other apps tracked under the same category.
