Pretrained GPT2 Turkish model that is meant to be an entry point for fine-tuning on other texts.
-
Updated
Jan 11, 2021
#
byte-level-bpe
Here are 3 public repositories matching this topic...
Text tokenization service (Rust, Axum) (Q1:2025)
-
Updated
May 2, 2025 - Rust
Open byte-level BPE tokenizer for Uzbek (16k vocab) — 1.85 tokens/word vs GPT-4's 2.62, at ~12× smaller vocab. Built-in apostrophe/okina normalization; loads with Hugging Face Transformers.
nlp tokenizer transformers uzbek low-resource-languages bpe huggingface byte-level-bpe subword-tokenization uzbek-nlp
-
Updated
Jun 14, 2026 - Python
Improve this page
Add a description, image, and links to the byte-level-bpe topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the byte-level-bpe topic, visit your repo's landing page and select "manage topics."