@yuvalreif
PhD student @ The Hebrew University | Focused on tokenization, efficiency and multilingual LLMs
AI
A post says CoBPE uses six tokens where BPE uses 13 for a Hobbit line, without changing the text.
AI
