Article
There is no such thing as a tokenizer-free lunch
By
•
•
84Catherine Arnett
catherinearnett
AI & ML interests
multilingual NLP, tokenization
Recent Activity
upvoted
a
paper
40 minutes ago
Global PIQA: Evaluating Physical Commonsense Reasoning Across 100+
Languages and Cultures
updated
a collection
about 2 hours ago
Global PIQA
updated
a collection
about 2 hours ago
Global PIQA
Organizations
published
an
article
about 1 month ago
published
an
article
about 1 month ago
Article
An Analysis of Multilingual Models on Hugging Face
By
and 1 other
•
•
4
published
an
article
6 months ago
Article
Best Practices for Open Multilingual LLM Evaluation
By
•
•
3
published
an
article
11 months ago
Article
They Said It Couldn’t Be Done
By
and 2 others
•
•
89
published
an
article
12 months ago
Article
Releasing the largest multilingual open pretraining dataset
By
and 2 others
•
•
104
published
an
article
12 months ago
Article
Detoxifying the Commons
By
•
•
6
published
an
article
about 1 year ago
Article
wHy DoNt YoU jUsT uSe ThE lLaMa ToKeNiZeR??
By
•
•
50