Tokenizers tranined with NFC normalization.
Guilherme L Mello
guilhermelmello
·
AI & ML interests
Language Models
Recent Activity
updated
a collection
15 days ago
Tokenizers Qwen-PT
updated
a collection
15 days ago
Tokenizers Qwen-PT
updated
a collection
15 days ago
Tokenizers Qwen-PT
Organizations
models
27
guilhermelmello/tokenizer-unigram-pt-15k
Updated
guilhermelmello/tokenizer-bpe-pt-15k
Updated
guilhermelmello/tokenizer-unigram-pt-50k
Updated
guilhermelmello/tokenizer-unigram-pt-30k
Updated
guilhermelmello/tokenizer-unigram-pt-10k
Updated
guilhermelmello/tokenizer-unigram-pt-8k
Updated
guilhermelmello/tokenizer-unigram-pt-5k
Updated
guilhermelmello/tokenizer-bpe-pt-50k
Updated
guilhermelmello/tokenizer-bpe-pt-30k
Updated
guilhermelmello/tokenizer-bpe-pt-10k
Updated