Hugging Face
Models
Datasets
Spaces
Buckets
new
Docs
Enterprise
Pricing
Website
Tasks
HuggingChat
Collections
Languages
Organizations
Community
Blog
Posts
Daily Papers
Hardware
Learn
Discord
Forum
GitHub
Solutions
Team & Enterprise
Hugging Face PRO
Enterprise Support
Inference Providers
Inference Endpoints
Storage Buckets
Log In
Sign Up
hafeez007
/
balochi-tokenizers
like
0
Text Generation
balochiml/balochi-dedup-corpus
Baluchi
English
sentencepiece
tokenizer
wordpiece
bpe
balochi
southern-balochi
low-resource-nlp
perso-arabic
nlp
gemma
bert
roberta
License:
apache-2.0
Model card
Files
Files and versions
xet
Community
Copy to bucket
new
main
balochi-tokenizers
76.6 MB
Ctrl+K
Ctrl+K
1 contributor
History:
2 commits
hafeez007
Update tokenizer models and README
e899795
verified
2 months ago
Ablation
Update tokenizer models and README
2 months ago
Code
Update tokenizer models and README
2 months ago
Models
Update tokenizer models and README
2 months ago
Tokens
Update tokenizer models and README
2 months ago
.gitattributes
Safe
1.52 kB
initial commit
2 months ago
Balochi_BPE_Tokenizer_64000.json
5.03 MB
Update tokenizer models and README
2 months ago
Balochi_Sentence_Piece_Tokenizer_64000.model
1.53 MB
xet
Update tokenizer models and README
2 months ago
Balochi_Sentence_Piece_Tokenizer_Vocab_64000.vocab
1.28 MB
Update tokenizer models and README
2 months ago
Balochi_Word_Piece_Tokenizer_64000.json
1.72 MB
Update tokenizer models and README
2 months ago
Balochi_bpe_47000_tokenizer.json
Safe
3.43 MB
Update tokenizer models and README
2 months ago
Balochi_bpe_tokenizer_80000.json
6.35 MB
Update tokenizer models and README
2 months ago
Balochi_sentencepiece_47000_tokenizer.model
1.19 MB
xet
Update tokenizer models and README
2 months ago
Balochi_sentencepiece_47000_tokenizer.vocab
Safe
1.03 MB
Update tokenizer models and README
2 months ago
Balochi_wordpiece_47000_tokenizer.json
Safe
1.2 MB
Update tokenizer models and README
2 months ago
README.md
13 kB
Update tokenizer models and README
2 months ago
