metadata
library_name: transformers
datasets:
- HuggingFaceFW/fineweb
Model Card for Model ID
This is a BPE tokenizer with 10,048 tokens trained on a portion of FineWeb's 10B token sample.