Hugging Face's logo

baebee
/
nuph-tokenizer

Transformers
Transformers.js
tokenizers
experimental
auto tokenizer
llama
llama 2
custom code
Model card Files
xet
 Community
How to use from the
Use from the
Transformers library
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("baebee/nuph-tokenizer", dtype="auto")
Quick Links

Experimental tokenizer

  • Derived from the LLaMa-2 tokenizer
  • Inspired by the ChatML & Zephyr tokenizers
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support