Instructions to use sign/utf8-lm-tiny with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use sign/utf8-lm-tiny with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="sign/utf8-lm-tiny")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("sign/utf8-lm-tiny")
model = AutoModelForCausalLM.from_pretrained("sign/utf8-lm-tiny", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use sign/utf8-lm-tiny with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "sign/utf8-lm-tiny"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sign/utf8-lm-tiny",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/sign/utf8-lm-tiny

SGLang

How to use sign/utf8-lm-tiny with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "sign/utf8-lm-tiny" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sign/utf8-lm-tiny",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "sign/utf8-lm-tiny" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sign/utf8-lm-tiny",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use sign/utf8-lm-tiny with Docker Model Runner:
```
docker model run hf.co/sign/utf8-lm-tiny
```

utf8-lm-tiny / README.md

AmitMY

Update README.md

0d6bc7b verified 8 months ago

preview code

Raw

History Blame Contribute Delete

2.75 kB

metadata

library_name: transformers
license: mit
base_model: sbintuitions/tiny-lm
tags:
  - generated_from_trainer
datasets:
  - HuggingFaceFW/fineweb
model-index:
  - name: output-tiny-lm-fineweb
    results: []
language:
  - en

UTF8-LM-tiny

This model is a fine-tuned version of sbintuitions/tiny-lm on the HuggingFaceFW/fineweb dataset.

Using this training script, from utf8-tokenizer.

The repository includes the joined model for ease of use, and the bit_projection_weights.pt for further analysis.

Usage

from transformers import AutoModelForCausalLM
import torch

from utf8_tokenizer import UTF8Tokenizer

model_id = "sign/utf8-lm-tiny"

tokenizer = UTF8Tokenizer()
model = AutoModelForCausalLM.from_pretrained(model_id)

prompt = "My name is"

inputs = tokenizer([prompt], return_tensors="pt",
                   padding=True,
                   add_special_tokens=True)
inputs["input_ids"] = inputs["input_ids"].to(torch.long)
# We need to remove the EOS token
inputs["input_ids"] = inputs["input_ids"][:, :-1]
inputs["attention_mask"] = inputs["attention_mask"][:, :-1]


with torch.no_grad():
    out = model.generate(
        **inputs,
        max_new_tokens=64,
    )

print(tokenizer.decode(out[0], skip_special_tokens=False))

Training procedure

python run_clm.py \
  --use_bit_embeddings True \
  --output_dir ./output-tiny-lm-fineweb \
  --dataset_name HuggingFaceFW/fineweb \
  --streaming True \
  --dataloader_num_workers 1 \
  --dataloader_prefetch_factor 4 \
  --dataloader_pin_memory True \
  --dataloader_persistent_workers True \
  --do_train True \
  --save_strategy steps \
  --max_steps 20000 \
  --save_steps 1000 \
  --save_total_limit 2 \
  --logging_steps 100 \
  --logging_strategy steps \
  --model_name_or_path sbintuitions/tiny-lm \
  --per_device_train_batch_size 128 \
  --block_size 256 \
  --optim adamw_torch_fused \
  --learning_rate 3e-4 \
  --lr_scheduler_type cosine \
  --warmup_ratio 0.01 \
  --weight_decay 0.1 \
  --adam_beta1 0.9 \
  --adam_beta2 0.95 \
  --max_grad_norm 1.0 \
  --gradient_checkpointing True \
  --bf16 True \
  --seed 42 \
  --report_to wandb \
  --include_num_input_tokens_seen True

Framework versions

Transformers 4.57.3
Pytorch 2.9.1+cu130
Datasets 4.4.1
Tokenizers 0.22.1