Text Generation
Transformers
PyTorch
TensorBoard
Safetensors
bloom
Eval Results (legacy)
text-generation-inference
Instructions to use bigscience/bloom with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use bigscience/bloom with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="bigscience/bloom")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("bigscience/bloom") model = AutoModelForCausalLM.from_pretrained("bigscience/bloom") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use bigscience/bloom with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "bigscience/bloom" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/bigscience/bloom
- SGLang
How to use bigscience/bloom with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "bigscience/bloom" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "bigscience/bloom" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigscience/bloom", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use bigscience/bloom with Docker Model Runner:
docker model run hf.co/bigscience/bloom
Commit History
Add arrows for code evaluation (#28) 60e49ad
Add evaluation (#27) bb3556d
updated opening blurb 168ece4
Teven Le Scao commited on
fixed pie chart link bfd53df
Teven Le Scao commited on
Removed non-standard simplified/traditional language tags 79bc63b
Teven Le Scao commited on
Updated opening blurb f689bf8
Teven Le Scao commited on
Fixed example a7ef188
Teven Le Scao commited on
Added reasoning example 5d3b298
Teven Le Scao commited on
Added widget examples a6cebd9
Teven Le Scao commited on
Added widget examples 2d60202
Teven Le Scao commited on
Updated pie chart 6adf557
Teven Le Scao commited on
Add link to interactive corpus treemap (#26) 0140768
Add TensorBoard Traces (#25) 5556040
ybelkada commited on
Update README.md (#23) eb49b9c
modify padding side c42411c
add pad token id 0142556
Younes Belkada commited on
Add image for how to use (#20) 46e5752
Remove .dev from transformers version (#21) 85e429f
ybelkada commited on
Add correct dtype on config fbda88e
Younes Belkada commited on
new data acbc8df
Updating italics to the HTML equivalent. 177dc5b
Fixing italics that aren't rendering for "BigScience Large Open-science Open-access Multilingual Language Model" 833b211
Update README.md (#18) 44651d2
ybelkada commited on
Update README.md (#15) 0aa20cd
ybelkada commited on
Update README.md f25baa0
Younes Belkada commited on
Update README.md 290df47
Younes Belkada commited on
Update README.md (#16) 45cf2c9
ybelkada commited on
Adding blurb about mistaking sentience. (#12) dc534f6
Update README.md (#14) 9d151d5
Update README.md a339d03
Younes Belkada commited on
Update README.md 826ae56
Younes Belkada commited on
Merge branch 'main' of https://huggingface.co/bigscience/bloom into main c2f6b0a
new data 18894c7
new data 2690f53
Update README.md 215bee9
Edit intermediary-checkpoint warning 55d00a5
Teven Le Scao commited on
Update README.md e8fa22e
Younes Belkada commited on
Update README.md cf39b50
Younes Belkada commited on
Update README.md 39b727a
Younes Belkada commited on
Update README.md 945bfb3
Younes Belkada commited on
Update README.md 4320d65
Younes Belkada commited on
Update README.md 3351c7c
Younes Belkada commited on
Update README.md 6dbe969
Younes Belkada commited on