Instructions to use microsoft/wavecoder-ultra-6.7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use microsoft/wavecoder-ultra-6.7b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="microsoft/wavecoder-ultra-6.7b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("microsoft/wavecoder-ultra-6.7b")
model = AutoModelForCausalLM.from_pretrained("microsoft/wavecoder-ultra-6.7b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use microsoft/wavecoder-ultra-6.7b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "microsoft/wavecoder-ultra-6.7b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/wavecoder-ultra-6.7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/microsoft/wavecoder-ultra-6.7b

SGLang

How to use microsoft/wavecoder-ultra-6.7b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "microsoft/wavecoder-ultra-6.7b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/wavecoder-ultra-6.7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "microsoft/wavecoder-ultra-6.7b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "microsoft/wavecoder-ultra-6.7b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use microsoft/wavecoder-ultra-6.7b with Docker Model Runner:
```
docker model run hf.co/microsoft/wavecoder-ultra-6.7b
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

🌊 WaveCoder: Widespread And Versatile Enhanced Code LLM

[📜 Paper] • [🐱 GitHub]
[🐦 Twitter] • [💬 Reddit] • [🍀 Unofficial Blog]

Repo for "WaveCoder: Widespread And Versatile Enhanced Instruction Tuning with Refined Data Generation"

🔥 News

[2024/04/10] 🔥🔥🔥 WaveCoder repo, models released at 🤗 HuggingFace!
[2023/12/26] WaveCoder paper released.

💡 Introduction

WaveCoder 🌊 is a series of large language models (LLMs) for the coding domain, designed to solve relevant problems in the field of code through instruction-following learning. Its training dataset was generated from a subset of code-search-net data using a generator-discriminator framework based on LLMs that we proposed, covering four general code-related tasks: code generation, code summary, code translation, and code repair.

Model	HumanEval	MBPP(500)	HumanEval Fix(Avg.)	HumanEval Explain(Avg.)
GPT-4	85.4	-	47.8	52.1
🌊 WaveCoder-DS-6.7B	65.8	63.0	49.5	40.8
🌊 WaveCoder-Pro-6.7B	74.4	63.4	52.1	43.0
🌊 WaveCoder-Ultra-6.7B	79.9	64.6	52.3	45.7

🪁 Evaluation

Please refer to WaveCoder's GitHub repo for inference, evaluation, and training code.

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("microsoft/wavecoder-ultra-6.7b")
model = AutoModelForCausalLM.from_pretrained("microsoft/wavecoder-ultra-6.7b")

📖 License

This code repository is licensed under the MIT License. The use of DeepSeek Coder models is subject to the its License.

☕️ Citation

If you find this repository helpful, please consider citing our paper:

@article{yu2023wavecoder,
  title={Wavecoder: Widespread and versatile enhanced instruction tuning with refined data generation},
  author={Yu, Zhaojian and Zhang, Xin and Shang, Ning and Huang, Yangyu and Xu, Can and Zhao, Yishujie and Hu, Wenxiang and Yin, Qiufeng},
  journal={arXiv preprint arXiv:2312.14187},
  year={2023}
}

Note

WaveCoder models are trained on the synthetic data generated by OpenAI models. Please pay attention to OpenAI's terms of use when using the models and the datasets.

Downloads last month: 902

Model tree for microsoft/wavecoder-ultra-6.7b

Quantizations

6 models

Paper for microsoft/wavecoder-ultra-6.7b

WaveCoder: Widespread And Versatile Enhanced Instruction Tuning with Refined Data Generation

Paper • 2312.14187 • Published Dec 20, 2023 • 49