mwitiderrick/SwahiliAlpaca
Viewer • Updated • 34.7k • 16
How to use mwitiderrick/SwahiliInstruct-v0.1 with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("text-generation", model="mwitiderrick/SwahiliInstruct-v0.1")
messages = [
{"role": "user", "content": "Who are you?"},
]
pipe(messages) # Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("mwitiderrick/SwahiliInstruct-v0.1")
model = AutoModelForCausalLM.from_pretrained("mwitiderrick/SwahiliInstruct-v0.1")
messages = [
{"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
tokenize=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))How to use mwitiderrick/SwahiliInstruct-v0.1 with vLLM:
# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mwitiderrick/SwahiliInstruct-v0.1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "mwitiderrick/SwahiliInstruct-v0.1",
"messages": [
{
"role": "user",
"content": "What is the capital of France?"
}
]
}'docker model run hf.co/mwitiderrick/SwahiliInstruct-v0.1
How to use mwitiderrick/SwahiliInstruct-v0.1 with SGLang:
# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
--model-path "mwitiderrick/SwahiliInstruct-v0.1" \
--host 0.0.0.0 \
--port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "mwitiderrick/SwahiliInstruct-v0.1",
"messages": [
{
"role": "user",
"content": "What is the capital of France?"
}
]
}'docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "mwitiderrick/SwahiliInstruct-v0.1" \
--host 0.0.0.0 \
--port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "mwitiderrick/SwahiliInstruct-v0.1",
"messages": [
{
"role": "user",
"content": "What is the capital of France?"
}
]
}'How to use mwitiderrick/SwahiliInstruct-v0.1 with Docker Model Runner:
docker model run hf.co/mwitiderrick/SwahiliInstruct-v0.1
This is a Mistral Instruct that has been fine-tuned on 1 epoch of the SwahiliPlatypus dataset.
### Maelekezo:
{query}
### Jibu:
<Leave new line for model to respond>
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
tokenizer = AutoTokenizer.from_pretrained("mwitiderrick/SwahiliInstruct-v0.1")
model = AutoModelForCausalLM.from_pretrained("mwitiderrick/SwahiliInstruct-v0.1", device_map="auto")
query = "Nipe maagizo ya kutengeneza mkate wa mandizi"
text_gen = pipeline(task="text-generation", model=model, tokenizer=tokenizer, max_length=200, do_sample=True, repetition_penalty=1.1)
output = text_gen(f"### Maelekezo:\n{query}\n### Jibu:\n")
print(output[0]['generated_text'])
"""
Maagizo ya Kutengeneza Mkate wa Mandazi
1.Washa tanuri kamili hadi 350 ° F (175°C) na uweka sufuria ya kuoka katikati.
2. Katika bakuli, changanya mikasi yote pamoja na chumvi kitamu.
3. Ongeza mayai, siagi na sukari, koroga hadi viwe sawasawa.
4. Panda mchanganyiko kwenye pan katika mkusanyo unaotuleta tayari kwa mtiririko.
5. Kata misingo kuburudisha kila mahali na uwatazama ikiwa yanapasuka y
"""
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2")
model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2", device_map="auto")
query = "Nipe maagizo ya kutengeneza mkate wa mandizi"
text_gen = pipeline(task="text-generation", model=model, tokenizer=tokenizer, max_length=500)
output = text_gen(f"### Instruction:\n{query}\n### Response:\n")
print(output[0]['generated_text'])
"""
To make a pancake, follow these steps:
1. In a large bowl, mix together 1 1/2 cups of all-purpose flour, 3 1/2 teaspoons of baking powder, 1 teaspoon of salt, and 1 tablespoon of sugar.
2. Make a well in the center of the flour mixture and pour in 1 1/2 cups of milk, 1 1/4 cups of buttermilk, 2 eggs, and 3 tablespoons of melted butter.
3. Mix the ingredients together until they are just combined. Do not overmix.
4. Let the batter rest for 30 minutes.
5. Heat a griddle or large skillet over medium heat. Brush it with oil or melted butter.
6. Using a 1/4 cup measure, scoop the batter onto the griddle or skillet. Cook until bubbles form on the surface and the edges start to look set, then flip and cook until the other side is golden brown.
7. Serve the pancakes warm with your favorite toppings, such as maple syrup, whipped cream, or fresh fruit. Enjoy!
"""
Base model
mistralai/Mistral-7B-Instruct-v0.2