Instructions to use arcee-ai/Meraj-Mini with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use arcee-ai/Meraj-Mini with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="arcee-ai/Meraj-Mini")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("arcee-ai/Meraj-Mini")
model = AutoModelForCausalLM.from_pretrained("arcee-ai/Meraj-Mini")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use arcee-ai/Meraj-Mini with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "arcee-ai/Meraj-Mini"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "arcee-ai/Meraj-Mini",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/arcee-ai/Meraj-Mini

SGLang

How to use arcee-ai/Meraj-Mini with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "arcee-ai/Meraj-Mini" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "arcee-ai/Meraj-Mini",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "arcee-ai/Meraj-Mini" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "arcee-ai/Meraj-Mini",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use arcee-ai/Meraj-Mini with Docker Model Runner:
```
docker model run hf.co/arcee-ai/Meraj-Mini
```

Upload folder using huggingface_hub

by MaziyarPanahi - opened Oct 6, 2024

base: refs/heads/main

←

from: refs/pr/1

Discussion Files changed

+151697

-0

MaziyarPanahi

Arcee AI org Oct 6, 2024

•

edited Oct 6, 2024

Upload folder using huggingface_hub

Multi commit ID: 71b71cb1cea0ea42551a562d4600cfca444f0d7216225e78de1d566924617ba8

Scheduled commits:

Upload 1 file(s) totalling 5.0G (56364a94bc336690042d1f6625b48b0d51deb0ed71ad45ac3fb5eb6ff0156874)
Upload 1 file(s) totalling 4.9G (8c1d3e50253922af21f30d08df763ea4ad5fbd89d9de11c499558afae90417fa)
Upload 1 file(s) totalling 5.0G (c3f4ecc0470b742f699f8970cd0031332aa52bb326740f30710dd826a6e5631d)
Upload 10 file(s) totalling 346.2M (2a77f22b24f18017040ad82235845e912906477b9be6ad674c85b124e43ad8f8)

This is a PR opened using the huggingface_hub library in the context of a multi-commit. PR can be commented as a usual PR. However, please be aware that manually updating the PR description, changing the PR status, or pushing new commits, is not recommended as it might corrupt the commit process. Learn more about multi-commits in this guide.

56364a94bc336690042d1f6625b48b0d51deb0ed71ad45ac3fb5eb6ff015687439998cf7

8c1d3e50253922af21f30d08df763ea4ad5fbd89d9de11c499558afae90417facf7760a6

c3f4ecc0470b742f699f8970cd0031332aa52bb326740f30710dd826a6e5631d58d0fb93

MaziyarPanahi changed pull request title from [WIP] Upload folder using huggingface_hub (multi-commit 71b71cb1cea0ea42551a562d4600cfca444f0d7216225e78de1d566924617ba8) to Upload folder using huggingface_hub Oct 6, 2024

2a77f22b24f18017040ad82235845e912906477b9be6ad674c85b124e43ad8f8a035c9a2

MaziyarPanahi

Arcee AI org Oct 6, 2024

Multi-commit is now completed! You can ping the repo owner to review the changes. This PR can now be commented or modified without risking to corrupt it.

This is a comment posted using the huggingface_hub library in the context of a multi-commit. Learn more about multi-commits in this guide.

MaziyarPanahi changed pull request status to open Oct 6, 2024

MaziyarPanahi

Arcee AI org Oct 6, 2024

create_pr=False has been passed so PR is automatically merged.

This is a comment posted using the huggingface_hub library in the context of a multi-commit. Learn more about multi-commits in this guide.

MaziyarPanahi changed pull request status to merged Oct 6, 2024

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment