Instructions to use athirdpath/Iambe-20b-DARE-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use athirdpath/Iambe-20b-DARE-v2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="athirdpath/Iambe-20b-DARE-v2")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("athirdpath/Iambe-20b-DARE-v2")
model = AutoModelForCausalLM.from_pretrained("athirdpath/Iambe-20b-DARE-v2")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use athirdpath/Iambe-20b-DARE-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "athirdpath/Iambe-20b-DARE-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "athirdpath/Iambe-20b-DARE-v2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/athirdpath/Iambe-20b-DARE-v2

SGLang

How to use athirdpath/Iambe-20b-DARE-v2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "athirdpath/Iambe-20b-DARE-v2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "athirdpath/Iambe-20b-DARE-v2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "athirdpath/Iambe-20b-DARE-v2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "athirdpath/Iambe-20b-DARE-v2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use athirdpath/Iambe-20b-DARE-v2 with Docker Model Runner:
```
docker model run hf.co/athirdpath/Iambe-20b-DARE-v2
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Strange quirk: This model seems to need a context size of EXACTLY 4096 ONLY. I'm assuming this is a dares_ties effect?

Iambe-20b-DARE-v2

Alpaca prompt formatting

Description

Named after a charming daughter of Echo and Pan in Greek myth, Iambe-20b-DARE-v2 is an improved DARE merge building on my recent experiments.

Iambe is intended to have the best realistically possible understanding of anatomy and of a scene's state for a 20b merge, while remaining personable and authentic in "voice".

Update Methodology

Noromaid and the general "no-robots" vibe didn't come through like I'd hoped in v1. My hypothesis is that the "soul" MythoMax and Noromaid have is probably distributed widely over many low-value deltas, due to the "ephemeral" nature of such a thing.

My old base model was likely giving DARE conniption fits, so I replaced that with a truly vanilla 20b base model.

CleverGirl was updated to the DARE version, as Sir Hillary said, simply because it was there.

Without a large base of dare_ties models to compare to, I'm basically feeling my way through this intuitively, so here's to good results!

Recipe

merge_method: dare_ties

base_model: athirdpath/BigLlama-20b-v1.1
model: Noromaid-20b-v0.1.1

weight: 0.38 / density: 0.60
model: athirdpath/athirdpath/Eileithyia-20b

weight: 0.22 / density: 0.40
model: athirdpath/CleverGirl-20b-Blended-v1.1-DARE

weight: 0.40 / density: 0.33

int8_mask: true

dtype: bfloat16

Downloads last month: 86

Safetensors

Model size

20B params

Tensor type

BF16

Model tree for athirdpath/Iambe-20b-DARE-v2

Quantizations

2 models

Collection including athirdpath/Iambe-20b-DARE-v2

Roleplay Deployment Models

Collection

Models I think are cool and worth using. A lot of what I make is intended only as a part in a further merge or as a test, these are the others. • 13 items • Updated Sep 3, 2024 • 3