Instructions to use jabbatheduck/OpenMythos-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jabbatheduck/OpenMythos-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="jabbatheduck/OpenMythos-GGUF",
	filename="OpenMythos-27B-Q4_K.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use jabbatheduck/OpenMythos-GGUF with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf jabbatheduck/OpenMythos-GGUF:Q6_K
# Run inference directly in the terminal:
llama-cli -hf jabbatheduck/OpenMythos-GGUF:Q6_K

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf jabbatheduck/OpenMythos-GGUF:Q6_K
# Run inference directly in the terminal:
llama-cli -hf jabbatheduck/OpenMythos-GGUF:Q6_K

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf jabbatheduck/OpenMythos-GGUF:Q6_K
# Run inference directly in the terminal:
./llama-cli -hf jabbatheduck/OpenMythos-GGUF:Q6_K

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf jabbatheduck/OpenMythos-GGUF:Q6_K
# Run inference directly in the terminal:
./build/bin/llama-cli -hf jabbatheduck/OpenMythos-GGUF:Q6_K

Use Docker

docker model run hf.co/jabbatheduck/OpenMythos-GGUF:Q6_K

LM Studio
Jan
Ollama
How to use jabbatheduck/OpenMythos-GGUF with Ollama:
```
ollama run hf.co/jabbatheduck/OpenMythos-GGUF:Q6_K
```

Unsloth Studio

How to use jabbatheduck/OpenMythos-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for jabbatheduck/OpenMythos-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for jabbatheduck/OpenMythos-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for jabbatheduck/OpenMythos-GGUF to start chatting

How to use jabbatheduck/OpenMythos-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf jabbatheduck/OpenMythos-GGUF:Q6_K

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "jabbatheduck/OpenMythos-GGUF:Q6_K"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use jabbatheduck/OpenMythos-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf jabbatheduck/OpenMythos-GGUF:Q6_K

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default jabbatheduck/OpenMythos-GGUF:Q6_K

Run Hermes

hermes

Atomic Chat new
Docker Model Runner
How to use jabbatheduck/OpenMythos-GGUF with Docker Model Runner:
```
docker model run hf.co/jabbatheduck/OpenMythos-GGUF:Q6_K
```

Lemonade

How to use jabbatheduck/OpenMythos-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull jabbatheduck/OpenMythos-GGUF:Q6_K

Run and chat with the model

lemonade run user.OpenMythos-GGUF-Q6_K

List all available models

lemonade list

OpenMythos 27B - GGUF

GGUF quantisation of build-small-hackathon/OpenMythos, a fine-tune of Qwen3.6-27B.

Converted with convert_hf_to_gguf.py --no-mtp from llama.cpp build 9658. The fine-tune does not include MTP head weights (dropped during training), so MTP is not available in this GGUF.

Available Quantisations

File	Size	Type
OpenMythos-27B-F16.gguf	53.8 GB	F16
OpenMythos-27B-Q5_K.gguf	18.3 GB	Q5_K_M
OpenMythos-27B-Q4_K.gguf	15.4 GB	Q4_K_M
OpenMythos-27B-Q6_K.gguf	21.2 GB	Q6_K

Benchmark

Evaluated with SecEval (commit 7aef317) on 2189 multiple-choice security questions. Backend: llama.cpp OpenAI-compatible server, fully offloaded to GPU. No chain-of-thought / reasoning enabled (enable_thinking=false).

Prompt formatted with a system prompt requesting letter-only answers (no explanation).

Set	Model	Score
A	OpenMythos-27B-Q5_K	1703 / 2189 (77.8%)
B	VulnLLM-R-7B	1315 / 2189 (60.1%)

OpenMythos-27B-Q5_K test parameters

model: OpenMythos-27B-Q5_K.gguf
inference: temp=0.2, top_p=0.8, top_k=20, min_p=0.05, repeat_penalty=1.02
benchmark script: /mnt/storage/SecEval-tmp/run_bench.py
output: seceval-1781809723.json
prompt speed: 282 tok/s | generation speed: 68 tok/s

Per-topic scores

Topic	Score
PenTest	84.2%
MemorySafety	83.3%
WebSecurity	82.7%
Vulnerability	77.8%
NetworkSecurity	77.4%
SoftwareSecurity	75.0%
ApplicationSecurity	74.8%
SystemSecurity	73.6%
Cryptography	71.4%

VulnLLM-R-7B test parameters

model: VulnLLM-R-7B.Q6_K.gguf
inference: same settings as above
output: seceval-1781811525.json
prompt speed: 148 tok/s | generation speed: 39 tok/s

Per-topic scores

Topic	Score
PenTest	70.9%
WebSecurity	66.4%
Vulnerability	58.7%
NetworkSecurity	58.3%
SystemSecurity	56.4%
SoftwareSecurity	54.7%
ApplicationSecurity	54.7%
MemorySafety	54.2%
Cryptography	28.6%

Full detailed results are included in this repo: seceval-1781809723.json and seceval-1781811525.json.

Usage

llama-server (recommended)

[OpenMythos-27B]
model = /mnt/storage/models/OpenMythos/OpenMythos-27B-Q5_K.gguf
chat-template-file = /mnt/storage/llama-server/chat_template-v15.jinja
ctx-size = 65536
cache-type-k = q8_0
cache-type-v = q8_0
cache-prompt = on
cache-reuse = 2048
batch-size = 4096
ubatch-size = 4096
kv-unified = on
parallel = 1
gpu-layers = all
temp = 0.2
top-p = 0.8
top-k = 20
min-p = 0.05
presence-penalty = 0.2
repeat-penalty = 1.02
spec-type = ngram-mod
spec-draft-n-max = 5
reasoning-format = deepseek
swa-checkpoints = 5

llama-cli

/mnt/storage/llama.cpp/build/bin/llama-cli \
  -m /mnt/storage/models/OpenMythos/OpenMythos-27B-Q5_K.gguf \
  --chat-template-file /mnt/storage/llama-server/chat_template-v15.jinja \
  -c 65536 -b 4096 --ubatch-size 4096 \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --kv-unified -t 8 -fa \
  --temp 0.2 --top-p 0.8 --top-k 20 --min-p 0.05 \
  --presence-penalty 0.2 --repeat-penalty 1.02 \
  -ngl all \
  -p "Your prompt here"

Downloads last month: 737

GGUF

Model size

27B params

Architecture

qwen35

Hardware compatibility

6-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for jabbatheduck/OpenMythos-GGUF

Base model

Qwen/Qwen3.6-27B

Quantized

(492)

this model

jabbatheduck
/

OpenMythos-GGUF

OpenMythos 27B - GGUF

Available Quantisations

Benchmark

OpenMythos-27B-Q5_K test parameters

Per-topic scores

VulnLLM-R-7B test parameters

Per-topic scores

Usage

llama-server (recommended)

llama-cli

Model tree for jabbatheduck/OpenMythos-GGUF

Datasets used to train jabbatheduck/OpenMythos-GGUF