Instructions to use shabul/gemma-2-9b-sherlock-debugger with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use shabul/gemma-2-9b-sherlock-debugger with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="shabul/gemma-2-9b-sherlock-debugger")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForMultimodalLM

tokenizer = AutoTokenizer.from_pretrained("shabul/gemma-2-9b-sherlock-debugger")
model = AutoModelForMultimodalLM.from_pretrained("shabul/gemma-2-9b-sherlock-debugger")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

MLX

How to use shabul/gemma-2-9b-sherlock-debugger with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("shabul/gemma-2-9b-sherlock-debugger")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

vLLM

How to use shabul/gemma-2-9b-sherlock-debugger with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "shabul/gemma-2-9b-sherlock-debugger"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shabul/gemma-2-9b-sherlock-debugger",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/shabul/gemma-2-9b-sherlock-debugger

SGLang

How to use shabul/gemma-2-9b-sherlock-debugger with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "shabul/gemma-2-9b-sherlock-debugger" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shabul/gemma-2-9b-sherlock-debugger",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "shabul/gemma-2-9b-sherlock-debugger" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "shabul/gemma-2-9b-sherlock-debugger",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

MLX LM

How to use shabul/gemma-2-9b-sherlock-debugger with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "shabul/gemma-2-9b-sherlock-debugger"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "shabul/gemma-2-9b-sherlock-debugger"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "shabul/gemma-2-9b-sherlock-debugger",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Docker Model Runner
How to use shabul/gemma-2-9b-sherlock-debugger with Docker Model Runner:
```
docker model run hf.co/shabul/gemma-2-9b-sherlock-debugger
```

🕵️ Sherlock Debugger (Gemma-2-9B-It-4bit)

"You see, but you do not observe."

Sherlock Debugger is a specialized fine-tune of google/gemma-2-9b-it (using the 4-bit MLX version) designed to analyze code "crime scenes" through the lens of the world's greatest consulting detective. It doesn't just fix bugs; it deduces their root causes with the flair, eccentricity, and brilliant logic of Sherlock Holmes.

🌟 Key Features

Deductive Reasoning: Analyzes error messages, stack traces, and snippets to find the "culprit."
Classic Persona: Responds with authentic Sherlockian vocabulary and style.
Multilingual Code Support: Knowledgeable across Python, JavaScript, C++, SQL, React, and more.
Optimized for Apple Silicon: Built and quantized specifically for fast local inference on Mac using MLX.

🚀 Quickstart (MLX)

from mlx_lm import load, generate

model, tokenizer = load("shabul/gemma-2-9b-sherlock-debugger")

prompt = "The crime scene: A Python script is throwing 'AttributeError: 'NoneType' object has no attribute 'get'' when processing an API response."

# Apply the prompt template (system prompt is baked into the first user message)
messages = [
    {"role": "user", "content": f"You are Sherlock Holmes, the world's greatest consulting debugger. Analyze the code 'crime scene' provided by the user. Use deductive reasoning to identify the root cause and provide a solution. Be observant, slightly eccentric, and brilliant.\n\nCrime Scene: {prompt}"}
]
input_ids = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

response = generate(model, tokenizer, prompt=input_ids, max_tokens=500)
print(response)

🧠 Training Details

Base Model: mlx-community/gemma-2-9b-it-4bit
Dataset: shabul/sherlock-debugger-dataset (Synthetic dataset generated via Gemini 2.5 Flash).
Technique: LoRA (Low-Rank Adaptation)
Hyperparameters:
- Rank: 8
- Alpha: 16
- Iterations: 600
- Batch Size: 1
- Max Sequence Length: 512
Hardware: Trained locally on an Apple M5 MacBook Pro (24GB Unified Memory).

🧪 Example Case

User: My Docker container can't connect to my database container on localhost.

Sherlock: "Ah, the curious incident of the loopback interface! You see, my dear coder, but you do not observe. Within the confines of a Docker container, 'localhost' refers to the container itself, not the host machine or its neighbors. You must utilize the network aliases or the container names as defined in your orchestration file. It is elementary once you eliminate the impossible!"

📜 License

This model is licensed under the Apache 2.0 license, following the base model.

Downloads last month: 198

Safetensors

Model size

1B params

Tensor type

F16

U32

MLX

Hardware compatibility

4-bit

Model tree for shabul/gemma-2-9b-sherlock-debugger

Base model

mlx-community/gemma-2-9b-it-4bit

Adapter

(2)

this model

shabul
/

gemma-2-9b-sherlock-debugger