File size: 3,600 Bytes
39fa001 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 | ---
license: apache-2.0
base_model: Qwen/Qwen2.5-Coder-1.5B-Instruct
tags:
- code
- function-calling
- tool-use
- agent
- small-language-model
datasets:
- NousResearch/hermes-function-calling-v1
language:
- en
pipeline_tag: text-generation
---
# smolcode-coder-1.5b-tools
A LoRA fine-tune of **Qwen2.5-Coder-1.5B-Instruct** that teaches the model to emit
**native `<tool_call>` function calls**, so a 1.5B *coder* model can actually drive an
agentic write β run β fix β verify loop.
Built for [**smolcode**](https://gitea.poyner.ai/sean/smolcode) β an SLM-optimized
agentic coding assistant β for the Hugging Face **Build Small** hackathon.
## Why
Out of the box, small Qwen-Coder models describe tool calls as plain-text/```json
instead of emitting the native `<tool_call>` token (id 151657) that runtimes (Ollama,
llama.cpp) parse into OpenAI-style `tool_calls` β which breaks agentic loops. This
fine-tune closes that gap on a tiny (1.5B) model: **100% native `<tool_call>` emission**
in free generation on held-out prompts (base model: 0%).
## Results
- **Native tool-call rate:** 100% (16/16 held-out prompts) β the release gate.
- **Agentic bench (smolcode pass@1, 10 tasks):** 9/10 as the entry tier of a
1.5Bβ8Bβ30B ladder, solving **7/10 entirely on its own** (2β16s each). For
comparison the all-Granite ladder (3B entry) scores 10/10 β the 1.5B carries the
same standalone load as a 2Γ-larger 3B.
- **Train loss:** 0.138 (3 epochs, assistant-only loss).
## Training
- **Base:** Qwen/Qwen2.5-Coder-1.5B-Instruct
- **Method:** bf16 LoRA (r=16, Ξ±=32) on attention + MLP projections, **plus full
training of `embed_tokens` + `lm_head`** (`modules_to_save`) β required so the model
can *output* the `<tool_call>` special token, which LoRA on attention/MLP alone
cannot. **Assistant-only loss** (loss on tool calls + final answers only).
- **Data:** NousResearch/hermes-function-calling-v1 (breadth) + synthetic smolcode
tool-use trajectories (sharpness), all rendered through the *same*
`apply_chat_template(tools=...)` used at inference β training target is byte-identical
to the served prompt (fixes the v1 train/inference template mismatch).
- **Schedule:** 3 epochs, full 2048 sequence length. Trained on Modal (A100).
## Serving β read this, two non-obvious requirements
1. **Serve via the GGUF, not the safetensors directly.** Ollama's bf16-safetensors
auto-import produces garbage (`??????`) for this model. Use the included
`smolcode-1.5b-q4_k_m.gguf` (converted with llama.cpp `convert_hf_to_gguf.py`):
```bash
ollama create smolcode-coder-1.5b:tools -f Modelfile # Modelfile is in this repo
```
2. **`repeat_penalty` / `repetition_penalty` MUST be 1.0.** The tool system prompt
literally contains the `<tool_call>` token, so any penalty > 1 suppresses the model
from emitting it (you'll see a stray token + bare JSON instead). The included
`Modelfile` sets `PARAMETER repeat_penalty 1.0`. For raw `transformers.generate`,
pass `repetition_penalty=1.0`.
With those, Ollama's `/v1/chat/completions` returns proper native `tool_calls`.
## Use (transformers)
Standard Qwen2.5 chat template with `tools=`; greedy, `repetition_penalty=1.0`. The
model responds with `<tool_call>{"name": ..., "arguments": ...}</tool_call>`.
## Files
- `model.safetensors` + tokenizer/config β the merged model (lm_head untied).
- `smolcode-1.5b-q4_k_m.gguf` β quantized GGUF for serving.
- `Modelfile` β Ollama import recipe (template + `repeat_penalty 1.0`).
## License
Apache-2.0 (inherits from the base model).
|