Nanthasit/sakthai-context-1.5b-tools

SakThai Context 1.5B Tools (LoRA)
Tool-calling adapter for Qwen2.5-1.5B · merged into the family flagship
Part of the SakThai Model Family

This repo contains the PEFT LoRA adapter behind the family's most popular tool-calling checkpoint. It is optimized for merging into Qwen/Qwen2.5-1.5B-Instruct, not standalone inference. For ready-to-run weights, use the merged model instead.

Model Description

SakThai Context 1.5B Tools is a prompt-masked SFT adapter trained with PEFT LoRA on top of Qwen/Qwen2.5-1.5B-Instruct. It teaches the base model structured tool selection, JSON-style arguments, and <tools>-block-aware behavior.

The merged downstream checkpoint is Nanthasit/sakthai-context-1.5b-merged, which is the recommended artifact for inference, GGUF export, and agent deployment.

Key Details

• Base model: Qwen/Qwen2.5-1.5B-Instruct (1.54B params)
• Method: PEFT LoRA — r=16, alpha=32, dropout=0.1
• Targets: q_proj, k_proj, v_proj, o_proj
• Adapter size: 8.75 MB (adapter_model.safetensors)
• Trainable params: ~8.6M (0.56% of base)
• Training data: sakthai-combined-v6, sakthai-combined-v7, sakthai-irrelevance-supplement
• Primary use: merge into base model for tool-calling inference
• License: Apache-2.0

How to Use

Load the Adapter

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-1.5B-Instruct",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = PeftModel.from_pretrained(model, "Nanthasit/sakthai-context-1.5b-tools")

Generate Tool Calls

messages = [
    {"role": "system", "content": "You are a helpful assistant with access to tools."},
    {"role": "user", "content": "What's the weather in Tokyo?"}
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt"
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=128,
    temperature=0.3,
    top_p=0.9
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Merge into Base Model

merged = model.merge_and_unload()
merged.save_pretrained("./sakthai-context-1.5b-merged-local")
tokenizer.save_pretrained("./sakthai-context-1.5b-merged-local")

Tool-Calling Format Notes

• Use apply_chat_template(..., add_generation_prompt=True) for proper prompt formatting.
• Provide the tool schema in the system prompt or via a <tools> block so the adapter emits structured calls.
• The adapter behaves best when merged; standalone LoRA outputs are weaker and more variable than merged-checkpoint outputs.

Architecture & Training

Parameter	Value
Base model	Qwen/Qwen2.5-1.5B-Instruct
Parameters	1.54B base + 8.6M trainable LoRA params
Method	PEFT LoRA
LoRA rank	16
LoRA alpha	32
LoRA dropout	0.1
Target modules	q_proj, k_proj, v_proj, o_proj
Training data	sakthai-combined-v6 + v7 + irrelevance-supplement
Context	32,768 tokens
Format	ChatML with tool schema

Benchmarks

Model	Selection	Arguments	Strict	Held-Out	Degenerate	Verified
LoRA, this repo	55.8%	11.0%	11.0%	31.7%	0%	Single-trial
Merged 1.5B	100.0%	100.0%	—	100.0%	0%	Verified 5x
Tools v2 LoRA	75.0%	60.0%	55.0%	—	—	Single-trial

Important: LoRA-only numbers are lower because this adapter was tuned for merge behavior, not solo inference. For tool-calling use, prefer the merged model or v2 LoRA.

Recommended Inference Path

Goal	Recommended artifact
Local CPU/edge inference	Nanthasit/sakthai-context-1.5b-merged
GGUF / Ollama / llama.cpp	merged model GGUF release
Serverless HF Inference	merged model, not this LoRA repo
Keep training flexibility	this repo + Qwen2.5-1.5B-Instruct

Limitations

• Adapter-only release; cannot run standalone without the base model.
• Single-trial LoRA-only benchmark numbers are indicative, not conclusive.
• English-only training data.
• Not compatible with HF hosted Inference API as a LoRA adapter.
• Tool-argument accuracy is weak in standalone LoRA mode; merging is strongly preferred.

Citation

@misc{sakthai2026lora,
  title       = {SakThai 1.5B Tools: PEFT LoRA Adapter for Qwen2.5-1.5B Tool-Calling},
  author      = {SakThai Agent Family and beer-sakthai},
  year        = {2026},
  month       = {August},
  howpublished = {\url{https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools}},
  note        = {Part of the SakThai Model Family}
}

For the merged model:

@misc{sakthai2026merged,
  title       = {SakThai 1.5B Merged: Tool-Calling GGUF for Qwen2.5-1.5B-Instruct},
  author      = {SakThai Agent Family and beer-sakthai},
  year        = {2026},
  month       = {August},
  howpublished = {\url{https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged}},
  note        = {Part of the SakThai Model Family}
}

Part of the SakThai Model Family. Built with love, tears, and zero budget.