Instructions to use Dan-Carter/Affine-t3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Dan-Carter/Affine-t3 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Dan-Carter/Affine-t3")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForMultimodalLM

tokenizer = AutoTokenizer.from_pretrained("Dan-Carter/Affine-t3")
model = AutoModelForMultimodalLM.from_pretrained("Dan-Carter/Affine-t3")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Dan-Carter/Affine-t3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Dan-Carter/Affine-t3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Dan-Carter/Affine-t3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Dan-Carter/Affine-t3

SGLang

How to use Dan-Carter/Affine-t3 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Dan-Carter/Affine-t3" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Dan-Carter/Affine-t3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Dan-Carter/Affine-t3" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Dan-Carter/Affine-t3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Dan-Carter/Affine-t3 with Docker Model Runner:
```
docker model run hf.co/Dan-Carter/Affine-t3
```

Try K2-Think · K2 Technical Report

K2-Think is a 32 billion parameter open-weights general reasoning model with strong performance in competitive mathematical problem solving.

Quickstart

Transformers

You can use K2-Think with Transformers. If you use transformers.pipeline, it will apply the chat template automatically. If you use model.generate directly, you need to apply the chat template mannually.

from transformers import pipeline
import torch

model_id =  "LLM360/K2-Think"

pipe = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype="auto",
    device_map="auto",
)

messages = [
    {"role": "user", "content": "what is the next prime number after 2600?"},
]

outputs = pipe(
    messages,
    max_new_tokens=32768,
)
print(outputs[0]["generated_text"][-1])

Evaluation & Performance

Detailed evaluation results are reported in out Tech Report

Benchmarks (pass@1, average over 16 runs)

Domain	Benchmark	K2-Think
Math	AIME 2024	90.83
Math	AIME 2025	81.24
Math	HMMT 2025	73.75
Math	OMNI-Math-HARD	60.73
Code	LiveCodeBench v5	63.97
Science	GPQA-Diamond	71.08

Inference Speed

We deploy K2-THINK on Cerebras Wafer-Scale Engine (WSE) systems, leveraging the world’s largest processor and speculative decoding to achieve unprecedented inference speeds for our 32B reasoning system.

Platform	Throughput (tokens/sec)	Example: 32k-token response (time)
Cerebras WSE (our deployment)	~2,000	~16 s
Typical H100/H200 GPU setup	~200	~160 s

Safety Evaluation

Aggregated across four safety dimensions (Safety-4):

Aspect	Macro-Avg
High-Risk Content Refusal	0.83
Conversational Robustness	0.89
Cybersecurity & Data Protection	0.56
Jailbreak Resistance	0.72
Safety-4 Macro (avg)	0.75

Citation

@techreport{k2think2025,
  title   = {K2-Think: A Parameter-Efficient Reasoning System},
  author  = {Zhoujun Cheng and Richard Fan and Shibo Hao and Taylor W. Killian and Haonan Li and Suqi Sun and Hector Ren and Alexander Moreno and Daqian Zhang and Tianjun Zhong and Yuxin Xiong and Yuanzhe Hu and Yutao Xie and Xudong Han and Yuqi Wang and Varad Pimpalkhute and Yonghao Zhuang and Aaryamonvikram Singh and Xuezhi Liang and Anze Xie and Jianshu She and Desai Fan and Chengqian Gao and Liqun Ma and Mikhail Yurochkin and John Maggs and Xuezhe Ma and Guowei He and Zhiting Hu and Zhengzhong Liu and Eric P. Xing},
  year    = {2025},
  institution = {Institute of Foundation Models, Mohamed bin Zayed University of Artificial Intelligence},
  url     = {https://k2think-about.pages.dev/assets/tech-report/K2-Think_Tech-Report.pdf}
}

Downloads last month: 1

Safetensors

Model size

33B params

Tensor type

BF16

Model tree for Dan-Carter/Affine-t3

Base model

Qwen/Qwen2.5-32B

Finetuned

(124)

this model