Instructions to use cs-552-2026-middle-west/general_knowledge_model with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use cs-552-2026-middle-west/general_knowledge_model with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="cs-552-2026-middle-west/general_knowledge_model")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("cs-552-2026-middle-west/general_knowledge_model")
model = AutoModelForCausalLM.from_pretrained("cs-552-2026-middle-west/general_knowledge_model")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use cs-552-2026-middle-west/general_knowledge_model with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "cs-552-2026-middle-west/general_knowledge_model"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cs-552-2026-middle-west/general_knowledge_model",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/cs-552-2026-middle-west/general_knowledge_model

SGLang

How to use cs-552-2026-middle-west/general_knowledge_model with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "cs-552-2026-middle-west/general_knowledge_model" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cs-552-2026-middle-west/general_knowledge_model",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "cs-552-2026-middle-west/general_knowledge_model" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cs-552-2026-middle-west/general_knowledge_model",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use cs-552-2026-middle-west/general_knowledge_model with Docker Model Runner:
```
docker model run hf.co/cs-552-2026-middle-west/general_knowledge_model
```

aryan-ahadinia commited on Jun 2

Commit

0f03fa8

verified ·

1 Parent(s): 4acb383

Upload folder using huggingface_hub

Browse files

Files changed (5) hide show

chat_template.jinja +1 -1
config.json +2 -2
generation_config.json +0 -1
model.safetensors +1 -1
training_args.bin +1 -1

chat_template.jinja CHANGED Viewed

@@ -1,5 +1,5 @@
-{%- set default_system = "You are a knowledgeable assistant answering multiple-choice general-knowledge questions. Work through the question step by step, but stay focused: reason only as much as the question needs and do not over-explain or pad your answer. End your reply with the answer on its own line as \\boxed{X}, where X is the single capital letter of the best option (e.g. \\boxed{C}). Even if you are unsure, pick the single most likely option and box it; never finish without a \\boxed{...}, and never give the answer only in words." -%}
 {%- if messages[0].role == 'system' -%}
 {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' -}}
 {%- set messages = messages[1:] -%}

+{%- set default_system = "You are a knowledgeable assistant answering multiple-choice general-knowledge questions. Think through the question step by step, then commit to an answer. Keep your reasoning factual and focused and concise enough that you always reach your final answer well within the limited budget. Do not ramble or over-explain. End your reply with the answer on its own line as \\boxed{X}, where X is the single capital letter of the best option (e.g. \\boxed{C}). Even if you are unsure, pick the single most likely option and box it; never finish without a \\boxed{...}, and never give the answer only in words." -%}
 {%- if messages[0].role == 'system' -%}
 {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' -}}
 {%- set messages = messages[1:] -%}

config.json CHANGED Viewed

@@ -4,7 +4,7 @@
   ],
   "attention_bias": false,
   "attention_dropout": 0.0,
-  "bos_token_id": 151643,
   "dtype": "bfloat16",
   "eos_token_id": 151645,
   "head_dim": 128,
@@ -48,7 +48,7 @@
   "num_attention_heads": 16,
   "num_hidden_layers": 28,
   "num_key_value_heads": 8,
-  "pad_token_id": null,
   "rms_norm_eps": 1e-06,
   "rope_parameters": {
     "rope_theta": 1000000,

   ],
   "attention_bias": false,
   "attention_dropout": 0.0,
+  "bos_token_id": null,
   "dtype": "bfloat16",
   "eos_token_id": 151645,
   "head_dim": 128,
   "num_attention_heads": 16,
   "num_hidden_layers": 28,
   "num_key_value_heads": 8,
+  "pad_token_id": 151643,
   "rms_norm_eps": 1e-06,
   "rope_parameters": {
     "rope_theta": 1000000,

generation_config.json CHANGED Viewed

@@ -1,5 +1,4 @@
 {
-  "bos_token_id": 151643,
   "do_sample": true,
   "eos_token_id": [
     151645,

 {
   "do_sample": true,
   "eos_token_id": [
     151645,

model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:40cbd67685c0c01cceec78a985dfa868ee8e37d48c79b6c1352583490056bf1c
 size 3441185608

 version https://git-lfs.github.com/spec/v1
+oid sha256:6aa3299cc44ad75416a1fba3e7a6028447977aaea4965cfed5883ac01504c961
 size 3441185608

training_args.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f1379a36761bd362916511760963f0f7cff6d6cfe4c5413908355c290289db80
 size 5329

 version https://git-lfs.github.com/spec/v1
+oid sha256:cf1ad2064860908a5884f6539f502ce15bc244f496ef37ce8c699d5031ec3b91
 size 5329