README.md

---
license: apache-2.0
language:
- en
tags:
- slm
- arithmetic
- math
- causal-lm
- text-generation
- custom_code
- safetensors
library_name: transformers
pipeline_tag: text-generation
metrics:
- accuracy
model-index:
- name: Arithmetic-SLM
  results:
  - task:
      type: text-generation
      name: Arithmetic continuation
    dataset:
      type: AxiomicLabs/ArithMark-2.0
      name: ArithMark-2
    metrics:
    - type: accuracy
      name: Overall
      value: 78.6
datasets:
- WhirlwindAI/Arithmetic
---


![image](https://cdn-uploads.huggingface.co/production/uploads/6975505c60cf607407afe2c0/g4yF4mQR39XYMoUehSTsX.png)

# Scores

<div align="center">

<table>
  <tr>
    <th align="center">Model</th>
    <th align="center">Parameters</th>
    <th align="center">Overall Score</th>
  </tr>
  <tr>
    <td align="center"><code>Qwen/Qwen2.5-Math-1.5B</code></td>
    <td align="center">1.54B</td>
    <td align="center"><strong>82.08%</strong></td>
  </tr>
  <tr>
    <td align="center"><code>WhirlwindAI/Arithmetic-SLM</code></td>
    <td align="center">31.70M</td>
    <td align="center"><strong>78.60%</strong></td>
  </tr>
  <tr>
    <td align="center"><code>Qwen/Qwen2.5-3B</code></td>
    <td align="center">3.09B</td>
    <td align="center">78.44%</td>
  </tr>
  <tr>
    <td align="center"><code>Qwen/Qwen2.5-1.5B</code></td>
    <td align="center">1.54B</td>
    <td align="center">77.72%</td>
  </tr>
  <tr>
    <td align="center"><code>Qwen/Qwen2.5-Coder-1.5B</code></td>
    <td align="center">1.54B</td>
    <td align="center">74.88%</td>
  </tr>
  <tr>
    <td align="center"><code>HuggingFaceTB/SmolLM2-1.7B</code></td>
    <td align="center">1.71B</td>
    <td align="center">66.12%</td>
  </tr>
  <tr>
    <td align="center"><code>Qwen/Qwen2.5-0.5B</code></td>
    <td align="center">494M</td>
    <td align="center">63.04%</td>
  </tr>
  <tr>
    <td align="center"><code>facebook/MobileLLM-R1-140M-base</code></td>
    <td align="center">140M</td>
    <td align="center">53.88%</td>
  </tr>
  <tr>
    <td align="center"><code>SupraLabs/Supra-50M-Base</code></td>
    <td align="center">52M</td>
    <td align="center">27.12%</td>
  </tr>
</table>

</div>

# Arithmetic-SLM

Arithmetic-SLM is a small language model specialized for arithmetic continuation. It is designed to be highly efficient on numerical operations with mostly two-digit numbers in patterns such as:

```text
a op b op c op d
```

where:

```text
op = +, -, *, /
```

The goal is not to make a general chatbot. The goal is to train a compact model that can learn arithmetic patterns, operator priority, parentheses, and numerical continuation with very few parameters.


## Calculation Patterns

### 1. Single operation

```text
59 + 45 = 104
26 - 2 = 24
12 * 7 = 84
84 / 12 = 7
```

### 2. Two operations without parentheses

```text
16 + 4 * 3 = 28
95 - 8 * 0 = 95
84 / 12 - 3 = 4
```

### 3. Two operations with parentheses

```text
(16 / 4) + 44 = 48
(10 + 28) * 3 = 114
1 * (16 + 28) = 44
```

### 4. Three operations without parentheses

```text
3 * 9 + 12 / 1 = 39
60 + 49 - 18 + 8 = 99
43 + 10 * 2 - 8 = 55
```

### 5. Three operations with parentheses

```text
(132 / 12) + (46 - 15) = 42
(46 + 34) - (1 + 7) = 72
(21 + 27) * (14 - 7) = 336
```

### 6. Decimal arithmetic

```text
0.5 * 0.5 = 0.25
1 / 10 = 0.1
7 / 2 = 3.5
```

## Example Outputs with `inference.py`

### Example 1 — Raw arithmetic prompt

```bash
python3 inference.py \
  --model WhirlwindAI/Arithmetic-SLM \
  --prompt "59 + 45 =" \
  --max-new-tokens 32 \
  --temperature 0.6 \
  --top-k 50 \
  --top-p 0.97 \
  --print-full
```

Expected style:

```text
59 + 45 = 104
```

### Example 2 — Production `/no think` format

```bash
python3 inference.py \
  --model WhirlwindAI/Arithmetic-SLM \
  --prompt "0.5 * 0.5 =" \
  --no-think \
  --max-new-tokens 48 \
  --temperature 0.6 \
  --top-k 50 \
  --top-p 0.97 \
  --repetition-penalty 1 \
  --frequency-penalty 0.0 \
  --no-repeat-ngram-size 0 \
  --seed -1 \
  --print-full
```

Example output:

```text
[IM_START]user
0.5 * 0.5 = /no think[IM_END]
[IM_START]assistant
<think>
</think>
0.5 * 0.5 = 0.25[IM_END]
```

### Example 3 — Operator priority

```bash
python3 inference.py \
  --model WhirlwindAI/Arithmetic-SLM \
  --prompt "8 * 5 + 4 / 4 =" \
  --no-think \
  --max-new-tokens 48 \
  --temperature 0.6 \
  --top-k 50 \
  --top-p 0.97 \
  --print-full
```

Expected style:

```text
8 * 5 + 4 / 4 = 41
```

### Example 4 — Parentheses

```bash
python3 inference.py \
  --model WhirlwindAI/Arithmetic-SLM \
  --prompt "(85 - 45) + 56 =" \
  --no-think \
  --max-new-tokens 48 \
  --temperature 0.5 \
  --top-k 40 \
  --top-p 0.95 \
  --print-full
```

Expected style:

```text
(85 - 45) + 56 = 96
```

### Example 5 — Three-operation expression

```bash
python3 inference.py \
  --model WhirlwindAI/Arithmetic-SLM \
  --prompt "3 * 9 + 12 / 1 =" \
  --no-think \
  --max-new-tokens 48 \
  --temperature 0.4 \
  --top-k 20 \
  --top-p 0.85 \
  --print-full
```

Expected style:

```text
3 * 9 + 12 / 1 = 39
```

## Next Research Directions

We will continue improving our dataset engineering, but more importantly, we want to teach the model what most models are never explicitly taught:

- **Binary calculation:** Neural Application Binary Interface, or **NABI**, with 16-bit numerical structures, including floats.
- **FP16 to BASE-65,536 conversion:** a `float16` value is represented by 2 bytes, meaning 65,536 possible bit patterns. Base 65,536 also contains 65,536 possible integer values, making exact bit-level mapping possible.
- **Dot-product learning:** explicit learning of scalar products on `float16` vectors with 16, 8, 4, and 2 dimensions.
- **Learning the dynamics of its own learning:** training the model to predict its own weights and gradients over time, including its own gradient descent dynamics.

This project does not claim to be a revolution.

It is an experiment in making small models learn precise arithmetic, numerical structure, and eventually parts of their own learning dynamics.

**By Science AND FOR SCIENCE <3**