Image-Text-to-Text
Transformers
Safetensors
inkling_mm_model
conversational
audio-text-to-text
Mixture of Experts
Eval Results
Instructions to use thinkingmachines/Inkling with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use thinkingmachines/Inkling with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="thinkingmachines/Inkling") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("thinkingmachines/Inkling") model = AutoModelForMultimodalLM.from_pretrained("thinkingmachines/Inkling", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Inference
- HuggingChat
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use thinkingmachines/Inkling with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "thinkingmachines/Inkling" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thinkingmachines/Inkling", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/thinkingmachines/Inkling
- SGLang
How to use thinkingmachines/Inkling with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "thinkingmachines/Inkling" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thinkingmachines/Inkling", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "thinkingmachines/Inkling" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thinkingmachines/Inkling", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use thinkingmachines/Inkling with Docker Model Runner:
docker model run hf.co/thinkingmachines/Inkling
Lucis
#3
by Ubuku - opened
- .eval_results/aime_2026.yaml +0 -7
- .eval_results/gpqa.yaml +0 -7
- .eval_results/hle.yaml +0 -7
- .eval_results/mmmu_pro.yaml +0 -7
- .eval_results/swe-bench_pro.yaml +0 -7
- .eval_results/swe-bench_verified.yaml +0 -7
- README.md +9 -10
.eval_results/aime_2026.yaml
DELETED
|
@@ -1,7 +0,0 @@
|
|
| 1 |
-
- dataset:
|
| 2 |
-
id: MathArena/aime_2026
|
| 3 |
-
task_id: MathArena/aime_2026
|
| 4 |
-
value: 97.1
|
| 5 |
-
source:
|
| 6 |
-
url: https://huggingface.co/thinkingmachines/Inkling
|
| 7 |
-
name: Model Card
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.eval_results/gpqa.yaml
DELETED
|
@@ -1,7 +0,0 @@
|
|
| 1 |
-
- dataset:
|
| 2 |
-
id: Idavidrein/gpqa
|
| 3 |
-
task_id: diamond
|
| 4 |
-
value: 87.2
|
| 5 |
-
source:
|
| 6 |
-
url: https://huggingface.co/thinkingmachines/Inkling
|
| 7 |
-
name: Model Card
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.eval_results/hle.yaml
DELETED
|
@@ -1,7 +0,0 @@
|
|
| 1 |
-
- dataset:
|
| 2 |
-
id: cais/hle
|
| 3 |
-
task_id: hle
|
| 4 |
-
value: 46
|
| 5 |
-
source:
|
| 6 |
-
url: https://huggingface.co/thinkingmachines/Inkling
|
| 7 |
-
name: Model Card
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.eval_results/mmmu_pro.yaml
DELETED
|
@@ -1,7 +0,0 @@
|
|
| 1 |
-
- dataset:
|
| 2 |
-
id: MMMU/MMMU_Pro
|
| 3 |
-
task_id: mmmu_pro_standard_10_options
|
| 4 |
-
value: 73.5
|
| 5 |
-
source:
|
| 6 |
-
url: https://huggingface.co/thinkingmachines/Inkling
|
| 7 |
-
name: Model Card
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.eval_results/swe-bench_pro.yaml
DELETED
|
@@ -1,7 +0,0 @@
|
|
| 1 |
-
- dataset:
|
| 2 |
-
id: ScaleAI/SWE-bench_Pro
|
| 3 |
-
task_id: SWE_Bench_Pro
|
| 4 |
-
value: 54.3
|
| 5 |
-
source:
|
| 6 |
-
url: https://huggingface.co/thinkingmachines/Inkling
|
| 7 |
-
name: Model Card
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.eval_results/swe-bench_verified.yaml
DELETED
|
@@ -1,7 +0,0 @@
|
|
| 1 |
-
- dataset:
|
| 2 |
-
id: SWE-bench/SWE-bench_Verified
|
| 3 |
-
task_id: swe_bench_%_resolved
|
| 4 |
-
value: 77.6
|
| 5 |
-
source:
|
| 6 |
-
url: https://huggingface.co/thinkingmachines/Inkling
|
| 7 |
-
name: Model Card
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
README.md
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
license_link: https://www.apache.org/licenses/LICENSE-2.0
|
| 4 |
pipeline_tag: image-text-to-text
|
| 5 |
tags:
|
|
@@ -7,7 +7,6 @@ tags:
|
|
| 7 |
- image-text-to-text
|
| 8 |
- audio-text-to-text
|
| 9 |
- moe
|
| 10 |
-
library_name: transformers
|
| 11 |
---
|
| 12 |
|
| 13 |
# Inkling
|
|
@@ -17,8 +16,8 @@ library_name: transformers
|
|
| 17 |
<p align="center">
|
| 18 |
<a href="https://huggingface.co/thinkingmachines/Inkling">BF16</a> |
|
| 19 |
<a href="https://huggingface.co/thinkingmachines/Inkling-NVFP4">NVFP4</a> |
|
| 20 |
-
<a href="https://tinker.thinkingmachines.ai/playground">Playground</a> |
|
| 21 |
<a href="https://github.com/thinking-machines-lab/tinker-cookbook">Tinker Cookbook</a> |
|
|
|
|
| 22 |
<a href="https://thinkingmachines.ai/model-acceptable-use-policy">Acceptable Use</a>
|
| 23 |
</p>
|
| 24 |
|
|
@@ -30,15 +29,15 @@ Inkling is a general-purpose multimodal model that accepts text, image and audio
|
|
| 30 |
|
| 31 |
## 2. Getting Started
|
| 32 |
|
| 33 |
-
|
| 34 |
|
| 35 |
Inkling supports local deployment using the following open-source libraries:
|
| 36 |
|
| 37 |
-
* SGLang ([recipe](https://docs.sglang.io/cookbook/autoregressive/ThinkingMachines/Inkling))
|
| 38 |
-
* vLLM ([recipe](https://recipes.vllm.ai/thinkingmachines/
|
| 39 |
-
* TokenSpeed ([recipe](https://lightseek.org/tokenspeed/recipes/models#Inkling))
|
| 40 |
-
* Unsloth ([recipe](https://unsloth.ai/docs/models/inkling))
|
| 41 |
-
* Huggingface ([recipe](https://hf.co/blog/thinkingmachines-inkling))
|
| 42 |
|
| 43 |
API access is also available through third party inference providers.
|
| 44 |
|
|
@@ -101,7 +100,7 @@ Inkling results are reported at effort=0.99. Comparison scores are generated Jul
|
|
| 101 |
| | IFBench | 79.8% | 81.4% | 70.2% | 76.0% | 73.3% | 76.5% | 77.1% | 63.5% | 72.7% |
|
| 102 |
| | Global-MMLU-Lite | 88.7% | 85.6% | 84.0% | 88.4% | 89.2% | 89.3% | 92.7% | 93.3% | 91.8% |
|
| 103 |
| **Vision** | | | | | | | | | | |
|
| 104 |
-
| | MMMU Pro (Standard 10) | 73.
|
| 105 |
| | Charxiv RQ | 78.1% | β | 77.5% | 80.4% | β | β | 80.2% | 86.5% | 84.7% |
|
| 106 |
| | Charxiv RQ (with python) | 82.0% | β | 78.7% | 86.7% | β | β | 89.9% | 89.4% | 87.8% |
|
| 107 |
| **Audio** | | | | | | | | | | |
|
|
|
|
| 1 |
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
license_link: https://www.apache.org/licenses/LICENSE-2.0
|
| 4 |
pipeline_tag: image-text-to-text
|
| 5 |
tags:
|
|
|
|
| 7 |
- image-text-to-text
|
| 8 |
- audio-text-to-text
|
| 9 |
- moe
|
|
|
|
| 10 |
---
|
| 11 |
|
| 12 |
# Inkling
|
|
|
|
| 16 |
<p align="center">
|
| 17 |
<a href="https://huggingface.co/thinkingmachines/Inkling">BF16</a> |
|
| 18 |
<a href="https://huggingface.co/thinkingmachines/Inkling-NVFP4">NVFP4</a> |
|
|
|
|
| 19 |
<a href="https://github.com/thinking-machines-lab/tinker-cookbook">Tinker Cookbook</a> |
|
| 20 |
+
<a href="https://tinker-docs.thinkingmachines.ai/cookbook/inkling/">Documentation</a> |
|
| 21 |
<a href="https://thinkingmachines.ai/model-acceptable-use-policy">Acceptable Use</a>
|
| 22 |
</p>
|
| 23 |
|
|
|
|
| 29 |
|
| 30 |
## 2. Getting Started
|
| 31 |
|
| 32 |
+
For accessing Inkling via Tinker: You can get started by referring to the Tinker Cookbook [here](https://github.com/thinking-machines-lab/tinker-cookbook) and associated documentation [here](https://tinker-docs.thinkingmachines.ai/cookbook/inkling/).
|
| 33 |
|
| 34 |
Inkling supports local deployment using the following open-source libraries:
|
| 35 |
|
| 36 |
+
* SGLang ([recipe](https://docs.sglang.io/cookbook/autoregressive/ThinkingMachines/Inkling), [PR](https://github.com/sgl-project/sglang/pull/31358))
|
| 37 |
+
* vLLM ([recipe](https://recipes.vllm.ai/thinkingmachines/inkling), [PR](https://github.com/vllm-project/vllm/pull/48768))
|
| 38 |
+
* TokenSpeed ([recipe](https://lightseek.org/tokenspeed/recipes/models#Inkling), [PR](https://github.com/lightseekorg/tokenspeed/pull/689))
|
| 39 |
+
* Unsloth ([recipe](https://unsloth.ai/docs/models/inkling), [PR](https://github.com/ggml-org/llama.cpp/pull/25731))
|
| 40 |
+
* Huggingface ([recipe](https://hf.co/blog/thinkingmachines-inkling), [PR](https://github.com/huggingface/transformers/pull/47347))
|
| 41 |
|
| 42 |
API access is also available through third party inference providers.
|
| 43 |
|
|
|
|
| 100 |
| | IFBench | 79.8% | 81.4% | 70.2% | 76.0% | 73.3% | 76.5% | 77.1% | 63.5% | 72.7% |
|
| 101 |
| | Global-MMLU-Lite | 88.7% | 85.6% | 84.0% | 88.4% | 89.2% | 89.3% | 92.7% | 93.3% | 91.8% |
|
| 102 |
| **Vision** | | | | | | | | | | |
|
| 103 |
+
| | MMMU Pro (Standard 10) | 73.3% | β | 75.0% | 79.0% | β | β | 82.0% | 84.2% | 83.0% |
|
| 104 |
| | Charxiv RQ | 78.1% | β | 77.5% | 80.4% | β | β | 80.2% | 86.5% | 84.7% |
|
| 105 |
| | Charxiv RQ (with python) | 82.0% | β | 78.7% | 86.7% | β | β | 89.9% | 89.4% | 87.8% |
|
| 106 |
| **Audio** | | | | | | | | | | |
|