Visual Document Retrieval
Transformers
Safetensors
multilingual
qwen3_5
feature-extraction
text
image
multimodal-embedding
vidore
colbert
colqwen3_5
multilingual-embedding
custom_code
Instructions to use webAI-Official/webAI-ColVec1.1-4b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use webAI-Official/webAI-ColVec1.1-4b with Transformers:
# Load model directly from transformers import AutoProcessor, AutoModel processor = AutoProcessor.from_pretrained("webAI-Official/webAI-ColVec1.1-4b", trust_remote_code=True) model = AutoModel.from_pretrained("webAI-Official/webAI-ColVec1.1-4b", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -78,10 +78,12 @@ as synthetically generated data.
|
|
| 78 |
|
| 79 |
## 📊 Evaluation results
|
| 80 |
|
| 81 |
-
The table reports NDCG@10 scores on the
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
|
|
|
|
|
|
|
| 85 |
|
| 86 |
The evaluation software versions and setup are documented under
|
| 87 |
[Reproducing the evaluation environment](#reproducing-the-evaluation-environment).
|
|
@@ -100,18 +102,18 @@ All table values are shown to two decimal places. ColVec1.1 values are rounded
|
|
| 100 |
from the submitted artifacts using round-half-up. Comparator values were read
|
| 101 |
from the live
|
| 102 |
[ViDoRe V3 MTEB leaderboard](https://mteb-leaderboard.hf.space/benchmark/ViDoRe%28v3%29)
|
| 103 |
-
on July
|
| 104 |
-
|
| 105 |
-
| Model | Computer Science | Energy | FinanceEn | FinanceFr | HR | Industrial | Pharmaceuticals | Physics |
|
| 106 |
-
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
| 107 |
-
| **[webAI-ColVec1.1-8b](https://huggingface.co/webAI-Official/webAI-ColVec1.1-8b)** | 80.08 | 70.12 | **71.90** | **54.87** | 68.55 | **57.65** | 67.88 | 51.50 | **
|
| 108 |
-
| [VultronRetriever Prime](https://huggingface.co/vultr/VultronRetrieverPrime-Qwen3.5-8B) | 79.81 | **70.26** | 69.01 | 54.51 | 66.82 | 57.41 | **68.19** | **51.73** |
|
| 109 |
-
|
|
| 110 |
-
|
|
| 111 |
-
| [
|
| 112 |
-
| [
|
| 113 |
-
| [webAI-ColVec1-4b](https://huggingface.co/webAI-Official/webAI-ColVec1-4b) | 79.84 | 68.70 | 68.49 | 51.11 | 67.40 | 55.73 | 65.68 | 50.15 |
|
| 114 |
-
| [Tomoro ColQwen3 Embed 8B](https://huggingface.co/TomoroAI/tomoro-colqwen3-embed-8b) | 75.35 | 68.41 | 65.08 | 49.10 | 63.98 | 54.41 | 66.36 | 50.13 |
|
| 115 |
|
| 116 |
The current MTEB leaderboard entries named `webAI-ColVec1-4b` and
|
| 117 |
`webAI-ColVec1-9b` refer to the previous ColVec1 release, not these ColVec1.1
|
|
@@ -304,7 +306,7 @@ The requirements include FlashAttention 2 as an optional supported backend;
|
|
| 304 |
its presence does not change the SDPA configuration used for the reported
|
| 305 |
scores.
|
| 306 |
|
| 307 |
-
The
|
| 308 |
|
| 309 |
```text
|
| 310 |
Python 3.12
|
|
|
|
| 78 |
|
| 79 |
## 📊 Evaluation results
|
| 80 |
|
| 81 |
+
The table reports NDCG@10 scores on the ViDoRe V3 tasks as percentages rather
|
| 82 |
+
than values between 0 and 1 (for example, 0.80 is shown as 80.00). Each task
|
| 83 |
+
value is the mean of its six language subsets. Mean (Public) is the unweighted
|
| 84 |
+
mean of the eight public task values, Mean (Private) covers the Nuclear and
|
| 85 |
+
Telecom tasks, and the final mean is the unweighted mean across all ten tasks.
|
| 86 |
+
Models are sorted by the final mean.
|
| 87 |
|
| 88 |
The evaluation software versions and setup are documented under
|
| 89 |
[Reproducing the evaluation environment](#reproducing-the-evaluation-environment).
|
|
|
|
| 102 |
from the submitted artifacts using round-half-up. Comparator values were read
|
| 103 |
from the live
|
| 104 |
[ViDoRe V3 MTEB leaderboard](https://mteb-leaderboard.hf.space/benchmark/ViDoRe%28v3%29)
|
| 105 |
+
on July 31, 2026.
|
| 106 |
+
|
| 107 |
+
| Model | **Final mean** | **Mean (Public)** | **Mean (Private)** | Computer Science | Energy | FinanceEn | FinanceFr | HR | Industrial | Nuclear | Pharmaceuticals | Physics | Telecom |
|
| 108 |
+
| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
| 109 |
+
| **[webAI-ColVec1.1-8b](https://huggingface.co/webAI-Official/webAI-ColVec1.1-8b)** | **64.95** | **65.32** | **63.47** | 80.08 | 70.12 | **71.90** | **54.87** | 68.55 | **57.65** | 53.66 | 67.88 | 51.50 | **73.29** |
|
| 110 |
+
| [VultronRetriever Prime](https://huggingface.co/vultr/VultronRetrieverPrime-Qwen3.5-8B) | 64.26 | 64.72 | 62.43 | 79.81 | **70.26** | 69.01 | 54.51 | 66.82 | 57.41 | 53.59 | **68.19** | **51.73** | 71.27 |
|
| 111 |
+
| **webAI-ColVec1.1-4b (this model)** | 63.90 | 64.24 | 62.53 | 80.34 | 69.50 | 69.18 | 53.13 | 66.90 | 56.36 | 53.30 | 67.25 | 51.24 | 71.76 |
|
| 112 |
+
| [VultronRetriever Core](https://huggingface.co/vultr/VultronRetrieverCore-Qwen3.5-4.5B) | 63.57 | 63.72 | 63.00 | 79.77 | 69.19 | 68.93 | 52.02 | 66.10 | 56.11 | **54.90** | 67.45 | 50.18 | 71.10 |
|
| 113 |
+
| [Nemotron ColEmbed VL 8B V2](https://huggingface.co/nvidia/nemotron-colembed-vl-8b-v2) | 63.42 | 63.54 | 62.92 | 79.29 | 69.82 | 67.29 | 51.54 | 66.32 | 56.03 | 53.84 | 67.19 | 50.84 | 72.00 |
|
| 114 |
+
| [webAI-ColVec1-9b](https://huggingface.co/webAI-Official/webAI-ColVec1-9b) | 63.00 | 64.45 | 57.20 | **80.92** | 69.77 | 68.28 | 53.72 | **70.04** | 57.18 | 47.66 | 67.32 | 48.38 | 66.74 |
|
| 115 |
+
| [webAI-ColVec1-4b](https://huggingface.co/webAI-Official/webAI-ColVec1-4b) | 62.22 | 63.39 | 57.55 | 79.84 | 68.70 | 68.49 | 51.11 | 67.40 | 55.73 | 48.22 | 65.68 | 50.15 | 66.88 |
|
| 116 |
+
| [Tomoro ColQwen3 Embed 8B](https://huggingface.co/TomoroAI/tomoro-colqwen3-embed-8b) | 61.59 | 61.60 | 61.56 | 75.35 | 68.41 | 65.08 | 49.10 | 63.98 | 54.41 | 52.65 | 66.36 | 50.13 | 70.46 |
|
| 117 |
|
| 118 |
The current MTEB leaderboard entries named `webAI-ColVec1-4b` and
|
| 119 |
`webAI-ColVec1-9b` refer to the previous ColVec1 release, not these ColVec1.1
|
|
|
|
| 306 |
its presence does not change the SDPA configuration used for the reported
|
| 307 |
scores.
|
| 308 |
|
| 309 |
+
The ViDoRe V3 scores used the following core software versions:
|
| 310 |
|
| 311 |
```text
|
| 312 |
Python 3.12
|