webbrain-one/webbrain-compass-tiny-v2.1

WebBrain Compass Tiny v2.1

Private experimental q4f16/WebGPU release of the tested v24 candidate. Based on WebBrain Compass Tiny v2, a fine-tune of OpenBMB MiniCPM5-2B (~2.6B parameters). v2.1 is an export/runtime revision, not a new training run. The merged BF16 source remains webbrain-one/webbrain-compass-tiny-v2 at 54bcab6731939137d0489ac01463c9404dff9da8. The previous v23 release is separate and unchanged.

What improved

Only the ONNX graph's 211 MatMulNBits accuracy_level attributes changed from 4 to 2, disabling the dynamic INT8-activation path used in the pinned WebGPU runtime. Quantized weight shards, tokenizer, config and vendored runtime remain byte-identical to v23.

Check	v23	v2.1 / v24
Quantized PyTorch vs WebGPU numerical checks	2/12 pass	12/12 pass
Worst relative RMS logit difference; unchanged limit 2%	6.5274%	1.8334%
Minimum cosine; unchanged limit 0.999	0.998065500	0.999833960

Generation smoke (8 cases), bounded 16,384-total-token context/cache checks, and a cold-browser restart with model downloads disabled passed. All three cold-cache native tool-call outputs were unchanged. Numerical tests cover three prompts, prefill plus three cached steps each, with identical token prefixes. They establish runtime parity in those probes—not BF16 equivalence, universal numerical accuracy, or a percentage loss in task accuracy. INT4 quality loss remains a separate issue; a Turkish smoke response remained awkward despite passing structural checks.

Fixed next-response routing results

Metric	BF16 reference	v2.1 / v24 WebGPU
First-turn structured calls	94/100	98/100
Strict exact action	16/89	15/89
Loose tool-family match	41/89	43/89

This is not end-to-end browser-agent task success or an Online-Mind2Web result. One run: 100 first-turn cases, 89 scored scenarios and 11 predetermined skips; 194 raw responses including 5 smoke requests were retained locally. The 193 pinned input files match the BF16 reference, but precision, runtime and JS/Torch random generators differ. No output repair, model helper, fallback or result-based retry.

The full predeclared behavioral gate failed: 3 explicitly discouraged actions versus a limit of 1. These were retrying an access-blocked URL, continuing a listing before reporting available results, and inventing a pagination URL. Other declared gates passed, with zero transport errors, zero native-decoder parser errors and zero length-limited responses. The owner authorized this separate release with these findings disclosed; it is not certified for unattended consequential actions.

Exact package

• GPTQ asymmetric INT4 weights, group size 32; FP16 scales/activations/KV cache.
• Default graph: onnx/model_q4f16.onnx. Both external data files are required: model_q4f16.onnx_data and model_q4f16.onnx_data_1 (1,868,992,512 bytes combined).
• The inherited BF16 dtype and 131072 architecture field in config.json do not describe this quantized graph or establish a tested browser context limit.
• Exact tested runtime under runtime/: Transformers.js 4.2.0, ONNX Runtime Web 1.27.0, WebBrain worker/parser and dependency licences. Tested with Chrome 150.0.7871.187, NVIDIA RTX 5090. Other devices/runtimes are not verified here.
• Generation-only, last-token logits; thinking disabled. No cloud or visual model.
• Tests used a local model transport alias; this repository name is the release identifier. All 39 tested model/runtime files are mapped and hashed in provenance.
• validation/ contains results and compact audit summaries. Complete original local audit hashes are recorded; private training/calibration data, browser profiles, credentials and raw benchmark requests are not included.

Integration

1. Download the pinned commit returned with the release, using an HF account with private-repository access:

   hf download webbrain-one/webbrain-compass-tiny-v2.1 --revision <PINNED_COMMIT> --local-dir ./compass-tiny-v2.1
   

   Keep onnx/, tokenizer/config files and both external shards together. Never embed a personal/write token in an extension, webpage, source file or log. Use a trusted authenticated asset proxy or a private local mirror pinned to that same commit. Anonymous HF downloads cannot access this private release.

2. In WebBrain, choose the custom model ID webbrain-one/webbrain-compass-tiny-v2.1, device webgpu, dtype q4f16, default filename stem model. Use the supplied runtime assets. The unchanged worker has no revision option: pin the asset proxy, or explicitly pass revision at pipeline initialization in your integration. The new model ID has a separate cache key; do not alias it onto v23's cached graph.

   Equivalent library initialization, with executable runtime assets served locally:

   import { env, pipeline } from './runtime/vendor/transformers.web.js';
   import { parseMiniCpmToolCalls } from './integration/minicpm5-tool-parser.mjs';
   
   env.backends.onnx.wasm.numThreads = 1;
   env.backends.onnx.wasm.wasmPaths = {
     mjs: new URL('./runtime/vendor/ort-wasm-simd-threaded.asyncify.mjs', import.meta.url).href,
     wasm: new URL('./runtime/vendor/ort-wasm-simd-threaded.asyncify.wasm', import.meta.url).href,
   };
   // Configure your trusted authenticated asset proxy before accessing this private repo.
   const generator = await pipeline('text-generation',
     'webbrain-one/webbrain-compass-tiny-v2.1', {
       revision: '<PINNED_COMMIT>', device: 'webgpu', dtype: 'q4f16',
       session_options: { extra: {
         'ep.webgpuexecutionprovider.storageBufferCacheMode': 'simple',
       } },
     });
   try {
     const result = await generator(messages, {
       tools, do_sample: false, max_new_tokens: 256,
       tokenizer_encode_kwargs: { enable_thinking: false },
     });
     const generated = result[0].generated_text;
     const text = Array.isArray(generated) ? generated.at(-1).content : generated;
     const calls = parseMiniCpmToolCalls(text, tools);
     // Retain raw text. Validate argument schemas and user authorization before dispatch.
     // An empty calls list is not automatically successful task completion.
   } finally {
     await generator.dispose();
   }
   

   The native custom-model worker defaults to greedy generation and caps output at 256 tokens. The routing benchmark used its documented test hook (4096-token budget, temperature 0.15/0.3, seed 3407); do not claim those scores for the default UI settings. Budget input plus output within the tested 16K limit and start with short tasks.

3. Parser integration is explicit, not silently changed. The exact bundled parser under runtime/ matches 162/194 recorded native-decoder responses. It can leave Python-style False and ['green', 'amber'] as strings. The optional, separately audited integration/minicpm5-tool-parser.mjs matches 194/194 recorded responses and passed its allowlist checks; it is not automatically wired into the worker. Pass the full tool schemas so string-valued JSON bodies remain strings. The helper does not execute literals, validate every schema constraint, or grant action approval. Preserve downstream validation, confirmations and raw-output/error logging.

   CPU-only helper checks: node --test integration/parser.test.mjs.

This release remains subject to ATTRIBUTIONS.md, including the project's noncommercial-research restrictions. Private hosting does not expand usage rights.