scienceverse/bibr-paper-classifier

bibr OECD/paper-type classifier — DeepSeek teacher, 2026-07-19

This is the DeepSeek-v4-Flash-teacher MiniLM candidate for bibr's multitask scientific-paper classifier. It predicts OECD Level 1, OECD Level 2, and paper type from title plus abstract.

Architecture

• Encoder: sentence-transformers/all-MiniLM-L6-v2
• Input template: title_abstract_v1
• Maximum input length: 256 tokens
• Heads: OECD L1, OECD L2, paper type
• Model SHA-256: 7a46c595cf3bb8e1eff39303786e8b7d16c95ea87f9f06b12d4a753f0be1e580

Training provenance

• Training corpus rows: 122,363
• Deterministic split: 91,771 train / 12,237 validation / 18,355 test
• DeepSeek teacher changed 16,220 L1 labels relative to the matched OpenAlex baseline.
• Teacher-labelled rows are training supervision, not evaluation gold.

Evaluation

On the frozen 1,500-row Phase-A panel:

Model	Accuracy	Macro-F1
Matched OpenAlex MiniLM baseline	0.7333	0.7280
This checkpoint	0.8120	0.8096
Previously shipped SPECTER2 checkpoint	0.6887	0.6874

The matched MiniLM gain was +0.0787 accuracy and +0.0816 macro-F1. The paired correctness table contained 158 teacher-only correct rows and 40 baseline-only correct rows; exact McNemar p-value was 8.687e-18.

The Phase-A reference is explicitly provisional_unadjudicated_codex_panel. It is not human-adjudicated gold, so these figures support candidate selection but not a final scientific-quality claim.

Bundle contract

The bibr loader requires:

• model.safetensors
• tokenizer.json
• tokenizer_config.json
• label_maps.json
• inference_config.json

test_metrics.json, test_l1_diagnostics.json, and phaseA_metrics.json provide training and provisional-panel diagnostics.