| # 📗 SPECTER2 – Social Sciences Classifier (Binary Classification) |
|
|
| This model is a fine-tuned version of **allenai/specter2_base** for identifying whether a scientific publication belongs to the **Social Sciences** domain. |
| |
| It achieves the following results on the evaluation set: |
| |
| - Loss: 0.1382 |
| - Accuracy: 0.9670 |
| - F1 Micro: 0.9670 |
| - F1 Macro: 0.9480 |
| - F1 Weighted: 0.9670 |
| |
| ## Model description |
| |
| This model performs **binary document classification** and predicts whether a publication belongs to the **Social Sciences** domain. |
| |
| The model was trained using title and abstract text from multiple openly available datasets with native disciplinary annotations, including: |
| |
| - [MAG / SciDocs](https://github.com/allenai/scidocs) |
| - [Elsevier Open Access (ASJC subject areas)](https://researchcollaborations.elsevier.com/en/datasets/elsevier-oa-cc-by-corpus/) |
| - [ERC panel datasets (publications and funded projects)](https://huggingface.co/datasets/SIRIS-Lab/erc-classification-dataset) |
| |
| Each dataset was converted into a common binary label indicating whether a document belongs to the Social Sciences according to mappings from the original classification systems. :contentReference[oaicite:0]{index=0} |
| |
| **Key characteristics** |
| |
| - Base model: `allenai/specter2_base` |
| - Task: binary document classification |
| - Labels: |
| - `False` → Non-Social Sciences |
| - `True` → Social Sciences |
| - Activation: softmax |
| - Loss: CrossEntropyLoss |
| |
| ## Intended uses |
| |
| This model is suitable for: |
| |
| - Identifying Social Sciences publications |
| - Research information systems |
| - Funding portfolio analysis |
| - Metadata enrichment |
| - Bibliometric analyses |
| |
| The model accepts: |
| |
| - title |
| - abstract |
| - title + abstract (recommended) |
| |
| ## Training data |
| |
| Training data combines approximately **20,000** documents sampled from multiple sources: |
|
|
| - MAG/SciDocs |
| - ERC panel datasets (publications and projects) |
| - Elsevier Open Access publications |
|
|
| Each source provides its own disciplinary taxonomy. Categories corresponding to **Social Sciences** were mapped into a common binary classification problem. :contentReference[oaicite:1]{index=1} |
|
|
| ## Training procedure |
|
|
| ### Preprocessing |
|
|
| - Input text: `title + abstract` |
| - Maximum sequence length: **512 tokens** |
| - Tokenization using the SPECTER2 tokenizer |
|
|
| ### Training hyperparameters |
|
|
| - learning_rate: 2e-5 |
| - train_batch_size: 32 |
| - eval_batch_size: 32 |
| - num_epochs: 4 |
| - max_length: 512 |
| - optimizer: AdamW |
| - metric for best model: F1 Macro |
| |
| ## Evaluation results |
| |
| | Metric | Value | |
| |--------|------:| |
| | Accuracy | 0.9670 | |
| | F1 Micro | 0.9670 | |
| | F1 Macro | 0.9480 | |
| | F1 Weighted | 0.9670 | |
| |
| ## Limitations |
| |
| - The model predicts whether a publication belongs to the **Social Sciences** domain only. |
| - It does not distinguish between individual Social Sciences disciplines. |
| - Labels are derived from mappings between different disciplinary taxonomies and should be interpreted as high-level domain assignments. |
| |
| ## Framework versions |
| |
| - Transformers 4.57.1 |
| - PyTorch 2.8.0 |
| - Datasets 3.6.0 |
| - Tokenizers 0.22.1 |
