TruthCheck-BERT-hybrid

This model is a fine-tuned version of bert-base-uncased on a combined dataset of Fake and Real News (Kaggle) and LIAR.
It achieves the following results on the evaluation set:

• Accuracy: ~0.93
• F1 Score: ~0.93
• Precision: ~0.93
• Recall: ~0.93

Model description

Base Model: bert-base-uncased, a transformer-based language model developed by Google.

Architecture:

• BERT encoder (all layers fine-tuned)
• BiLSTM layer for sequential context
• Attention mechanism for interpretability
• Fully connected classification head

Fine-tuning:
All BERT layers and additional layers were trained on the combined fake/real news dataset.

Intended uses & limitations

Intended Use

• Fake News Detection: Classify news articles as real or fake.
• Content Moderation: Assist platforms in flagging potentially misleading news.
• Research: Serve as a baseline for further research in misinformation detection.

Limitations

• Domain Dependency: Performance may drop on news types or topics not present in the training data.
• Bias: The model may inherit biases from the datasets used.
• Language: Trained on English news articles; performance on other languages is not guaranteed.

Training and evaluation data

• Datasets:
  • Kaggle Fake and Real News Dataset
  • LIAR dataset
• Preprocessing:
  • Text cleaning, tokenization, lemmatization, and stopword removal.

Training procedure

Training hyperparameters

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: AdamW
• lr_scheduler_type: linear
• num_epochs: 3
• early_stopping_patience: 2

Training results

Metric	Value
Accuracy	~0.93
F1 Score	~0.93
Precision	~0.93
Recall	~0.93

Framework versions

• Transformers 4.44.2
• Pytorch 2.5.1+cu121
• Datasets 3.0.0
• Tokenizers 0.19.1

---

Citation

If you use this model, please cite the original datasets and this repository.

---

License

MIT

---

Contact

For questions or support, contact Adnan Tariq. ```