Adaptation of Deep Bidirectional Multilingual Transformers for Russian Language
Paper • 1905.07213 • Published
# Load model directly
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("yassin99/rubertForRussian")
model = AutoModel.from_pretrained("yassin99/rubertForRussian")Quick Links
rubert-base-cased
RuBERT (Russian, cased, 12‑layer, 768‑hidden, 12‑heads, 180M parameters) was trained on the Russian part of Wikipedia and news data. We used this training data to build a vocabulary of Russian subtokens and took a multilingual version of BERT‑base as an initialization for RuBERT[1].
08.11.2021: upload model with MLM and NSP heads
[1]: Kuratov, Y., Arkhipov, M. (2019). Adaptation of Deep Bidirectional Multilingual Transformers for Russian Language. arXiv preprint arXiv:1905.07213.
- Downloads last month
- 8
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="yassin99/rubertForRussian")