slovakbert EU AI Act Compliance Profile

SlovakBERT (base-sized model)

SlovakBERT pretrained model on Slovak language using a masked language modeling (MLM) objective. This model is case-sensitive: it makes a difference between slovensko and Slovensko.

Intended uses & limitations

You can use the raw model for masked language modeling, but it's mostly intended to be fine-tuned on a downstream task. IMPORTANT: The model was not trained on the “ and ” (direct quote) character -> so before tokenizing the text, it is advised to replace all “ and ” (direct quote marks) with a single "(double quote marks).

How to use

You can use this model directly with a pipeline for masked language modeling:

from transformers import pipeline
unmasker = pipeline('fill-mask', model='gerulata/slovakbert')
unmasker("Deti sa <mask> na ihrisku.")

[{'sequence': 'Deti sa hrali na ihrisku.',
  'score': 0.6355380415916443,
  'token': 5949,
  'token_str': ' hrali'},
 {'sequence': 'Deti sa hrajú na ihrisku.',
  'score': 0.14731724560260773,
  'token': 9081,
  'token_str': ' hrajú'},
 {'sequence': 'Deti sa zahrali na ihrisku.',
  'score': 0.05016357824206352,
  'token': 32553,
  'token_str': ' zahrali'},
 {'sequence': 'Deti sa stretli na ihrisku.',
  'score': 0.041727423667907715,
  'token': 5964,
  'token_str': ' stretli'},
 {'sequence': 'Deti sa učia na ihrisku.',
  'score': 0.01886524073779583,
  'token': 18099,
  'token_str': ' učia'}]

Here is how to use this model to get the features of a given text in PyTorch:

from transformers import RobertaTokenizer, RobertaModel
tokenizer = RobertaTokenizer.from_pretrained('gerulata/slovakbert')
model = RobertaModel.from_pretrained('gerulata/slovakbert')
text = "Text ktorý sa má embedovať."
encoded_input = tokenizer(text, return_tensors='pt')
output = model(**encoded_input)

and in TensorFlow:

from transformers import RobertaTokenizer, TFRobertaModel
tokenizer = RobertaTokenizer.from_pretrained('gerulata/slovakbert')
model = TFRobertaModel.from

Your obligations as a slovakbert deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Library