bert-large-portuguese-cased EU AI Act Compliance Profile

BERTimbau Large (aka "bert-large-portuguese-cased")

Introduction

BERTimbau Large is a pretrained BERT model for Brazilian Portuguese that achieves state-of-the-art performances on three downstream NLP tasks: Named Entity Recognition, Sentence Textual Similarity and Recognizing Textual Entailment. It is available in two sizes: Base and Large.

For further information or requests, please go to BERTimbau repository.

Available models

| Model | Arch. | #Layers | #Params | | ---------------------------------------- | ---------- | ------- | ------- | | neuralmind/bert-base-portuguese-cased | BERT-Base | 12 | 110M | | neuralmind/bert-large-portuguese-cased | BERT-Large | 24 | 335M |

Usage

from transformers import AutoTokenizer  # Or BertTokenizer
from transformers import AutoModelForPreTraining  # Or BertForPreTraining for loading pretraining heads
from transformers import AutoModel  # or BertModel, for BERT without pretraining heads

model = AutoModelForPreTraining.from_pretrained('neuralmind/bert-large-portuguese-cased')
tokenizer = AutoTokenizer.from_pretrained('neuralmind/bert-large-portuguese-cased', do_lower_case=False)

Masked language modeling prediction example

from transformers import pipeline

pipe = pipeline('fill-mask', model=model, tokenizer=tokenizer)

pipe('Tinha uma [MASK] no meio do caminho.')
# [{'score': 0.5054386258125305,
#   'sequence': '[CLS] Tinha uma pedra no meio do caminho. [SEP]',
#   'token': 5028,
#   'token_str': 'pedra'},
#  {'score': 0.05616172030568123,
#   'sequence': '[CLS] Tinha uma curva no meio do caminho. [SEP]',
#   'token': 9562,
#   'token_str': 'curva'},
#  {'score': 0.02348282001912594,
#   'sequence': '[CLS] Tinha uma parada no meio do caminho. [SEP]',
#   'token': 6655,
#   'token_str': 'parada'},
#  {'score':

Your obligations as a bert-large-portuguese-cased deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Library