PubMedBERT MNLI MedNLI EU AI Act Compliance Profile

PubMedBERT-MNLI-MedNLI

This model is a fine-tuned version of PubMedBERT on the MNLI dataset first and then on the MedNLI dataset. It achieves the following results on the evaluation set:

• Loss: 0.9501
• Accuracy: 0.8667

Model description

More information needed

Intended uses & limitations

The model can be used for NLI tasks related to biomedical data and even be adapted to fact-checking tasks. It can be used from the Huggingface pipeline method as follows:

from transformers import TextClassificationPipeline, AutoModel, AutoTokenizer, AutoModelForSequenceClassification
model = AutoModelForSequenceClassification.from_pretrained("pritamdeka/PubMedBERT-MNLI-MedNLI", num_labels=3, id2label = {1: 'entailment', 0: 'contradiction',2:'neutral'})
tokenizer = AutoTokenizer.from_pretrained("pritamdeka/PubMedBERT-MNLI-MedNLI")
pipe = TextClassificationPipeline(model=model, tokenizer=tokenizer, return_all_scores=True, device=0, batch_size=128)

pipe(['ALDH1 expression is associated with better breast cancer outcomes',
      'In a series of 577 breast carcinomas, expression of ALDH1 detected by immunostaining correlated with poor prognosis.'])

The output for the above will be:

[[{'label': 'contradiction', 'score': 0.10193759202957153},
  {'label': 'entailment', 'score': 0.2933262586593628},
  {'label': 'neutral', 'score': 0.6047361493110657}],
 [{'label': 'contradiction', 'score': 0.21726925671100616},
  {'label': 'entailment', 'score': 0.24485822021961212},
  {'label': 'neutral', 'score': 0.5378724932670593}]]

Training and evaluation data

More information needed

Tr

Your obligations as a PubMedBERT MNLI MedNLI deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Library

Base Model