gatortron base EU AI Act Compliance Profile

Developed by a joint effort between the University of Florida and NVIDIA, GatorTron-Base is a clinical language model of 345 million parameters, pre-trained using a BERT architecure implemented in the Megatron package (https://github.com/NVIDIA/Megatron-LM).

GatorTron-Base is pre-trained using a dataset consisting of:

• 82B words of de-identified clinical notes from the University of Florida Health System,
• 6.1B words from PubMed CC0,
• 2.5B words from WikiText,
• 0.5B words of de-identified clinical notes from MIMIC-III

The Github for GatorTron is at : https://github.com/uf-hobi-informatics-lab/GatorTron

This model is converted to Hugginface from : https://catalog.ngc.nvidia.com/orgs/nvidia/teams/clara/models/gatortron_og

Model | Parameter | Maximum input --- | --- | --- gatortron-base-2k | 345 million | 2048 gatortron-base (this model) | 345 million | 512 gatortronS | 345 million | 512 gatortron-medium | 3.9 billion | 512 gatortron-large | 8.9 billion | 512

from transformers import AutoModel, AutoTokenizer, AutoConfig

tokenizer= AutoTokenizer.from_pretrained('UFNLP/gatortron-base')
config=AutoConfig.from_pretrained('UFNLP/gatortron-base')
mymodel=AutoModel.from_pretrained('UFNLP/gatortron-base')

encoded_input=tokenizer("Bone scan:  Negative for distant metastasis.", return_tensors="pt")
encoded_output = mymodel(**encoded_input)
print (encoded_output)

• An NLP pacakge using GatorTron for clinical concept extraction (Named Entity Recognition): https://github.com/uf-hobi-informatics-lab/ClinicalTransformerNER
• An NLP pacakge using GatorTron for Relation Extraction: https://github.com/uf-hobi-informatics-lab/ClinicalTransfor

Your obligations as a gatortron base deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Library