nucleotide transformer 500m human ref EU AI Act Compliance Profile

nucleotide-transformer-500m-human-ref model

The Nucleotide Transformers are a collection of foundational language models that were pre-trained on DNA sequences from whole-genomes. Compared to other approaches, our models do not only integrate information from single reference genomes, but leverage DNA sequences from over 3,200 diverse human genomes, as well as 850 genomes from a wide range of species, including model and non-model organisms. Through robust and extensive evaluation, we show that these large models provide extremely accurate molecular phenotype prediction compared to existing methods

Part of this collection is the nucleotide-transformer-500m-human-ref, a 500M parameters transformer pre-trained on the human reference genome. The model is made available both in Tensorflow and Pytorch.

Developed by: InstaDeep, NVIDIA and TUM

Model Sources

• Repository: Nucleotide Transformer
• Paper: The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human Genomics

How to use

Until its next release, the transformers library needs to be installed from source with the following command in order to use the models:

pip install --upgrade git+https://github.com/huggingface/transformers.git

A small snippet of code is given here in order to retrieve both logits and embeddings from a dummy DNA sequence.

from transformers import AutoTokenizer, AutoModelForMaskedLM
import torch

# Import the tokenizer and the model
tokenizer = AutoTokenizer.from_pretrained("InstaDeepAI/nucleotide-transformer-500m-human-ref")
model = AutoModelForMaskedLM.from_pretrained("InstaDeepAI/nucleotide-transformer-500m-huma

Your obligations as a nucleotide transformer 500m human ref deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Library