ProstT5 EU AI Act Compliance Profile

Model Card for ProstT5

ProstT5 is a protein language model (pLM) which can translate between protein sequence and structure.

Model Details

Model Description

ProstT5 (Protein structure-sequence T5) is based on ProtT5-XL-U50, a T5 model trained on encoding protein sequences using span corruption applied on billions of protein sequences. ProstT5 finetunes ProtT5-XL-U50 on translating between protein sequence and structure using 17M proteins with high-quality 3D structure predictions from the AlphaFoldDB. Protein structure is converted from 3D to 1D using the 3Di-tokens introduced by Foldseek. In a first step, ProstT5 learnt to represent the newly introduced 3Di-tokens by continuing the original span-denoising objective applied on 3Di- and amino acid- (AA) sequences. Only in a second step, ProstT5 was trained on translating between the two modalities. The direction of the translation is indicated by two special tokens ("<fold2AA>" for translating from 3Di to AAs, “<AA2fold>” for translating from AAs to 3Di). To avoid clashes with AA tokens, 3Di-tokens were cast to lower-case (alphabets are identical otherwise).

• Developed by: Michael Heinzinger (GitHub @mheinzinger; Twitter @HeinzingerM)
• Model type: Encoder-decoder (T5)
• Language(s) (NLP): Protein sequence and structure
• License: MIT
• Finetuned from model: ProtT5-XL-U50

Uses

1. The model can be used for traditional feature extraction. For this, we recommend using only the encoder in half-precision (f

Your obligations as a ProstT5 deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Library