ernierna EU AI Act Compliance Profile

ERNIE-RNA

Pre-trained model on non-coding RNA (ncRNA) using a masked language modeling (MLM) objective.

Disclaimer

This is an UNOFFICIAL implementation of the ERNIE-RNA: An RNA Language Model with Structure-enhanced Representations by Weijie Yin, Zhaoyu Zhang, Liang He, et al.

The OFFICIAL repository of ERNIE-RNA is at Bruce-ywj/ERNIE-RNA.

[!TIP] The MultiMolecule team has confirmed that the provided model and checkpoints are producing the same intermediate representations as the original implementation.

The team releasing ERNIE-RNA did not write this model card for this model so this model card has been written by the MultiMolecule team.

Model Details

ERNIE-RNA is a bert-style model pre-trained on a large corpus of non-coding RNA sequences in a self-supervised fashion. This means that the model was trained on the raw nucleotides of RNA sequences only, with an automatic process to generate inputs and labels from those texts. Please refer to the Training Details section for more information on the training process.

Variants

• multimolecule/ernierna: The ERNIE-RNA model pre-trained on non-coding RNA sequences.
• multimolecule/ernierna-ss: The ERNIE-RNA model fine-tuned on RNA secondary structure prediction.

Model Specification

| Num Layers | Hidden Size | Num Heads | Intermediate Size | Num Parameters (M) | FLOPs (G) | MACs (G) | Max Num Tokens | | ---------- | ----------- | --------- | ----------------- | ------------------ | --------- | -------- | -------------- | | 12 | 768 | 12 | 3072 | 85.67 | 22.37 | 11.18 | 1024 |

Links

• Code: [multimolecule.ernierna](https://github.com/DLS5-Omics/m

Your obligations as a ernierna deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Library