BioLORD 2023 EU AI Act Compliance Profile

| 🙏 If you are able to, please help me fund my open research. 🙏 Thank you for your generosity! 🤗 | |-----------------------------------------------------------------------------------------------------------------------------------|

FremyCompany/BioLORD-2023

This model was trained using BioLORD, a new pre-training strategy for producing meaningful representations for clinical sentences and biomedical concepts.

State-of-the-art methodologies operate by maximizing the similarity in representation of names referring to the same concept, and preventing collapse through contrastive learning. However, because biomedical names are not always self-explanatory, it sometimes results in non-semantic representations.

BioLORD overcomes this issue by grounding its concept representations using definitions, as well as short descriptions derived from a multi-relational knowledge graph consisting of biomedical ontologies. Thanks to this grounding, our model produces more semantic concept representations that match more closely the hierarchical structure of ontologies. BioLORD-2023 establishes a new state of the art for text similarity on both clinical sentences (MedSTS) and biomedical concepts (EHR-Rel-B).

This model is based on sentence-transformers/all-mpnet-base-v2 and was further finetuned on the BioLORD-Dataset and LLM-generated definitions from the Automatic Glossary of Clinical Terminology (AGCT).

Sibling models

This model is accompanied by other models in the BioLORD-2023 series, which you might want to check:

• BioLORD-2023-M (multilingual model; distilled from BioLORD-2023)
• BioLORD-2023 (best model after model averaging; this model)

Your obligations as a BioLORD 2023 deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Library