gte modernbert base EU AI Act Compliance Profile

gte-modernbert-base

We are excited to introduce the gte-modernbert series of models, which are built upon the latest modernBERT pre-trained encoder-only foundation models. The gte-modernbert series models include both text embedding models and rerank models.

The gte-modernbert models demonstrates competitive performance in several text embedding and text retrieval evaluation tasks when compared to similar-scale models from the current open-source community. This includes assessments such as MTEB, LoCO, and COIR evaluation.

Model Overview

• Developed by: Tongyi Lab, Alibaba Group
• Model Type: Text Embedding
• Primary Language: English
• Model Size: 149M
• Max Input Length: 8192 tokens
• Output Dimension: 768

Model list

| Models | Language | Model Type | Model Size | Max Seq. Length | Dimension | MTEB-en | BEIR | LoCo | CoIR | |:--------------------------------------------------------------------------------------:|:--------:|:----------------------:|:----------:|:---------------:|:---------:|:-------:|:----:|:----:|:----:| | gte-modernbert-base | English | text embedding | 149M | 8192 | 768 | 64.38 | 55.33 | 87.57 | 79.31 | | gte-reranker-modernbert-base | English | text reranker | 149M | 8192 | - | - | 56.19 | 90.68 | 79.99 |

Usage

[!TIP] For transformers and sentence-transformers, if your GPU supports it, the efficient Flash Attention 2 will be used automatically if you have flash_attn installed. It is not mandatory.

pip install flash_attn

Use with transformers

# Requires transformers>=4.48.0

import torch.nn.functional as F
from transformers import AutoModel, AutoTokenizer

input_texts = [
    "what is the capit

Your obligations as a gte modernbert base deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Library

Base Model