GTE ModernColBERT v1 EU AI Act Compliance Profile

GTE-ModernColBERT-v1

PyLate model based on Alibaba-NLP/gte-modernbert-base

This is a PyLate model trained on the ms-marco-en-bge-gemma dataset. It maps sentences & paragraphs to sequences of 128-dimensional dense vectors and can be used for semantic textual similarity using the MaxSim operator.

Model Details

Model Description

• Model Type: PyLate model
• Base model: Alibaba-NLP/gte-modernbert-base <!-- at revision bc02f0a92d1b6dd82108036f6cb4b7b423fb7434 -->
• Document Length: 300 tokens
• Query Length: 32 tokens
• Output Dimensionality: 128 dimensions
• Similarity Function: MaxSim
• Training Dataset:
  • ms-marco-en-bge-gemma
• Language: English
• License: Apache 2.0

Document length

GTE-ModernColBERT has been trained with knowledge distillation on MS MARCO with a document length of 300 tokens, explaining its default value for documents length.

However, as illustrated in the ModernBERT paper, ColBERT models can generalize to documents lengths way beyond their training length and GTE-ModernColBERT actually yields results way above SOTA in long-context embedding benchmarks, see LongEmbed results.

Simply change adapt the document length parameter to your needs when loading the model:

model = models.ColBERT(
    model_name_or_path=lightonai/GTE-ModernColBERT-v1,
    document_length=8192,
)

ModernBERT itself has only been trained on 8K context length, but it seems that GTE-ModernColBERT can generalize to even bigger context sizes, though it is not guaranteed so please perform your own benches!

Model So

Your obligations as a GTE ModernColBERT v1 deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Library

Base Model