stella pl retrieval 8k EU AI Act Compliance Profile

This is an embedding model based on stella_en_1.5B_v5 and further fine-tuned for retrieval tasks in Polish. It transforms texts into 1024-dimensional vectors. The model training consisted of two stages:

• In the first stage, we adapted the model to support the Polish language using the multilingual knowledge distillation method, leveraging a diverse corpus of 20 million Polish-English text pairs.
• The original Stella model and the output of the first stage were limited to a short context of 512 tokens. In the second stage, we extended the context to 8192 tokens and then fine-tuned the model using contrastive loss on a dataset comprising 1.5 million queries. Positive and negative passages for each query have been selected with the help of BAAI/bge-reranker-v2.5-gemma2-lightweight reranker. The model was trained for three epochs with a batch size of 1024 queries.

Usage (Sentence-Transformers)

The model utilizes the same prompts as the original stella_en_1.5B_v5.

For retrieval, queries should be prefixed with "Instruct: Given a web search query, retrieve relevant passages that answer the query.\nQuery: ".

For symmetric tasks such as semantic similarity, both texts should be prefixed with "Instruct: Retrieve semantically similar text.\nQuery: ".

Please note that the model uses a custom implementation, so you should add trust_remote_code=True argument when loading it. You can use the model like this with sentence-transformers:

from sentence_transformers import SentenceTransformer
from sentence_transformers.util import cos_sim

model = SentenceTransformer(
    "sdadas/stella-pl-retrieval-8k",
    trust_remote_code=True,
    device="cuda"
)
mode

Your obligations as a stella pl retrieval 8k deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Library