embedic base EU AI Act Compliance Profile

djovak/embedic-base

Say hello to Embedić, a group of new text embedding models finetuned for the Serbian language!

These models are particularly useful in Information Retrieval and RAG purposes. Check out images showcasing benchmark performance, you can beat previous SOTA with 5x fewer parameters!

Although specialized for Serbian(Cyrillic and Latin scripts), Embedić is Cross-lingual(it understands English too). So you can embed English docs, Serbian docs, or a combination of the two :)

This is a sentence-transformers model: It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search.

Usage (Sentence-Transformers)

Using this model becomes easy when you have sentence-transformers installed:

Then you can use the model like this:

from sentence_transformers import SentenceTransformer
sentences = ["ko je Nikola Tesla?", "Nikola Tesla je poznati pronalazač", "Nikola Jokić je poznati košarkaš"]

model = SentenceTransformer('djovak/embedic-base')
embeddings = model.encode(sentences)
print(embeddings)

Important usage notes

• "ošišana latinica" (usage of c instead of ć, etc...) significantly deacreases search quality
• The usage of uppercase letters for named entities can significantly improve search quality

Training

• Embedić models are fine-tuned from multilingual-e5 models and they come in 3 sizes (small, base, large).

• Training is done on a single 4070ti super GPU

• 3-step training: distillation, training on (query, text) pairs and finally fine-tuning with triplets.

Evaluation

Model description:

| Model Name | Dimension | Sequence Length | Parameters |:----:|:---:|:---:|:---:| | intfloat/multilingual-e5-small | 384 | 512 | 117M | [djovak/embedic-small](ht

Your obligations as a embedic base deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Library

Base Model