Giga-Embeddings-instruct EU AI Act Compliance Profile

Giga-Embeddings-instruct

• Base Decoder-only LLM: GigaChat-3b
• Pooling Type: Latent-Attention
• Embedding Dimension: 2048

Для получения более подробной информации о технических деталях, пожалуйста, обратитесь к нашей статье.

Использование

Ниже приведен пример кодирования запросов и текстов.

Requirements

pip install -q transformers==4.51.0 sentence-transformers==5.1.1 flash-attn langchain_community langchain_huggingface langchain_gigachat

Transformers

import torch
import torch.nn.functional as F

from torch import Tensor
from transformers import AutoTokenizer, AutoModel


def get_detailed_instruct(task_description: str, query: str) -> str:
    return f'Instruct: {task_description}\nQuery: {query}'

# Each query must come with a one-sentence instruction that describes the task
task = 'Given a web search query, retrieve relevant passages that answer the query'

queries = [
    get_detailed_instruct(task, 'What is the capital of Russia?'),
    get_detailed_instruct(task, 'Explain gravity')
]
# No need to add instruction for retrieval documents
documents = [
    "The capital of Russia is Moscow.",
    "Gravity is a force that attracts two bodies towards each other. It gives weight to physical objects and is responsible for the movement of planets around the sun."
]
input_texts = queries + documents

# We recommend enabling flash_attention_2 for better acceleration and memory saving.
tokenizer = AutoTokenizer.from_pretrained(
    'ai-sage/Giga-Embeddings-instruct',
    trust_remote_code=True
)
model = AutoModel.from_pretrained(
    'ai-sage/Giga-Embeddings-instruct', 
    attn_implementation="flash_attention_2", 
    torch_dtype=torch.bfloat16,
    trust_remote_code=True
)
model.eval()
model.cuda()

max_length = 4096

# Tokenize the input texts
batch_dict = tokenizer(
    input_texts,
    padding=True,
    truncation=True,
    max_length=max_length,
    return_tensors="pt",
)
batch_dic

Your obligations as a Giga-Embeddings-instruct deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Library