splade pt br EU AI Act Compliance Profile

SPLADE-PT-BR

SPLADE (Sparse Lexical AnD Expansion) model fine-tuned for Portuguese text retrieval. Based on BERTimbau and trained on Portuguese question-answering datasets.

GitHub Repository: https://github.com/AxelPCG/SPLADE-PT-BR

Model Description

SPLADE is a neural retrieval model that learns to expand queries and documents with contextually relevant terms while maintaining sparsity. Unlike dense retrievers, SPLADE produces sparse vectors (typically ~99% sparse) that are:

• Interpretable: Each dimension corresponds to a vocabulary token
• Efficient: Can use inverted indexes for fast retrieval
• Effective: Combines lexical matching with semantic expansion

Key Features

• Base Model: neuralmind/bert-base-portuguese-cased (BERTimbau)
• Vocabulary Size: 29,794 tokens (Portuguese-optimized)
• Training Iterations: 150,000
• Final Training Loss: 0.000047
• Sparsity: ~99.5% (100-150 active dimensions per vector)
• Max Sequence Length: 256 tokens

Training Details

Training Data

• Training Dataset: mMARCO Portuguese (unicamp-dl/mmarco)
• Validation Dataset: mRobust (unicamp-dl/mrobust)
• Format: Triplets (query, positive document, negative document)

Training Configuration

Learning Rate: 2e-5
Batch Size: 8 (effective: 32 with gradient accumulation)
Gradient Accumulation Steps: 4
Weight Decay: 0.01
Warmup Steps: 6,000
Mixed Precision: FP16
Optimizer: AdamW

Regularization

FLOPS regularization is applied to enforce sparsity:

• Lambda Query: 0.0003 (queries are more sparse)
• Lambda Document: 0.0001 (documents less sparse for better recall)

Performance

Dataset: mRobust (528k docs, 250 queries)

| Metric | Score | |--------|-------| | MRR@10 | 0.453 |

Usage

Installation

pip install torch transformers

Basic Usage

**Option 1: Using HuggingFace Hub (Recommende

Your obligations as a splade pt br deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Base Model