Splade PP en v1 EU AI Act Compliance Profile

<br><br>

Independent Implementation of SPLADE++ Model (a.k.a splade-cocondenser* and family) for the Industry setting.

This work stands on the shoulders of 2 robust researches: Naver's From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective paper and Google's SparseEmbed. Props to both the teams for such a robust work.

1. What are Sparse Representations and Why learn one?

Beginner ? expand this. Expert in Sparse & Dense representations ? feel free skip to next section 2,

1. Lexical search:

Lexical search with BOW based sparse vectors are strong baselines, but they famously suffer from vocabulary mismatch problem, as they can only do exact term matching. Here are the pros and cons:

• ✅ Efficient and Cheap.
• ✅ No need to fine-tune models.
• ✅️ Interpretable.
• ✅️ Exact Term Matches.
• ❌ Vocabulary mismatch (Need to remember exact terms)

2. Semantic Search:

Learned Neural / Dense retrievers (DPR, Sentence transformers*, BGE* models) with approximate nearest neighbors search has shown impressive results. Here are the pros and cons:

• ✅ Search how humans innately think.
• ✅ When finetuned beats sparse by long way.
• ✅ Easily works with Multiple modals.
• ❌ Suffers token amnesia (misses term matching),
• ❌ Resource intensive (both index & retreival),
• ❌ Famously hard to interpret.
• ❌ Needs fine-tuning for OOD data.

3. The big idea:

Getting pros of both searches made sense and that gave rise to interest in learning sparse representations for queries and documents with some interpretability. The sparse representations also double as implicit or

Your obligations as a Splade PP en v1 deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Library