pubmedbert base splade EU AI Act Compliance Profile

PubMedBERT SPLADE

This is a SPLADE Sparse Encoder model finetuned from PubMedBERT-base using sentence-transformers. It maps sentences & paragraphs to a 30522-dimensional sparse vector space and can be used for semantic search and sparse retrieval.

The training dataset was generated using a random sample of PubMed title-abstract pairs along with similar title pairs.

PubMedBERT SPLADE produces higher quality sparse embeddings than generalized models for medical literature. Further fine-tuning for a medical subdomain will result in even better performance.

Usage (txtai)

This model can be used to build embeddings databases with txtai for semantic search and/or as a knowledge source for retrieval augmented generation (RAG).

Note: txtai 9.0+ is required for sparse vector scoring support

import txtai

embeddings = txtai.Embeddings(
  sparse="neuml/pubmedbert-base-splade",
  content=True
)
embeddings.index(documents())

# Run a query
embeddings.search("query to run")

Usage (Sentence-Transformers)

Alternatively, the model can be loaded with sentence-transformers.

from sentence_transformers import SparseEncoder
sentences = ["This is an example sentence", "Each sentence is converted"]

model = SparseEncoder("neuml/pubmedbert-base-splade")
embeddings = model.encode(sentences)
print(embeddings)

Evaluation Results

Performance of this model compared to the top base models on the MTEB leaderboard is shown below. A popular smaller model was also evaluated along with the most downloaded PubMed similarity model on the Hugging Face Hub.

The following datasets were used to evaluate model performance.

• [PubMed QA](http

Your obligations as a pubmedbert base splade deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Library

Base Model