finemath classifier EU AI Act Compliance Profile

FineMath classifier

Model summary

This is a classifier for evaluating mathematical reasoning and deduction in web pages, fine-tuned from intfloat/multilingual-e5-small. It was developed to filter and curate mathematical content from web datasets and was trained on 1M annotations generated by LLama3-70B-instruct for web samples from Common Crawl, which were extracted using the OpenWebMath text extraction pipeline. To ensure a balanced dataset, we upsampled pages containing mathematical content in the annotations, using a preliminary math classifier on 5M samples.

We used this classifier to build FineMath dataset.

How to use in transformers

To load the FineMath classifier, use the following code:

from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("HuggingFaceTB/finemath-classifier")
model = AutoModelForSequenceClassification.from_pretrained("HuggingFaceTB/finemath-classifier")

text = "This is a test sentence."
inputs = tokenizer(text, return_tensors="pt", padding="longest", truncation=True)
outputs = model(**inputs)
logits = outputs.logits.squeeze(-1).float().detach().numpy()
score = logits.item()
result = {
    "text": text,
    "score": score,
    "int_score": int(round(max(0, min(score, 5)))),
}

print(result)
# {'text': 'This is a test sentence.', 'score': 0.07964489609003067, 'int_score': 0}

Training

The classifier was trained on 1M pairs of web samples and their scores from 0 to 5, generated by Llama3. The samples were annotated based on their usefulness for studyin

Your obligations as a finemath classifier deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Library

Base Model