bias detector EU AI Act Compliance Profile

Citation

Paper Link: https://arxiv.org/abs/2505.13010

If you are using this model, please cite this paper:

@misc{ghosh2025biasbiasdetectingbias,
      title={To Bias or Not to Bias: Detecting bias in News with bias-detector}, 
      author={Himel Ghosh and Ahmed Mosharafa and Georg Groh},
      year={2025},
      eprint={2505.13010},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2505.13010}, 
}

This is a RoBERTa-based binary classification model fine-tuned on the BABE (URL: https://huggingface.co/datasets/mediabiasgroup/BABE) dataset for bias detection in English news statements. The model predicts whether a given sentence contains biased language (LABEL_1) or is unbiased (LABEL_0). It is intended for applications in media bias analysis, content moderation, and social computing research.

• Example usage with Hugging Face’s pipeline:

  from transformers import pipeline
  
  classifier = pipeline("text-classification", model="himel7/bias-detector", tokenizer="roberta-base")
  result = classifier("Immigrants are criminals.")
  

Evaluation

The model was evaluated on the entire BABE dataset with a K-fold Cross Validation and yielded the following metrics at K=5:

• Accuracy: 0.9202
• Precision: 0.9615
• Recall: 0.8927
• F1 Score: 0.9257

Model Details

Model Description

This model is a fine-tuned version of roberta-base trained to detect linguistic bias in English-language news statements. The task is framed as binary classification: the model outputs LABEL_1 for biased statements and LABEL_0 for non-biased statements.

Fine-tuning was performed on the BABE dataset, which contains annotated news snippets across various topics and political leanings. The annotations focus on whether the language used expresses subjective bias rather than factual reporting.

The goal of this model is to assist in detecting subtle forms of bias in media content, such as emoti

Your obligations as a bias detector deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Library

Base Model