nci-binary-detector-v2 EU AI Act Compliance Profile

NCI Binary Propaganda Detector v2

This model is Stage 1 of the NCI (Narrative Control Index) two-stage propaganda detection pipeline. It performs binary classification to detect whether text contains ANY propaganda techniques.

Model Description

• Model Type: Binary text classifier
• Base Model: answerdotai/ModernBERT-base
• Training Data: synapti/nci-binary-classification (24,517 train, 1,727 validation, 1,729 test)
• Language: English
• License: Apache 2.0

Performance

| Metric | Value | |--------|-------| | Accuracy | 99.4% | | Precision | 98.9% | | Recall | 100.0% | | F1 Score | 99.4% | | False Positive Rate | 1.47% | | False Negative Rate | 0.00% |

Confusion Matrix (Test Set, n=1,729)

Threshold Analysis

| Threshold | Accuracy | Precision | Recall | F1 | |-----------|----------|-----------|--------|-----| | 0.3 | 99.2% | 98.6% | 100% | 99.3% | | 0.4 | 99.2% | 98.7% | 100% | 99.3% | | 0.5 | 99.4% | 98.9% | 100% | 99.4% | | 0.6 | 99.7% | 99.4% | 100% | 99.7% | | 0.7 | 99.7% | 99.5% | 100% | 99.7% |

Recommended threshold: 0.5 (default) or 0.6 for reduced false positives

Training Details

• Loss Function: Focal Loss (gamma=2.0, alpha=0.25) for class imbalance
• Optimizer: AdamW with weight decay 0.01
• Learning Rate: 2e-5 with warmup ratio 0.1
• Batch Size: 16 (effective 32 with gradient accumulation)
• Epochs: 5 with early stopping (patience=3)
• Best Model Selection: Based on F1 score on validation set

Usage

With Transformers Pipeline

from transformers import pipeline

detector = pipeline(
    "text-classification",
    model="synapti/nci-binary-detector-v2"
)

resul

Your obligations as a nci-binary-detector-v2 deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.