Fine R1 7B EU AI Act Compliance Profile

Fine-R1: Make Multi-modal LLMs Excel in Fine-Grained Visual Recognition by Chain-of-Thought Reasoning

This is the official model repository for the paper Fine-R1: Make Multi-modal LLMs Excel in Fine-Grained Visual Recognition by Chain-of-Thought Reasoning.

Introduction

Fine-R1 is a Multi-modal Large Language Model (MLLM) specifically designed for Fine-Grained Visual Recognition (FGVR). While general MLLMs often struggle with distinguishing between highly similar sub-categories, Fine-R1 bridges the gap between generative models and specialized discriminative models (like CLIP) through an R1-style training framework.

Key Innovations:

• Chain-of-Thought Supervised Fine-tuning (CoT-SFT): The model is trained on high-quality FGVR CoT datasets, teaching it to perform visual analysis, consider candidate sub-categories, and compare them before predicting.
• Triplet Augmented Policy Optimization (TAPO): This includes Intra-class Augmentation to handle visual variance and Inter-class Augmentation to maximize distinction between similar sub-categories.

With only 4-shot training, Fine-R1 excels in identifying both seen and unseen sub-categories, outperforming many general reasoning MLLMs and contrastive models.

Resources

• Paper: Hugging Face Papers
• GitHub: PKU-ICST-MIPL/FineR1_ICLR2026

Usage

This model is compatible with the Hugging Face transformers library. For detailed instructions on environment setup, training scripts, and evaluation pipelines (closed-world and open-world), please refer to the official GitHub Repository.

Citation

If you find Fine-R1 helpful in your research, please cite the following paper:

@article{he2026finer1,
  title={Fine-R1: Make Multi-modal LLMs Excel in Fine-Grained Visual Recognition by C

Your obligations as a Fine R1 7B deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Library