midnight EU AI Act Compliance Profile

Kaiko midnight

Midnight - Training State-of-the-Art Pathology Foundation Models with Orders of Magnitude Less Data

This repository contains the model checkpoints for the Midnight-12k model presented in our paper titled "Training state-of-the-art pathology foundation models with orders of magnitude less data." Our approach achieves competitive performance compared to leading pathology foundation models (FMs), despite being trained on significantly fewer whole slide images (WSIs).

Overview

We propose a refined self-supervised training framework based on DINOv2 with modifications that optimize model performance specifically for computational pathology. Our main contributions include:

• Three novel pathology FMs trained with significantly reduced data (up to 100x fewer WSIs).
• Introduction of high-resolution post-training to enhance embedding quality.

Model Highlights

• Midnight-12k: Trained exclusively on the publicly available TCGA dataset (12k WSIs).
• Midnight-92k: Trained on TCGA and an additional proprietary dataset from the Netherlands Cancer Institute (NKI-80k).
• Midnight-92k/392: Our top-performing model fine-tuned with high-resolution post-training.

Model Weights

• Midnight-12k: Publicly available under the permissive MIT license.
• Midnight-92k & Midnight-92k/392: Trained on proprietary datasets and subject to restricted access.

Usage

Our models are trained on 224x224 images normalized with a mean of (0.5, 0.5, 0.5) and a standard deviation of (0.5, 0.5, 0.5). Please ensure you apply these exact normalization parameters when preparing your datasets for embedding extraction.

from transformers import AutoImageProcessor, AutoModel
from PIL import Image
import requests
from torchvision.transforms import v2

url = 'https://upload.wikimedia.org/wikipedia/commons/8/80/Breast_DCIS_histopathology_%281%29.jpg'
image = Image.open(requests.get(url, stream=True).raw

Your obligations as a midnight deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Base Model