Florence-2-base EU AI Act Compliance Profile

Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks

Model Summary

This Hub repository contains a HuggingFace's transformers implementation of Florence-2 model from Microsoft.

Florence-2 is an advanced vision foundation model that uses a prompt-based approach to handle a wide range of vision and vision-language tasks. Florence-2 can interpret simple text prompts to perform tasks like captioning, object detection, and segmentation. It leverages our FLD-5B dataset, containing 5.4 billion annotations across 126 million images, to master multi-task learning. The model's sequence-to-sequence architecture enables it to excel in both zero-shot and fine-tuned settings, proving to be a competitive vision foundation model.

Resources and Technical Documentation:

• Florence-2 technical report.
• Jupyter Notebook for inference and visualization of Florence-2-large model

| Model | Model size | Model Description | | ------- | ------------- | ------------- |
| Florence-2-base[HF] | 0.23B | Pretrained model with FLD-5B
| Florence-2-large[HF] | 0.77B | Pretrained model with FLD-5B
| Florence-2-base-ft[HF] | 0.23B | Finetuned model on a colletion of downstream tasks | Florence-2-large-ft[HF] | 0.77B | Finetuned model on a colletion of downstream tasks

How to Get Started with the Model

Use the code below to get started with the model. All models are trained with float16.

import requests

from PIL import Image
from transformers import AutoProcessor, AutoModelForCausalLM 

device = "cuda:0" if torch.cuda.is_available() else "cpu"
torch_dtype = torch.float16 if torch.cuda.is_available()

Your obligations as a Florence-2-base deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Library