fast vit hd EU AI Act Compliance Profile

🏎️ FastViT-HD Image Encoder

A Hugging Face-compatible wrapper around the FastViT-HD vision backbone from
FastVLM: Efficient Vision Encoding for Vision-Language Models (Apple CVPR 2025).
This repo exposes only the image encoder – no text tower, no projection head – so you can plug it into any downstream pipeline that needs per-image embeddings.

✨ What you get

• 3072-D global embedding for any resolution (default 1024 × 1024).
• Runs out-of-the-box with transformers.
• Much faster than vanilla ViT-L/14 for high-res images (see original paper).

| Variant | #Params (enc.) | Output dim | Patch size | Global pool |
|----------------------|---------------:|-----------:|-----------:|------------:|
| FastViT-HD (this) | ~272 M | 3 072 | 64 | Yes |

🚀 Quick start

conda create --name fast-vit-hd python=3.10
conda activate fast-vit-hd
pip install torch torchvision transformers timm pillow

Then, run the following code to get a 3 072-D embedding for your image:

from transformers import AutoModel, AutoImageProcessor
import torch, PIL.Image

device = "cuda"  # or "cpu" / "mps"

model = AutoModel.from_pretrained(
    "kevin510/fast-vit-hd", trust_remote_code=True
).to(device).eval()

processor = AutoImageProcessor.from_pretrained(
    "kevin510/fast-vit-hd", trust_remote_code=True
)

img = PIL.Image.open("your_image.jpg")
px  = processor(img, do_center_crop=False, return_tensors="pt")["pixel_values"].to(device)   # (1,3,1024,1024)

emb = model(px)
print(emb.shape)   # (1, D, 3072)

D is the number of patches (e.g. 16 × 16 for 1024 × 1024 input). For 1024 × 1024 input, D = 16 × 16 = 256.

🛠️ Implementation details

• Wrapper – FastViTImageEncoder extends PreTrainedModel; we keep the

Your obligations as a fast vit hd deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags