MolmoPoint 8B EU AI Act Compliance Profile

MolmoPoint-8B

MolmoPoint-8B is a fully-open VLM developed by the Allen Institute for AI (Ai2) that support image, video and multi-image understanding and grounding. It has new pointing mechansim that improves image pointing, video pointing, and video tracking, see our technical report for details.

Note the huggingface MolmoPoint model does not support training, see our github repo for the training code.

Quick links:

• 🖥️ Demo
• 💬 Code
• 📂 All Models
• 📃 Paper
• 📝 Blog

Quick Start

Setup Conda Environment

Inference

We recommend running MolmoPoint with logits_processor=model.build_logit_processor_from_inputs(model_inputs) to enforce points tokens are generated in a valid way.

In MolmoPoint, instead of coordinates points will be generated as a series of special tokens, decoding the tokens back into points requires some additional metadata from the preprocessor. The metadata is returned by the preprocessor using the return_pointing_metadata flag. Then model.extract_image_points and model.extract_video_points do the decoding, they return a list of ({image_id|timestamps}, object_id, pixel_x, pixel_y) output points.

Image Pointing Example:

from transformers import AutoProcessor, AutoModelForImageTextToText
import torch
import numpy as np

checkpoint_dir = "allenai/MolmoPoint-8B"  # or path to a converted HF checkpoint

model = AutoModelForImageTextToText.from_pretrained(
    checkpoint_dir,
    trust_remote_code=True,
    dtype="auto",
    device_map="auto",
)

processor = AutoProcessor.from_pretrained(
    check

Your obligations as a MolmoPoint 8B deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Base Model