Heron NVILA Lite 1B hf EU AI Act Compliance Profile

Heron-NVILA-Lite-1B-hf

Heron-NVILA-Lite-1B-hf is a vision language model trained for Japanese, based on the NVILA-Lite architecture. This implementation is compatible with transformers.AutoModelForImageTextToText.

Model Overview

• Developer: Turing Inc.
• Vision Encoder: paligemma-siglip-so400m-patch14-448
• Projector: mlp_downsample_2x2_fix
• LLM: Qwen2.5-0.5B-Instruct
• Supported Languages: Japanese, English

Setup

# We have developed this implementation with `transformers==4.57.1`. Other versions of transformers may also work, but we have not tested them.
pip install "transformers[torch]==4.57.1" pillow

Usage

from transformers import AutoConfig, AutoModelForImageTextToText, AutoProcessor

MODEL_NAME = "turing-motors/Heron-NVILA-Lite-1B-hf"

# you can use config
config = AutoConfig.from_pretrained(MODEL_NAME, trust_remote_code=True)
model = AutoModelForImageTextToText.from_config(config, trust_remote_code=True)

# or directly from_pretrained
model = AutoModelForImageTextToText.from_pretrained(MODEL_NAME, trust_remote_code=True, device_map="auto")

# load processor
processor = AutoProcessor.from_pretrained(MODEL_NAME, trust_remote_code=True)

# show chat_template
print(processor.tokenizer.chat_template)

def generate_content(content: str, images: list | None = None, **kwargs) -> str:
    conversation = [{"role": "user", "content": content}]
    text = processor.apply_chat_template(conversation, add_generation_prompt=True, tokenize=False)
    encoding = processor(text=text, images=images, return_tensors="pt").to(model.device)
    output = model.generate(**encoding, **kwargs)
    return processor.decode(output[0, len(encoding["inp

Your obligations as a Heron NVILA Lite 1B hf deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Base Model