tribev2 EU AI Act Compliance Profile

TRIBE v2

A Foundation Model of Vision, Audition, and Language for In-Silico Neuroscience

📄 Paper ▶️ Demo | 🤗 Weights

TRIBE v2 is a deep multimodal brain encoding model that predicts fMRI brain responses to naturalistic stimuli (video, audio, text). It combines state-of-the-art feature extractors — LLaMA 3.2 (text), V-JEPA2 (video), and Wav2Vec-BERT (audio) — into a unified Transformer architecture that maps multimodal representations onto the cortical surface.

Quick start

Load a pretrained model from HuggingFace and predict brain responses to a video:

from tribev2 import TribeModel

model = TribeModel.from_pretrained("facebook/tribev2", cache_folder="./cache")

df = model.get_events_dataframe(video_path="path/to/video.mp4")
preds, segments = model.predict(events=df)
print(preds.shape)  # (n_timesteps, n_vertices)

Predictions are for the "average" subject (see paper for details) and live on the fsaverage5 cortical mesh (~20k vertices). You can also pass text_path or audio_path to model.get_events_dataframe — text is automatically converted to speech and transcribed to obtain word-level timings.

For a full walkthr

Your obligations as a tribev2 deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags