hallucination_evaluation_model EU AI Act Compliance Profile

<img src="https://huggingface.co/vectara/hallucination_evaluation_model/resolve/main/candle.png" width="50" height="50" style="display: inline;"> In Loving memory of Simon Mark Hughes...

Quickstart: try HHEM-2.1-Open Live Demo

👉 Launch Interactive Demo - No setup required, runs in your browser

💡 Quick test: Try inputting "The capital of France is Berlin" as premise and "The capital of France is Paris" as hypothesis to see HHEM detect this factual but hallucinated case.

HHEM-2.1-Open is the latest open source version of Vectara's HHEM series models for detecting hallucinations in LLMs. These are particularly useful in the context of building retrieval-augmented-generation (RAG) applications or Agentic workflows, where a set of facts is summarized by an LLM, and HHEM can be used to measure the extent to which this summary is factually consistent with the facts.

Hallucination Detection 101

By "hallucinated" or "factually inconsistent", we mean that a text (hypothesis, to be judged) is not supported by another text (evidence/premise, given). You always need two pieces of text to determine whether a text is hallucinated or not. When applied to RAG or AI Agents, the LLM is provided with several pieces of text (often called facts or context) retrieved from some dataset, and a hallucination would indicate that the summary (hypothesis) is not supported by those facts (evidence).

A common type of RAG hallucination is factual but hallucinated. For example, given the premise "The capital of France is Berlin", the hypothesis "The capital of France is Paris" is hallucinated -- although it is true in the world knowledge. This happens

Your obligations as a hallucination_evaluation_model deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Library

Base Model