metricx 24 hybrid large v2p6 bfloat16 EU AI Act Compliance Profile

MetricX-24 (Large, bfloat16)

This is not an officially supported Google product.

ℹ️ For the full-precision (float32) variant of this model, see MetricX-24 (Large).

GitHub repository: https://github.com/google-research/metricx

The repository contains the code for running inference on MetricX-24 models, a family of models for automatic evaluation of translations that were proposed in the WMT'24 Metrics Shared Task submission MetricX-24: The Google Submission to the WMT 2024 Metrics Shared Task. The models were trained in T5X and then converted for use in PyTorch.

Available Models

There are 3 MetricX-24 models available on Hugging Face that vary in the number of parameters. Unlike the MetricX-23 models, the MetricX-24 models are all hybrid models that can do both reference-based and reference-free (also known as quality estimation, or QE) inference:

• MetricX-24-Hybrid-XXL
• MetricX-24-Hybrid-XL
• MetricX-24-Hybrid-Large

We recommend using the XXL model versions for the best agreement with human judgments of translation quality, the Large versions for best speed, and the XL for an intermediate use case.

Changes to the WMT'24 Submission

The MetricX-24 models available here are most similar to the primary submission to the WMT'24 Metrics Shared Task. They are initialized with mT5, then fine-tuned on a combination of direct assessment and MQM data from WMT'15-'22. However, we made a couple of small changes that make these models different from the WMT'24 submissions.

First, the metric scores get automatically clipped at 0 and 25, to ensure they are strictly in the [0, 25] ra

Your obligations as a metricx 24 hybrid large v2p6 bfloat16 deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags