vllm translategemma 12b it EU AI Act Compliance Profile

vLLM-Compatible TranslateGemma Configuration

This is a modified version of google/translategemma-12b-it optimized for deployment with vLLM.

Changes from Original Model

Chat Template

The original TranslateGemma model requires a structured payload with dedicated source_lang_code and target_lang_code fields:

{
  "role": "user",
  "content": [
    {
      "type": "text",
      "source_lang_code": "cs",
      "target_lang_code": "de-DE",
      "text": "V nejhorším případě i k prasknutí čočky."
    }
  ]
}

However, vLLM does not support these custom content parameters. To maintain compatibility, the chat template has been modified to encode language codes directly in the message content using a delimiter-based format:

{
  "model": "model",
  "messages": [
    {
      "role": "user",
      "content": "<<<source>>>cs<<<target>>>de-DE<<<text>>>V nejhorším případě i k prasknutí čočky."
    }
  ]
}

Format: <<<source>>>{source_lang}<<<target>>>{target_lang}<<<text>>>{text_to_translate}

If you need to provide a custom prompt input

Format: <<<custom>>>{text}

Model Configuration (RoPE)

The original model uses the new Transformers RoPE configuration format with separate attention type settings:

"rope_parameters": {
  "full_attention": {
    "factor": 8.0,
    "rope_type": "linear"
  },
  "sliding_attention": {
    "rope_type": "default"
  }
}

This has been simplified for vLLM compatibility:

"rope_parameters": {
  "factor": 8.0,
  "rope_type": "linear"
}

Generation Configuration

The EOS token has been corrected from <end_of_turn> to <eos> to ensure proper sequence termination.

Original readme

Resources and Technical Documentation:

• Technical Report
• Responsible Generative AI Toolkit
• [TranslateGemma

Your obligations as a vllm translategemma 12b it deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Library