voyage 4 nano ONNX EU AI Act Compliance Profile

voyage-4-nano-ONNX

ONNX export of voyageai/voyage-4-nano optimized for CPU inference with sentence-transformers and Vespa.

Note: This is an ONNX-only model. Ignore the auto-generated snippet above - use the code below with backend="onnx".

Model Variants

| Variant | File | Size | Notes | |---------|------|------|-------| | FP32 | onnx/model.onnx + model.onnx.data | ~1.3GB | Highest accuracy, external data file | | INT8 | onnx/model_qint8_avx512.onnx | 332MB | Recommended - 4x smaller, cosine sim ~0.96 vs FP32 |

Specifications

• Embedding dimension: 2048
• Max sequence length: 32K tokens
• Pooling: Mean pooling
• Normalization: L2 normalized embeddings
• Model inputs: input_ids, attention_mask (no position_ids required)

Quick Start with sentence-transformers

from sentence_transformers import SentenceTransformer

# Load the ONNX model
model = SentenceTransformer(
    "thomasht86/voyage-4-nano-ONNX",
    backend="onnx",
    model_kwargs={"file_name": "onnx/model_qint8_avx512.onnx"},
)

# For documents (no prefix needed)
docs = ["The quick brown fox jumps over the lazy dog."]
doc_embeddings = model.encode(docs)

# For queries (use the prompt name)
queries = ["What animal jumps over the dog?"]
query_embeddings = model.encode(queries, prompt_name="query")

print(f"Embedding shape: {doc_embeddings.shape}")  # (1, 2048)

ONNX Runtime Direct Usage

import onnxruntime as ort
import numpy as np
from transformers import AutoTokenizer

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("thomasht86/voyage-4-nano-ONNX")
session = ort.InferenceSession("onnx/model_qint8_avx512.onnx")

# Tokenize
text = "The quick brown fox jumps over the lazy dog."
inputs = tokenizer(text, return_tensors="np", padding=True, truncation=True)

# Run inference (only input_ids and attention_mask needed)
outputs = session.run(
    None,
    {

Your obligations as a voyage 4 nano ONNX deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Library