moonshine base ONNX EU AI Act Compliance Profile

Usage

Transformers.js

If you haven't already, you can install the Transformers.js JavaScript library from NPM using:

npm i @huggingface/transformers

Example: Automatic speech recognition w/ Moonshine base.

import { pipeline } from "@huggingface/transformers";

const transcriber = await pipeline("automatic-speech-recognition", "onnx-community/moonshine-base-ONNX");
const output = await transcriber("https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/jfk.wav");
console.log(output);
// { text: 'And so my fellow Americans ask not what your country can do for you as what you can do for your country.' }

ONNXRuntime

import numpy as np
import onnxruntime as ort
from transformers import AutoConfig, AutoTokenizer
import librosa

# Load config and tokenizer
model_id = 'onnx-community/moonshine-base-ONNX'
config = AutoConfig.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)

# Load encoder and decoder sessions
encoder_session = ort.InferenceSession('./onnx/encoder_model_quantized.onnx')
decoder_session = ort.InferenceSession('./onnx/decoder_model_merged_quantized.onnx')

# Set config values
eos_token_id = config.eos_token_id
num_key_value_heads = config.decoder_num_key_value_heads
dim_kv = config.hidden_size // config.decoder_num_attention_heads

# Load audio
audio_file = 'jfk.wav'
audio = librosa.load(audio_file, sr=16_000)[0][None]

# Run encoder
encoder_outputs = encoder_session.run(None, dict(input_values=audio))[0]

# Prepare decoder inputs
batch_size = encoder_outputs.shape[0]
input_ids = np.array([[config.decoder_start_token_id]] * batch_size)
past_key_values = {
    f'past_key_values.{layer}.{module}.{kv}': np.zeros([batch_size, num_key_value_heads, 0, dim_kv], dtype=np.float32)
    for layer in range(config.decoder_num_hidden_layers)
    for module in ('decoder', 'encod

Your obligations as a moonshine base ONNX deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Library

Base Model