llmlingua-2-bert-base-multilingual-cased-meetingbank EU AI Act Compliance Profile

LLMLingua-2-Bert-base-Multilingual-Cased-MeetingBank

This model was introduced in the paper LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression (Pan et al, 2024). It is a BERT multilingual base model (cased) finetuned to perform token classification for task agnostic prompt compression. The probability $p_{preserve}$ of each token $x_i$ is used as the metric for compression. This model is trained on the extractive text compression dataset constructed with the methodology proposed in the LLMLingua-2, using training examples from MeetingBank (Hu et al, 2023) as the seed data.

You can evaluate the model on downstream tasks such as question answering (QA) and summarization over compressed meeting transcripts using this dataset.

For more details, please check the project page of LLMLingua-2 and LLMLingua Series.

Usage

from llmlingua import PromptCompressor

compressor = PromptCompressor(
    model_name="microsoft/llmlingua-2-bert-base-multilingual-cased-meetingbank",
    use_llmlingua2=True
)

original_prompt = """John: So, um, I've been thinking about the project, you know, and I believe we need to, uh, make some changes. I mean, we want the project to succeed, right? So, like, I think we should consider maybe revising the timeline.
Sarah: I totally agree, John. I mean, we have to be realistic, you know. The timeline is, like, too tight. You know what I mean? We should definitely extend it.
"""
results = compressor.compress_prompt_llmlingua2(
    original_prompt,
    rate=0.6,
    force_tokens=['\n', '.', '!', '?', ','],
    chun

Your obligations as a llmlingua-2-bert-base-multilingual-cased-meetingbank deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Library