industry bert contracts v0.1 EU AI Act Compliance Profile

industry-bert-contracts

industry-bert-contracts-v0.1 is part of a series of industry-fine-tuned sentence_transformer embedding models.

Model Description

industry-bert-contracts-v0.1 is a domain fine-tuned BERT-based 768-parameter Sentence Transformer model, intended to as a "drop-in" substitute for contractual and legal domains. This model was trained on a wide range of publicly available commercial contracts, including open source contract datasets.

• Developed by: llmware
• Model type: BERT-based Industry domain fine-tuned Sentence Transformer architecture
• Language(s) (NLP): English
• License: Apache 2.0
• Finetuned from model [optional]: BERT-based model, fine-tuning methodology described below.

Model Use

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("llmware/industry-bert-contracts-v0.1") model = AutoModel.from_pretrained("llmware/industry-bert-contracts-v0.1")

Bias, Risks, and Limitations

This is a semantic embedding model, fine-tuned on public domain contracts and related documents. Results may vary if used outside of this domain, and like any embedding model, there is always the potential for anomalies in the vector embedding space. No specific safeguards have put in place for safety or mitigate potential bias in the dataset.

Training Procedure

This model was fine-tuned using a custom self-supervised procedure and custom dataset that combined contrastive techniques with stochastic injections of distortions in the samples. The methodology was derived, adapted and inspired primarily from three research papers cited below: TSDAE (Reimers), DeClutr (Giorgi), and Contrastive Tension (Ca

Your obligations as a industry bert contracts v0.1 deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Library