bart large chinese cluecorpussmall EU AI Act Compliance Profile

Chinese BART

Model description

This model is pre-trained by UER-py, which is introduced in this paper. Besides, the models could also be pre-trained by TencentPretrain introduced in this paper, which inherits UER-py to support models with parameters above one billion, and extends it to a multimodal pre-training framework.

You can download the set of Chinese BART models either from the UER-py Modelzoo page, or via HuggingFace from the links below:

| | Link | | ----------------- | :----------------------------: | | BART-Base | [L=6/H=768 (Base)][base] | | BART-Large | [L=12/H=1024 (Large)][large] |

How to use

You can use this model directly with a pipeline for text2text generation (take the case of BART-Base):

>>> from transformers import BertTokenizer, BartForConditionalGeneration, Text2TextGenerationPipeline
>>> tokenizer = BertTokenizer.from_pretrained("uer/bart-base-chinese-cluecorpussmall")
>>> model = BartForConditionalGeneration.from_pretrained("uer/bart-base-chinese-cluecorpussmall")
>>> text2text_generator = Text2TextGenerationPipeline(model, tokenizer)  
>>> text2text_generator("中国的首都是[MASK]京", max_length=50, do_sample=False)
    [{'generated_text': '中 国 的 首 都 是 北 京'}]

Training data

CLUECorpusSmall is used as training data.

Training procedure

The model is pre-trained by UER-py on Tencent Cloud. We pre-train 1,000,000 steps with a sequence length of 512. Taking the case of BART-Base

Your obligations as a bart large chinese cluecorpussmall deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Library