phobert-base-v2 EU AI Act Compliance Profile

Table of contents

1. Introduction
2. Using PhoBERT with transformers
   • Installation
   • Pre-trained models
   • Example usage
3. Using PhoBERT with fairseq
4. Notes

<a name="introduction"></a> PhoBERT: Pre-trained language models for Vietnamese

Pre-trained PhoBERT models are the state-of-the-art language models for Vietnamese (Pho, i.e. "Phở", is a popular food in Vietnam):

• Two PhoBERT versions of "base" and "large" are the first public large-scale monolingual language models pre-trained for Vietnamese. PhoBERT pre-training approach is based on RoBERTa which optimizes the BERT pre-training procedure for more robust performance.
• PhoBERT outperforms previous monolingual and multilingual approaches, obtaining new state-of-the-art performances on four downstream Vietnamese NLP tasks of Part-of-speech tagging, Dependency parsing, Named-entity recognition and Natural language inference.

The general architecture and experimental results of PhoBERT can be found in our paper:

Please CITE our paper when PhoBERT is used to help produce published results or is incorporated into other software.

<a name="transformers"></a> Using PhoBERT with transformers

Installation <a name="install2"></a>

• Install transformers with pip: pip install transformers, or [install transformers from source](https://huggingface.co/docs/transformers/ins

Your obligations as a phobert-base-v2 deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Library