nbailab base ner scandi EU AI Act Compliance Profile

ScandiNER - Named Entity Recognition model for Scandinavian Languages

Check out a demo of the model here.

This model is a fine-tuned version of NbAiLab/nb-bert-base for Named Entity Recognition for Danish, Norwegian (both Bokmål and Nynorsk), Swedish, Icelandic and Faroese. It has been fine-tuned on the concatenation of DaNE, NorNE, SUC 3.0 and the Icelandic and Faroese parts of the WikiANN dataset. It also works reasonably well on English sentences, given the fact that the pretrained model is also trained on English data along with Scandinavian languages.

The model will predict the following four entities:

| Tag | Name | Description | | :------ | :------- | :-------------- | | PER | Person | The name of a person (e.g., Birgitte and Mohammed) | | LOC | Location | The name of a location (e.g., Tyskland and Djurgården) | | ORG | Organisation | The name of an organisation (e.g., Bunnpris and Landsbankinn) | | MISC | Miscellaneous | A named entity of a different kind (e.g., Ūjķnustu pund and Mona Lisa) |

Quick start

You can use this model in your scripts as follows:

>>> from transformers import pipeline
>>> import pandas as pd
>>> ner = pipeline(task='ner', 
...                model='saattrupdan/nbailab-base-ner-scandi', 
...                aggregation_strategy='first')
>>> result = ner('Borghild kjøper seg inn i Bunnpris')
>>> pd.DataFrame.from_records(result)
  entity_group     score      word  start  end
0          PER  0.981257  Borghild      0    8
1          ORG  0.974099  Bunnpris     26   34

Performance

The following is the Micro-F1 NER performance on Scandinavian NER test datasets, compared with the current state

Your obligations as a nbailab base ner scandi deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Library