lt-wikidata-comp-multi EU AI Act Compliance Profile

{MODEL_NAME}

This is a LinkTransformer model. At its core this model this is a sentence transformer model sentence-transformers model- it just wraps around the class. It is designed for quick and easy record linkage (entity-matching) through the LinkTransformer package. The tasks include clustering, deduplication, linking, aggregation and more. Notwithstanding that, it can be used for any sentence similarity task within the sentence-transformers framework as well. It maps sentences & paragraphs to a 768 dimensional dense vector space and can be used for tasks like clustering or semantic search. Take a look at the documentation of sentence-transformers if you want to use this model for more than what we support in our applications.

This model has been fine-tuned on the model : sentence-transformers/paraphrase-multilingual-mpnet-base-v2. It is pretrained for the language : - de

• en
• zh
• ja
• hi
• ar
• bn
• pt
• ru
• es
• fr
• ko.

This model was trained on a dataset consisting of company aliases from wiki data using the LinkTransformer framework. It was trained for 70 epochs using other defaults that can be found in the repo's LinkTransformer config file - LT_training_config.json

Usage (LinkTransformer)

Using this model becomes easy when you have LinkTransformer installed:

Then you can use the model like this:

import linktransformer as lt
import pandas as pd

##Load the two dataframes that you want to link. For example, 2 dataframes with company names that are written differently
df1=pd.read_csv("data/df1.csv") ###This is the left dataframe with key CompanyName for instance
df2=pd.read_csv("data/df2.csv") ###This is the right dataframe with key CompanyName for instance

###Merge the two dataframes on the key

Your obligations as a lt-wikidata-comp-multi deployer

Track your progress

Sign up to track obligations, receive deadline reminders, and generate audit packages.

Tags

Languages

Library