Based on a scenario, Duplicate Check will compare the contents of one or more fields between records, and determines potential duplicates, taking relative importance of each field into account.
In a scenario, you:
- Set the fields that need to be compared
- Select a way to compare the fields (a matching method)
- Define how empty fields should be handled
- Determine the importance of each field, and
- Define the minimum required matching percentage.
Configuring a scenario
Each entity comes with a predefined scenario. You can use this scenario as is, edit it to better suit your situation, or even delete it and create a fully new one.
The steps below assume you are editing an existing scenario.
To configure a scenario:
- In the Duplicate Check app, go to Manage Entities. At the entity you want to configure a scenario for, click Manage at right.
- At the <Entity> Scenarios tab, find the scenario you want to configure.
- At right, click Edit Scenario to change the scenario name and description.
- Set a minimum matching percentage at Minimum required matching percentage to define a potential duplicate record.
For example:- A percentage of 100% will only show a record as duplicate if all set fields match.
- A percentage of 80% will show a record as duplicate when all set fields have a total duplicate score of 80% or more. Records that match to some extent, but with a matching percentage below this threshold, will not show up as potential duplicates.
- Select one or more Fields to Compare to compare the values of for determining duplicates. (Fold the scenario open with the chevron button at left if you don't see any scenario field settings yet)
Add more fields to compare by clicking + Add Field , or remove a field by deleting it with its Delete button. - For each field, set a Matching Method. See the Matching Methods page for the specifics of each method.
- At Disregard field if empty, decide how empty fields should be handled.
- If Disregard Field If Empty is set to 'Yes', the field is not taken into account for finding duplicates if it is empty. This lets you find duplicate records that are not fully filled out. However, it can also trigger false positives.
- If Disregard Field If Empty is set to 'No', and the field is empty, the field is taken into account and will count as a 0% match.
If the same field is also empty on another record, these fields will not be considered equal to each other, and count 0% towards determining duplicates. When two fields are empty, this can be because the value is not known yet, and when filled in later the values might differ. In the same way, if the same field is not empty on another record, the field counts 0% towards determining duplicates.
- Determine the Field Importance.
The Field Importance determines to what extent the field should count towards calculating the matching percentage of records. A field with a Field Importance that is higher than the other fields has more weight in calculating the matching percentage score.
See below for some example calculations. - Click Save to save your settings, and Publish Now to use the scenario straight away.
Adding a scenario
To add a scenario:
- At the <Entity> Scenarios tab, at top right, click + Add Scenario .
- In the pop-up window, enter a scenario name and description.
- Decide if you want to have default fields added for you in the new scenario. These are predefined Fields to Compare, with predefined settings, suggested based on best practices. Of course you can always edit or remove these fields.
The option to add default fields is available for the most commonly used entities.
- Click Add .
- Configure the new scenario as described above at 'Configuring a Scenario'.
Deleting a scenario
If you want to stop using a scenario, there is no need to delete the scenario altogether. Start by removing it from the Duplicate Prevention settings on tab Feature Activation & Setup. This way you can start using it again at a later date, and you won't need to set it up all over again. But this also means that in theory someone else can add it back to the settings and take it into use again.
To make sure a scenario cannot be used at all anymore, delete it permanently. Do note that it cannot be retrieved.
To delete a scenario:
- At the <Entity> Scenarios tab, find the scenario you want to delete.
- At the top right of the scenario, click Delete Scenario to delete the scenario.
- If the scenario is still in use, for example in one of the settings on the Feature Activation & Setup tab, the scenario cannot be deleted. An error message will appear stating where the scenario is currently in use. Remove the scenario from those places and try again.
- In the confirmation pop-up, click Delete to permanently delete the scenario.
Examples
Examples for 'Disregard Field if Empty'
These example scores are calculated with an equal field importance for all fields.
Disregard Field if Empty = "Yes":
Disregarding empty fields means that a field is not taken into account for finding duplicates if it is empty, and is ignored in the total matching score calculation. This lets you find duplicate records that are not fully filled out. However, it can also trigger false positives.
First Name |
Last Name |
Company Name |
Email Address |
Phone Number |
Score |
Jennifer |
Blake |
Loareet Corporation |
j.blake@loareet.com |
1-817-297-7931 |
100% |
Jennifer |
Blake |
Loareet Corporation |
Empty |
1-817-297-7931 |
Or
First Name |
Last Name |
Company Name |
Email Address |
Phone Number |
Score |
Jennifer |
Blake |
Loareet Corporation |
j.blake@loareet.com |
1-817-297-7931 |
100% |
Jennifer |
Empty |
Empty |
Empty |
Empty |
Depending on your 'Empty Fields' setting, your scenario could calculate a 100% score based on one field, even if there are more fields defined in your scenario.
Disregard Field if Empty = "No":
Taking empty fields into account will assign empty fields a matching score of 0%, i.e. the difference between the two field values is considered a non-match. As a result, the total matching score will be lower.
Note that this also applies if a field is empty on both records. When two fields are empty, this can be because the value is not known yet, and when filled in later the values might differ.
First Name |
Last Name |
Company Name |
Email Address |
Phone Number |
Score |
Jennifer |
Blake |
Loareet Corporation |
j.blake@loareet.com |
1-817-297-7931 |
80% |
Jennifer |
Blake |
Loareet Corporation |
Empty |
1-817-297-7931 |
Or
First Name |
Last Name |
Company Name |
Email Address |
Phone Number |
Score |
Jennifer |
Blake |
Loareet Corporation |
Empty |
1-817-297-7931 |
80% |
Jennifer |
Blake |
Loareet Corporation |
Empty |
1-817-297-7931 |
Examples for 'Field Importance'
Use the Field Importance setting to decide which fields matter the most when comparing records. A field with a higher importance impacts the total matching score more than a field with a lower importance.
All fields set to be equally important
Field |
Matching Method |
Field Importance |
First Name |
Exact |
3 ( 3/15 = 20%) |
Last Name |
Exact |
3 ( 3/15 = 20%) |
Company Name |
Exact |
3 ( 3/15 = 20%) |
Email |
Exact |
3 ( 3/15 = 20%) |
Phone | Exact |
3 ( 3/15 = 20%) |
First Name |
Last Name |
Company Name |
Email Address |
Phone Number |
Score |
Jennifer |
Blake |
Loareet Corporation |
j.blake@loareet.com |
1-817-297-7931 |
80% |
Jennifer |
Blake |
Loareet Corporation |
different@mail.com |
1-817-297-7931 |
More importance to Email Address
Field |
Matching Method |
Field Importance |
First Name |
Exact |
3 (3/18 = 16.6%) |
Last Name | Exact |
3 (3/18 = 16.6%) |
Company Name |
Exact |
3 (3/18 = 16.6%) |
Exact |
6 (6/18 = 33.3%) |
|
Phone |
Exact |
3 (3/18 = 16.6%) |
First Name |
Last Name |
Company Name |
Email Address |
Phone Number |
Score |
Jennifer |
Blake |
Loareet Corporation |
j.blake@loareet.com |
1-817-297-7931 |
67% |
Jennifer |
Blake |
Loareet Corporation |
Empty |
1-817-297-7931 |