DC Job - Find Duplicates in Batch

Last published at: September 3rd, 2024
Delete

Duplicate Check Job (DC Job) provides you with extensive options to find and process duplicate records in batch.

  • Run a DC Job on all records of an object, or a subset of records. Find duplicates across different objects (Cross Object), including custom objects.
  • Schedule a DC Job to run repeatedly at set times.
  • Automatically merge or convert found duplicates if they meet set criteria.
  • View the job results per job or per object to quickly process (merge, convert) the duplicates.
  • Run large volume jobs on Plauti Cloud or DC Local to speed up the process.

Prerequisites

  • You have configured one or more Scenarios‍ to define which records are considered potential duplicates.
  • If you want to find duplicates between different objects, you have configured Cross Object‍.

Start a DC Job

To find duplicate records in batch, start a DC Job as follows:

  1. In Duplicate Check, go to tab DC Job. A few Duplicate Check menu tabs, with the DC Job tab highlighted. 
  2. At top right, click + New Job. Button titled "+ New Job"
  3. Enter a Job Name. For future reference it's recommended to have the name describe what the job does.
  4. At Select Object, select the Object to find duplicate records in. If the object is not in the list, you can configure it on the DC Setup page‍.
  5. If enabled for the selected object, click Add Cross ObjectButton titled "+ Add Cross Object". to search for duplicates across objects(optional).
  6. Select one or more Scenarios to determine when a record qualifies as a duplicate.
    If multiple scenarios are used, two records are considered duplicates if they score as duplicates on at least one of the scenarios. Read more about Scenarios here‍.
  7. Click Add Filter to search for duplicates within a subset of records (optional but recommended). Read more about job filters here‍.
  8. Click Next.
  9. Click Add Schedule if you want to run the job repeatedly at a set schedule. Read more about Scheduled Jobs.
  10. Click Add Auto Merge (for single object jobs) or Add Auto Convert (for Cross Object jobs) to automatically merge or convert found duplicates (optional).
    Delete

    Adding Auto Merge or Auto Convert here is only recommended if you are absolutely sure of the results you are going to receive. Merging or converting records cannot be undone.
    You can also decide to manually start an Auto Merge‍ or Auto Convert‍ at a later point, after reviewing your results. 


    Add Auto Merge:

    1. Set an Auto Merge Threshold. Duplicate pairs that have a matching percentage equal or higher than this threshold will be automatically merged.
      Delete

      Do not set the threshold too low: merging records cannot be undone!

    2. Click + Add <Object> Filter to merge only those records that meet certain filter criteria (optional). Add one or more filter lines, and use filter logic if needed. Read more here.
    3. Click + Add Duplicate Group Filter to merge only records from duplicate groups that meet certain filter criteria, e.g. only groups that contain two records. Read more here.

    Add Auto Convert:
    1. Set a Status After Conversion.
    2. Set an Auto Convert Threshold. Duplicate pairs that have a matching percentage equal or higher than this threshold will be automatically converted.
      Delete

      Do not set the threshold too low: converting records cannot be undone!

  11. Click Next.
  12. Select a Processing Option:
    • Run on Salesforce Platform: for smaller jobs, without any data transfer outside of Salesforce.
    • Run on DC Local: for large data volume jobs, to speed up the duplicate search process. Runs on your local machine. Read more about DC Local‍.
    • Run on Plauti Cloud: for large data volume jobs, to speed up the duplicate search process. Runs on a secure cloud server and has some advantages over DC Local. Read more about Plauti Cloud‍.
  13. Click Start.

You are returned to the DC Job overview page. The DC Job you just created appears at the top.

Delete

Start the job on DC Local or Plauti Cloud

If you selected 'Run on DC Local' or 'Run on Plauti Cloud': open DC Local or Plauti Cloud and Start the Job.
Run a DC Local Duplicate Check job
Run a DC Job on Plauti Cloud

Note that if you added Auto Merge or Auto Convert, the Auto Merge/Convert job that follows after the DC job is finished needs to be started manually as well, because the user permissions for merging need to be checked first.

A DC Job or Auto Merge/Convert job that is still waiting to be started will show status "Holding" in the DC Job Overview.

A running job will have status "Processing" on the DC Job overview page. Once it's done, it will show status "Completed". 

You can now view the job results, and merge or convert duplicate records. See DC Job Overview‍ and DC Job Results Overview‍ for all options.

Delete

Notifications for finished jobs

You can set up an email notification to be sent once a job has finished. Read more here.‍ 


Delete

Running both D365 and Salesforce jobs

If you are using both Dynamics 365 and Salesforce, you can use Duplicate Check in both environments. Both DC Local and Plauti Cloud can handle DC Jobs from both sources intermingled. However, DC Local cannot receive jobs from both sources at the same time.

When using DC Local, you need to log out and into DC Local to switch between jobs coming from D365 and Salesforce. This also applies when Auto Processing is enabled!

Plauti Cloud can receive jobs from both sources at the same time, both for manual and auto running.