Microsoft Dynamics 365 | Integration, Dataverse...

Azure Synapse Link and Copying Data to Azure SQL not loading all records.

(1) Share

Report

Posted on by JFulfordMS

193

I'm using the following guide to setup Azure Synapse Link to copy D365/Dataverse data down to an Azure SQL database. Everything is working as expected except that the process doesn't seem to be ingesting all records. As a test I configured for Accounts, Campaigns and Products to be included in the data replication. However, the counts in the database are much lower. I've tried a manual run of the process by specifying the original folder that was created which has all of the underlying Account data and it didn't force a full load. Can someone please give me some insight on how to either force a full load, or to make sure that the supported template forces a full load of the data?

https://learn.microsoft.com/en-us/power-apps/maker/data-platform/azure-synapse-link-pipelines?tabs=data-factory

The counts in the database of products are correct, the count of campaign is off by about 100, and the total count of accounts is 46.

Categories:

Dynamics 365 general

I have the same question (0)

All responses (4)

Answers (0)

André Arnaud de Cal... 303,730 Super User 2026 Season 1 on at

Like (0)

Report
Copy link

Link copied!

Hi @JFulfordMS,

Do you have all records in your data lake and not in your BYOD SQL database? If so, then check the data flow responsible for the copy. What settings do you have top copy the data from the lake to your DB?

Was this reply helpful? Yes No
JFulfordMS 193 on at

Like (0)

Report
Copy link

Link copied!

The records are in the data lake, they're being exported by the Azure Synapse Link configured in Power Apps. The data flow being used is the one given by Microsoft in the link. All settings are based on that article with no changes.

Was this reply helpful? Yes No
Suggested answer

11manish 365 on at

Like (0)

Report
Copy link

Link copied!
Root Cause

When using Azure Synapse Link for Dataverse:

Data is not always fully loaded by default

It starts with a snapshot + incremental sync model

Some records may be skipped due to:

Sync not fully completed

Filters or configuration issues

Initial load not triggered correctly

How to fix:

Reinitialize the table

Go to Synapse Link configuration

Remove the affected tables (Accounts, Campaigns, Products)

Re-add them

This triggers a fresh full load + new snapshot

Was this reply helpful? Yes No
Suggested answer

ManoVerse 935 Super User 2026 Season 1 on at

Like (1)

Report
Copy link

Link copied!
Azure Synapse Link pipeline does not replay the initial full snapshot, It only processes incremental changes after the pipeline’s watermark is established and
the watermark is established when the pipeline starts reading data not when synapse link finishes creating the initial snapshot. so there can be some records which might missed in that duration if those were already existing records and never changed after that.

When you run pipelines manually it does not ignore watermark and does not force full reload so you are still missing those records.

How you can force a full load :

Option 1 : delete and recreate the synapse link profile.

Forces a brand‑new full snapshot

Guarantees correct counts

Disruptive, but supported by Microsoft

Note : WAIT until All tables show Active , Initial synchronization is complete for large tables (Accounts especially)

Option 2 : One time backfill

Temporarily disable watermark logic in the pipeline

Truncate Azure SQL tables

Reprocess all table folders in ADLS

Re‑enable incremental mode

Was this reply helpful? Yes No