Change Tracking Entities – Concept and Understanding in Dataverse

In the dynamic world of business applications, managing and monitoring data changes efficiently is crucial. Microsoft Dataverse, a key component of the Power Platform, offers robust capabilities to track changes in data entities. This feature, known as Change Tracking, is indispensable for ensuring data integrity, facilitating integration with external systems, and optimizing performance. In this post, we’ll delve into the concept of Change Tracking in Dataverse, its benefits, and how you can leverage it effectively.

What is Change Tracking in Dataverse?

Change Tracking in Dataverse is a feature that allows you to keep a record of changes—creations, updates, and deletions—made to data entities. This functionality is essential for applications that need to synchronize data with external systems, generate audit logs, or trigger workflows based on data changes.

Key Benefits of Change Tracking

1. Data Synchronization:
   • Efficiency: Change Tracking ensures that only modified data is synchronized between Dataverse and external systems, reducing the volume of data transfers and improving performance.
   • Consistency: It helps maintain data consistency across different systems, which is critical for operations involving multiple databases or services.
2. Performance Optimization:
   • By tracking changes, Dataverse can minimize the resources required for data synchronization and integration tasks, leading to better overall system performance.
3. Audit and Compliance:
   • Traceability: Change Tracking enables detailed audit logs, providing a clear history of data modifications which is crucial for compliance and forensic analysis.
   • Accountability: It ensures accountability by recording who made changes and when, which is essential for maintaining data governance.

How Change Tracking Works

Change Tracking in Dataverse works at the entity level. When you enable Change Tracking for an entity, Dataverse begins to record changes to records within that entity. Here’s a step-by-step outline of how it functions:

1. Enable Change Tracking:
   • You can enable Change Tracking for specific entities via the Power Platform admin center or through configuration tools like the Power Apps maker portal.
2. Recording Changes:
   • Once enabled, Dataverse records changes in a dedicated Change Tracking table. This table keeps track of the type of change (create, update, delete) and metadata such as timestamps and user information.
3. Retrieving Changes:
   • Applications can query the Change Tracking table to retrieve a list of changes since a specific version or timestamp. This is typically done using the Dataverse Web API or through SDKs provided by Microsoft.

Enabling Change Tracking for an Entity

To enable Change Tracking for an entity in Dataverse, follow these steps:

1. Navigate to the Power Apps Maker Portal:
   • Go to make.powerapps.com and select your environment.
2. Select the Entity:
   • In the left navigation pane, select “Data” > “Entities” and choose the entity you want to enable Change Tracking for.
3. Enable Change Tracking:
   • Open the entity’s settings and look for the “Change Tracking” option. Toggle it to enable, and save the changes.

Example Use Case: Synchronizing with an External System

Imagine you have a Customer Relationship Management (CRM) system built on Dataverse, and you need to synchronize customer data with an external billing system. By enabling Change Tracking on the customer entity, you can efficiently detect and propagate changes to the billing system without transferring the entire dataset each time.

1. Initial Sync:
   • Perform an initial synchronization of all customer data.
2. Incremental Sync:
   • Use the Change Tracking feature to retrieve changes (new customers, updates to existing customers, or deletions) since the last sync and apply these changes to the billing system.

Best Practices for Using Change Tracking

1. Select Entities Wisely:
   • Enable Change Tracking only on entities where it is necessary to avoid unnecessary overhead.
2. Monitor Performance:
   • Regularly monitor the performance of your Dataverse environment, especially if you have many entities with Change Tracking enabled.
3. Implement Error Handling:
   • Ensure robust error handling in your synchronization logic to manage potential issues during data transfer.

Working Demo

The below sample code can be used to get changed records using C# console app.

FIRST RUN – All Data Retrieve

pass DataVersion as Null to pull all records of the entity

static void Main(string[] args)
        {

            Console.WriteLine("Console App started");

            IOrganizationService orgService;

            orgService = GetOrganizationServiceClientSecret(
                 "24ab773b-d8e0-4621-ba64-f75c695f434534c",
                 ".6a8Q~6UxoVIkzLBYK8BIIjurdBXLWOrEEDvkdfj",
                 "https://devboxsoftchief.crm8.dynamics.com/");


            RetrieveEntityChangesResponse response = null;
            // pass dataversion as null to fetch all records
            string dataVersion = null;
            ColumnSet columns = new ColumnSet(true);

            RetrieveEntityChangesRequest request = new RetrieveEntityChangesRequest()
            {
                EntityName = "seven_medicinemaster",
                Columns = columns,
                PageInfo = new PagingInfo()
                {
                    PageNumber = 1,
                    Count = 50,
                    ReturnTotalRecordCount = false
                },
                DataVersion = dataVersion,
            };

            response = (RetrieveEntityChangesResponse)orgService.Execute(request);

            var count = response.EntityChanges.Changes.Count;
            //use datatoken next time call. and pass in datavrsion
            var dataver = response.EntityChanges.DataToken;

        }

When you run you will see all records are returning, I have 5 records total so its returing 5 count.

Note the Data Token from this run to use in second run.

SECOND RUN using previous DataVersion timestamp ( No data change scenario) – I have not changed any data

You will get 0 count as no data changed, note that we passed the last tun datatoken in dataversion.

For multiple runs and testing demo wetch the video below.

Sample code for multiple runs

   static void Main(string[] args)
   {

       Console.WriteLine("Console App started");

       IOrganizationService orgService;

       orgService = GetOrganizationServiceClientSecret(
            "24ab773b-d8e0-4621-ba64-rerer",
            ".6a8Q~errerererererere",
            "https://errt.crm8.dynamics.com/");


       RetrieveEntityChangesResponse response = null;
       string dataVersion = "15772301!05/20/2024 07:49:09";
       ColumnSet columns = new ColumnSet(true);

       RetrieveEntityChangesRequest request = new RetrieveEntityChangesRequest()
       {
           EntityName = "seven_medicinemaster",
           Columns = columns,
           PageInfo = new PagingInfo()
           {
               PageNumber = 1,
               Count = 50,
               ReturnTotalRecordCount = false
           },
           DataVersion = dataVersion,
       };

       response = (RetrieveEntityChangesResponse)orgService.Execute(request);

       var count = response.EntityChanges.Changes.Count;
       //use datatoken next time call. and pass in datavrsion
       var dataver = response.EntityChanges.DataToken;

   }

Hope it helps.