Monday, January 25, 2021

Dual-write learning series - Dual-write initial sync is the data integrator

One of the features of the dual-write is initial sync where you copy data from the source app (Finance Operation OR DataVerse) to the target app (Finance Operation OR DataVerse) depending on the selection in Master for initial sync option. 

This initial sync is the Data Integrator service running behind the scene and copies your data over. You configure the application id for data integrator and add it both apps (Finance Operation OR DataVerse), I have documented it my previous post The Dual Write implementation - Part 2 - understand and action pre-requisites

Master for initial sync can be either Common Data Service (Dataverse) or Finance and Operations apps. For example, If I choose Finance and Operations app in below example where I am syncing Functional Locations then all records will be copied from Finance and operations to Dataverse.








Initial Sync is a full push means if an individual row fails to sync, you cannot resync only failed ones. If the initial synchronization only partially succeeds, a second synchronization runs for all the rows, not just the rows that failed to be synced during the initial synchronization.

For example;

1st initial sync Run for 1000 records from FO to CDS à 700 passed and 300 failed

2nd initial sync Run will again run for 1000 records 

Do check Considerations for initial Sync from Microsoft Docs 

Initial Sync runs against all legal entities configured for dual-write. If you have entered a filter for a specific legal entity in a table map at Finance and Operations app side, as shown below as an example, this will not work for initial sync as it will run against all legal entities configured for dual write under environment details.



Wednesday, January 13, 2021

D365FO: Right click on any control at D365FO browser takes you directly to the control in AOT

Last week I explored very Interesting feature especially for developers where you right click on any field/control on the form and follow these steps.












This opens the visual studio in non admin mod, Opens the correct form, and takes you directly at the control in AOT.

NOTE: You can only get this feature within Development VM where your browser and Visual Studio are in same machine. I am at 10.0.14 but not sure when this great feature was available first :(



Sunday, December 27, 2020

Another step closer - Finance Operations data in Power Platform - Virtual Entities

This post focuses on the integration technologies available to have the Microsoft Dynamics 365 Finance Operations data available in Dataverse/Common Data Services/CDS. What could be better then having The biggest ERP system's data in Power Platform. You can Power Portals, Power Apps, Power BI analytical reports, use power virtual agents for inventory closing and year end closing processes, manage expenses and employee/contractors time entry processes, most of these processes can be even without logging to MS ERP (Dynamics 365 Finance Operation) so can safe on license cost too. 

Let's see what options are available to integrate F&O data with Power Platform however, this post is dedicated to Virtual Entities. 

3 Options available out-the-box to integrate F&O data with Power Platform;

👉 Data Integrator - Click on link to read more

👉 Dual-Write - Click on link to read more

👉 Virtual Entities - MS Tech Talk on Virtual entities 




Before we jump to the installation and configuration part, let's see when were the virtual entities available and what features these have come up with compared to other two integrations technologies.

Virtual Entities Generally available

✔️ Finance and Supply Chain Management App: 10.0.12 or later

✔️ Dataverse: Service update 189 or later

Virtual Entities features

✔️ Finance and Operations is available as a data source in Dataverse

✔️ No Finance and Operations data replication in Dataverse

✔️ Access all public data entities of Finance and Operations in Dataverse

✔️ Support all CRUD operations

Install Virtual Entities solution

Head to this link https://appsource.microsoft.com/en-us/product/dynamics-365/mscrm.finance_and_operations_virtual_entity and Get it Now










Enter your work or school account and Sign in

Choose the environment where you want to install this solution













Wait for finish to installing









Finance and Operations Virtual Entity solution shows as Enabled









Finance and Operations Virtual Entity solution is installed successfully - Hurray!! that was easy








Register an App in Azure Active Directory

The AAD application must be created on the same tenant as F&O.

  1. Log on to http://portal.azure.com
  2. Azure Active Directory > App registration
  3. New Registration


    Define these attributes
    1. Name
    2. Account type
    3. Redirect URI - leave blank
    4. Select Register
    5. Make note of the Application (Client) ID, you will need it later


Register an App



Create a symmetric key for the application, Save and note it for later use.


Steps to follow in Dataverse environment 

Log on to Dataverse environment and click on Advance settings


Go to Administrator













Choose Virtual entity data sources


Finance and operations is available as of the data source in Dataverse




















Click on Finance and Operations and following screen pops up, this is where the connections established


























Configuration in Finance and Operations

  1. Log on to Finance and Operations and go to System Administration | Users | Users
  2. Create a new user and assign 'CDS virtual entity application' role to it - don't assign system admin role to this user - This user is used to look at the metadata of the data entities from the Dataverse instance.
  3. Enter Application Id in System Administration | Setup | Azure Active Directory applications screen with the User ID = <The user created in step 1>


Test Finance and Operations data in Dataverse

Log on to Dataverse instance and click on a little funnel to open advance find and look for 'Available Finance and Operations Entities' in the list of tables in Dataverse instance. 


















By default not all the entities are enabled this is to avoid cluttering the user experience in Dataverse but individual entities can be enabled e.g. I enabled DataManagementDefinitionGroupEntity and mark visible to make this as a virtual entity in Dataverse.















To illustrate this example, I created an export data project in Finance and Operations under Data Management with the name 'CDSVirtualEntitiesExport' - The data entity behind this data export projects is DataManagementDefinitionGroupEntity which is marked as virtual entity in above step. 
















Restart the Advance find in Dataverse instance and look for Definition Group (mserp) table map and Run to see the output












This is it for today, with next post I will explain how to do customization/extension in F&O and get data into Dataverse using Virtual Entities. 

Hope you may have enjoyed the post, please do provide your feedback. Enjoy your break!!


Monday, November 30, 2020

D365FO: Entity cannot be deleted while dependent Entities for a processing group exist. Delete dependent Entities for a processing group and try again.

Scenario:

There are times when you want to delete an entity from target entity list and when you do so, you face an error message which does not tell you where exactly the entity has been used. 

"Entity cannot be deleted while dependent Entities for the processing group exist. Delete dependent Entities for a processing group and try again."

Solution:

Browse the environment by appending this part /?mi=SysTableBrowser&TableName=DMFDefinitionGroupEntity&cmp=USMF at the end. 

For example; if the environment URL is https://daxture.sandbox.operations.dynamics.com then the complete URL will be https://daxture.sandbox.operations.dynamics.com/?mi=SysTableBrowser&TableName=DMFDefinitionGroupEntity&cmp=USMF

Filter for Entity and it will give you the DefinitionGroup where the entity has been added or used in data management import/export projects.

Get the DefinitionGroup name and search in the export/import projects under data management and either delete the whole project or remove the entity from the group. Try deleting/removing entity from target entity list and it should be good now.







Sunday, November 29, 2020

How to run dual-write table map when underline entity table is setup under cross-company data sharing policy

Scenario: 

Project groups are shared across all legal entities in D365 finance & operations app so have setup under one of the cross-company data sharing policies. You are also required to setup dual-write table map for project groups to sync from FO to CDS. However, you get following error message when you try to Run the table map

"Copying pre-existing data completed with errors.

For additional details, go to initial sync details tab."

Follow these steps to overcome this issue as a workaround. 

Steps to follow:

  1. Disable cross-company data sharing policy where ProjGroup table has been used
  2. Choose Yes at next pop up window
  3. Go to Data management > Dual write > select Project Groups table map and Run
  4. Enable cross-company data sharing policy for project groups
  5. Choose No at the next pop up window unless you want to copy data across all companies

Saturday, November 21, 2020

Steps to follow when refresh dual-write integrated environments (FO and CE)

Scenario: 

The D365 finance & operations app is linked to Common Data Services through dual-write in development or sandbox environments. You are required to refresh databases in D365 FO and CDS both from another production environment (dual-write integrated (linked) environments in the scenario)

Steps to follow:

  1. Before initial refresh, log on to target environment
  2. Go to Data management > Dual write and stop all jobs
  3. Refresh FO and CDS environments from the same integrated source environments
  4. Go to Data management > Dual write and Unlink environments
  5. Clean data from following tables in D365FO if data exists
    1. DualWriteProjectConfiguration - A sample URL would be like this to access tables from D365FO environment <Environemnt URL>/?mi=SysTableBrowser&prt=initial&limitednav=true&TableName=DualWriteProjectConfiguration&cmp=<CompanyId>
    2. DualWriteProjectFieldConfiguration- A sample URL would be like this to access tables from D365FO environment <Environemnt URL>/?mi=SysTableBrowser&prt=initial&limitednav=true&TableName=DualWriteProjectFieldConfiguration&cmp=<CompanyId>
  6. Clean data from DualWriteRunTimeConfiguration Table in Dataverse [Customer Engagement] - Click on little funnel on top right side of the CE environment to open table browser or Advance find and look for Dual write Runtime configuration table





  7. Link Environments
  8. Apply MS dual-write solutions and your custom solution if there is any
  9. Start jobs and no need for initial sync, both environments are already sync as were copied from another dual-write integrated source environments




Saturday, October 24, 2020

Dual-write connection set error: An item with the same key has already been added

If you happen to see this error message then you have duplicate records in cdm_company entity in CDS environment. Check for cdm_companycode field this is normally not allowed but have a look and delete the ones with duplicates.



Dual-write integration for Cross-Company data in D365 Finance and SCM



Dual-write does not work with the cross company data sharing policies in D365 FinOps (there are so many names but I am using this name for reference 😊).

Brief overview about cross company data sharing policy first to set the base for the readers, the Cross-Company data sharing lets you have your data accessible from multiple legal entities (companies in D365 FinOps). For example, if you setup a policy for vendors to be crossed-company then whenever you create a new vendor it will be created in all data sharing legal entities.

https://docs.microsoft.com/en-us/dynamics365/fin-ops-core/dev-itpro/sysadmin/cross-company-data-sharing

Now, What happened when you a table under cross-company data sharing (e.g. VendTable) and want to sync vendors through dual-write?

Since, VendTable is one of the data sources for Vendors data entity (dual-write entity map) and we know dual-write does not work well with cross-company data sharing by design. You get following error message when you try to Run the entity map.

"Copying pre-existing data completed with errors.
For additional details, go to initial sync details tab."




The error message is confusing and does not reflect the actual issue behind the scene - you will never be able to figured it out what is wrong until you raise it with MS dual-write team and share the activity Id of the job with them to investigate the telemetry (you don't have access to check this one 😒) then they share the root cause.

However, You can also investigate by putting a breakpoint in method validateDataSharingEnabledForEntityTableBeforDualWriteEnable() of class SysDataSharingValidation.


/// <summary>

    /// Validates that cross company data sharing is not enabled when enabling Dual Write.

    /// </summary>

    /// <param name = "_entityName">The name of the entity containing the table being enabled</param>

    /// <param name = "_tableName">Table in entity</param>

    /// <param name = "_dataAreaId">Company info</param>

    [SubscribesTo(classStr(BusinessEventsRegistrationBase), staticdelegatestr(BusinessEventsRegistrationBase, onTableEnabled))]

    public static void validateDataSharingEnabledForEntityTableBeforDualWriteEnable(str _entityName, str _tableName, DataAreaId _dataAreaId)

    {

        SysDataSharingOrganization sysDataSharingOrganizationTable;

        SysDataSharingRuleEnabled sysDataSharingRuleEnabledTable;

 

        select firstonly SharedTableName from sysDataSharingRuleEnabledTable

                where sysDataSharingRuleEnabledTable.SharedTableName == _tableName

                    join DataSharingPolicy, DataSharingCompany from sysDataSharingOrganizationTable

                         where sysDataSharingRuleEnabledTable.DataSharingPolicy == sysDataSharingOrganizationTable.DataSharingPolicy

                &&  sysDataSharingOrganizationTable.DataSharingCompany == _dataAreaId;

 

        if (sysDataSharingRuleEnabledTable)

        {

            throw error(strFmt("@DataSharing:CrossCompanySharingError", _entityName, sysDataSharingOrganizationTable.DataSharingPolicy));

        }

    }


Dual-write learning series - Dual-write initial sync is the data integrator

One of the features of the dual-write is initial sync where you copy data from the source app (Finance Operation OR DataVerse) to the target...