Maximizing Dynamics 365 Data Migration Performance: 6 Best Practices

6 Best Practices for Dynamics 365 Data Migration

As businesses continue to adopt Microsoft Dynamics 365, many face the challenge of migrating their existing data and processes to the new cloud platform. While the migration process can be complex, following the best practices can help ensure a smooth transition and maximize the benefits of Dynamics 365. We will explore some of Dynamics 365 migration best practices that organizations should consider when moving their data to this platform.

1. Use Batch Size with Multi-threaded Writing

One key consideration for improving performance is using a batch size in conjunction with multi-threaded writing. When doing so, it is essential to use the right combination of batch size and thread number. The recommended batch size for CRM entities is 10, while for attachment entities, such as annotation or activitymimeattachment, it’s 1. Keep in mind that the maximum batch size is 1000, but higher batch sizes don’t always offer better performance and may cause service call timeouts.

Also, enable multi-threaded writing whenever possible – the thread number for an on-premises Dynamics 365/CRM server installation can be as high as the application and database servers can sustain the load. However, for CDS or Dynamics 365 online instances, the number of threads that can be used depends on the server-side throttling that Microsoft has put in place. Typically, the recommended thread number is 10, and after a certain number, further performance improvement would stop.

2. Minimize the Number of CDS/CRM Fields

An option to improve the performance of the CDS/CRM destination component is to minimize the number of CDS/CRM fields that you want to write to and reduce network latency. When migrating data to Dynamics 365 CE/CRM, it is common to have large amounts of data that need to be processed. Writing to a large number of fields can significantly increase the amount of time it takes to complete the migration process. By identifying and minimizing the number of fields that need to be written to, you can reduce the amount of time it takes to complete the migration.

3. Reduce the Network Latency

To lower network latency to your Dynamics 365 CE/CRM server, ensure your computer and server are connected to a fast and reliable network. Consider using a cloud-based Dynamics 365 CE/CRM server, which can help reduce latency and improve data transfer speed. Furthermore, you can reduce network latency by running the integration on an Azure SSIS-IR or Azure Windows Server virtual machine.

4. Run the Integration on an Azure SSIS-IR or Azure Windows Server Virtual Machine

Consider running the integration on an Azure SQL Server Integration Services Integration Runtime (SSIS-IR) or an Azure Windows Server virtual machine. Azure SSIS-IR provides a scalable and secure way to run SQL Server Integration Services (SSIS) packages in the cloud while reducing the cost and complexity of managing on-premises infrastructure. Running the integration on an Azure SSIS-IR can help to improve the performance of the migration process by leveraging the scalability and processing power of the cloud infrastructure.

Alternatively, running the integration on an Azure Windows Server virtual machine can also provide a scalable and secure way to run the integration in the cloud. Azure virtual machines can be configured with a variety of hardware specifications, including CPU, RAM, and storage capacity, providing the flexibility to choose the optimal configuration for the migration workload with a range of features and tools for managing and monitoring the migration process, including automatic scaling, high availability, and built-in monitoring and diagnostics.

5. Upgrade the Server Capacity and Increase the Network Bandwidth

When migrating large amounts of data, migration performance can be affected by the server’s capacity. If the server can’t handle the amount of data being processed, it can slow down the migration and result in longer processing times. To upgrade the server capacity, increase the server processing power by adding more RAM or upgrade the storage capacity. Upgrading the server capacity can also improve the performance of other applications running on the server, resulting in faster processing times for all tasks.

The network bandwidth can be a bottleneck that slows down the migration process. Network bandwidth can be increased by upgrading the network hardware, such as network adapters or switches, or by increasing the current bandwidth through the internet service provider. Another way is to optimize the data transfer protocols used for the migration process. For example, using a more efficient data transfer protocol such as TCP/IP or HTTP can improve data transfer speed.

6. Select the Correct Matching Fields for Update or Upsert Actions

This option will improve the performance of the CDS/CRM destination component during the migration. When performing an update or upsert action in Dynamics 365 CE/CRM, the migration tool will compare the source data with the existing data in the target system to determine whether a record needs to be updated or inserted. This comparison process is typically performed by matching fields between the source and target systems; selecting the correct matching fields can help ensure the comparison process is accurate and efficient without the need for a complex search algorithm.

Also, take into account data types and formatting of the matching fields when selecting the correct fields. Incompatible data types or formats can lead to errors and processing delays. Selecting the correct matching fields can help improve the quality of the migrated data, ensuring data is accurate and consistent between the source and target systems. This will reduce the need for manual data cleaning or correction after the completed migration.

A focus on the destination component should be considered to improve the performance of the Dynamics 365 CE/CRM server and database server.

Additional Recommendations

• Identify any data dependencies that may exist between different applications and processes, as well as any external data sources that may need to be integrated with Dynamics 365.
• Plan data migration in stages, starting with the most critical data and applications. This will allow testing and validation of the migration process before moving on to more complex or less critical data.
• Ensure that manually specified fields used for Update or Upsert actions are covered by database indexes.
• Use straight Update or Create actions instead of Upsert (except when Alternate Key matching is used) to avoid the extra service call that queries the target system.
• Use the “Ignore Unchanged Fields” option with Upsert to skip unchanged fields and improve integration performance.
• Choose the right cache mode when using the text lookup feature.
• Sort input data by impersonation fields to better organize records for batching.
• Use automated tools and scripts if possible to help with data mapping, extraction, transformation, and loading.
• Evaluate the performance impact of plugins and workflows and revisit their design if necessary.
• Leverage process optimization options to disable plugins, audits, and workflows during data load.
• Avoid using the “Remove Unresolvable References” option unless necessary.
• Avoid relying on the Duplicate Detection option.
• Test the migration process thoroughly to ensure that data has been migrated correctly and that all applications and processes are working as expected.

Overall, these best practices aim to improve the performance of the data migration process and ensure a smooth transition to the CDS/CRM destination component.

This was not a complete list but just an introduction to best practices for data migration in Dynamics 365.  To learn more, read our advanced topics article.  The tools offered by Kingwaysoft.com will guide you through an effective and efficient data migration project. Contact us for details.

by KingswaySoft www.kingswaysoft.com

The post Maximizing Dynamics 365 Data Migration Performance: 6 Best Practices appeared first on CRM Software Blog | Dynamics 365.