Dynamics 365 finance and operations Regression testing with Playwright

Introduction

Regression testing is essential for Dynamics 365 Finance and Operations, especially when applying standard service updates or rolling out new customizations. Its purpose is to ensure that recent changes don’t break existing functionality. However, performing these tests manually is time-consuming, repetitive, and error-prone. Automating regression testing with the right tools significantly improves efficiency. It speeds up test execution, reduces manual effort, and enables reusability of test scripts across releases—saving time and effort in the long run.
 
There are many regression testing tools available in the market. Among them, Microsoft’s Regression Suite Automation Tool (RSAT) stands out as a purpose-built solution for Dynamics 365 Finance and Operations. RSAT integrates seamlessly with Azure DevOps, offering a comprehensive and streamlined testing experience. If you're looking for a complete solution to manage and automate testing within the Microsoft ecosystem, RSAT should be your primary consideration.
 
In addition to RSAT, Microsoft has introduced Playwright, a powerful open-source framework for end-to-end (E2E) testing and browser automation. Playwright is exceptionally well-suited for testing modern web applications including Dynamics 365 Finance and Operations apps. It supports modern browsers like Chromium, Firefox, and WebKit, and provides a unified API for consistent cross-browser testing. Playwright supports multiple programming languages, including JavaScript, TypeScript, Python, .NET (C#), and Java, making it highly versatile. Its robust API and ability to handle complex web interactions make it especially suitable for testing modern applications like Microsoft Dynamics 365. For regression testing, Playwright offers fast, repeatable, and reliable automation capabilities, making it an ideal choice for testing business-critical processes.

 
In this blog post, I will show you how Playwright to automate regression testing for Dynamics 365 Finance and operations apps, using an end-to-end (E2E) scenario: Sales Order Creation → Packing Slip → Invoicing.

Pre-requisites
 
Before getting started, the following are necessary for installed and configured:

 
You can refer to the official Playwright documentation for installation guidance.
 
Once installed successfully, you can verify the setup by checking the installed version of Playwright using: 

npx playwright --version

 
To set up a new Playwright project, use the following command:

npm init playwright@latest

This command creates a sample Playwright project structure with essential configuration files and sample tests. From here, you can begin customizing and building your regression test suite tailored for Dynamics 365.
 

 
 
Recording Tests in Playwright

 

Make sure that the red “REC” icon is lighted in the browser, this indicates that recording is active. As you interact with the application, such as by clicking buttons, entering text, or navigating between pages, Playwright will automatically generate the corresponding test code. This code is saved into a new test file within your project. Once you’ve completed the desired actions, you can stop the recording and close the browser window. The recorded script is then ready for further refinement and execution directly from within VS Code.
 
In my case, I used this feature to record a full sales order creation process, starting from Login, order creation through to packing slip generation and final invoicing. For applications like Dynamics 365 Finance and Operations, it’s important to begin recording from the login page to capture authentication and session initialization. The script is generated live as you interact with the application.
 

 
After recording, it’s important to perform some additional steps to ensure the quality and maintainability of your test scripts.

 
Playwright Test Runner
 
Playwright tests can be executed in two primary ways: directly from Visual Studio Code using the built-in UI, or from the command line using terminal commands. Both methods are useful and serve different purposes, depending on the stage of your testing workflow. Here's a comparison between running Playwright tests via UI and via the command line:

 
Completed steps will display their execution duration. Steps that are still running will show a "Waiting" status.

In practice, it's best to use the Visual Studio Code UI during the development phase—for creating and debugging tests—and then switch to the command line for running larger test suites or automating your regression testing as part of continuous integration workflows.
 
Playwright Trace Viewer
 
The Playwright Trace Viewer is a powerful visual debugging tool that allows you to replay and inspect your test runs step by step. It captures detailed information during test execution, including screenshots, DOM snapshots, console logs, and network activity. This makes it much easier to understand what happened during a test, especially when diagnosing failures or analyzing complex UI behavior.
 
To use the Trace Viewer, you first need to enable tracing in your test script. This opens an interactive viewer that visually displays each step the test performed. Within the viewer, you can examine actions, locators, and network requests along a timeline. You can also see what the application looked like at each step through screenshots, which helps you understand exactly when and where a test might have failed.

 
When you run Playwright tests from the Visual Studio Code UI with tracing enabled, the Trace Viewer becomes available automatically after execution. It provides a clear breakdown of the test flow, allowing you to inspect each interaction and better understand how the script is behaving. The Playwright Trace Viewer is especially valuable when debugging flaky tests or troubleshooting issues in dynamic, UI-heavy applications like Microsoft Dynamics 365. By offering full visibility into what happened during the test, it significantly shortens the time needed to pinpoint and fix issues.
 
For example, when I highlighted my steps during the execution, there are displays to show the script is referring to Posting Invoice's OK button.

 
Playwright Test Report
 
The Playwright Test Report, an automatically generated interactive HTML summary, presents test results including passed, failed, and skipped tests. This report enables quick identification of issues, review of test steps, and analysis of errors through detailed information like test duration, error messages, and stack traces. You can access the report after test execution by running npx playwright show-report to visually review and analyze results. The report serves as a valuable tool for debugging and tracking regression testing progress.

 
Upon completing regression testing, you can validate your results using the Playwright Test Report. The report’s detailed page includes execution logs, screenshots, and trace files to aid in issue analysis, and you can download it for further examination.

Microsoft Playwright Testing
 
Microsoft Playwright Testing is a fully managed, cloud-based service built on the open-source Playwright framework. It empowers teams to run Playwright test suites at scale—across multiple browsers, operating systems, and regions—without changing test code or local configurations.
 
Here is Key Capabilities:

• Scalable Test Execution - Run your Playwright tests in the cloud with up to 50 parallel workers, dramatically reducing test cycle time from hours to minutes.
• Cross-Environment Validation - Test your applications across a wide range of browser-OS combinations and mobile emulations, whether hosted publicly or behind a private network.
• CI/CD Integration - Seamlessly integrate with GitHub Actions, Azure Pipelines, and other CI/CD tools to automate testing with every code commit.
• Advanced Debugging Tools - Leverage screenshots, videos, and trace files via the Playwright portal’s Trace Viewer for fast and effective issue resolution.
• Developer Productivity - Use the Playwright Test extension for Visual Studio Code or the Playwright CLI for streamlined test authoring and automation.

When you connect your Regression testing project with Microsoft Playwright Testing, you can execute your regression testing with cloud-based browser with up to 50 parallel worker, which can distribute your testing workload to enhance the completion time, and you can connect to browsers in the region closest to the location where tests are initiated. This region may differ from the region where the workspace is created. Regional affinity can help minimize network latency during test execution.
 
By connecting your regression testing project to Microsoft Playwright Testing, you can, Execute tests using cloud-based browsers with up to 50 parallel workers. And also distribute workloads efficiently to accelerate completion time, you can also select browser regions closest to test initiation points, independent of workspace location, to reduce network latency and improve performance.


 

 
What is Benefits of Microsoft Playwright Testing

• Exceptional Scalability: Run tests across up to 50 cloud web browsers in parallel on Azure infrastructure, significantly cutting execution time.
• Comprehensive Coverage: Automatically test across Chromium, Firefox, and WebKit on Windows and Linux, with mobile emulation support for consistent validation.
• Rich Reporting & Artifacts: Access centralized dashboards with detailed logs, screenshots, and trace files for simplified debugging and root-cause analysis.
• Seamless Integration: No code changes required—your existing Playwright tests work out of the box. Integrates with VS Code, Playwright CLI, GitHub Actions, Azure Pipelines, and more.
• Built-in Visual Comparison: Manage visual snapshot comparisons across cloud environments to reduce false positives and streamline visual regression testing.