Introduction

As software development continues to evolve, ensuring the quality of your Canvas App’s user interface is paramount. With Test Studio, building end-to-end UI tests for your canvas app has never been easier. Not only can you validate your app’s functionality when you initially build it, but you can also maintain its quality by continually testing it as new changes or updates are deployed. In this article, we will explore how Test Studio can help you streamline your canvas app’s UI testing process, ensuring that your users have a seamless experience.

Estimated reading time: 18 minutes

Table of contents

Getting Started with Test Studio

Test Studio Terminology

Test Cases

Test cases consist of a series of test steps that are executed to validate the functionality of an app or specific features within an app. For instance, in an Expense app, you may want to ensure that expense reports can only be submitted if they have an associated actual cost. Writing a test case can help verify that this requirement is consistently met.

In Test Studio, test steps are written using the Power Apps expression language. Test expressions can include app functions and additional expressions to support automated testing. These expressions help simulate user actions and interactions with the app and enable the test case to verify that the app is functioning as expected.

Test Suites

Test suites are used to organize test cases into specific features or functionality. As the number of test cases in an app increases, grouping them in test suites becomes essential. For instance, one test suite can focus on validating expense report submissions, while another can focus on expense approvals.

When test cases are contained in a test suite, they run sequentially, and the app state is persisted across all test cases within the suite. For example, if a test case completes on Screen 5 in the app, the next test case in the suite will begin running from Screen 5. This allows for breaking down a complex test scenario into multiple test cases within a single suite, and the state is shared across all test cases.

If a test case expects to begin at the start screen of the app, you can navigate to the start screen as the first step in the test case. It’s important to note that the app is not reloaded at the beginning of every test case in a test suite when planning test execution. Therefore, designing test cases with the app state in mind is essential to ensure that tests are executed correctly.

Test Assertions

Every test case should have a clear expected result. To validate the expected result against the actual result of the test, test assertions are used. An assertion is an expression that evaluates to true or false in the test. If the assertion returns false, the test case will fail.

In the Expense app example, an assertion can be written to validate whether an expense report is created with an expense line item having zero cost associated. This assertion would help to verify that the app is behaving as expected, and if it fails, it indicates a problem in the app that needs to be fixed.

Test assertions can be used to verify various conditions, such as checking the correctness of calculations, the visibility of elements, or the behavior of app controls. By using assertions, we can ensure that the app is functioning as intended and identify any issues or bugs that need to be addressed.

How To Create Your First UI Test with Test Studio

In this quickstart, you’ll create tests for a canvas app called Suggestion Box. You can also explore and discover testing concepts and apply them to writing tests for your own canvas apps.

Open Test Studio

1. Sign in to Power Apps.
2. Create a new app or edit an existing app.
3. Save your app to open Test Studio. Please note, you must save an app before you can write tests for the app.
4. Select Advanced tools in the left navigation bar.
5. Select Open tests to open the Test Studio for this application. This action opens Test Studio in a new browser tab.

Tests created in Test Studio are published and stored within the app package. When you export and import a canvas app package to another environment, all the test definitions such as test suites and test cases you have created are included.

This makes it easy to transfer tests between environments and ensures that tests can be executed in the target environment without needing to recreate them. It’s important to note that when importing a canvas app package with tests into a new environment, you may need to update any connection references or environment-specific settings to ensure the tests function correctly.

Create a Test Suite

When you start using Test Studio, a default test suite and test case are created for you. Test suites are used to organize your test cases into logical groups. An app can contain one or more test suites.

You can start writing your tests immediately using the default test suite and test case, or you can create a new test suite based on your testing needs. Using multiple test suites allows you to organize your test cases based on functionality or features, making it easier to manage and execute tests.

It’s important to note that the default test suite and test case are provided as a starting point and may not fit your testing requirements. Creating new test suites and test cases based on your specific testing needs is recommended to ensure effective testing of your app.

Before you create your first test suits, make sure Formula-level error management setting is enabled.

1. Select New suite.
2. Update the Test suite name and description by selecting the fields on the main grid.

Create a Test Case

To effectively organize and group your tests, you can create multiple test cases within a test suite. Each test case can be designed to test a specific feature or a subset of functionalities in your application, depending on your testing needs.

1. To create a new test case, first select the desired test suite.
2. Then, navigate to the top menu and select New Case.
3. From there, you can update the Test case name and description by selecting the appropriate fields on the main grid. This will help ensure that your tests are well-organized and that each case is focused on testing a specific aspect of your application.

Record a Test Case

A test case comprises of a set of test steps, where each step consists of an action that is performed using Power Apps expressions. To create these actions, you can use the recorder feature in Power Apps, which automatically generates test steps as you interact with your application.

Once you have recorded the actions, you can modify the test case as needed by adding or deleting steps and writing test assertions to validate the outcome of the test. It’s essential to ensure that the app is published before recording a test case since only the published version of the app will be available for recording. If the recent changes are not published, the test case will be recorded using the last published version of the app.

Note: To avoid discrepancies while recording, make sure to publish any recent changes to the app before starting to record a test case.

1. Select Record from the top menu. This action opens the published app with recording mode in a new browser tab.

2. Interact with your application, and your actions will be documented in the left panel.
3. Upon completion, click Done. If desired, click Cancel to return to Test Studio without saving the recorded interactions.

4. View the test steps and the expressions that were automatically generated for you in Test Studio.
5. If needed, modify the step description text within the main grid. To update test step actions, simply select the corresponding formula on the main grid.

Add Test Steps and Test Assertions

Each test case should include an expected outcome. In the Suggestion Box example, a successful Idea send should result in a new record created. To update the test case and add steps for verifying record creation, follow these steps:

• At the beginning of the test case, initialize a variable to store the Submitted Idea record count in the collection.
• At the end of the test case, initialize another variable to store the Submitted Idea record count in the collection.
• Create a test assertion expression to confirm the count has increased by one. If the count does not increment by one, the test assertion and test case will fail.

To include test steps and assertions in the Suggestion Box app:

1. Choose Step 1 or the step where you want to insert a new step above.
2. Click Insert a step above in the top menu or select the option from the active row. This will generate an empty step.

3. Update the step description. For example, Count Ideas Before Test.
4. Enter an expression or formula into the action input to count the records in the database before executing the test.You can use any supported expression. You can also query any data sources, collections, variables, or run flows that are contained in your app, and create new global variables or collections to use in your tests.

Set(IdeasBeforeTest, CountRows(Ideas))

5. Select Step 3 or the step above which you want to insert a new step.
6. Select Insert a step above from the top menu or by selecting the option from the active row. This action creates an empty step.
7. Enter an expression or formula in the action input to Trace and write the kudosBeforeTest value to test the results record.

Trace("Submitted Ideas Before Test: " & IdeasBeforeTest)

8. Go to the bottom of the test case and insert a new step to count the records in the database after the test has completed.

Set(IdeasAfterTest, CountRows(Ideas))

9. Add a final step to validate that the record count in the database has increased by a count of 1, and enter the following assertion action to verify:

Assert(IdeasAfterTest = IdeasBeforeTest + 1, "Submitted Ideas count incorrect. Expected : " & IdeasBeforeTest + 1  & " Actual :" & IdeasAfterTest)

10. Save the test case from the top-right menu in Test Studio.

Running and Analyzing UI Tests in Test Studio

Playback your test

To verify your application’s functionality, you can playback your recorded tests. Either play back all tests within a single test suite or focus on a single test case. However, prior to playing back the recording with the latest modifications, ensure that the application has been published.

1. Select Publish to automatically save and publish your test.
2. Select either a test suite or a single test case.
3. Select Play. The published app opens in Play mode, and you can see your test steps playing back automatically. A green check mark indicates when a test step is executed successfully. If a step fails, a red failure indicator along with a failure message is displayed.

4. Select Done to return to Test Studio.

Failed Assertion

In this section, you will intentionally modify the test assertion to induce a test failure:

1. Locate the assertion step and click on the expression box.
2. Change the test action by updating the “+ 1” to “+ 2”. This alteration signifies that the test anticipates the creation of two records, which is inaccurate. A successful test should result in the creation of only one record in the database.

Assert(IdeasAfterTest = IdeasBeforeTest + 2, "Submitted Ideas count incorrect. Expected : " & IdeasBeforeTest + 2  & " Actual :" & IdeasAfterTest)

3. Click on Publish.
4. Click on Play.
5. Observe the test playback in progress. Upon reaching the final step, the test will fail, displaying an error message and the custom message you input during the assertion step.

Running Tests in a Browser with Persistent Links

Utilizing Test Studio, you can generate a shareable link to play a test in a separate browser. This feature is particularly beneficial for integrating your tests into a continuous build and release pipeline, such as Azure DevOps.

The play link for a selected test remains constant, ensuring that neither the test suite nor the test case is affected. This enables you to update your tests without having to modify your build and release processes, streamlining your workflow within the Azure DevOps environment.

To play tests in your browser:

1. Select a test suite or test case in the right pane.
2. Select Copy play link.
3. You’re prompted to publish your tests if there are any unpublished changes.
4. You can select to skip the publish process and copy the play link. New test changes don’t play if you skip.
5. Open a browser and paste the URL into the address bar to play the test.
6. View your test playing back.

Set Up Your Tests With Ease

The OnTestCaseStart property in a Test Suite helps you set up your tests efficiently. This expression triggers for each test case in the suite before execution, saving you from repeating test steps. Customize this property to handle tasks common to all cases, like:

• Always starting from the first screen
• Initializing common collections or variables
• Fetching test data for the current test

In the following example, we will tailor the OnTestCaseStart property to ensure each test case starts at your app’s first screen. Additionally, you’ll retrieve test data from a data source for use in the test case steps.

1. Select Test in the left pane or View on the suite header.

2. Select the OnTestCaseStart action.

3. Input an expression to Navigate to first screen and fetch the test data for your test.

//Start every cases on the first screen in the Kudos app
Navigate('Dashboard Screen');

//Initialize my test data for the current case. 
Set(currentTestData, LookUp(SuggestionBoxTestData, TestCase = TestCaseInfo.TestCaseName));

Reviewing Test Results Outside Test Studio

When playing back tests in browser, the test panel is not visible, making it difficult to see the specific test step executed or whether a test passes or fails.

To process test results outside Test Studio, use the OnTestCaseComplete and OnTestSuiteComplete properties in the test object. These properties are useful when integrating tests into a continuous build and release pipeline, like Azure DevOps, to decide whether to proceed with app deployment.

The expressions for these properties trigger when each case or suite finishes. Customize them to process and send test results to various data sources or services, such as:

• SQL Server
• Dataverse
• Power Automate
• Email with Office 365

These settings apply to every test suite or case in your app. After completion, test results and trace messages are available in the TestCaseResult and TestSuiteResult records.

In this quickstart, we will set up two custom tables in the Dataverse database for storing test results by modifying the OnTestCaseComplete and OnTestSuiteComplete properties.

1. Select Test in the left pane or View on the suite header.
2. Select the OnTestCaseComplete action.
3. Input an expression to process the results of your test. The following sample saves each test case’s results to the custom AppTestResults table in Dataverse. The test results can optionally be stored to SQL, SharePoint, or any other data source. You might need to set or increase the Trace field in your data source as required.

//Save to Dataverse
Patch(AppTestResults
, Defaults(AppTestResults)
, {
         TestPass: TestCaseResult.TestCaseName & ":" & Text(Now())
         ,TestSuiteId: TestCaseResult.TestSuiteId
         ,TestSuiteName: TestCaseResult.TestSuiteName
         ,TestCaseId: TestCaseResult.TestCaseId
         ,TestCaseName: TestCaseResult.TestCaseName
         ,StartTime: TestCaseResult.StartTime
         ,EndTime: TestCaseResult.EndTime
         ,TestSuccess: TestCaseResult.Success
         ,TestTraces: JSON(TestCaseResult.Traces)
         ,TestFailureMessage: TestCaseResult.TestFailureMessage
}
);

4. Select the OnTestSuiteComplete action.
5. Input an expression to process the results of your test. In the following sample, you’ll save each test suite’s results to the custom AppTestSuiteResults table in Dataverse.

//Save to Dataverse
Patch(AppTestSuiteResults
    , Defaults(AppTestSuiteResults)
    , {
         TestSuiteId: TestSuiteResult.TestSuiteId
         ,TestSuiteName: TestSuiteResult.TestSuiteName
         ,StartTime: TestSuiteResult.StartTime
         ,EndTime: TestSuiteResult.EndTime
         ,TestPassCount: TestSuiteResult.TestsPassed
         ,TestFailCount: TestSuiteResult.TestsFailed
    }
);

Also we could receive an app notification of the test result:

Notify(TestCaseResult.TestCaseName & " : "
        & If( TestCaseResult.Success
            , " Passed"
            , TestCaseResult.TestFailureMessage)
        ,If(  TestCaseResult.Success
            , NotificationType.Success
            , NotificationType.Error)
)

Best Practices for UI Testing Your Canvas App

To improve the quality of your canvas apps when testing with Test Studio, consider the following refined best practices:

1. Determine which tests to automate: Don’t rely solely on automation and prioritize tests that are repetitive, high-impact, stable, require multiple data sets, or are time-consuming to perform manually.
2. Keep test cases small: Instead of writing one large test case, break it down into smaller test cases, each focusing on a specific feature or functionality. This makes it easier to isolate failures and test other functionality independently.
3. Use single test actions: It’s best to keep test actions to a single expression to simplify debugging and error isolation. Consider dividing large multi-action test steps into smaller, single-action test steps.
4. Set expected results: Each test case should have one or more expected results, and test assertions should be used to verify the expected outcomes against actual outcomes.
5. Use test suites: Group similar test cases together and describe the purpose and expected results of your test for easier maintenance.

Test Studio Limitation

Please note that Test Studio has certain limitations when it comes to Power Apps UI testing.

• Components.
• Code components written in the Power Apps Component Framework.
• Nested galleries.
• Media controls.
• Formula-level error management experimental feature needs to be turned on for the app.
• Support for controls not listed in the Select and SetProperty functions.
• Person-type columns.
• Test Studio is not compatible with the experimental Git version control feature, and will not work properly if that feature is enabled.

Conclusion

In conclusion, Test Studio is an invaluable tool for conducting UI tests on your Canvas App, ensuring a smooth and efficient user experience. By leveraging features you can optimize test setup, execution, and result analysis. Remember to consider the bigger picture and utilize various data sources and services to make the most of your testing process. With Test Studio, you can elevate the quality of your Canvas App and create a product that stands out in today’s competitive market. Happy testing!

In the upcoming article We’ll dive into setting up and running your Canvas App tests created in Test Studio using pipelines. We’ll explore the seamless integration of these tests with Azure Pipelines, allowing you to streamline your testing process effectively. Additionally, we’ll discuss generating realistic demo data to ensure your tests are comprehensive and reliable, ultimately helping you maintain a high-quality Canvas App. Stay tuned for a detailed guide on optimizing your testing process with Azure Pipelines and Test Studio.

Resource Links

These links below gave me inspiration and use cases in this post:

• Microsoft Build: Test Studio
• Chris Baldwin: Power Apps & Automated Testing
• David Uhlmann: Creating realistic Demo Data for Demo/POC scenarios