What if you wanted your test cases to be executed under different scenarios like on different platforms (OS/Hardware) or in different environments (testing/production) or on different web browsers Well... all this can be easily managed with test plan configurations, you have to have custom fields of dropdown or multiple choice types that can be "Used for test plan configuration"
How configurations help?
With configurations you can make sure whether a test case renders the expected result under different conditions if not then how much the results vary? How much time it takes for a test case to run in a particular scenario? On that basis you can analyse what can be done to reduce the gaps?
Let us take some examples, suppose you have custom fields like platform , environment and web browser added to your project and all these can be used to define configurations.
Example 1: Configurations based on single field
Web Browser: Chrome, Firefox and IE
Here 0, 1, 2 or all 3 configurations can be selected and assigned to your testers. You can see results for test cases in Chrome, Firefox and IE in a single Test Plan.
Example 2: Configurations based on multiple fields
Here are the selections -
Web Browser: Chrome and Firefox
Platform: Linux, Windows and Mac
Now you have option to either have none, or only a few, or all of the above combinations as configurations, i.e. you can have 0 to 6 configurations.
Manual definition of configurations for test plan
As on writing this, we have introduced manual configuration definition option; herein if the number of possible combinations initially exceeds 30. With this the individual values can be selected from all (or some) of the fields to define a configuration manually while adding a test plan.
Once a Test Plan with configuration is created you will see a configuration selection dropdown each in each Test Plan-
Once configurations are in place, here is what they can be basis of:
Create test plans so that they can be tested in different scenarios
Assign test cases to testers, based on configurations
See results for test plan for each configuration
Testers can run test plan in each configuration
Exporting test results and compare result for each configuration
Filter Test Plan reports for each configuration
Here is a video that explains the process in detail:
Note: Configurations are premium feature so they may not be the part of all pricing plans.