Announcing Test Parameterization for Early Access & Jira Integration Improvements

We’re excited to announce the general availability of TestRail 7.6 for cloud customers on the Early Access track.

We just made it even easier to complete your work in less time with the introduction of Test Parameterization for TestRail Enterprise customers, letting you leverage dynamic data with Test Parameterization for faster test case design and less maintenance. Users on all TestRail plans can now streamline workflows through an improved Jira integration that allows all customers to edit Jira issues directly from within TestRail and provides increased visibility into associated TestRail data from within Jira.

Please note: At this time, this release is available for cloud customers on the Early Access release track only. Click here to switch to Early Access and receive TestRail 7.6 now. Learn more >

Overview

Test Parameterization lets you leverage dynamic data to scale up test case design and test planning, helping you increase test coverage and reduce the amount of time you spend on test maintenance. By managing datasets and variables that can be reused across your test case repository, it’s easier than ever to ensure each of your test scenarios are executed accurately, yielding results that you can rely on. We also include a few new features in the Test Parameterization management interface that builds on our tradition at TestRail to make it the most intuitive test management solution available today.

New improvements to the TestRail integration with Jira helps teams using Jira save time and simplify their workflows by editing Jira cases directly from within TestRail and enjoying faster loading speeds in Jira. Teams using any requirements tracker can also increase traceability by associating Test Plans and Milestones with requirements.

Finally, teams using Jira can add visibility into testing across their organization by viewing them directly from within Jira – even for Jira users without a TestRail account.

Joins us on June 1, 2022 for the release webinar to learn more about what’s new in 7.6 for Early Access >

Test Parameterization

Prior to TestRail 7.6, TestRail users were required to manually enter test case criteria into each and every test case. The core of the new functionality is the ability for testers to use centrally-managed reusable variable data for testing purposes. By referencing variables in test cases, you can reduce not only the amount of time spent writing test cases for various permutations of test criteria, but also greatly reduce the overall quantity of test cases in their TestRail instance – all with fewer chances of human error. 

Administrators can assign add/modify/delete permissions to roles, ensuring the Test Parameterization data is being managed by the proper users. Testers with adequate permissions assigned to their roles will have a central area for managing Test Parameterization variables and datasets within their projects, allowing them to view and update data as needed so they can quickly and easily create test runs using their datasets.

Test Data and Variables

Within TestRail, the term “test data” refers to the general practice of reusing variables, datasets, and values specific to each project that can be called to quickly insert variables into test cases and to generate test runs based on existing datasets. As we’ll soon demonstrate, when a dataset is used by a test run, testers will see their test data appear in their tests, greatly increasing the flexibility of test cases and allowing for greater test coverage with fewer cases to manage.

Test Data Management

Each project in TestRail includes a dashboard dedicated to viewing and managing test data. You can access the dashboard from your lists of test runs, test suites, or test cases by clicking the View Test Data button.

C6vzClkgFPpOwUHVjmafKTCw4EZqjfywtKPdBFnGk6Vwoye8vI6CFOd8JGPf1t8QC9MuLe9W2NSH492Lg5Q4htEBG2sk t9AxHeXSPt45KnuDJozPm82LTU4tP7tw2X7qql3xEad tKko sj2A

The test data dashboard displays your project’s test data represented by a table that displays your existing datasets and variables. Note that test data is unique to each project.

j51jzq4noZNtqPfK6epqAgfYCXhilR0A3ZJzP3KUmt9P8dtW0VwWbpmIKfKOFNqUHWTMQH9JTdDYfUAih 3Z6LNN4ZAge boB3haRgRmbZdGJaL Q

  • A Variable is a placeholder that you insert into your test cases. You can have up to 500 variables per dataset.
  • A Dataset is a collection of values corresponding to each variable. There is always one default dataset and you can add up to 99 more datasets for a total of 100. 

Datasets are NOT required to have a value for every variable. If a dataset’s value is blank, then any associated test cases use the value from the default dataset, if it exists.

The TestRail dashboard allows you to perform the following actions:

  • Edit your test data – click Edit to open the edit mode for your test data. 
  • To add a new variable, type a new variable name into a blank row of the table’s Variable column. 
  • To add a new dataset, type a new dataset name into the header row of the table. You can then add new values to your new datasets.
  • Click on the row number or column header to select the entire row or column.  To delete the entire selected row or column, click delete the values.
  • You must click Save to retain your changes.

IMPORTANT NOTE: Open test runs always retrieve the current value of the dataset and variable. Editing the values inside of your datasets affects any tests in open test runs. Closing a test run or test plan captures and preserves the dataset values at the time of closure, which creates a static record.

  • Create test runs – click Create Test Runs to create test runs based on your data sets. This action opens the Create Test Runs dialog:

Select the dataset(s) you would like to use for your test run(s), and then click Continue. TestRail now allows you to select what you want to do with the selected dataset(s), as shown in the following dialog.

N3eGtbd5iHSRu ITclziMAavLd4KGTKKXGS8JDNIGqbs4e5EgysTo69pABWtb5kTeizymOANE2qR1BNmprsacdzK4j0 koRsGkmL6dB6xfUwEuwxxSRAYRnEv2WBAX5eDeR YmhL6wbh E6wjA

This dialog provides you with the following options for your new run(s):

  • Create a test plan containing your test runs
    • This option creates one test plan that contains a test run for each of the datasets selected. TestRail then opens the view for creating a new test plan with your new test runs included.
  • Add your test runs to a current test plan
    • This option creates a new test run for each of the datasets selected, but also allows you to add these test runs into an existing test plan. Choosing this option prompts you for the target test plan, and then adds these runs automatically to the selected plan.
  • Create individual test runs
    • This option creates a new, separate test run for each of the datasets selected. You can select test cases for these runs, but all of the test runs are created with the same selection of test cases.

IMPORTANT NOTE: Please keep in mind that it is not possible to add these options to a new or existing test run later.

  • Import test data

Click Import CSV to import a CSV file of datasets and values. 

NWFpZhgnlLwtgM8Qo3dTwIjtUoayUz8mzb7wSdp Ku uAR2KQPROZxxQP2T4Hbd34FDqDz6K hKc NmenvxNjBt8ZBBwXUazOr1ISXrPsMZtwOosbsA8musXUX9Pn cKU0 66d925m9DWzNVIQ

Select your file and the appropriate file encoding, and then select the CSV delimiter to begin the import process.

TestRail displays a preview of the imported data. Verify that the preview looks accurate before importing the data. Any items appearing in green reflect an updated or new value.

wHc40ATwPyYV1HXQRRjVquhzelQhZ fYD0dFw3U d9z7AeYVbiSR9h6dr3S5HiVEcoegfDl9RK4qREFEGWN zEDq2yeQ933nm1

IMPORTANT NOTE: Importing test data via CSV overwrites any values corresponding to matching datasets and variables, including blank values.

  • Export your test data
    • Click Export CSV to export your datasets to a CSV file. In the Export CSV dialog, select the dataset(s) you would like to export, and then click Export.

LxEXv0dL3m qgSJyDOj0abOHX4dEzhGHx2

User Permissions

Test data is specific to each project in your TestRail instance. Administrators use roles and permissions to manage how testers can use this test data. You can create roles such as Lead, Tester, and Guest and apply permissions based on what actions a person in each role performs. 

The Test Data and Variables permission highlighted below affects how your user roles in each project are able to manage this data. 

  • Add/Edit: Allows users to add and edit variables and datasets within the Test Data management area.
  • Delete: Allows users to delete variables and/or datasets entirely.

VWe02LD42Zg RDdLB1j2w3 IaEPj1IKixD PPXQCRZX yhgJgAtnnPpqOAps6piIjcCMuSLx2bTELi9AiYrykhn1ZO 1wY8LRLUOIRYZ59N64n5 peDzBo8noFuGluhoLAB

Using Variables in Test Cases

When you have datasets and variables inside of your project, you can then use this data across your project’s test cases. When editing a test case, you can insert a variable into any markdown-enabled text, string, and separate step field, including shared test steps.

TestRail does not support using variables in any fields not mentioned above, in test case titles, or any in fields outside of test cases (Section names, Test Plan descriptions, etc.).

Insert a Variable into a test case

When editing a test case, you can quickly insert a variable name by typing the name directly into the field, surrounded by percent symbols ‘%’, or by using the variable button to select an existing variable.

Using Variables in Test Runs

When a test run utilizes a dataset and test cases in the test run call variables, TestRail automatically uses the associated dataset values inside of the run’s tests. These values are highlighted in your test, allowing you to quickly identify which variable is in use:

wPkmYklKX9U6 gwxUhDAYweeeOdADHdVsfUQLFsH1YTA3RvXjzkkRno9AG4BUOEw4 pIvwTVh85537s4sO

IMPORTANT NOTE: Open test runs always retrieve the current value of the dataset and variable. Editing the values inside of your datasets affects any tests in open test runs. Closing a test run or test plan captures and preserves the dataset values at the time of closure, which creates a static record. You can configure test runs to utilize datasets by clicking Edit on each one and changing the dataset from the drop-down option, as shown in the following image:

You also can configure test runs inside of test plans to utilize datasets by clicking Description, References & Data, and then click the drop-down menu to choose the dataset, as shown in the following image:

Test Parameterization API Endpoints

TestRail’s test data feature is also supported by the TestRail API. The following API endpoints are available to view and manage your test data:

  • get_dataset
  • get_datasets
  • add_dataset
  • update_dataset
  • delete_dataset
  • get_variables
  • add_variable
  • update_variable
  • delete_variable

Please review our API documentation for further details.

Jira Integration Improvements

Inside TestRail, testers on all TestRail plans now have the ability to update linked Jira issues without needing to switch between tools. When hovering over a linked Jira issue ID inside TestRail, testers will now see an Edit button for making updates to the Jira entity. We’ve also added a References field to Test Plans, which will give testers the ability to link an entire Test Plan in TestRail to external requirements, user stories, or other entities. This field will be available for any integrated tool, including Jira.

The TestRail Integration for Jira add-on received a few updates for this release as well. To accompany support for linked Test Plans in TestRail, the add-on now has a Plans tab available for viewing Test Plan data. We also added a Milestones tab to the add-on, to support the available References field in TestRail’s milestones. Finally, the TestRail Integration for Jira add-on will load inside Jira issues in a collapsed view, giving Jira users more screen space if they aren’t interested in linked TestRail data.

Editing Jira Issues

If your project is integrated with Jira using the Jira Defect Plugin, testers will now be able to edit a linked Jira issue. To edit the Jira entity, simply hover over the linked ID and press the Edit button available in the hover window. Pressing the Edit button will open a form, similar to the Push dialog for creating new Jira issues.

8gS13OLnxn5UszSwAuyGhcVw2P0AeJNZO65nV3hamF7rh1nAVQU4hP X9IR6PNJXwYma3YtFHfDTuTfgdIbnR0I Lj1YmSm73EyP5fWOkAZJnsiwERrbJ83BswOc3x3Sg wtwHLu4Kgvp202OQ

Test Plan References

To allow greater flexibility for users, TestRail now has a References field available for Test Plans. This field will be available for any Test Plan, regardless of the integrated tool. With the addition of References on Test Plans, TestRail now supports linking Test Results (using the Defects field), Test Cases, Test Runs, Test Runs inside Test Plans, Test Plans, and Milestones to external entities, such as User Stories, Bugs, or Epics.

New Tabs on the TestRail Integration for Jira Add-on

The TestRail Integration for Jira add-on now has tabs for both Milestones and Test Plans, allowing Jira users to retrieve the most recent data from their TestRail instance and navigate directly to the linked TestRail item.

7qtqLhwICj77vVGndZwFx0 Jy8vocI

Other Minor Changes

  • The Jira SOAP Defect and Reference plugins will no longer be distributed with TestRail’s installation files.
  • The TestRail Integration for the Jira add-on will load in a collapsed view inside Jira issues. This provides more screen space to Jira users who aren’t interested in TestRail information and reduces the load on the connected TestRail instance.

PVSMynYbQz0X8iTxyy8erD97za9DuoBMQrI4wlwodDTpkKFU9GGQqq4j S9IENtdT35NxZN36axr7dX7ahKAqLYcBgwC4q0cQf69oeJXwVHkSGM96pefHtXSjqsobAnZAC6UZXkayh n6WfNhw

Getting the Updated TestRail Integration for Jira Add-on

Jira Cloud users will automatically receive the updated add-on through Jira.

Jira Server administrators can download the latest version of the add-on through the Jira Administration area, or the Atlassian Marketplace.

Getting TestRail 7.6 on Early Access

You can start a 14-day free trial of TestRail here (cloud or server): https://secure.gurock.com/customers/testrail/trial/. Once your instance is ready, send a request here to get your instance upgraded to the Early Access track. 

If you want to create a subscription for TestRail Cloud, you can do so from within TestRail via Administration > Subscription. If you want to order TestRail Server licenses, you can do so from our website here: https://secure.gurock.com/customers/shop/annual/purchase/

Registered customers can download the full version from our customer portal: http://www.gurock.com/portal/

TestRail Cloud

TestRail Cloud instances are automatically updated according to the currently chosen release track. Read more about Quarterly Release release tracks here. You can check your version via the TestRail Help & Feedback > About TestRail menu item.

TestRail Server

If you are using the on-premise version of TestRail, you can update to the new version as usual by installing it over your existing TestRail installation (there’s no need to uninstall your existing installation). The database upgrade wizard is automatically started when you access TestRail with your web browser. Please see the update instructions for details: http://docs.gurock.com/testrail-admin/installation-upgrading/

If you’re using Docker, please refer to the instructions here: http://docs.gurock.com/testrail-admin/installation-docker

Please ensure you make a backup of your current on-premise installation before upgrading to the new version.

TestRail Enterprise

If you’d like to learn more about SSO, Project Level Administration, Test Case Reviews, or any of the other features on TestRail Enterprise, please email us for a trial or quote via [email protected], or you can use the contact form here: https://secure.gurock.com/customers/support/.

Not sure which TestRail plan you’re on? Reach out and we’d be happy to help.  Learn more about Quarterly Release release tracks here to receive TestRail 7.6 now.

In This Article:

Sign up for our newsletter

Share this article

Other Blogs

Announcement, Integrations, Webinar

How to Increase Visibility and Efficiency of Your Test Automation with TestRail and Sauce Labs

Most software development teams use some form of automation in their testing processes to improve the efficiency, speed, and accuracy of repetitive regression tests. Using automation allows you to perform a larger number of tests in a shorter time frame and...

Announcement, TestRail

Introducing TestRail’s New Brand

We are thrilled to announce the launch of TestRail’s new brand.  You have probably noticed a few new things already: updated colors, a refreshed logo, and our new “testrail.com” website.  But our new brand goes beyond just colors and design....

Announcement, Automation

Announcing the TestRail CLI Tool

The TestRail team is proud to announce the release of the TestRail CLI Tool. Sending automated test results to TestRail just got a lot easier. By parsing and uploading JUnit style test results into TestRail directly from the command line or by running the C...