We’re excited to announce the general availability of TestRail 7.5 for TestRail Cloud and Server customers.
TestRail 7.5 introduces webhooks, which give you the ability to trigger real-time notifications based on events in TestRail like saving new test cases, creating new test runs, adding test results, and more. This allows you to build custom integrations with other tools in your tech stack—like triggering CI builds, programmatically kicking off test automation, and getting notifications in communications channels like Slack—so that you can streamline the connection between TestRail and the rest of your software development toolchain and increase visibility into testing activities for everyone on your team.
TestRail 7.5 Server is now in full parity with features available in TestRail Cloud 7.5, including security improvements with expanded SSO implementation standards (available on the Enterprise edition), Multi-Factor Authentication (MFA), and new API endpoints for enhanced User management.
At the heart of the new webhooks functionality is the ability to automatically send notifications and payloads to external systems or applications when events occur in TestRail, like when new test cases are added, test runs are created, or test results get updated. This way, you can automatically trigger an automated test build or pipeline in your CI tool through its API or post real-time updates to a Slack channel.
With webhooks, you can now integrate TestRail with the rest of your tech stack, save time, and give everyone on your team more visibility into the activity and progress of your testing. You can create a webhook globally across all projects or choose specific projects that you want to apply your new workflow to.
For TestRail Enterprise Server customers, we’ve expanded Single Sign On (SSO) functionality by adding support for OpenID Connect (OIDC) and OAuth 2.0 standards, adding to TestRails existing support for SAML 2.0. Now more organizations can enforce SSO policies to help ensure users’ credentials aren’t lost, stolen, or reused.
Speaking of keeping data safe within your TestRail instance, all TestRail Server customers can now make sure users log in with Multi-Factor Authentication (MFA). By enforcing authentication with email and popular authenticator apps like Google Authenticator, Microsoft Authenticator, and Duo mobile, you’re better protected from stolen credentials being used to access sensitive data.
That’s not all! We just made it easier to create users and user groups with your LDAP service, like Azure Active Directory, with new API endpoints. You can also now control which projects users and user groups can access and retrieve and update additional user information, such as group membership and administrator access. As an added layer of security for the TestRail API, administrators now have the option to require the use of API keys instead of account passwords.
Head over to the TestRail Administration console and navigate to the Integrations tab. From there, choose the Webhooks tab. The new webhooks management console makes it easy to create new webhooks as well as test, monitor, and manage webhooks you have already created.
In this example, we have created two webhooks already. The first one sends a notification to Travis CI that triggers a CI/CD server build job that runs a suite of automated tests. The second one sends notifications to Slack when new test cases are designed to help communicate to teams as our test coverage grows for maximum visibility into our testing efforts.
Good news! The status column shows that both of our webhooks are currently working because it is receiving a HTTP response code of 2-something. If the webhook receives a different response, like a 404 or a 500, the status for the webhook would be flagged red, indicating we need to investigate the problem.
Each webhooks has a name and some additional information relating to the TestRail projects with which they are associated with as well as the events which will spawn a webhook payload. This way it’s easy to parse through your list and recognize each one.
Need to update a webhook, deactivate it for the time being, or delete it? You can do that under the actions column on the right of the webhook that’s misbehaving or has reached the end of its days.
But how did we set up these sweet, sweet webhooks? Let’s take a look!
To create a new webhook, click on Add Webhook. Here’s one of the webhooks you saw above, which triggers a build in Travis CI:
To create this webhook we:
As you can see, our webhook doesn’t have much of a payload yet. We’ll show you how to prepare that below. Travis-CI happens to not require much in the way of a payload, hence it’s empty in this example.
On the right, you can see our recent webhook deliveries displayed, providing some visibility to what our webhook’s been up to lately. The Test button at the top right of this section lets you test your webhook configuration to make sure it’s behaving the way you want it to.
In the settings further down the page:
Once you’ve got your webhook all configured, you can save it, test it, and leave it to run per your configuration.
In the example I’ve provided so far, my webhook payload has mostly been empty (apart from the headers and URL content). Depending on your needs though, you may need to define some payload content – so let’s take a look at that.
The webhook payload field is designed to be flexible so that you can construct and send more or less any kind of payload you wish, using the webhooks console.
Because you can customize webhooks and webhooks payloads from TestRail just about any way you would like, there is an almost endless number of ways you can use webhooks to connect TestRail with other tools in your tech stack. That said, two of the most common use cases we anticipate TestRail users to adopt are one, using webhooks to trigger CI/CD build jobs for test automation and two, posting updates to communications tools like Slack or Microsoft Teams.
A common use case for webhooks is using them to trigger a CI/CD server build job, which may run a suite of automated tests, using Jenkins or, in this example, TravisCI.
The TravisCI documentation states that all you need to do to trigger a build is to use their API and ensure your request includes the relevant repository and job details, along with the appropriate authentication:
This simple example will have the effect of triggering the build job and executing it without any further instructions. More complicated scenarios can be accomplished by adding JSON formatted instructions to the webhook payload.
For additional scenarios and instructions related to your specific CI/CD build server, we recommend consulting the relevant documentation.
Another use case for webhooks might be using them to notify users of events in TestRail via a messaging app, like Slack or Microsoft Teams.
I set up a Slack app (instructions for doing so can be found here) to accept incoming webhook payloads from TestRail, and created a webhook in the TestRail webhooks console:
The payload field is a bit squished, so we’ve included the full JSON payload below to give you an idea of the kind of things you can do with webhooks. (Slack provides some further instructions for accomplishing different message formats here.)
“text”: “%event_creator% created a new test case:”,
“text”: “*%event_creator% created a new test case:*”
“text”: “*<%url%|%name%>* n Priority: %case_priority% n Type: %case_type%”
When the webhook is executed, you’ll end up with something resembling the below message from Slack:
As demonstrated in the example above, your payloads can be supplemented with data from TestRail, by using variables.
We’ve made the following variables available to use in your webhook payloads:
The variables can be accessed by adding them to the payload field and will be populated with the corresponding TestRail data when the webhook is triggered:
The ability to add variables to webhook payloads is a beta feature at the moment and something we’ll be looking to improve on in a future release. If you have any feedback, suggestions (e.g. for additional variables), or requests for how to improve the functionality – we’d be very happy to hear from you about them!
On that note, there are a couple of known issues we are aware of.
If you spot any others, please do let us know by contacting us at [email protected]. Improvements are planned for an update release soon.
TestRail’s SSO feature allows administrators to integrate TestRail with their preferred SSO identity provider (IDP) using SAML 2.0, OAuth 2.0, and OpenID Connect protocols. In practice, this means that the management of users can be streamlined, by creating them once in the IDP and then providing them with access to whichever applications are required by the user – TestRail, in this case.
You can enable SSO by navigating to Administration > Site Settings > SSO, choosing your preferred protocol, and filling out the required settings. You can use the SSO configuration page to integrate with any identity provider that supports SAML 2.0, OAuth 2.0, or OpenID Connect.
While SSO is not a new feature in TestRail, support for OAuth 2.0 and OIDC is now available in 7.4, adding to the previous implementation of SAML 2.0. We’ve updated our existing documentation for configuring SSO and have guides for a variety of popular tools using each available protocol:
Click here to learn more about Enterprise SSO implementation
To enhance security for TestRail instances, we’ve added Multi-Factor Authentication (MFA) so you can ensure users authenticate their login via email or popular authenticator apps. This can be enabled for your TestRail instance and then enabled on a per-user basis. Any administrator can enable MFA for TestRail under Site Settings > Security:
To then require MFA for your users, you can enable this setting on a per-user basis or by editing your users in bulk. Requiring MFA will also require API keys for your TestRail users who utilize the API.
Once MFA is enabled, users will be required to enter a one-time use code when logging into TestRail:
By default, this code will be sent by email to the user:
Alternatively, users can connect their individual accounts to an authenticator app that supports the Time-Based One-Time Password algorithm, such as Authy, Google Authenticator, or Microsoft Authenticator. To configure an authenticator app, users should follow these steps:
2. Follow the instructions in the dialog:
Click here to learn more about MFA
The TestRail API received a number of updates to enhance permission management for individual users, user groups, and projects. User endpoints now return additional user details and project endpoints now return more permission information for users and groups. New endpoints have been added to allow for the creation and management of TestRail users and user groups.
Here’s a brief breakdown of updates to existing endpoints:
get_user and get_users support the following additional fields for users:
get_project, get_projects, and update_project now support the
The groups and user fields will contain user and group information specific to the project.
We’ve added the following endpoints to TestRail’s API to enhance user and group management
Please Note: If Multi-Factor Authentication is required for users, this will require API keys for these users, regardless of the Site Setting enabled.
You can start a 14-day free trial of TestRail here (cloud or server): https://secure.gurock.com/customers/testrail/trial/
If you want to create a subscription for TestRail Cloud, you can do so from within TestRail via Administration > Subscription. Or, if you want to order TestRail Server licenses, you can do so from our website here: https://secure.gurock.com/customers/shop/annual/purchase/
Registered customers can download the full version from our customer portal: http://www.gurock.com/portal/
To learn more about SSO, Project Level Administration, or any of the other features on TestRail Enterprise, please email us for a trial or quote via [email protected], or you can use the contact form here: https://secure.gurock.com/customers/support/.
Not sure which TestRail plan you’re on? Reach out and we’d be happy to help.
TestRail Cloud instances are automatically updated to the latest version. You can check your version via the TestRail Help > About TestRail menu item.
Registered customers can download the full version from our customer portal: http://www.gurock.com/portal/
Help us improve this page!
What problem are you trying to solve?
We are thrilled to announce the launch of TestRail’s new brand. You have probably noticed a few new things already: updated colors, a refreshed logo, and our new “testrail.com” website. But our new brand goes beyond just colors and design....
We’re excited to announce the general availability of TestRail 7.6 for cloud customers on the Early Access track. We just made it even easier to complete your work in less time with the introduction of Test Parameterization for TestRail Enterprise custome...
The TestRail team is proud to announce the release of the TestRail CLI Tool. Sending automated test results to TestRail just got a lot easier. By parsing and uploading JUnit style test results into TestRail directly from the command line or by running the C...