A green Jenkins build tells you whether the suite passed.
But what it doesn’t tell you is which 12 cases ran, which one flaked yesterday and passed today, or how this build’s coverage compares to last week’s.
For most QA teams, that information often gets buried in the build console log or scattered across one-off test reports: thousands of lines of stdout where every regression run looks identical and every flake disappears the next time the pipeline turns green. Test case management closes that gap by turning short-lived build output into durable, queryable history. Each automated run becomes a structured record. Each case carries its own pass-fail history. Test coverage and release readiness stop being a feeling and start being a metric you can defend in a release meeting.
This guide covers what changes when you wire Jenkins to TestRail, how to do the wiring without rebuilding your pipeline, and what to do after the integration is working so the data actually shapes release decisions.
| Already running tests in Jenkins and want them landing in TestRail with full history? Learn how to connect your first Jenkins job to TestRail using the TestRail CLI. |
Key takeaways
- Most green-build blindness is a reporting problem, not a testing problem. The tests are running. The history is just trapped in places that are hard to query, compare, or explain in a release meeting.
- TestRail integrates with Jenkins through three paths: the official TestRail CLI (TRCLI), third-party Jenkins plugins like Railflow, or or Agiletestware Pangolin, or direct REST API calls. The CLI is the officially supported approach documented by TestRail for Jenkins pipeline and freestyle jobs.
- Mapping JUnit results to TestRail cases happens one of two ways: code-first (classname.name becomes the automation_id) or specification-first (case IDs like C123 embedded in the test name or set as a test_id property).
- The integration is worth the setup only if the output drives release decisions. A pass-rate trend, a flaky-case watchlist, and per-feature coverage are the views that justify the work.
- Most teams run into a few predictable issues the first time through: case ID mismatches, missing Jenkins credentials, and JUnit XML that lacks the fields TRCLI expects.
What is Jenkins test case management?
Jenkins test case management is the practice of routing test results from Jenkins build jobs into a dedicated test case management system, where each result is associated with a documented test case, a release, and a history.
Jenkins itself is a CI server. It runs whatever build and test commands you give it and reports whether the suite passed or failed. It does not natively understand the concept of a test case as a versioned artifact with documentation, requirements traceability, owners, and historical pass rate. A test case management platform like TestRail does. When you wire the two together, every Jenkins run becomes a structured test run inside TestRail: each case shows whether it passed, failed, or was skipped on that build, and the result joins a longer history you can query, report on, and use for release decisions.
The integration matters because it changes what stakeholders outside QA can see. A green build tells engineering the pipeline is finished. A linked TestRail run tells the release manager which 247 cases ran, which three flaked over the last two weeks, and which feature areas are still gapped.
Step 1: Get the Jenkins side ready to report
Before any wiring to TestRail, your Jenkins job needs to produce a results file in a format TestRail understands. The standard is JUnit XML. Most test frameworks (Pytest, JUnit 5, TestNG, NUnit, Cypress, Playwright, Mocha) either emit it natively or have an adapter.
Emit JUnit XML during the test run. For Pytest, use pytest –junitxml reports/junit-report.xml. For JUnit 5, the Surefire or Gradle test plugin generates compatible reports by default. For TestNG, configure the JUnit XML reporter rather than the TestNG-only reporter. For NUnit, use a JUnit-compatible reporter, adapter, or transform so the output can be parsed by the TestRail CLI.
Publish results in Jenkins. The Jenkins JUnit Plugin reads the XML and renders the per-test view, pass/fail trends, and the standard test report on the build page. Add this to your Jenkinsfile inside the post { always { … } } block:
| post { always { junit ‘**/reports/junit-report.xml’ archiveArtifacts artifacts: ‘reports/*.xml’, fingerprint: true }} |
The archiveArtifacts step preserves the XML for later review or troubleshooting in Jenkins. The TestRail CLI can read the same report file from the workspace when you call it in the pipeline.
Tag builds with metadata. TestRail can store context like the Jenkins build URL, branch, commit SHA, and build number with each test run when you pass that information into the run description. For example: –run-description “Build: ${BUILD_URL} | Branch: ${GIT_BRANCH} | Commit: ${GIT_COMMIT}”. Without that, you can see results in TestRail, but you cannot easily jump back to the Jenkins job that produced them.
Step 2: Set up the TestRail project
TestRail organizes test cases into projects, suites, sections, and cases. Before wiring Jenkins, your TestRail instance needs to be structured so results have somewhere to land.
Create or open the project. In TestRail, go to Administration > Projects and create a project for the application under test. The project decides where test runs live and who has access.
Choose a suite mode. TestRail offers single-suite, single-suite with baselines, or multi-suite mode. Single-suite is the default and works for most teams. Multi-suite is useful when you need separate case repositories for distinct products under one project.
Structure sections by feature area. Inside the suite, create one section per feature area, module, or service. When Jenkins results land, each case maps to a section, and reports can break down pass rate or flakiness by feature without manual tagging.
Decide on a matching strategy. This is the most important decision before the integration. The choice determines how Jenkins results map to TestRail cases:
| Strategy | How it works | When to use it |
|---|---|---|
| Code-first (auto matcher) | TRCLI builds an automation_id from the JUnit classname.name and matches it against the automation_id custom field in TestRail. New cases can be auto-created for unmatched tests. | You write tests in code first, then let TestRail mirror them. |
| Specification-first (name matcher) | The case ID is embedded in the JUnit test name (e.g., C123_test_login). TRCLI reads the C-prefixed ID and posts the result against that case. | You author cases in TestRail first, then automate against them. |
| Specification-first (property matcher) | The JUnit <testcase> includes a <property name=”test_id” value=”C123″/> element. TRCLI uses the property value to identify the case. | You author cases in TestRail first but do not want to alter test names. This is often cleaner than embedding IDs in names. |
The specification-first approaches give you tighter control over what maps where. The code-first approach is faster to set up but requires the automation_id custom field configured in TestRail and discipline around test naming, because renaming a test can break the mapping or create duplicate cases if the new automation ID is treated as a new test.
Step 3: Wire Jenkins to TestRail
With Jenkins producing JUnit XML and TestRail structured, the wiring step posts results from one to the other. There are three viable paths, and the officially supported one is the TestRail CLI.
The TestRail CLI (TRCLI). This is the official command-line tool for posting results to TestRail, maintained by Gurock on GitHub. It is a Python package installable with pip:
| pip install TRCLI |
In your Jenkinsfile, call it inside the post { always { … } } block so results upload regardless of whether the build passed or failed:
| post { always { withCredentials([usernamePassword( credentialsId: ‘testrail-api’, usernameVariable: ‘TR_USER’, passwordVariable: ‘TR_KEY’)]) { sh ”’ TRCLI -y \ -h https://YOUR-INSTANCE.testrail.io \ –project “Your Project Name” \ –username “$TR_USER” \ –password “$TR_KEY” \ parse_junit \ –suite-id 10 \ –case-matcher “property” \ –title “Build ${BUILD_NUMBER} – ${GIT_BRANCH}” \ –run-description “Build URL: ${BUILD_URL}” \ -f “reports/junit-report.xml” ”’ } }} |
Store TestRail credentials in Jenkins via the Credentials Binding plugin, then bind them to environment variables in your pipeline with withCredentials. Never commit API keys to your repo.
Third-party Jenkins plugins. The Railflow plugin and Agiletestware Pangolin Connector are Jenkins plugin options for teams that prefer UI-based configuration over Jenkinsfile code. Before adopting a plugin, review its maintenance status, setup requirements, and licensing. For example, Pangolin requires a Pangolin Server, while Railflow has its own integration workflow. Plugins suit teams that prefer UI configuration over pipeline code. The CLI suits teams whose pipelines are version-controlled and reviewed.
Direct API calls. TestRail exposes a REST API that the CLI uses internally. If your framework needs to post results outside the standard JUnit-to-TRCLI flow (parametrized runs across configurations, custom result fields populated per case, attachments per test), a direct API integration gives you full control. It is more code to maintain, so use it only when the CLI and plugins do not fit.
| Want to see what a clean Jenkins-to-TestRail integration looks like end to end? Walk through TestRail’s Jenkins integration page or wire your first job inside a free trial. |
Step 4: Build the views that justify the integration
Wiring results into TestRail is half the work. The other half is making the results inform release decisions instead of sitting in another dashboard nobody opens.
Watch a run land. Trigger a Jenkins build. After the post-build step completes, open TestRail and go to Test Runs & Results. You should see a new run with the title from your –title argument. Click in and each case shows its status, elapsed time, and any failure stack trace pulled from the JUnit XML.
Drill into a failed case. Click a failed case and TestRail shows its full history: every previous run, every prior status, and any comments or defects linked to it. This is where a flaky case becomes obvious. A case that has passed 47 times and failed twice in the last fortnight tells a different story than a case that has failed five times in a row.
Build a release-readiness view. TestRail’s reports pull from multiple test runs across multiple Jenkins jobs. Configure a report that aggregates the runs you care about for a release. Pass rate trend over the last N builds, a flaky case watchlist, and coverage by section turn the integration into a release artifact rather than a logging upgrade.
| What to put on the dashboard | Why it matters |
|---|---|
| Pass rate trend across the last 20 builds | Catches gradual regression that no single build surfaces. |
| Flaky case watchlist (cases that changed status more than once in 10 runs) | Surfaces cases that are not reliable enough to gate a release. |
| Coverage by feature area | Identifies feature areas where automation has thinned out. |
| Open defects linked to failed cases | Routes failures to the people who can fix them without manual triage. |
Three common gotchas (and how to fix them)
These three issues account for most failed first-time integrations. Each one has a clean fix.
Case ID mismatch. Symptom: TRCLI runs, posts results, and the test run in TestRail is empty or contains brand-new auto-created cases instead of updating the ones you already have. Cause: the –case-matcher strategy does not match how your cases are identified. Fix: if you are using auto, confirm the automation_id custom field exists in TestRail and is populated for each case. If you are using name, confirm test names start with C<ID>_ (e.g., C123_test_login_invalid). If you are using property, confirm each <testcase> in your JUnit XML has the <property name=”test_id” value=”C123″/> element.
Missing TestRail credentials in Jenkins. Symptom: TRCLI fails with a 401 Unauthorized or authentication error. Cause: credentials are not bound into the pipeline environment, or the API key was generated for a user without permission on the project. Fix: store the TestRail email and API key in Jenkins Credentials. Bind them with withCredentials and pass them to TRCLI as environment variables, not as literal arguments. Generate the API key on a TestRail user account with permission to add runs and results in the target project.
JUnit XML missing required fields. Symptom: TRCLI parses the file without errors but the resulting TestRail run is missing test details, failure messages are blank, or elapsed times do not appear. Cause: the test framework’s JUnit output omits the time, failure, or system-out elements that TRCLI reads from. Fix: verify your framework’s JUnit configuration produces standard JUnit XML. Pytest’s –junitxml output is compatible by default. For TestNG, use the standard JUnit reporter (not the TestNG-only one). For elapsed times in milliseconds rather than seconds, pass –allow-ms to TRCLI.
Wire your first Jenkins job
Most of what makes a Jenkins-to-TestRail integration valuable lives downstream of the wiring: the release-readiness view, the flaky-case watchlist, the historical pass rate per feature. The wiring itself takes a couple of hours once your tests have already emitted JUnit XML. Everything in this guide can be done inside a 30-day trial.
If you are running tests in Jenkins and want the results to build a usable history in TestRail by the end of the week, that is the setup we see most often.
Start a free trial, install TRCLI, and post your first build results before the end of your next pipeline run.
Frequently asked questions
Does TestRail have an official Jenkins plugin?
The officially supported integration is the TestRail CLI, or TRCLI, which runs as a shell step inside a Jenkins pipeline or freestyle job. Several third-party Jenkins plugins exist, including Railflow and Agiletestware Pangolin Connector, both available in the Jenkins Plugin Manager. If you choose a plugin, review its current maintenance status and setup requirements before standardizing on it.
What test frameworks does the TestRail CLI support?
The parse_junit command works with any framework that produces JUnit-compatible XML, including Pytest, JUnit 4 and 5, TestNG, Cypress, Playwright, and NUnit when configured to export JUnit-style reports. The CLI also has dedicated parsers for Robot Framework, using parse_robot, and Cucumber, using parse_cucumber.
How does TestRail handle parallel Jenkins builds writing to the same test run?
Use the add_run command to create the test run before the parallel stages start, then pass the resulting run ID to each parallel node so they all post into the same run with –run-id. Without this, each parallel node creates its own run and fragments the results.
Can I map one Jenkins test to multiple TestRail cases?
Yes. Add a comma-separated list of case IDs to the test_id property: <property name=”test_id” value=”C101, C102, C103″/>. The CLI creates a separate result for each referenced case in the test run.
Does the integration work with Jenkins freestyle jobs as well as pipelines?
Yes. TRCLI runs the same way in both: a shell step in a pipeline post { always { } } block, or an Execute shell build step in a freestyle job. Gurock documents both flows. Pipelines give you tighter control over when results post relative to other steps.
