TestRail User Survey Results 2020: Part 1

We are excited to share the insights from the 2020 TestRail User Survey. We are listening and want to share the results to help us all better understand the community that makes up TestRail.

In total, we received over 2,500 responses to this year’s survey (2,549 to be exact)! To break it down, here are some key demographic highlights from this year’s survey.

  • Most people who answered the survey were testers or QA engineers, followed by test leads and test automation engineers (SDETs)
  • They represented 30+ industries, with the largest being computer software
  • Survey participants represented almost 100 countries around the world
  • 80% of those who took the survey use TestRail multiple times per week—50% use TestRail every day!

Let’s dive into the details.

A bit about you, our community

Our survey included questions to get a better sense of who we were hearing from. Here is some of what we learned.

Your role and how we fit in

What job responsibilities and/or titles do you hold? Based on the participants in this year’s survey, there is a good chance you fall into the category of a Test / QA Engineer (48%) or a Test Lead/ QA Lead / Manager / Director (22%).

Mx8MznwsFgHzwyuJwhlR8YhSON206dkRpRufUF42fMATkOL9FIibzkUyiHFpSJ29m4rw HvNtZUr bSmu3KXH9lbLFj QoVZ L7jlv3t0BG8NPE6ZBpALso6VRihTgoRsLQ17Lp s0

How large is your QA/ testing team? 56% of you have teams of 1-10 testers and 28% have teams of 11-50 testers.

ULKH3wDayrX8ROqwRVLPJzMlZgT8p0KRd36nH7hWZuLM3vYH89bsLHxnXzuBp0sr XD426 Athot9Z1zkjNj2cdCOz YFQJ62IA81KY2Kdg4ukt LNHMFR 0ZNDR1qb8y Aq 4lts0

What industry does your organization work in? Computer software, Telecommunications, Technology, Internet & Electronics ranks highest with 51% and the rest were a mix between various industries.

WAkQ4AinwGBF3kwaSsbDO aq5z6F 5zj2FY13AL Ld720MKx KvTK5K3PEDnIG5JsKHwzvpgluMcOXKuLf2KRRtbL3B57rMsEPTvrSBG8hMiKNX0pmKWe2 JKizrBQeKppCj VVbs0

Your usage of TestRail

When asked how long you’ve been using TestRail? 44% said that you have been using TestRail for 1-3 years, while 29% said that you’ve been using TestRail for less than 1 year (Welcome to those folks who have been using TR for less than a year!).

lAxObGLm d4uYtzNanm7spZH3hdHt2UaU8xg

When asked how often you use TestRail? 49% answered that you use Testrail every day, while 29% answered that you use TestRail multiple times a week.

U98BrdUPYqckYndFg2dUd5X9DT3SQ1DlfMyCbwLUHiQT GMDUJUNZYW2qUgs5T0AgUKeehvwhJg7yKiEa7FUamax1JjFXbtpwpZYlTt8lkxqCHXWCxFDv lH3ry1

Testing types, techniques, and/or development methodologies your team currently uses

About 82% of you have embraced the Agile testing methodology and 65% use Scrum. This shows us that the industry is shifting towards faster development and release pipelines. The growing popularity of these testing methodologies shows us the demand being placed on QA teams for faster delivery cycles and reflects the need for test management tools to be an essential part of the quality management process. The reality of today’s development world is that fast turnaround and overall development efficiency are of utmost importance, and the test management you use for agile testing must allow you and your team to meet your quality goals and complete testing on time.

Ss7bZfStDFoLkJaM0fFROhlyys5vmJSxRcgwmNGJiOOHA 6R20mRHFJ7aEoeY186SX3xb7pxx wJpRK0g4lpITeBBIMnyCe0kqqS6vUN bGnjfv3SZHAlk4MJmPzvT IW9Im789ns0

Metrics or KPIs your team report on

In terms of the key KPIs you and your teams use to measure your testing, over 50% of you said that your teams track KPIs related to testing activities such as testing progress (were pass/fail/blocked %), the percentage of their tests that are automated vs. manual, test coverage, total number of tests executed, and the number of defects reported during testing. A smaller percentage of participants reported tracking quality metrics around code that has been deployed to production, such as production incidences or test effectiveness (% bugs found during testing vs. production).

krkt98KWvQHISvFZufihAjy3EKCFeBAA97Z4y0oOZk8S3 b9JHlLnTjvT4ScPfsNZc8VYyiRoIdtqgWWQoYi3qpX1iXybbJ4a1hviiOsPBeuom i

Percentage of automated vs. manual tests

35% of your tests are automated. When asked, “What percentage of your tests do you expect to automate in the next year?” you answered that you expect to be automating around 54% of your tests by the next year.

h4fQ2lZ7K6MX yMGyQoYVXecCflsAdhrpe33o9z0A8OTStHSJ2SFMsDcihjmtqeEKFofQdtkXkvdnlBS1ayj 4XpvlBV7Z4mzzp3bbkfKVVNDebFKk2R8bSklhhAMuuMJOYD7kVis0

This tells us how important it is to have a comprehensive test case management system in place. This data point resonates with conversations we’ve had with you in TestRail Community forums, interviews and through webinar Q&As over the last year or two: QA engineers are under ever-increasing pressure to do more testing in less time, and test automation promises to help take some of the most repetitive tasks off your plate so that you can focus your efforts around QA craft management (quality risk analysis, test design and planning, and analyzing data that comes back from testing).

These data from the survey also highlight why it’s so important to make sure you have a test management solution in place as you automate more of your test suite: test management systems give you a way to keep track of which tests have already been automated and which are still manual, help you highlight opportunities for further automation, and help maintain consistency around reporting on test progress and results across all of your human-centric and robot-driven testing activities.

Why do you use TestRail?

When asked about the primary reasons why you and your team use TestRail, a whopping 65% of you answered that one of the reasons is to maintain a repository of reusable test cases, 63% reported that you use TestRail to document test plans, and 59% said that it was to make it easier to track the testing status. Moreover, 46% of you reported that you use TestRail to maintain a record of historical test results, while 42% said that you use the tool to both be more productive/efficient during testing and to get more visibility into the testing process.

This tells us how TestRail plugs into your overall QA process. With software development evolving rapidly, companies need to look for ways to optimize testing processes and improve speed without compromising quality. From allowing you to maintain a repository of reusable test cases, to giving you more visibility into the testing process, TestRail is used to handle the complete lifecycle of software testing and empowers you with the visibility, flexibility, and insights required to increase efficiency in your testing processes.

Our genuine thanks go out to those who took the time to answer the survey. It is immeasurably valuable to us, and we’re looking forward to applying it to all the amazing TestRail releases we have coming your way in 2021.

We’ve learned so much, it doesn’t fit into one post. Stay tuned for Part 2: Your tech stack, your biggest challenges & your goals for the year.

Thank you again to everyone who participated in this year’s user survey!

It’s with great gratitude that we share the list of individuals who were picked in the prize drawing to receive a €25 or €100 Amazon Gift Card (who have permitted us to share their first names). To everyone below and the 13 other individuals who asked us not to share any of their personal information, congratulations and thank you again for participating!

Amanda
Donovan
Daphne
Chris
Tim
Srilata
Nicola
David
Julia
Yumiko
Srilatha
Rick
Chetan
Tatyana
Nayeli
Carleen
Steven
Martina
Emily
Catherine

If you didn’t win this time, keep your eye out for more chances to participate in the TestRail community and potentially win other prizes over the coming year!

In This Article:

Sign up for our newsletter

Share this article

Other Blogs

How to Write Effective Test Cases (With Templates)
Software Quality, Agile

How to Write Effective Test Cases (With Templates)

A test case is a fundamental element of software testing that documents a sequence of steps to verify that a specific feature, functionality, or requirement of a software application behaves as expected.  Test cases serve as a blueprint, outlining the wha...
Introduction to DevOps
Agile, Automation, Software Quality

How to Choose the Right Automation Framework: Types & Examples 

In modern software development, a test automation framework gives you structured guidelines and tools to execute tests efficiently. But beyond that, it provides many benefits that improve your testing processes and the quality of your software. Types of test a...
How to Improve Bug Reporting & Management for Better Results
Software Quality, Automation, Continuous Delivery, Jira

How to Improve Bug Reporting & Management for Better Results

Maintaining product quality and functionality is crucial in software development, and effective bug reporting plays a key role in this process. Whether you’re an experienced developer, tester, or new to the industry, mastering bug reporting is essential....