Drawing on insights from thousands of quality assurance (QA) teams worldwide, TestRail presents the findings from an in-depth exploration into the current state of QA to offer a comprehensive look into the steady evolution of software testing in 2023.
This report examines survey findings across three main areas:
- Current trends in QA and development, processes, and testing
- QA responsibilities, challenges, and priorities
- The future of testing
Since its inception in 2018, the annual user survey has played a pivotal role in our mission to understand and address the evolving needs of QA teams worldwide. By connecting with thousands of QA teams globally, we gain valuable insights into their current practices, challenges, and priorities, allowing us to deliver timely solutions and drive excellence in test management systems.
While many had expectations coming into 2023 that technologies such as machine learning (ML) and artificial intelligence (AI) would drastically impact software testing, the reality of the results we gathered indicates that these technologies still have a long way to go to live up to their hype.
Now more than ever, organizations are focusing on proven tools and technologies that will allow teams to develop and deliver faster all while maintaining set standards of software quality.
This report underscores a nuanced evolution within the QA landscape where thorough testing processes drive incremental improvements over time, fostering steady progress instead of sudden transformations.
Here’s a preview of the third edition of the Software Testing & Quality Report.
Want the full report now? You can find and download the full Software Testing & Quality Report at the end of this post.
Trends in development and testing: Automated testing
Each year we ask respondents to share the percentage of tests they currently automate versus the percentage of tests they desire to automate in the future. More companies are steadily incorporating automated tests into their daily routines, signaling a shift towards greater efficiency and reliability.
In previous years, the gap between current and desired automation levels has been about 20%. Slowly but surely that gap has narrowed with this year’s results seeing just a 16% difference. On average, respondents reported automating 40% of their tests but aim to increase that to 56% next year. This trend suggests that teams are actively working to close the gap between current and desired automation levels.
QA processes and benchmarks: Release and deployment cadence
As we dive into the intricacies of release cadences it’s clear that there’s no one-size-fits-all approach. Each organization crafts its own release cycle, tailored to its unique needs and strategies. From daily updates to quarterly or longer intervals, the diversity in deployment cycles reflects the dynamic nature of the industry.
Yet, amidst this diversity, a common thread emerges: the value of frequent releases. Our data shows that the majority (22%) of survey respondents indicated that their organization deploys bi-weekly. Weekly and monthly release deployments are also prevalent, each accounting for 18% of responses.
Overall, a staggering 69% of organizations opt for monthly or more frequent deployments, highlighting the collective recognition of the benefits of rapid iteration and adaptation.
Challenges, priorities, and KPIs: Top QA objectives
“What are your team’s top objectives around QA right now?” This question demonstrates a focused approach toward enhancing software quality and operational efficiency.
In 2023, the top objective around testing & QA was reducing bugs in production, this underscores a continuous effort to enhance product reliability, highlighting the significance placed on minimizing post-release issues.
Following closely are objectives aimed at streamlining testing processes and expanding test automation. This signals a strategic shift towards more efficient and automated test environments, again, aligning with the industry’s push for greater efficiency and reliability.
Comprehensive test coverage also remains a significant objective, tied with automation. This underscores a commitment to thorough testing across all aspects of software, ensuring comprehensive quality assurance.
Another notable objective is the incorporation of testing earlier in the development cycle, known as “shift left”. This reflects a growing adoption of proactive testing strategies, aiming to catch and address issues earlier in the development process.
Overall, these insights offer a glimpse into an evolving landscape where quality, efficiency, and automation drive the shaping of testing strategies for the future.
The future of testing: AI in QA
This year, we delved into how QA teams harness AI within testing and QA processes to explore this evolving landscape. Surprisingly, more than half (54%) of respondents revealed that they do not currently integrate AI technology into their quality assurance efforts, indicating that a substantial portion of the industry is still teetering on the brink of AI adoption.
However, there’s a promising shift: 22% of teams are currently utilizing AI are using it to craft test cases or scenarios, while 19% employ it to generate test automation scripts. This showcases an open-mindedness within the industry towards AI’s potential to streamline and enhance testing practices.
A smaller yet significant fraction of respondents (14% and 12% respectively) are leveraging AI for managing test data and debugging test code, recognizing its ability to address more intricate aspects of modern software testing.
Looking ahead into the future of testing, the adoption of AI in quality assurance holds promise for enhancing testing methods and efficiency. There is a growing recognition of AI’s potential to streamline test automation, test case creation, and test data management. As teams explore AI integration, they stand to redefine the quality processes and efficiency in testing as they seek to adopt new methodologies and techniques.
Bottom Line
The third edition of the Software Testing & Quality Report reveals a steady evolution in QA practices. Despite initial anticipation of groundbreaking impacts from advanced technologies, organizations continue to prioritize proven tools and methods to accelerate development while maintaining software quality standards.
The report emphasizes the enduring relevance of manual testing alongside the steadily rising usage of automation, promoting a balanced approach for efficiency and reliability. It also highlights diverse release cadences, with a consensus on the value of frequent deployments.
The top QA objectives in 2023 focused on reducing production bugs, streamlining testing processes, expanding automation, ensuring comprehensive test coverage, and shifting testing earlier in the development cycle.
Although AI adoption in QA is still nascent, there’s potential for significant improvement in testing methods and efficiency as teams explore AI integration for various tasks.
This report shows that teams are sticking to their game plan of making testing more efficient, ensuring better quality, and increasing automation. This change didn’t happen overnight; it’s been happening slowly over the years as teams realize the importance of making their processes the best they can be.
This means that teams are actively looking for new ways to improve, showing that they’re staying proactive and keeping up with the ever-changing world of software development and quality assurance.
To dive deeper into the insights and trends shaping the future of software testing and quality assurance, download the full Software Testing & Quality Report here! Gain valuable insights into current practices, challenges, and priorities, and discover how your organization can drive excellence in test management systems.
