5 Steps to Successful Testing in DevOps

This is a guest post by Rachel Kibler.

First things first: as your team adopts DevOps, testing requires a lot more communication and collaboration. People skills are important, and asking questions is important.

There are some other practices that are good ideas for testing in DevOps too, and if your company wants to be successful with DevOps, you should try to implement most of them. Use this quick checklist to identify which devops testing practices you’ve already implemented and get ideas for other ways you can improve.

1. Set the framework

a. Determine your definition of a bug and policy for bugs. Are bugs everything found in production that deviates from expectations? Will they be fixed first or prioritized? How will bugs that require immediate engagement be decided?

b. Make sure your rollback process is easy. (That’s it; that’s the tweet.)

2. Get the whole team engaged in writing stories

a. Split the stories into the smallest releasable (and testable) chunks. API calls that aren’t hooked up to the front end are fine and can be tested on their own before the UI is done.

b. Review the acceptance criteria carefully. Make sure the problem being solved is actually understood. Look for, as Elisabeth Hendrickson says, absolute statements, such as software “always”  or “never” doing something. These are potential landmines. Make sure everyone is on the same page and understands the approach.

c. Ask questions about risk — risk to the user, security, business, whatever you can think of.

d. Figure out what needs to be answered with the code change. Write some charters or other kinds of testing notes. Ask for feedback on this. Giving the developers and product owner an idea of how the change will be tested can lead to better code. By “better,” I mean cleaner, more testable code, and a smoother process for the people who need to use it and maintain it.

3. While the code is being written, there are a few things you can do

a. Pair with developers. If you can get a greater understanding of the code and what the changes touch, you’ll do better testing in the end. You can also raise issues during pairing, giving your developers a better idea of what you’ll be looking at.

b. Set up your test environment, or make sure you know how to access the test environment once it is available. Get some scenarios set up, prepare any data you might need for testing, and get users in the right state.

c. Start writing your automated testing, if applicable.

4. When the code is ready for you, it’s time to do what everyone thinks we always do

a. Review the code changes. Make sure the developer wrote whatever automated tests they’re expected to write, whether unit, integration or UI. If there are no tests, ask why.

b. Test the scenarios you’ve set up using all the notes you’ve collected and all the conversations you’ve had.

c. Make sure you touch all the areas of risk. Stuff can change during development, so talk with your developer to see what areas of code the change touches.

d. Run all the applicable automated tests. Typically, start by running high priority or smoke tests. Use automated tests to identify areas you might need to dig in more manually.

e. (If you don’t already know this) Ask your development team if they prefer for you to report bugs you find (one at a time? all at once? in Jira issues? with a summary email?). Depending on how they feel about context switching, they may want you to do as much testing as possible first, then tell them about the issues, or they may want to follow your progress and fix stuff as you find it.

f. Add your automated tests, either in a separate branch or the same one. Make sure you’re following the same code process that your developers are held to.

5. When the code is released, you’re not done

a. Check production to make sure the change got there successfully. Pipelines can be weird, and sometimes stuff gets stuck or doesn’t release properly. Depending on the stability of your pipeline, you might not see this much, but it’s worth checking. You don’t need to reproduce all your testing again; just a simple check will do it.

b. Make sure you’re reviewing error logs to see if there are spikes in errors caused by the change. The amount of time you look at the logs depends on the possibility of intermittency of errors. If you released a change to login, you probably don’t need hours of review. If you released a change to a flow that users see only a little bit, set a query, and come back to it later.

There it is! DevOps is fun and worth it, and with these proven practices, you can set yourselves up for success with testing in DevOps. Check out more of our DevOps blog posts here!

Modern Test Case Management Software for QA and Development Teams

Rachel Kibler is a tester at 1-800 Contacts. She can be found on Twitter @racheljoi or on her website at racheljoi.com.

In This Article:

Start free with TestRail today!

Share this article

Other Blogs

Building a Modern QA Team With T-Shaped Testers
Agile, Books, General, Software Quality

Building a Modern QA Team With T-Shaped Testers

The role of QA is changing—fast. Agile workflows, DevOps, and CI/CD pipelines have raised the bar for what’s expected of software testers. Today, being a specialist in just one area isn’t enough. Teams need people who can wear multiple hats, jump into differen...
T shaped tester
Uncategorized, General, Software Quality, TestRail, Webinar

Competing in a Modern QA Job Market: Insights from the T-shaped Tester Webinar

The QA job market is evolving rapidly. The rise of Agile, DevOps, and shift left methodologies demands a new breed of tester—one who combines deep expertise in specific areas with broad knowledge across various domains. This shift has paved the way for the con...
Key Factors to Consider When Selecting the Right Test Case Management Tool
General, Software Quality, TestRail

Choosing the Right Test Case Management Tool: Key Factors

Understanding the need you have is often the first step in defining the method for managing test cases that will work for you and your team.