Many agile teams understand acceptance criteria for a story. They become great at finishing the story. And, they became disappointed when the customers don’t actually want what they’ve delivered. This is the story of one team who learned how to define “done” for an iteration, a customer release, and a project.
The Acme team was new to agile ideas and approaches. They started with iterations. They carefully defined what they thought they could deliver in an iteration. They took a few iterations to discover their capacity. In the meantime, they were actually finishing features. Or, so they thought.
Get TestRail FREE for 30 days!
Define Done for an Iteration or Milestone
Peter, the Product Owner for the Acme team, reviewed the stories as the team completed them. He rejected half the stories as being “half-baked.” When he ran the stories as small features inside the product, the features didn’t always make sense.
Sometimes, he discovered that the team hadn’t finished the error paths. Sometimes, he discovered, that what the team thought was done wasn’t actually in the product. What worked on a team’s machine didn’t work on his machine.
He asked the team to conduct a root cause analysis on the most recent stories that he had to reject. Here is what they discovered:
- The code wasn’t always checked into the right place.
- Not all code had sufficient automated tests. The team was creating automated unit and system tests as a check on their work, but the team wasn’t spending enough time building test automation.
- The team “lost” some of the test automation because the tests weren’t checked in.
The team decided to recreate their working agreements, especially about story-level “done.” It wasn’t enough to meet the acceptance criteria for a story. In addition, while they were still learning about agile approaches, they needed to have several pairs of eyes on the code and tests; and their checkin process.
The team worked collaboratively over the next few iterations to smooth their story acceptance. Peter and the team got pretty good at defining relatively small stories. When the team thought it wasn’t worth breaking the story down more, the entire team worked on the story.
Peter started accepting virtually all the stories. He couldn’t accept all the stories. That’s because while the team delivered what he requests, it wasn’t what he needed. The team was okay with that because the stories were so small. They hadn’t invested too much time finishing work Peter didn’t want.
Define What’s Minimum for a Delivery
After another couple of months, the team was ready to release their Minimum Viable Product. They’d carefully defined their minimums for the various feature sets with Peter. They’d even refactored the deployment code to use the equivalent of a “one-button-push” to deliver their software.
They pushed the button. At first, nothing happened. About an hour later, Customer Support called Peter with challenging news. While the features were actually an MVP, the customers didn’t have enough features to use the MVP.
The team had to roll back their delivery. They were not happy.
Peter and the team learned that while the features they released worked, they hadn’t released enough features to make the customers happy.
In agile approaches, we have any number of minimums:
- MVE: Minimum Viable Experiment, to learn something small with an experiment
- MMF: Minimum Marketable Feature, the minimum a customer can use
- MVP: Minimum Viable Product, provides enough of a solution that the customer is satisfied.
The team had released an MMF, not an MVP. While the customers could use the product, the customers didn’t want to.
Peter was also surprised. He and the team worked with the product manager to refine the roadmap so they didn’t make that mistake again.
Release Criteria Help Bound a Project
The team had now finished about eight months of work on this project. They learned about getting to done for stories, for iterations, for customer release. They thought they’d finished everything except some final documentation. They felt great.
Peter’s was sure this project was now done, except for that last bit of documentation. It was time for the monthly product management demo.
He showed the product management team the demo. Peter’s boss, Bill, asked, “What about advanced search?”
Peter’s eyes shot up to his hairline. “Advanced search isn’t in the roadmap, never mind this project,” he said. “Why do you want it?”
Bill said, “Oh, I guess I didn’t tell you we decided it was part of this project a couple of months ago.”
When Peter recounted he said, “It felt as if we had snatched defeat from the jaws of victory. We were so disappointed.”
Feature acceptance criteria are necessary. And, they are not sufficient. Release criteria help define the minimum acceptance criteria for a project.
When a team starts a project, they might create a project vision. A vision helps guide the feature set development by clarifying what the customers will be able to do with the outcome of your project.
In addition, the release criteria tell you when you can complete the project. Release criteria are not the entire list of all features. Instead, think of the release criteria as the “bounding box” for what the project needs to deliver.
Here are some criteria Peter and the team might have created:
- Basic search works in Scenario 1, returning at least 10 results within 1 second.
- Basic search works in Scenario 2, returning a minimum of 20 results within 1 second.
- Basic search works in Scenario 3, returning a minimum of 50 results in 2 seconds.
When a team creates release criteria, they can continue to check those criteria against the project requirements. Sometimes, the project needs to expand the release criteria. Sometimes, the project needs to shrink the criteria.
If the team realized Scenario 2 would not be able to return a minimum of 50 results in two seconds, they could have told Peter as soon as they knew. Peter would be able to tell Bill and decide what to do.
Your team might need other release criteria. I often see teams use other reliability or scalability scenarios.
One of the nice things about naming the functionality such as “Basic search,” is that everyone understands the team will not deliver anything called “Advanced search.” In addition, Peter, on behalf of the team, can check with the relevant product manager or the customer.
Release criteria help you decide when a project is ready to end, regardless of the actual scope. Sometimes, the team can end early.
Join 34,000 subscribers and receive carefully researched and popular article on software testing and QA. Top resources on becoming a better tester, learning new tools and building a team.
Know What Done Means at All Levels for Your Project
I’m not a fan of using “done,” “done-done,” or “done-done-done” to discuss whether something is done or not. I find that when we define and use working agreements for stories, we can create a coherent product. When we know what the minimums are for a delivery, we can release something the customer can use. And, when we use release criteria, we can stop—or acknowledge—scope creep.
This article was written by Johanna Rothman. Johanna, known as the “Pragmatic Manager,” provides frank advice for your tough problems. Her most recent book is “Create Your Successful Agile Project: Collaborate, Measure, Estimate, Deliver.”
Test Automation – Anywhere, Anytime
