Three Levels of Acceptance Testing

At the Agile Open Europe conference I participated in a discussion on the subject of acceptance testing. One of the open questions was how to match "traditional" customers with the agile idea of delivering a potentially shippable product every few weeks.

Different Types of Customers
While some customers are delighted to receive a new release every month, there are others that don't even want to see or know about any of the intermediate releases. Some customers simply insist on working the traditional way. They hire a team of testers for a couple of weeks before the intended release date, and they require the development team to deliver one release candidate, just in time for the test period, and the team should then stop working on the system until the test team has finished testing.

After fifteen minutes of heated debate about expectations, responsibilities and mindwiping customers, it dawned on me that the participants in this discussion were using three different meanings of the term acceptance testing:

1. acceptance testing performed by testers in the project team, who try to read the customer's mind and attempt to represent him by checking if the delivered product is likely to be acceptable;
2. acceptance testing performed by testers on the customer's side, who need to verify on behalf of the customer that the budget was properly spent, and that there's no urgent need to sue the vendor;
3. acceptance testing performed by the users of the product, who might not even want to touch the new system with a long stick, despite the fact that twenty testers told them that the system is fine.

Forget the Words, but Check the Meaning
Depending on the type of organisation, type of product, and type of customer, a project will face one, two, or all three of these forms of acceptance testing. In our discussion at Agile Open it took a while for us to realize that some people were not agreeing with each other only because their ideas of acceptance testing followed different levels of definition. Once we identified that problem, communication (and agreement) seemed to be a lot easier.

Now, don't get me wrong, I have no intention to redefine acceptance testing! The testing community itself already has many fine definitions of all kinds of testing activities. But some people correctly pointed out to me that exact terms and wording are irrelevant as long as people agree on meanings. I have noticed similar confusion among development teams and customers leading to miscommunication about expectations and responsibilities.

Let's Understand Each Other
So… I advise you to check each of these "levels of acceptance testing" to see if all stakeholders agree on the way they are implemented in the process. It doesn't matter how you call it. Just make sure you're talking about the same thing.