Maybe I exaggerated the title of this post, but I tend to use it as a heuristic (rule of thumb) when answering testing status related questions. I never assume that I am being asked a yes or no question. So I don't answer with a yes or no. Here are some examples of the questions I am asked by project stakeholders and how I prefer to answer them at different occasions. Majority of the answers I have already used and some I intend to.
Are you done testing the feature?
- We are done testing the cases we wanted to test.
- We assumed certain important failure cases and tried to trigger those problems with our tests, within the time you gave. We did not find any problems with those.
- There are some important problems we found that gave us ideas to do more testing. Will we get more time?
- There are certain risks we still anticipate, and would want to run a few more tests. Will we get more time?
- The setup time took longer than expected and we could not execute all the tests we intended. But we did run the important tests first. Will we get more time?
Can we release?
- Well, we got some problems that may turn out to be serious issues when we release (I would sometimes give a brief outline of the issues). What do you think? Is it important to give the release anyway?
- We are still testing, but we are not finding many problems with our tests. But we don't anticipate the current problems as a major threat.
- There were some important problems we discovered and resolved, but that gave us new ideas to do more testing. Is it important to give the release anyway?
- I realize the bug count is very low, but we fear that the recent bug fixes may have caused side effects. We have witnessed some of these problems already. Do you think we can get the time to run some regression tests?
Do you need more time?
- More time would certainly help since I got some test ideas that could trigger some important problems.
- I can always come up with more tests if you give me more time.
- We did not find any problems. Do you want us to rethink the risks and try to come up with different tests?
You will notice that some of the answers are interchangeable between the questions. My intension was to give an overview. There are other questions that may be addressed with the above answers, such as:
- Is it stable?
- Are there any more bugs?