In one of my testing consultancy sessions, I was demonstrating the testing of a JavaScript calendar widget in a web application developed by the client. I started off explaining that we should use the windows calendar as an oracle to help us discover problems in the widget. I opened the webpage containing the widget and the windows calendar next to each other.
I was demonstrating this on a projector which only supported a resolution of 800x600. So if I wanted to keep the windows calendar and webpage always visible, my webpage real-estate becomes considerably reduced. To the surprise of my audience I started resizing the browser window so that I can see more of the region surrounding the widget, which seemed unnecessary since the widget only took a small portion of the entire webpage. I explained that I want to be aware of problems surrounding the widget while I am executing my tests on the widget. I did confess that I do not know what kind of problems could appear in those regions but it doesn't hurt to pay attention.
This was a demonstration of exploratory testing. I was narrating my choices of executing certain tests that I generated on the fly, and why I chose to ignore other tests that I considered as equivalent or not related to my current test objective. A few minutes into the testing, one of my colleagues discovered that a button's outline at the corner of the web page was blinking as I was invoking the widget. The audience appreciated this discovery since they could relate to my initial preemption. I myself did not notice this due to inattentional blindness.
Before we continue, I think a good way to explain inattentional blindness is with the example of a magician's performance. Ever notice how a magician sometimes tries to distract you away from his actual trickery by trying to focus your attention on something else. For example, when a coin that you can swear was in the palm of the magician's hand disappears into thin air when he unfolds his fist. In reality, the coin was swiftly removed while you were paying attention to where the illusionist wanted and hence the perception of magic. This is called inattentional blindness. You can't see what is right in front of you, simply because you were paying attention to something else.
The irony of my episode was that I did not discover the problem even though my intention was to pay attention to such peculiarities. Also, exploratory testing is supposed to be better at minimizing inattentional blindness. This symptom is advocated as being more common during scripted testing. So I tried to analyze this phenomenon and came to the following explanation.
Since I was sitting close to a large projection of the screen on the wall, my visual field was limited. This made me more prone to inattentional blindness. Also I was frequently alternating my attention:
- To find important problems in the widget.
- To explain to the audience the reasons for my actions.
- To make sure my explanations were comprehensible.
- To address queries from the audience.
Yes, that is true. The far you go, the wider the viewing angle become. I used to do observe things from a bit far when i wanted to see them all and realize the issues. "Inattentional blindness" greatly reduces when the programmer and the tester are two different person. As for my case, I make the programs and I myself test them. As a result I only look for bugs at those portions only where I expect them. The other bugs some time remain hidden for this "inattentional blindness". This would not be the case if the tester was a different person than me. We should be aware of where we are paying attention. You are quite right.
ReplyDeleteTarafder,
ReplyDeleteThank for sharing your experience. However I would want to make a slight correction to your comment.
I think what you mean is that when you have a second person do the same testing, there is a better chance that some of the problems that you overlooke due to inattentional blindness will be detected by the second pair of eyes. But that does not mean that your inattentional blindness reduces. Rather, the consequence of inattentional blindness reduces.
Do correct me if I misunderstood your comment.
Sajjadul,
ReplyDeleteBut that does not mean that your inattentional blindness reduces. Rather, the consequence of inattentional blindness reduces.
Yes, that's true. But what I meant was if I test other's program, my "Inattentional blindness" will be reduced as compared to the case if I test my own program myself. The fact is that you thought of the program as constant and the tester is varying and I thought the tester is constant and the programs he is testing are varying. In a few words, if I test my program, "Inattentional blindness" is high but if i test other's program, "Inattentional blindness" might be low enough compared to the first case.
Please do correct me if I misunderstood your reply.
Tarafder,
ReplyDeleteThank you for clarifying your comment. It is an interesting observation.
Sajjadul, what a wonderful example.
ReplyDeleteTesting is a lot like fishing. You try to create conditions that maximize your chances for catching something tasty. Exploratory testing is a little more like fishing, because just as with real life fishermen, the tester continually reads the conditions, draws inferences about the location of the fish, and adjusts the process.
I like your description of inattentional blindness, and I accept the point you made, but don't forget another dynamic that you put in motion-- one that is very much an exploratory strategy-- distributed cognition. Your testing comprised not only what you noticed, but what other people noticed at the same time.
By provide more room to view the application, you made it *possible* for you to see something unexpected, but by also by doing so and by doing the demonstration live and encouraging audience response, you made it even more *probable* that your *group* would discover those problems.
-- James
Good post Sajjad --
ReplyDelete>>Also, exploratory testing is supposed to be better at minimizing inattentional blindness.
Why do you think so? what elements in ET help you to minimize inattentional blindness?
Let me take a guess and answer --
May be it is "thinking between alternating polarities" while doing ET - doing vs explaining, fast vs slow, reading vs doing, focussing vs defocussing.
In my opinion inattentional blindness happens due to "heavy" focus on one or more "atomic" aspects of bigger object under observation. One possible solution is to defocus as often as you can.
A related phrase (a kind of antonym) I use is "your eyes will see what you would like to see". This is more dominant theme in scripted testing where a tester is "pre-programmed" observe only expected results.
Shrini
James,
ReplyDeleteYou have tickled my curiosity. Distributed Cognition is a great new perspective to my example. I WILL explore this.
Shrini,
ReplyDeleteYou have made some good points. Thanks.
"One possible solution is to defocus as often as you can"
That is a good advise. Can you please elaborate the following part of your comment:
"May be it is "thinking between alternating polarities" while doing ET - doing vs explaining, fast vs slow, reading vs doing, focussing vs defocussing."
I am not sure what you meant by "fast vs slow". Thanks again.
Sajjadul,
ReplyDeleteVery interesting post. After reading your post I was reading Inattentional blindness on Wikipedia. It made me think on how our existing knowledge about domain/technology (or lack of it) can act as a catalyst in increasing this blindness. If I already have any pre-conceived notion of how some thing should work (technically or functionally) It might be affecting me with this blindness. Having some one working with you (Testing in Pair) or continue collaboration with the developers/customers are something that can be used to make sure that our testing activities are not affected by this.
Nice post.
Testinggeek,
ReplyDeleteThat was a good outline of the reasons behind inattentional blindness. Also your example of pair testing is probably what James referred to as distributed cognition.
I still did not get the chance to further analyze this interesting concept of distributed cognition, but it is my homework. Thanks.
Great post Sajjadul!
ReplyDeleteInattentional Blindness is something, which many testers suffer from without ever realizing it! I had posted an article on this topic a couple of months before. Hope you would like it. Inattentional Blindness and Testers
Regards,
-Debasis
Software Testing Zone
Debasis,
ReplyDeleteThat was a great video. I had seen the ball passing part of the video from Cem Kaner and James Bach's online BBST lessons (testingeducation.org). But not your version of the video with the commentary. Thanks for sharing that.