December 22, 2007

Discombobulation Testing

Posted by Ben Simo

I do not like software that discombobulates users. I find that software error handling and reporting often confuses users more than it helps. Several months back, I created and wrote about a mnemonic of guideword heuristics that I have been using for testing error handling and reporting. I have found this useful in my own testing and have received some positive feedback from others that are using it. I created the following PowerPoint show to help demonstrate how I use the mnemonic. Take a look. It runs for just under seven minutes.



FAILURE.pps
(For those that don't have PowerPoint: try an experimental Flash version here.)


The next time you encounter an error, try the FAILURE heuristic.

  • F Does the error detecting, reporting, and handling function as expected?
  • A Is the error reported at the appropriate time in an appropriate manner?
  • I Is the impact to the user communicated?
  • L Is the error appropriately logged? ... or not logged if that is desired?
  • U Is the error appropriately reported in the user interface in terms that the user understands?
  • R Is the user told how to recover from the error condition?
  • E Does the error reporting evoke the desired emotions? Could changes in the error reporting and handling improve the users emotional response?

Merry Christmas, and Happy Testing!

  Edit

2 Comments:

January 10, 2008  
Madhukar Jain wrote:

Hi Ben,

Amazing piece of work, i appreciate the way you presented it to us.

Madhukar..

December 01, 2009  
Jonas Söderström wrote:

I would like to add a letter to the heuristic.

S - How is the system *situation* after the failure? Many error creates follow up failures because the system does not recover properly. Do extensive testing after the failure. Recreate the error several times and see if the system behaves the same.

Now it spells FAILURES! Even better.

Thanks for a great post, I added this heuristic to my personal Testing Toolbox document and will use this in the future.

Happy Testing / Jonas