tag:blogger.com,1999:blog-414482498098790205.post9177393123791302068..comments2023-05-18T03:46:07.779-06:00Comments on Questioning Software: Distracted by the machineryBen Simohttp://www.blogger.com/profile/11448600123169359955noreply@blogger.comBlogger6125tag:blogger.com,1999:blog-414482498098790205.post-19491194472368015862007-08-24T17:25:00.000-06:002007-08-24T17:25:00.000-06:00I believe we can compare manual and automated test...I believe we can compare manual and automated testing -- but only at a superficial level. In comparing them, we discover that they are different things. As James states in the article, they aren't two different ways to do the same thing.<BR/><BR/>Human testers can do things that automation cannot do. Automation can do things that people cannot do. <BR/><BR/>Automation has its place just as sapient work by people has its place. Smartly using both people and machines in our overall testing strategy gives us something better than either one without the other.<BR/><BR/>A human tester that does nothing more than follow a script will do things differently than a computer executing a script. They aren't doing the same thing. They will likely find different bugs. This makes real comparison difficult.<BR/><BR/>Dot Graham advocates calculating the equivalent manual test effort (EMTE) in terms of execution time for automated execution. This sounds like a good idea on the surface. However ... <BR/><BR/>We need to consider the manual work involved in coding, maintaining, executing, reviewing results, and investigating errors (testing).<BR/><BR/>We also need to consider that the automated and human test executors are doing different things. <BR/><BR/>An automated test that runs in half an hour with an EMTE of 2 hours is not necessarily equal to 2 hours of manual testing. It could be more valuable. It could be less valuable. It could be doing something so different that there is no comparison.<BR/><BR/>The assertion that we can quantify costs and benefits of anything is testing is only one of many assumptions in our business that frustrate me.<BR/><BR/>We can't really quantify things that are qualitative. Doing so requires that we dumb down the data used for making decisions. And that can lead us to make decisions based on the wrong things -- often without even realizing it.<BR/><BR/>Sometimes we have so much data that we need to use quantitative methods to help of focus on specific information. We just can't rely on numbers alone.<BR/><BR/>Do I buy one model of car instead of another based on counts of features in advertising brochures? Do I buy a car simple because of price? Due to gas mileage? Due to passenger capacity? No. I want to know details of those features. I want to know how the car meets my specific needs. Such decisions are based on quality, not just quantity.Ben Simohttps://www.blogger.com/profile/11448600123169359955noreply@blogger.comtag:blogger.com,1999:blog-414482498098790205.post-6805153556960296652007-08-24T16:17:00.000-06:002007-08-24T16:17:00.000-06:00Darn!Assuption: We can quantify the costs and bene...Darn!<BR/><BR/>Assuption: We can quantify the costs and benefits of manual vs. automated testing.<BR/><BR/>Ack! Now I'm in trouble. I can't figure out how to stand up for the manual testing that needs protecting and some balance between the two if I can't somehow quantify and compare. Can't I quantify just the time in vs. bugs that got fixed out total?<BR/><BR/>I'm looking for anyone who has ever tried this. If it always fails maybe I'd better stop trying.<BR/><BR/><A HREF="http://blog.testyredhead.com/" REL="nofollow">Testy Redhead</A>TestyRedheadhttps://www.blogger.com/profile/05188851077233522202noreply@blogger.comtag:blogger.com,1999:blog-414482498098790205.post-12778116815682863552007-05-15T13:49:00.000-06:002007-05-15T13:49:00.000-06:00Thinking that a tool can make someone a good teste...<I>Thinking that a tool can make someone a good tester (a claim that some tool vendors make) is like thinking that the right tools will make anyone a great doctor.</I><BR/><BR/>Cool! That means everyone are great pilots :) [ as per tool vendors ]Pradeep Soundararajanhttps://www.blogger.com/profile/17849721523107325938noreply@blogger.comtag:blogger.com,1999:blog-414482498098790205.post-2772383406768937752007-05-13T15:46:00.000-06:002007-05-13T15:46:00.000-06:00I wonder why the world is still thinking of automa...<I>I wonder why the world is still thinking of automation to be a faster approach than manual testing.</I><BR/><BR/>Some tasks can be executed faster by automation. The fallacy comes when people think automation is faster because it requires no human intervention. I see two flaws in this. The first is the assumption that automation can be created that does not require ongoing maintenance. The other is the loss of cognitive thought during execution when people are removed from testing.<BR/><BR/>Automation can be a great tool. I like tools. However, a great tool does not make testing great. Thinking that a tool can make someone a good tester (a claim that some tool vendors make) is like thinking that the right tools will make anyone a great doctor.<BR/><BR/>BenBen Simohttps://www.blogger.com/profile/11448600123169359955noreply@blogger.comtag:blogger.com,1999:blog-414482498098790205.post-59248917599672230892007-05-12T23:32:00.000-06:002007-05-12T23:32:00.000-06:00I wonder why the world is still thinking of automa...I wonder why the world is still thinking of automation to be a faster approach than manual testing.<BR/><BR/>Maybe many are bit by wrong snakes and the poison is having a long lasting effect.<BR/><BR/>The Snake Oil from James is a sure cure to the poisoned ones.Pradeep Soundararajanhttps://www.blogger.com/profile/17849721523107325938noreply@blogger.comtag:blogger.com,1999:blog-414482498098790205.post-85518017887115482662007-05-11T07:53:00.000-06:002007-05-11T07:53:00.000-06:00Ben - Here is a linkhttp://shrinik.blogspot.com/20...Ben - <BR/><BR/>Here is a link<BR/>http://shrinik.blogspot.com/2007/05/reliving-taso.html<BR/><BR/>ShriniShrini Kulkarnihttps://www.blogger.com/profile/10782753752478547381noreply@blogger.com