May 23, 2012

What is Performance?

Posted by Ben Simo

As we design and test for performance, let's look beyond speed. Let's look beyond basic stability. Let's look at the many facets of performance. Consider in which ways your system needs to perform. How fast does it need to be? How long does it need to keep running? Does it need to grow? Does it need to be available at all times? How much can we spend? Can we make it faster?

There are endless questions we could ask. Therefore, categorizing facets of performance and creating tests for each category can be helpful. However, let's not fail to look at the interaction between these categories.

OFAT (One Factor At a Time) testing (as exampled in the above performance testing checklist excerpt) often fails to provide information related to the interaction between the categories. Let's do some MFAT (Multiple Factors At a Time) testing and analysis. Let's look at the system as a whole. Let's mix it up. Let's consider how these facets interact. Let's create test scenarios that include multiple facets.



September 04, 2012  
Justin Hunter wrote:


I'm a huge fan of MFAT but pretty inexperienced in performance testing.

Would you be able to share an example or two where MFAT performance testing has been helpful in revealing information that OFAT tests may well have overlooked?

- Justin

October 11, 2012  
Anonymous wrote:

Hi Ben

Nice post,
I am very impressed with your way of presenting all the different variance of performance with these snaps and you have minutely collected all particle that builds performance. Since I am a tester so familiar with this word and every day I use to listen all about the performance and its root cause.

But still learning the pros and con of this kind of testing and this place has proved to a good mind feeder related to performance testing and its feature.

January 24, 2013  
Dzmitry wrote:

Interesting presentation, Ben, thank you.
I agree with you at the point, that any software system should be tested on multiple factors, as a whole one, because tuning one parameter we can destroy other or many others.
Do you use JMeter? I'd like to share with you article which was written by one of JMeter developers. It describes some tricks about performance testing with jmeter -

August 14, 2013  
ProITOnlineTraining ProIT wrote:

I learned about the OFAT and MFAT today. I was searching for it. meanwhile learned about the factors to be considered for better performance with the help of your article. Thanks.