03 Apdex method

For a good performance test we need to be able to measure the user experience in a consistent manner. The Application Performance Index (Apdex) is measuring the quality of experience by providing a uniform index that represents the satisfaction level and productivity of humans and their applications. Apdex defines a method that converts many measurements into one number on a 0-1 scale (0 = no users satisfied, 1 = all users satisfied).

Threshold value ‘T’                                                                            

The Apdex T-value is the threshold which defines the response time (in seconds) of a user action. Based on response time research it’s more like 1 to 2 seconds for most user steps.

The T-value in the performance test is determined by looking at the complexity of each step and the understanding and expectation a user has on how long this step will take. In the test results in chapter 4 it can be seen the T-value is set between 0,5 to 2 seconds with a few exception to 2,5 seconds for very complex steps.

Performance zones

There are three performance zones or levels of satisfaction for a task as defined by Apdex.

So if our T-value is 2 seconds, 8 seconds will be our frustration threshold. We might have some users frustrated and still have an acceptable performance.

Apdex formula

The Apdex formula is the number of satisfied samples plus half of the tolerating samples plus none of the frustrated samples, divided by all the samples:

Apdex = (Satisfied Count + (Tolerating Count / 2)) / Total Samples

There are counts of response time measurement samples within the above defined performance zones of:

• Satisfied_Count = number of satisfied response time samples,
• Tolerating_Count = number of tolerating response time samples,
• Total_Samples = number of all samples in the report group (satisfied+ tolerating+ frustrated)

Note that measurements in the frustrated zone are counted in the number of total user samples in the denominator. To achieve the optimal Apdex value of 1.00, all users must experience satisfactory performance. If some users see tolerating or frustrating performance, then the index rapidly dips below 1.00.

General the rating of the performance application as defined by Apdex is a follows:

The goal for Blueriq applications is to have an Apdex value for all tests > 0.85.

For more information on the Apdex method, please refer to: http://www.apdex.org/