How to calculate percentage improvement in response time for performance testing

How should I calculate the percentage improvement in response time.

I am getting 15306 ms response time for old code and 799 ms response for the updated code. What will be the percentage improvement in response time?

There are two ways to interpret "percentage improvement in response time". One is the classic and ubiquitous formula for computing a percentage change in a data point from an old value to a new value, which looks like this:

(new - old)/old*100%

So for your case:

(799 - 15306)/15306*100% = -94.78%

That means the new value is 94.78% smaller (faster, since we're talking about response time) than the old value.

The second way of interpreting the statement is to take the percentage of the old value that the new value "covers" or "reaches":

new/old*100%

For your case:

799/15306*100% = 5.22%

That means the new value is just 5.22% of the old value, which, for response time, means it takes just 5.22% of the time to respond, compared to the old response time.

The use of the word "improvement" suggests that you want the 94.78% value, as that shows how much of the lag in the old response time was eliminated ("improved") by the new code. But when it comes to natural language, it can be difficult to be certain about precise meaning without careful clarification.

> I think the accepted answer suffers from the original question not having > nice round numbers and that there are 3 different ways to state the > result.

Let's assume that the old time was 10 seconds and the new time is 5 seconds.

There's clearly a 50% reduction (or decrease) in the new time:

(old-new)/old x 100% = (10-5)/10 x 100% = 50%

But when you talk about an increase in performance, where a bigger increase is clearly better, you can't use the formula above. Instead, the increase in performance is 100%:

(old-new)/new x 100% = (10-5)/5 x 100% = 100%

The 5 second time is 2x faster than the 10 second time. Said a different way, you can do the task twice (2x) now for every time you used to be able to do it.

old/new = 10/5 = 2.0

---

So now let's consider the original question

The old time was 15306 ms and the new time is 799 ms.

There is a 94.7% reduction in time.

(old-new)/old x 100% = (15306-799)/15306 x 100% = 94.7%

There is a 1816% increase in performance:

(old-new)/new x 100% = (15306-799)/799 x 100% = 1815.6%

Your new time is 19x faster:

old/new = 15306/799 = 19.16

Actually performance is about how much can be done in the same amount of time.

So the formula is OLD/NEW - 1 In your case your performance increased by 1816% (i.e. you can do 18.16X more in the same time)

15306/799 - 1 = 1816%

Note: before you could do 1/15360, now 1/799 ...

your code's runtime is 94.78% shorter/improved/decreased:

(new - old) / old x 100%
(799 - 15306) / 15306 x 100% =~ -94.78% (minus represents decrease)

your code is 1816% faster:

(old - new) / new x 100%
(15306 - 799) / 799 x 100% =~ 1816%

Couple of responses already answered the question correctly, but let me expand those answers with some additional thoughts and a practical example.

Percentage improvement is usually calculated as ((NEW - OLD)/OLD) * 100

Let us think about it with some practical examples:

• If I make $10,000 in my current job and get a new job that offers $12,000 then I am getting 20% increase in salary with this new job ((12000 - 10000)/10000)*100
• If Train A travels at 100 miles per hour and Train B travels at 150 miles per hour, Train B is 50% faster than Train A. Simple, right?

It gets tricky when you try to measure a metric using another metric that has an inverse relationship with the metric you are trying to measure. Let me explain what I mean by this.

Let us try the second example again now using "time taken to reach destination". Let us say Train A takes 3 hours to reach the destination and Train B takes 2 hours to reach the same destination. Train B is faster than Train A by what percentage?

If we use the same formula we used in the above examples, we get ((2-3)/3)*100, which is -33%. This simply tells that Train B takes 33% less time to reach the destination than Train A, but that is not what we are trying to determine. Right? We are trying to measure the difference in speed by percentage. If we change the formula slightly and take the absolute value, we get 33%, which may seem right, but not really. (I'll explain why in a minute)

So, what do we do? The first thing we need to do is to convert the metric we have in hand to the metric we want to measure. After that, we should be able to use the same formula. In this example, we are trying to measure difference in speed. So let us first get the speed of each train. Train A travels at 1/3 of the distance per hour. Train B travels at 1/2 distance per hour. The difference in speed in percentage then is: ((1/2 - 1/3)/1/3) * 100 = ((1/2 - 1/3)*3)*100 = (3/2 - 3/3) * 100 = 50%

Which happens to be same as ((3 - 2)/2) * 100.

In short, when the metric we are measuring and the metric we have at hand have an inverse relationship, the formula should be

((OLD - NEW)/NEW) * 100

What is wrong with the original formula? Why can't we use the original formula and conclude that train B is only 33% faster? Because it is inaccurate. The original formula always yields a result that is less than 100%. Imagine a flight reaching the same destination in 15 mins? The first formula tells that flight is 91.6% faster while the second formula tells that the flight is 1100% faster (or 11 times faster), which is more accurate.

Using this modified formula, the percentage improvement for case posted in the original question is ((15306 - 799)/799) * 100 = 1815.6%

((old time - new time) / old time) * 100
This formula will give the Percentage Decreased in New Response time.

In your case, ((15306 - 799)/ 15306) * 100 = 94.78 %

The formula for finding the percentage of reduction is:

P = a/b × 100

Where P is the percentage of reduction, a is the amount of the reduction and b is the original amount that was reduced.

So to calculate a you do: old - new wichi will translate into:

P = ((OLD - NEW)/OLD)*100