How to transform negative elements to zero without a loop?

PythonNumpy

Python Problem Overview


If I have an array like

a = np.array([2, 3, -1, -4, 3])

I want to set all the negative elements to zero: [2, 3, 0, 0, 3]. How to do it with numpy without an explicit for? I need to use the modified a in a computation, for example

c = a * b

where b is another array with the same length of the original a

Conclusion

import numpy as np
from time import time

a = np.random.uniform(-1, 1, 20000000)
t = time(); b = np.where(a>0, a, 0); print ("1. ", time() - t)
a = np.random.uniform(-1, 1, 20000000)
t = time(); b = a.clip(min=0); print ("2. ", time() - t)
a = np.random.uniform(-1, 1, 20000000)
t = time(); a[a < 0] = 0; print ("3. ", time() - t)
a = np.random.uniform(-1, 1, 20000000)
t = time(); a[np.where(a<0)] = 0; print ("4. ", time() - t)
a = np.random.uniform(-1, 1, 20000000)
t = time(); b = [max(x, 0) for x in a]; print ("5. ", time() - t)
  1. 1.38629984856
  2. 0.516846179962 <- faster a.clip(min=0);
  3. 0.615426063538
  4. 0.944557905197
  5. 51.7364809513

Python Solutions


Solution 1 - Python

a = a.clip(min=0)

Solution 2 - Python

I would do this:

a[a < 0] = 0

If you want to keep the original a and only set the negative elements to zero in a copy, you can copy the array first:

c = a.copy()
c[c < 0] = 0

Solution 3 - Python

Another trick is to use multiplication. This actually seems to be much faster than every other method here. For example

b = a*(a>0) # copies data

or

a *= (a>0) # in-place zero-ing

I ran tests with timeit, pre-calculating the the < and > because some of these modify in-place and that would greatly effect results. In all cases a was np.random.uniform(-1, 1, 20000000) but with negatives already set to 0 but L = a < 0 and G = a > 0 before a was changed. The clip is relatively negatively impacted since it doesn't get to use L or G (however calculating those on the same machine took only 17ms each, so it is not the major cause of speed difference).

%timeit b = np.where(G, a, 0)  # 132ms  copies
%timeit b = a.clip(min=0)      # 165ms  copies
%timeit a[L] = 0               # 158ms  in-place
%timeit a[np.where(L)] = 0     # 122ms  in-place
%timeit b = a*G                # 87.4ms copies
%timeit np.multiply(a,G,a)     # 40.1ms in-place (normal code would use `a*=G`)

When choosing to penalize the in-place methods instead of clip, the following timings come up:

%timeit b = np.where(a>0, a, 0)             # 152ms
%timeit b = a.clip(min=0)                   # 165ms
%timeit b = a.copy(); b[a<0] = 0            # 231ms
%timeit b = a.copy(); b[np.where(a<0)] = 0  # 205ms
%timeit b = a*(a>0)                         # 108ms
%timeit b = a.copy(); b*=a>0                # 121ms

Non in-place methods are penalized by 20ms (the time required to calculate a>0 or a<0) and the in-place methods are penalize 73-83 ms (so it takes about 53-63ms to do b.copy()).

Overall the multiplication methods are much faster than clip. If not in-place, it is 1.5x faster. If you can do it in-place then it is 2.75x faster.

Solution 4 - Python

Use where

a[numpy.where(a<0)] = 0

Solution 5 - Python

And just for the sake of comprehensiveness, I would like to add the use of the Heaviside function (or a step function) to achieve a similar outcome as follows:

Let say for continuity we have

a = np.array([2, 3, -1, -4, 3])

Then using a step function np.heaviside() one can try

b = a * np.heaviside(a, 0)

Note something interesting in this operation because the negative signs are preserved! Not very ideal for most situations I would say.

This can then be corrected for by

b = abs(b)

So this is probably a rather long way to do it without invoking some loop.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionRuggero TurraView Question on Stackoverflow
Solution 1 - PythonRuggero TurraView Answer on Stackoverflow
Solution 2 - PythonDavid ZView Answer on Stackoverflow
Solution 3 - PythoncoderforlifeView Answer on Stackoverflow
Solution 4 - PythonAdam SchmidegView Answer on Stackoverflow
Solution 5 - PythonKris SternView Answer on Stackoverflow