ReLU Prime with NumPy array

Question

I want to pass multidimensional array to relu prime function

 def reluprime(x): if x > 0: return 1 else: return 0 

... where x is the entire array. He returns

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any () or a.all ()

I had this problem with the normal relu function, and instead of using the python max() function, I used np.max() and it worked. But with relu prime, it doesn't work anyway. I tried:

 def reluprime(x): if np.greater(x, 0): return 1 else: return 0 

... and it still returns the same error. How can i fix this? Thanks.

Answer 1

The if statement does not make sense, since it is evaluated only once for the entire array. If you need the equivalent of an if statement for each element of the array, you should do something like:

 def reluprime(x): return np.where(x > 0, 1.0, 0.0) 

Answer 2

Since relu prime returns 1 if the entry in the vector is greater than 0 and 0 otherwise, you can simply do:

 def reluprime(x): return (x>0).astype(x.dtype) 

in the above code, the input x array is considered a numpy array. For example, reluprime(np.array([-1,1,2])) returns array([0, 1, 1]) .

Answer 3

The "relu prime", or gradient of the ReLU function, is better known as the "step function".

Numpy 1.13 introduces ufunc for this:

 def reluprime(x): return np.heaviside(x, 0) # second value is value at x == 0 # note that ReLU is not differentiable at x==0, so there is no right value to # pass here 

The synchronization results on my machine show that this works rather poorly, offering more work there:

 In [1]: x = np.random.randn(100000) In [2]: %timeit np.heaviside(x, 0) #mine 1.31 ms ± 58.3 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each) In [3]: %timeit np.where(x > 0, 1.0, 0.0) # Jonas Adler's 658 µs ± 74.2 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each) In [4]: %timeit (x>0).astype(x.dtype) # Miriam Farber's 172 µs ± 34.8 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)