RuntimeWarning: divide by zero encountered in log

PythonNumpy

Python Problem Overview


I am using numpy.log10 to calculate the log of an array of probability values. There are some zeros in the array, and I am trying to get around it using

result = numpy.where(prob > 0.0000000001, numpy.log10(prob), -10)

However, RuntimeWarning: divide by zero encountered in log10 still appeared and I am sure it is this line caused the warning.

Although my problem is solved, I am confused why this warning appeared again and again?

Python Solutions


Solution 1 - Python

numpy.log10(prob) calculates the base 10 logarithm for all elements of prob, even the ones that aren't selected by the where. If you want, you can fill the zeros of prob with 10**-10 or some dummy value before taking the logarithm to get rid of the problem. (Make sure you don't compute prob > 0.0000000001 with dummy values, though.)

Solution 2 - Python

You can turn it off with seterr

numpy.seterr(divide = 'ignore') 

and back on with

numpy.seterr(divide = 'warn') 

Solution 3 - Python

Just use the where argument in np.log10

import numpy as np
np.random.seed(0)

prob = np.random.randint(5, size=4) /4
print(prob)

result = np.where(prob > 0.0000000001, prob, -10)
# print(result)
np.log10(result, out=result, where=result > 0)
print(result)

Output

[1.   0.   0.75 0.75]
[  0.         -10.          -0.12493874  -0.12493874]

Solution 4 - Python

I solved this by finding the lowest non-zero number in the array and replacing all zeroes by a number lower than the lowest :p

Resulting in a code that would look like:

def replaceZeroes(data):
  min_nonzero = np.min(data[np.nonzero(data)])
  data[data == 0] = min_nonzero
  return data

 ...

prob = replaceZeroes(prob)
result = numpy.where(prob > 0.0000000001, numpy.log10(prob), -10)

Note that all numbers get a tiny fraction added to them.

Solution 5 - Python

This solution worked for me, use numpy.sterr to turn warnings off followed by where

numpy.seterr(divide = 'ignore')
df_train['feature_log'] = np.where(df_train['feature']>0, np.log(df_train['feature']), 0)

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionGeauxEricView Question on Stackoverflow
Solution 1 - Pythonuser2357112View Answer on Stackoverflow
Solution 2 - Pythonjohn ktejikView Answer on Stackoverflow
Solution 3 - PythonMarkus DutschkeView Answer on Stackoverflow
Solution 4 - PythonRamon BalthazarView Answer on Stackoverflow
Solution 5 - PythonE.ZolduoarratiView Answer on Stackoverflow