mean, nanmean and warning: Mean of empty slice

PythonNumpyWarnings

Python Problem Overview


Say I construct two numpy arrays:

a = np.array([np.NaN, np.NaN])
b = np.array([np.NaN, np.NaN, 3])

Now I find that np.mean returns nan for both a and b:

>>> np.mean(a)
nan
>>> np.mean(b)
nan

Since numpy 1.8 (released 20 April 2016), we've been blessed with https://docs.scipy.org/doc/numpy/reference/generated/numpy.nanmean.html">nanmean</a>;, which ignores nan values:

>>> np.nanmean(b)
3.0

However, when the array has nothing but nan values, it raises a warning:

>>> np.nanmean(a)
nan
C:\python-3.4.3\lib\site-packages\numpy\lib\nanfunctions.py:598: RuntimeWarning: Mean of empty slice
  warnings.warn("Mean of empty slice", RuntimeWarning)

I don't like suppressing warnings; is there a better function I can use to get the behaviour of nanmean without that warning?

Python Solutions


Solution 1 - Python

I really can't see any good reason not to just suppress the warning.

The safest way would be to use the warnings.catch_warnings context manager to suppress the warning only where you anticipate it occurring - that way you won't miss any additional RuntimeWarnings that might be unexpectedly raised in some other part of your code:

import numpy as np
import warnings

x = np.ones((1000, 1000)) * np.nan

# I expect to see RuntimeWarnings in this block
with warnings.catch_warnings():
    warnings.simplefilter("ignore", category=RuntimeWarning)
    foo = np.nanmean(x, axis=1)

@dawg's solution would also work, but ultimately any additional steps that you have to take in order to avoid computing np.nanmean on an array of all NaNs are going to incur some extra overhead that you could avoid by just suppressing the warning. Also your intent will be much more clearly reflected in the code.

Solution 2 - Python

A NaN value is defined to not be equal to itself:

>>> float('nan') == float('nan')
False
>>> np.NaN == np.NaN
False

You can use a Python conditional and the property of a nan never being equal to itself to get this behavior:

>>> a = np.array([np.NaN, np.NaN])
>>> b = np.array([np.NaN, np.NaN, 3])
>>> np.NaN if np.all(a!=a) else np.nanmean(a)
nan
>>> np.NaN if np.all(b!=b) else np.nanmean(b)
3.0

You can also do:

import warnings
import numpy as np

a = np.array([np.NaN, np.NaN])
b = np.array([np.NaN, np.NaN, 3])

with warnings.catch_warnings():
    warnings.filterwarnings('error')
    try:
        x=np.nanmean(a)
    except RuntimeWarning:
        x=np.NaN    
print x    

Solution 3 - Python

I have got this runtime warning when I perform np.nanmean over a 3-D array, e.g. (time, lon, lat). Maybe not a direct answer to your question, but I would like to add this warning message in my case can be related to point cell value with all NaN value series.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionMichael CurrieView Question on Stackoverflow
Solution 1 - Pythonali_mView Answer on Stackoverflow
Solution 2 - PythondawgView Answer on Stackoverflow
Solution 3 - PythonChenyaoView Answer on Stackoverflow