Python RuntimeWarning: overflow encountered in long scalars

PythonNumpyAppendOverflowScalar

Python Problem Overview


I am new to programming. In my latest Python 2.7 project I encountered the following:

> RuntimeWarning: overflow encountered in long_scalars

Could someone please elaborate what this means and what I could do to fix that?

The code runs through, but I'm not sure if it is a good idea to just ignore the warning.

It happens during an append process like:

SomeList.append(VeryLongFormula)

Python Solutions


Solution 1 - Python

Here's an example which issues the same warning:

import numpy as np
np.seterr(all='warn')
A = np.array([10])
a=A[-1]
a**a

yields

RuntimeWarning: overflow encountered in long_scalars

In the example above it happens because a is of dtype int32, and the maximim value storable in an int32 is 2**31-1. Since 10**10 > 2**32-1, the exponentiation results in a number that is bigger than that which can be stored in an int32.

Note that you can not rely on np.seterr(all='warn') to catch all overflow errors in numpy. For example, on 32-bit NumPy

>>> np.multiply.reduce(np.arange(21)+1)
-1195114496

while on 64-bit NumPy:

>>> np.multiply.reduce(np.arange(21)+1)
-4249290049419214848

Both fail without any warning, although it is also due to an overflow error. The correct answer is that 21! equals

In [47]: import math

In [48]: math.factorial(21)
Out[50]: 51090942171709440000L

According to numpy developer, Robert Kern,

> Unlike true floating point errors (where the hardware FPU sets a > flag > whenever it does an atomic operation that overflows), we need to > implement the integer overflow detection ourselves. We do it on > the > scalars, but not arrays because it would be too slow to implement > for > every atomic operation on arrays.

So the burden is on you to choose appropriate dtypes so that no operation overflows.

Solution 2 - Python

An easy way to overcome this problem is to use 64 bit type

list = numpy.array(list, dtype=numpy.float64)

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestiontimkadoView Question on Stackoverflow
Solution 1 - PythonunutbuView Answer on Stackoverflow
Solution 2 - PythonKhaledView Answer on Stackoverflow