Histogram values of a Pandas Series

PythonPandasNumpyMatplotlib

Python Problem Overview


I have some values in a Python Pandas Series (type: pandas.core.series.Series)

In [1]: series = pd.Series([0.0,950.0,-70.0,812.0,0.0,-90.0,0.0,0.0,-90.0,0.0,-64.0,208.0,0.0,-90.0,0.0,-80.0,0.0,0.0,-80.0,-48.0,840.0,-100.0,190.0,130.0,-100.0,-100.0,0.0,-50.0,0.0,-100.0,-100.0,0.0,-90.0,0.0,-90.0,-90.0,63.0,-90.0,0.0,0.0,-90.0,-80.0,0.0,])

In [2]: series.min()
Out[2]: -100.0

In [3]: series.max()
Out[3]: 950.0

I would like to get values of histogram (not necessary plotting histogram)... I just need to get the frequency for each interval.

Let's say that my intervals are going from [-200; -150] to [950; 1000]

so lower bounds are

lwb = range(-200,1000,50)

and upper bounds are

upb = range(-150,1050,50)

I don't know how to get frequency (the number of values that are inside each interval) now... I'm sure that defining lwb and upb is not necessary... but I don't know what function I should use to perform this! (after diving in Pandas doc, I think cut function can help me because it's a discretization problem... but I'm don't understand how to use it)

After being able to do this, I will have a look at the way to display histogram (but that's an other problem)

Python Solutions


Solution 1 - Python

You just need to use the histogram function of NumPy:

import numpy as np
count, division = np.histogram(series)

where division is the automatically calculated border for your bins and count is the population inside each bin.

If you need to fix a certain number of bins, you can use the argument bins and specify a number of bins, or give it directly the boundaries between each bin.

count, division = np.histogram(series, bins = [-201,-149,949,1001])

to plot the results you can use the matplotlib function hist, but if you are working in pandas each Series has its own handle to the hist function, and you can give it the chosen binning:

series.hist(bins=division)

Edit: As mentioned by another poster, Pandas is built on top of NumPy. Since OP is explicitly using Pandas, we can do away with the additional import by accessing NumPy through Pandas:

count, division = pd.np.histogram(series)

Solution 2 - Python

Inorder to get the frequency counts of the values in a given interval binned range, we could make use of pd.cut which returns indices of half open bins for each element along with value_counts for computing their respective counts.

To plot their counts, a bar plot can be then made.

step = 50
bin_range = np.arange(-200, 1000+step, step)
out, bins  = pd.cut(s, bins=bin_range, include_lowest=True, right=False, retbins=True)
out.value_counts().plot.bar()

enter image description here

Frequency for each interval sorted in descending order of their counts:

out.value_counts().head()
[-100, -50)    18[0, 50)        16[800, 850)      2[-50, 0)        2[950, 1000)     1dtype: int64

To modify the plot to include just the lower closed interval of the range for aesthetic purpose, you could do:

out.cat.categories = bins[:-1]
out.value_counts().plot.bar()

enter image description here

Solution 3 - Python

If you say you want to get values of histogram, you are simply looking for frequency of each unique value in your series, if I'm not mistaken. In that case, you can simply do serie.value_counts(), which will give you:

 0.0      16
-90.0      8
-100.0     5
-80.0      3
 63.0      1
-50.0      1
 130.0     1
 190.0     1
 840.0     1
-48.0      1
 208.0     1
-64.0      1
 812.0     1
-70.0      1
 950.0     1

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionFemto TraderView Question on Stackoverflow
Solution 1 - PythonEnricoGiampieriView Answer on Stackoverflow
Solution 2 - PythonNickil MaveliView Answer on Stackoverflow
Solution 3 - Pythonst19297View Answer on Stackoverflow