pandas provides high level data manipulation tools built on top of NumPy. NumPy by itself is a fairly low-level tool, similar to MATLAB. pandas on the other hand provides rich time series functionality, data alignment, NA-friendly statistics, groupby, merge and join methods, and lots of other conveniences. It has become very popular in recent years in financial applications. I will have a chapter dedicated to financial data analysis using pandas in my upcoming book.

Numpy is required by pandas (and by virtually all numerical tools for Python).  Scipy is not strictly required for pandas but is listed as an &quot;optional dependency&quot;.  I wouldn&#39;t say that pandas is an alternative to Numpy and/or Scipy.  Rather, it&#39;s an extra tool that provides a more streamlined way of working with numerical and tabular data in Python.  You can use pandas data structures but freely draw on Numpy and Scipy functions to manipulate them.

Pandas offer a great way to manipulate tables, as you can make binning easy (https://stackoverflow.com/questions/16947336/binning-a-dataframe-in-pandas-in-python) and calculate statistics. Other thing that is great in pandas is the Panel class that you can join series of layers with different properties and combine it using groupby function.


Here is my code,
```
$url = &#39;url_to_post&#39;;
$data = array(
    &quot;first_name&quot; =&gt; &quot;First name&quot;,
    &quot;last_name&quot; =&gt; &quot;last name&quot;,
    &quot;email&quot;=&gt;&quot;email@gmail.com&quot;,
    &quot;addresses&quot; =&gt; array (
        &quot;address1&quot; =&gt; &quot;some address&quot;,
        &quot;city&quot; =&gt; &quot;city&quot;,
        &quot;country&quot; =&gt; &quot;CA&quot;,
        &quot;first_name&quot; =&gt;  &quot;Mother&quot;,
        &quot;last_name&quot; =&gt;  &quot;Lastnameson&quot;,
        &quot;phone&quot; =&gt; &quot;555-1212&quot;,
        &quot;province&quot; =&gt; &quot;ON&quot;,
        &quot;zip&quot; =&gt; &quot;123 ABC&quot;
    )
);
$data_string = json_encode($data);
$ch=curl_init($url);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, &quot;POST&quot;);
curl_setopt($ch, CURLOPT_POSTFIELDS, array(&quot;customer&quot;=&gt;$data_string));
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER,
    array(
        &#39;Content-Type:application/json&#39;,
        &#39;Content-Length: &#39; . strlen($data_string)
    )
);

$result = curl_exec($ch);
curl_close($ch);
```

        
And at other page, I am retrieving post data. 

        print_r ($_POST);

Output is 

    HTTP/1.1 200 OK
    Date: Mon, 18 Jun 2012 07:58:11 GMT
    Server: Apache
    X-Powered-By: PHP/5.3.6
    Vary: Accept-Encoding
    Connection: close
    Content-Type: text/html
    
    Array ( ) 

So, I am not getting proper data even at my own server, it&#39;s empty array. I want to implement REST using json as at http://docs.shopify.com/api/customer#create

How to POST JSON Data With PHP cURL?

I have just created an ASP.NET MVC 4 project and used Visual Studio 2012 RC to create a Controller and Razor Views for `Index` and `Create` Actions.

When I came to run the application, and browsed to the `Create` view, the following error was shown:

&gt; Compiler Error Message: CS0103: The name &#39;Scripts&#39; does not exist in
&gt; the current context

The problem is the following code which was added automatically to the bottom of the View:

    @section Scripts {
    	@Scripts.Render(&quot;~/bundles/jqueryval&quot;)
    }

Why does Scripts not exist?

I looked at the base Web Page class in Assembly System.Web.Mvc.dll, v4.0.0.0

I can see the following helper properties available:

 - Ajax
 - Html
 - Url

But nothing named Scripts.

Any ideas?

**EDIT:**

My Web.config file looks like this (untouched from the one that Visual Studio created):

      &lt;system.web.webPages.razor&gt;
        &lt;host factoryType=&quot;System.Web.Mvc.MvcWebRazorHostFactory, System.Web.Mvc, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35&quot; /&gt;
        &lt;pages pageBaseType=&quot;System.Web.Mvc.WebViewPage&quot;&gt;
          &lt;namespaces&gt;
            &lt;add namespace=&quot;System.Web.Mvc&quot; /&gt;
            &lt;add namespace=&quot;System.Web.Mvc.Ajax&quot; /&gt;
            &lt;add namespace=&quot;System.Web.Mvc.Html&quot; /&gt;
            &lt;add namespace=&quot;System.Web.Routing&quot; /&gt;
          &lt;/namespaces&gt;
        &lt;/pages&gt;
      &lt;/system.web.webPages.razor&gt;

**EDIT #2:**

People are blogging about using the `@Scripts` helper:

 - [SCOTT HANSELMAN Blog][1]
 - [Codebetter.com][2]

Yet having just installed Visual Studio 2012 RC onto a fresh Windows 8 install I am still unable to use `@Scripts` even though Visual Studio adds it to the generated View!

**Solutions are presented below.**

I am not sure how to close this, because in the end an update seemed to resolve the issue. I double checked I had performed a clean install, using a new project. But the same failing project I had made works fine now after various updates and no manual obvious intervention. Thanks for all of the thoughts but there was definitely an issue at the time ;)


  [1]: http://www.hanselman.com/blog/VisualStudio2012RCIsReleasedTheBigWebRollup.aspx
  [2]: http://codebetter.com/howarddierking/2012/06/04/web-optimization-in-visual-studio-2012-rc

MVC 4 @Scripts &quot;does not exist&quot;

They both seem *exceedingly* similar and I&#39;m curious as to which package would be more beneficial for financial data analysis. 

What are the differences between Pandas and NumPy+SciPy in Python?

<p>They both seem <em>exceedingly</em> similar and I'm curious as to which package would be more beneficial for financial data analysis.</p>


I have looked around for a while, and I was surprised not finding any information whether Gunicorn runs on Windows or not.
Does anyone know if that is the case, and if so, where can I find some documentation about it?

Does Gunicorn run on Windows

I&#39;m working on python 3.2.2.
Breaking my head more than 3 hours to sort a dictionary by it&#39;s keys.
I managed to make it a sorted list with 2 argument members, but can not make it a sorted dictionary in the end.

This is what I&#39;ve figured:

    myDic={10: &#39;b&#39;, 3:&#39;a&#39;, 5:&#39;c&#39;}
    sorted_list=sorted(myDic.items(), key=lambda x: x[0])

But no matter what I can not make a dictionary out of this sorted list. How do I do that? Thanks!

sorting dictionary python 3

What&#39;s the best way to download a python package and it&#39;s dependencies from pypi for offline installation on another machine? Is there any easy way to do this with pip or easy_install? I&#39;m trying to install the requests library on a FreeBSD box that is not connected to the internet. 


How to install packages offline?

Let&#39;s say I have a list of dictionaries:

    [
        {&#39;id&#39;: 1, &#39;name&#39;: &#39;john&#39;, &#39;age&#39;: 34},
        {&#39;id&#39;: 1, &#39;name&#39;: &#39;john&#39;, &#39;age&#39;: 34},
        {&#39;id&#39;: 2, &#39;name&#39;: &#39;hanna&#39;, &#39;age&#39;: 30},
    ]

How can I obtain a list of unique dictionaries (removing the duplicates)?

    [
        {&#39;id&#39;: 1, &#39;name&#39;: &#39;john&#39;, &#39;age&#39;: 34},
        {&#39;id&#39;: 2, &#39;name&#39;: &#39;hanna&#39;, &#39;age&#39;: 30},
    ]

List of unique dictionaries

I want to use pprint&#39;s output to show a complex data structure, but I would like to output it using the logging module rather than stdout.

    ds = [{&#39;hello&#39;: &#39;there&#39;}]
    logging.debug( pprint.pprint(ds) ) # outputs as STDOUT

Use logging print the output of pprint

I&#39;m trying to create required libraries in a package I&#39;m distributing. It requires both the [SciPy][1] and [NumPy][2] libraries.
While developing, I installed both using

    apt-get install scipy

which installed SciPy 0.9.0 and NumPy 1.5.1, and it worked fine.

I would like to do the same using `pip install` - in order to be able to specify dependencies in a setup.py of my own package.

The problem is, when I try:

    pip install &#39;numpy==1.5.1&#39;

it works fine.

But then

    pip install &#39;scipy==0.9.0&#39;

fails miserably, with

    raise self.notfounderror(self.notfounderror.__doc__)

    numpy.distutils.system_info.BlasNotFoundError:

    Blas (http://www.netlib.org/blas/) libraries not found.

    Directories to search for the libraries can be specified in the

    numpy/distutils/site.cfg file (section [blas]) or by setting

    the BLAS environment variable.

How do I get it to work?

  [1]: http://en.wikipedia.org/wiki/SciPy
  [2]: http://en.wikipedia.org/wiki/NumPy


Installing SciPy and NumPy using pip

I find myself typing `import numpy as np` almost every single time I fire up the python interpreter. How do I set up the python or ipython interpreter so that numpy is automatically imported?

Automatically import modules when entering the python or ipython interpreter

I have two numpy arrays that define the x and y axes of a grid.  For example:

    x = numpy.array([1,2,3])
    y = numpy.array([4,5])

I&#39;d like to generate the Cartesian product of these arrays to generate:

    array([[1,4],[2,4],[3,4],[1,5],[2,5],[3,5]])

In a way that&#39;s not terribly inefficient since I need to do this many times in a loop.  I&#39;m assuming that converting them to a Python list and using `itertools.product` and back to a numpy array is not the most efficient form.

Cartesian product of x and y array points into single array of 2D points

Are there any C++ (or C) libs that have NumPy-like arrays with support for slicing, vectorized operations, adding and subtracting contents element-by-element, etc.?

NumPy style arrays for C++?

It looks like the only 64 bit windows installer for Numpy is for Numpy version 1.3.0 which only works with Python 2.6

http://sourceforge.net/projects/numpy/files/NumPy/

It strikes me as strange that I would have to roll back to Python 2.6 to use Numpy on Windows, which makes me think I&#39;m missing something.

Am I?

Installing Numpy on 64bit Windows 7 with Python 2.7.3

I am looking for a simple function that can generate an array of specified random values based on their corresponding (also specified) probabilities. I only need it to generate float values, but I don&#39;t see why it shouldn&#39;t be able to generate any scalar. I can think of many ways of building this from existing functions, but I think I probably just missed an obvious SciPy or NumPy function. 

E.g.:

    &gt;&gt;&gt; values = [1.1, 2.2, 3.3]
    &gt;&gt;&gt; probabilities = [0.2, 0.5, 0.3]
    &gt;&gt;&gt; print some_function(values, probabilities, size=10)
    (2.2, 1.1, 3.3, 3.3, 2.2, 2.2, 1.1, 2.2, 3.3, 2.2)

Note: I found **[scipy.stats.rv_discrete][1]** but I don&#39;t understand how it works. Specifically, I do not understand what this (below) means nor what it should do:

    numargs = generic.numargs
    [ &lt;shape(s)&gt; ] = [&#39;Replace with resonable value&#39;, ]*numargs

If rv_discrete is what I should be using, could you please provide me with a simple example and an explanation of the above &quot;shape&quot; statement?


  [1]: http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.rv_discrete.html#scipy.stats.rv_discrete

Generating Discrete random variables with specified weights using SciPy or NumPy

I can&#39;t seem to find any python libraries that do multiple regression. The only things I find only do simple regression. I need to regress my dependent variable (y) against several independent variables (x1, x2, x3, etc.).

For example, with this data:

    print &#39;y        x1      x2       x3       x4      x5     x6       x7&#39;
    for t in texts:
        print &quot;{:&gt;7.1f}{:&gt;10.2f}{:&gt;9.2f}{:&gt;9.2f}{:&gt;10.2f}{:&gt;7.2f}{:&gt;7.2f}{:&gt;9.2f}&quot; /
       .format(t.y,t.x1,t.x2,t.x3,t.x4,t.x5,t.x6,t.x7)

(output for above:)

    

          y        x1       x2       x3        x4     x5     x6       x7
       -6.0     -4.95    -5.87    -0.76     14.73   4.02   0.20     0.45
       -5.0     -4.55    -4.52    -0.71     13.74   4.47   0.16     0.50
      -10.0    -10.96   -11.64    -0.98     15.49   4.18   0.19     0.53
       -5.0     -1.08    -3.36     0.75     24.72   4.96   0.16     0.60
       -8.0     -6.52    -7.45    -0.86     16.59   4.29   0.10     0.48
       -3.0     -0.81    -2.36    -0.50     22.44   4.81   0.15     0.53
       -6.0     -7.01    -7.33    -0.33     13.93   4.32   0.21     0.50
       -8.0     -4.46    -7.65    -0.94     11.40   4.43   0.16     0.49
       -8.0    -11.54   -10.03    -1.03     18.18   4.28   0.21     0.55

How would I regress these in python, to get the linear regression formula:

&gt; Y = a1x1 + a2x2 + a3x3 + a4x4 + a5x5 + a6x6 + +a7x7 + c



Multiple linear regression in Python

Is there any python package that allows the efficient computation of the PDF (probability density function) of a [multivariate normal distribution][1]?

It doesn&#39;t seem to be included in Numpy/Scipy, and surprisingly a Google search didn&#39;t turn up any useful thing.


  [1]: https://en.wikipedia.org/wiki/Multivariate_normal_distribution

Multivariate normal density in Python?

UPDATE:

I found a Scipy Recipe based in this question! So, for anyone interested, go straight to: [Contents &#187; Signal processing &#187; Butterworth Bandpass][1]

----------

I&#39;m having a hard time to achieve what seemed initially a simple task of implementing a Butterworth band-pass filter for 1-D numpy array (time-series).

The parameters I have to include are the sample_rate, cutoff frequencies IN HERTZ and possibly order (other parameters, like attenuation, natural frequency, etc. are more obscure to me, so any &quot;default&quot; value would do).

What I have now is this, which seems to work as a high-pass filter but I&#39;m no way sure if I&#39;m doing it right:

    def butter_highpass(interval, sampling_rate, cutoff, order=5):
        nyq = sampling_rate * 0.5
        
        stopfreq = float(cutoff)
        cornerfreq = 0.4 * stopfreq  # (?)
        
        ws = cornerfreq/nyq
        wp = stopfreq/nyq
        
        # for bandpass:
        # wp = [0.2, 0.5], ws = [0.1, 0.6]
        
        N, wn = scipy.signal.buttord(wp, ws, 3, 16)   # (?)
        
        # for hardcoded order:
        # N = order
        
        b, a = scipy.signal.butter(N, wn, btype=&#39;high&#39;)   # should &#39;high&#39; be here for bandpass?
        sf = scipy.signal.lfilter(b, a, interval)
        return sf

![enter image description here][2]

The docs and examples are confusing and obscure, but I&#39;d like to implement the form presented in the commend marked as &quot;for bandpass&quot;. The question marks in the comments show where I just copy-pasted some example without understanding what is happening.

I am no electrical engineering or scientist, just a medical equipment designer needing to perform some rather straightforward bandpass filtering on EMG signals.

  [1]: http://scipy-cookbook.readthedocs.io/items/ButterworthBandpass.html
  [2]: http://i.stack.imgur.com/CtpRF.png

How to implement band-pass Butterworth filter with Scipy.signal.butter

I have two `dataframes`, both indexed by `timeseries`.  I need to add the elements together to form a new `dataframe`, but only if the index and column are the same.  If the item does not exist in one of the `dataframe`s then it should be treated as a zero.

I&#39;ve tried using `.add` but this sums regardless of index and column.  Also tried a simple `combined_data = dataframe1 + dataframe2` but this give a `NaN` if both dataframes don&#39;t have the element.

Any suggestions?

Adding two pandas dataframes

I have data in different columns, but I don&#39;t know how to extract it to save it in another variable.

```none
index  a   b   c
1      2   3   4
2      3   4   5
```

How do I select columns `a`, `b` and save it in to `df1`?

Unsuccessful attempt:

    df1 = df[&#39;a&#39;:&#39;b&#39;]
    df1 = df.ix[:, &#39;a&#39;:&#39;b&#39;]

Selecting multiple columns in a Pandas dataframe

How do I change the column labels of a pandas DataFrame from:

    [&#39;$a&#39;, &#39;$b&#39;, &#39;$c&#39;, &#39;$d&#39;, &#39;$e&#39;]

to

    [&#39;a&#39;, &#39;b&#39;, &#39;c&#39;, &#39;d&#39;, &#39;e&#39;].



Renaming column names in Pandas

I have a pandas DataFrame with a column of string values. I need to select rows based on partial string matches.

Something like this idiom:

    re.search(pattern, cell_in_question) 

returning a boolean. I am familiar with the syntax of `df[df[&#39;A&#39;] == &quot;hello world&quot;]` but can&#39;t seem to find a way to do the same with a partial string match, say `&#39;hello&#39;`.

Filter pandas DataFrame by substring criteria

I have a csv file with the name `params.csv`. I opened up `ipython qtconsole` and created a pandas `dataframe` using:

    import pandas
    paramdata = pandas.read_csv(&#39;params.csv&#39;, names=paramnames)

where, `paramnames` is a python list of string objects. Example of `paramnames` (the length of actual list is 22):

    paramnames = [&quot;id&quot;,
    &quot;fc&quot;,
    &quot;mc&quot;,
    &quot;markup&quot;,
    &quot;asplevel&quot;,
    &quot;aspreview&quot;,
    &quot;reviewpd&quot;]


At the ipython prompt if I type `paramdata` and press enter then I do not get the dataframe with columns and values as shown in examples on [Pandas website][1]. Instead, I get information about the dataframe. I get:

    In[35]: paramdata
    Out[35]: 
    &lt;class &#39;pandas.core.frame.DataFrame&#39;&gt;
    Int64Index: 59 entries, 0 to 58
    Data columns:
    id                    59  non-null values
    fc                    59  non-null values
    mc                    59  non-null values
    markup                59  non-null values
    asplevel              59  non-null values
    aspreview             59  non-null values
    reviewpd              59  non-null values

If I type `paramdata[&#39;mc&#39;]` then I do get the values as expected for the `mc` column. I have two questions:

(1) In the examples on the pandas website (see, for example, the output of `df` here: [http://pandas.sourceforge.net/indexing.html#additional-column-access][2]) typing the name of the dataframe gives the actual data. Why am I getting information about the dataframe as shown above instead of the actual data? Do I need to set some output options somewhere?

(2) How do I output all columns in the dataframe to the screen without having to type their names, i.e., without having to type something like `paramdata[[&#39;id&#39;,&#39;fc&#39;,&#39;mc&#39;]]`. 

I am using pandas version 0.8. 

Thank you.


  [1]: http://pandas.sourceforge.net/indexing.html#additional-column-access
  [2]: http://pandas.sourceforge.net/indexing.html#additional-column-access

Content Type	Original Author	Original Content on Stackoverflow
Question	user1462733	View Question on Stackoverflow
Solution 1 - Python	Wes McKinney	View Answer on Stackoverflow
Solution 2 - Python	BrenBarn	View Answer on Stackoverflow
Solution 3 - Python	iury simoes-sousa	View Answer on Stackoverflow

What are the differences between Pandas and NumPy+SciPy in Python?

Python Problem Overview

Python Solutions

Solution 1 - Python

Solution 2 - Python

Solution 3 - Python

MVC 4 @Scripts "does not exist"

How to POST JSON Data With PHP cURL?

Attributions