There is a built in method which would be the fastest method also, calling `tolist` on the `.values` np array:

    df.values.tolist()

    [[0.0, 3.61, 380.0, 3.0],
     [1.0, 3.67, 660.0, 3.0],
     [1.0, 3.19, 640.0, 4.0],
     [0.0, 2.93, 520.0, 4.0]]

you can do it like this:

    map(list, df.values)

EDIT: [`as_matrix` is deprecated since version 0.23.0][1]

You can use the built in [`values`][2] or [`to_numpy`][3] (recommended option) method on the dataframe:

    In [8]:
    df.to_numpy()
    
    Out[8]:
    array([[  0.9,   7. ,   5.2, ...,  13.3,  13.5,   8.9],
       [  0.9,   7. ,   5.2, ...,  13.3,  13.5,   8.9],
       [  0.8,   6.1,   5.4, ...,  15.9,  14.4,   8.6],
       ..., 
       [  0.2,   1.3,   2.3, ...,  16.1,  16.1,  10.8],
       [  0.2,   1.3,   2.4, ...,  16.5,  15.9,  11.4],
       [  0.2,   1.3,   2.4, ...,  16.5,  15.9,  11.4]])

If you explicitly want lists and not a numpy array add `.tolist()`:

    df.to_numpy().tolist()

  [1]: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.as_matrix.html &quot;documentation reference&quot;
  [2]: http://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.values.html#pandas.DataFrame.values
  [3]: http://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_numpy.html#pandas.DataFrame.to_numpy

I know you can create a simple HTTP web server in Python 3 using

    python -m http.server

However is there a simple way to secure the connection to the WebServer, do i need to generate certificates? How would I do this?



Python 3 Simple HTTPS server

I&#39;ve found similar questions here but nothing works for me.

have inputs like:

    &lt;input type=&quot;text&quot; value=&quot;2&quot; name=&quot;pages_title[1]&quot;&gt;
    &lt;input type=&quot;text&quot; value=&quot;1&quot; name=&quot;pages_title[2]&quot;&gt;

trying to get these fields like:

    $(&#39;input[name=&quot;pages_title[]&quot;]&#39;)


but have empty results :(

how to get all fields?
I can only get it like `$(&#39;input[name=&quot;pages_title[1]&quot;]&#39;)`


Jquery get input array field

I have a pandas data frame like this:  
 
    admit   gpa  gre  rank   
    0  3.61  380     3  
    1  3.67  660     3  
    1  3.19  640     4  
    0  2.93  520     4

Now I want to get a list of rows in pandas like:  

    [[0,3.61,380,3], [1,3.67,660,3], [1,3.19,640,4], [0,2.93,520,4]]   
 
How can I do it?

Convert pandas DataFrame into list of lists

I have a pandas data frame like this:
<pre><code class="hljs">admit gpa gre rank 
0 3.61 380 3 
1 3.67 660 3 
1 3.19 640 4 
0 2.93 520 4
</code></pre>
Now I want to get a list of rows in pandas like:
<pre><code class="hljs language-lua">[[0,3.61,380,3], [1,3.67,660,3], [1,3.19,640,4], [0,2.93,520,4]] 
</code></pre>
How can I do it?

I need to iterate through the subdirectories of a given directory and search for files. If I get a file I have to open it and change the content and replace it with my own lines.

I tried this:

	import os
    
    rootdir =&#39;C:/Users/sid/Desktop/test&#39;
    
    for subdir, dirs, files in os.walk(rootdir):
        for file in files:
			f=open(file,&#39;r&#39;)
			lines=f.readlines()
			f.close()
			f=open(file,&#39;w&#39;)
			for line in lines:
				newline = &quot;No you are not&quot;
				f.write(newline)
			f.close()


but I am getting an error. What am I doing wrong? 
    


Iterating through directories with Python

I&#39;m having problems reading from a file, processing its string and saving to an UTF-8 File.

Here is the code:

    try:
        filehandle = open(filename,&quot;r&quot;)
    except:
        print(&quot;Could not open file &quot; + filename)
        quit() 
    
    text = filehandle.read()
    filehandle.close()

I then do some processing on the variable text.

And then 

    try:
        writer = open(output,&quot;w&quot;)
    except:
        print(&quot;Could not open file &quot; + output)
        quit() 
        
    #data = text.decode(&quot;iso 8859-15&quot;)    
    #writer.write(data.encode(&quot;UTF-8&quot;))
    writer.write(text)
    writer.close()

This output the file perfectly but it does so in iso 8859-15 according to my editor. Since the same editor recognizes the input file (in the variable filename) as UTF-8 I don&#39;t know why this happened. As far as my reasearch has shown the commented lines should solve the problem. However when I use those lines the resulting file has gibberish in special character mainly, words with tilde as the text is in spanish. I would really appreciate any help as I am stumped....

Python reading from a file and saving to utf-8

What is the efficient(probably vectorized with Matlab terminology) way to generate random number of zeros and ones with a specific proportion? Specially with Numpy?

As my case is special for `1/3`, my code is:

    import numpy as np 
    a=np.mod(np.multiply(np.random.randomintegers(0,2,size)),3)

But is there any built-in function that could handle this more effeciently at least for the situation of `K/N` where K and N are natural numbers?




Binary random array with a specific proportion of ones?

How do we get a particular filtered row as series?

Example dataframe:

    &gt;&gt;&gt; df = pd.DataFrame({&#39;date&#39;: [20130101, 20130101, 20130102], &#39;location&#39;: [&#39;a&#39;, &#39;a&#39;, &#39;c&#39;]})
    &gt;&gt;&gt; df
           date location
    0  20130101        a
    1  20130101        a
    2  20130102        c

I need to select the row where `location` is `c` **as a series**.

**I tried:**

    row = df[df[&quot;location&quot;] == &quot;c&quot;].head(1)  # gives a dataframe
    row = df.ix[df[&quot;location&quot;] == &quot;c&quot;]       # also gives a dataframe with single row

In either cases I can&#39;t the row as series.
    

Get particular row as series from pandas dataframe

Suppose I have a numpy array:

    data = np.array([[1,1,1],[2,2,2],[3,3,3]])

and I have a corresponding &quot;vector:&quot;

    vector = np.array([1,2,3])

How do I operate on `data` along each row to either subtract or divide so the result is:

    sub_result = [[0,0,0], [0,0,0], [0,0,0]]
    div_result = [[1,1,1], [1,1,1], [1,1,1]]

Long story short: How do I perform an operation on each row of a 2D array with a 1D array of scalars that correspond to each row?


Numpy: Divide each row by a vector element

I uploaded a file to Google spreadsheets (to make a publically accessible example IPython Notebook, with data) I was using the file in it&#39;s native form could be read into a Pandas Dataframe. So now I use the following code to read the spreadsheet, works fine but just comes in as string,, and I&#39;m not having any luck trying to get it back into a dataframe (you can get the data)

    import requests
    r = requests.get(&#39;https://docs.google.com/spreadsheet/ccc?key=0Ak1ecr7i0wotdGJmTURJRnZLYlV3M2daNTRubTdwTXc&amp;output=csv&#39;)
    data = r.content

The data ends up looking like: (1st row headers)

    &#39;,City,region,Res_Comm,mkt_type,Quradate,National_exp,Alabama_exp,Sales_exp,Inventory_exp,Price_exp,Credit_exp\n0,Dothan,South_Central-Montgomery-Auburn-Wiregrass-Dothan,Residential,Rural,1/15/2010,2,2,3,2,3,3\n10,Foley,South_Mobile-Baldwin,Residential,Suburban_Urban,1/15/2010,4,4,4,4,4,3\n12,Birmingham,North_Central-Birmingham-Tuscaloosa-Anniston,Commercial,Suburban_Urban,1/15/2010,2,2,3,2,2,3\n

The native pandas code that brings in the disk resident file looks like:

    df = pd.io.parsers.read_csv(&#39;/home/tom/Dropbox/Projects/annonallanswerswithmaster1012013.csv&#39;,index_col=0,parse_dates=[&#39;Quradate&#39;])

A &quot;clean&quot; solution would be helpful to many to provide an easy way to share datasets for Pandas use! I tried a bunch of alternative with no success and I&#39;m pretty sure I&#39;m missing something obvious again.

Just a Update note The new Google spreadsheet has a different URL pattern Just use this in place of the URL in the above example and or the below answer and you should be fine here is an example: 

    https://docs.google.com/spreadsheets/d/177_dFZ0i-duGxLiyg6tnwNDKruAYE-_Dd8vAQziipJQ/export?format=csv&amp;id

see solution below from @Max Ghenis which just used pd.read_csv, no need for StringIO or requests...

Getting Google Spreadsheet CSV into A Pandas Dataframe

I&#39;m looking for advice on how to show a title at the top of a collection of histogram plots that have been generated by a pandas df.hist() command. For instance, in the histogram figure block generated by the code below I&#39;d like to place a general title (e.g. &#39;My collection of histogram plots&#39;) at the top of the figure:

    data = DataFrame(np.random.randn(500).reshape(100,5), columns=list(&#39;abcde&#39;))
    axes = data.hist(sharey=True, sharex=True)

I&#39;ve tried using the *title* keyword in the hist command (i.e. title=&#39;My collection of histogram plots&#39;), but that didn&#39;t work.

The following code *does* work (in an ipython notebook) by adding text to one of the axes, but is a bit of a kludge.

    axes[0,1].text(0.5, 1.4,&#39;My collection of histogram plots&#39;, horizontalalignment=&#39;center&#39;,
                   verticalalignment=&#39;center&#39;, transform=axes[0,1].transAxes)

Is there a better way?

add title to collection of pandas hist plots

Assume I have two dataframes of this format (call them `df1` and `df2`):


    +------------------------+------------------------+--------+
    |        user_id         |      business_id       | rating |
    +------------------------+------------------------+--------+
    | rLtl8ZkDX5vH5nAx9C3q5Q | eIxSLxzIlfExI6vgAbn2JA |      4 |
    | C6IOtaaYdLIT5fWd7ZYIuA | eIxSLxzIlfExI6vgAbn2JA |      5 |
    | mlBC3pN9GXlUUfQi1qBBZA | KoIRdcIfh3XWxiCeV1BDmA |      3 |
    +------------------------+------------------------+--------+


I&#39;m looking to get a dataframe of all the rows that have a common `user_id` in `df1` and `df2`. (ie. if a `user_id` is in both `df1` and `df2`, include the two rows in the output dataframe)

I can think of many ways to approach this, but they all strike me as clunky. For example, we could find all the unique `user_id`s in each dataframe, create a set of each, find their intersection, filter the two dataframes with the resulting set and concatenate the two filtered dataframes.

Maybe that&#39;s the best approach, but I know Pandas is clever. Is there a simpler way to do this? I&#39;ve looked at `merge` but I don&#39;t think that&#39;s what I need.

Finding common rows (intersection) in two Pandas dataframes

I try to read the file into pandas.
The file has values separated by space, but with different number of spaces
I tried:

    pd.read_csv(&#39;file.csv&#39;, delimiter=&#39; &#39;)

but it doesn&#39;t work

How to read file with space separated values in pandas

I want to make all column headers in my pandas data frame lower case

## Example

If I have:

    data =
    
      country country isocode  year     XRAT          tcgdp
    0  Canada             CAN  2001  1.54876   924909.44207
    1  Canada             CAN  2002  1.56932   957299.91586
    2  Canada             CAN  2003  1.40105  1016902.00180
    ....

I would like to change XRAT to xrat by doing something like:

    data.headers.lowercase()

So that I get:

      country country isocode  year     xrat          tcgdp
    0  Canada             CAN  2001  1.54876   924909.44207
    1  Canada             CAN  2002  1.56932   957299.91586
    2  Canada             CAN  2003  1.40105  1016902.00180
    3  Canada             CAN  2004  1.30102  1096000.35500
    ....

I will not know the names of each column header ahead of time.

How can I make pandas dataframe column headers all lowercase?

I&#39;ve got a dataframe called `data`. How would I rename the only one column header? For example `gdp` to `log(gdp)`?

    data =
        y  gdp  cap
    0   1    2    5
    1   2    3    9
    2   8    7    2
    3   3    4    7
    4   6    7    7
    5   4    8    3
    6   8    2    8
    7   9    9   10
    8   6    6    4
    9  10   10    7




Rename specific column(s) in pandas

Say I import the following Excel spreadsheet into a dataframe:

    Val1 Val2 Val3
    1     2    3 
    5     6    7 
    9     1    2

How do I delete the column name row (in this case `Val1, Val2, Val3`) so that I can export a csv with no column names, just the data?

I have tried `df.drop()` and `df.ix[1:]` and have not been successful with either.

How do you remove the column name row when exporting a pandas DataFrame?

I have a very large dataframe (around 1 million rows) with data from an experiment (60 respondents).

I would like to split the dataframe into 60 dataframes (a dataframe for each participant). 

In the dataframe, `data`, there is a variable called `&#39;name&#39;`, which is the unique code for each participant.

I have tried the following, but nothing happens (or execution does not stop within an hour). What I intend to do is to split the `data` into smaller dataframes, and append these to a list (`datalist`):

    import pandas as pd

    def splitframe(data, name=&#39;name&#39;):
        
        n = data[name][0]

        df = pd.DataFrame(columns=data.columns)
    
        datalist = []

        for i in range(len(data)):
            if data[name][i] == n:
                df = df.append(data.iloc[i])
            else:
                datalist.append(df)
                df = pd.DataFrame(columns=data.columns)
                n = data[name][i]
                df = df.append(data.iloc[i])
            
        return datalist

I do not get an error message, the script just seems to run forever!

Is there a smart way to do it?

Content Type	Original Author	Original Content on Stackoverflow
Question	user2806761	View Question on Stackoverflow
Solution 1 - Python	EdChum	View Answer on Stackoverflow
Solution 2 - Python	Roman Pekar	View Answer on Stackoverflow
Solution 3 - Python	Daan	View Answer on Stackoverflow

Convert pandas DataFrame into list of lists

Python Problem Overview

Python Solutions

Solution 1 - Python

Solution 2 - Python

Solution 3 - Python

Jquery get input array field

Python 3 Simple HTTPS server

Attributions