python pandas flatten a dataframe to a list

PythonListNumpyPandasDataframe

Python Problem Overview


I have a df like so:

import pandas
a=[['1/2/2014', 'a', '6', 'z1'], 
   ['1/2/2014', 'a', '3', 'z1'], 
   ['1/3/2014', 'c', '1', 'x3'],
   ]
df = pandas.DataFrame.from_records(a[1:],columns=a[0])

I want to flatten the df so it is one continuous list like so:

['1/2/2014', 'a', '6', 'z1', '1/2/2014', 'a', '3', 'z1','1/3/2014', 'c', '1', 'x3']

I can loop through the rows and extend to a list, but is a much easier way to do it?

Python Solutions


Solution 1 - Python

You can use .flatten() on the DataFrame converted to a NumPy array:

df.to_numpy().flatten()

and you can also add .tolist() if you want the result to be a Python list.

Edit

In previous versions of Pandas, the values attributed was used instead of the .to_numpy() method, as mentioned in the comments below.

Solution 2 - Python

Maybe use stack?

df.stack().values
array(['1/2/2014', 'a', '3', 'z1', '1/3/2014', 'c', '1', 'x3'], dtype=object)

(Edit: Incidentally, the DF in the Q uses the first row as labels, which is why they're not in the output here.)

Solution 3 - Python

You can try with numpy

import numpy as np
np.reshape(df.values, (1,df.shape[0]*df.shape[1]))

Solution 4 - Python

you can use the reshape method

df.values.reshape(-1)

Solution 5 - Python

The previously mentioned df.values.flatten().tolist() and df.to_numpy().flatten().tolist() are concise and effective, but I spent a very long time trying to learn how to 'do the work myself' via list comprehension and without resorting built-in functions.

For anyone else who is interested, try:

[ row for col in df for row in df[col] ]

Turns out that this solution to flattening a df via list comprehension (which I haven't found elsewhere on SO) is just a small modification to the solution for flattening nested lists (that can be found all over SO):

[ val for sublst in lst for val in sublst ]

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionjasonView Question on Stackoverflow
Solution 1 - PythonSaullo G. P. CastroView Answer on Stackoverflow
Solution 2 - PythonmeloncholyView Answer on Stackoverflow
Solution 3 - PythonChitrasenView Answer on Stackoverflow
Solution 4 - Pythonahmed hindiView Answer on Stackoverflow
Solution 5 - PythonZwiTraderView Answer on Stackoverflow