Transform "list of tuples" into a flat list or a matrix
PythonListTuplesPython Problem Overview
With Sqlite, a select .. from
command returns the results output
, which prints:
>>print output
[(12.2817, 12.2817), (0, 0), (8.52, 8.52)]
It seems to be a list of tuples. I would like to either convert output
to a simple list:
[12.2817, 12.2817, 0, 0, 8.52, 8.52]
or a 2x3 matrix:
12.2817 12.2817
0 0
8.52 8.52
to be read via output[i][j]
The flatten command does not do the job for the 1st option, and I have no idea for the second one...
A fast solution would be appreciated, as the real data is much bigger.
Python Solutions
Solution 1 - Python
By far the fastest (and shortest) solution posted:
list(sum(output, ()))
About 50% faster than the itertools
solution, and about 70% faster than the map
solution.
Solution 2 - Python
List comprehension approach that works with Iterable types and is faster than other methods shown here.
flattened = [item for sublist in l for item in sublist]
l
is the list to flatten (called output
in the OP's case)
timeit tests:
l = list(zip(range(99), range(99))) # list of tuples to flatten
List comprehension
[item for sublist in l for item in sublist]
timeit result = 7.67 µs ± 129 ns per loop
List extend() method
flattened = []
list(flattened.extend(item) for item in l)
timeit result = 11 µs ± 433 ns per loop
sum()
list(sum(l, ()))
timeit result = 24.2 µs ± 269 ns per loop
Solution 3 - Python
In Python 2.7, and all versions of Python3, you can use itertools.chain
to flatten a list of iterables. Either with the *
syntax or the class method.
>>> t = [ (1,2), (3,4), (5,6) ]
>>> t
[(1, 2), (3, 4), (5, 6)]
>>> import itertools
>>> list(itertools.chain(*t))
[1, 2, 3, 4, 5, 6]
>>> list(itertools.chain.from_iterable(t))
[1, 2, 3, 4, 5, 6]
Solution 4 - Python
Update: Flattening using extend but without comprehension and without using list as iterator (fastest)
After checking the next answer to this that provided a faster solution via a list comprehension with dual for
I did a little tweak and now it performs better, first the execution of list(...) was dragging a big percentage of time, then changing a list comprehension for a simple loop shaved a bit more as well.
The new solution is:
l = []
for row in output: l.extend(row)
The old one replacing list
with []
(a bit slower but not much):
[l.extend(row) for row in output]
Older (slower):
Flattening with list comprehension
l = []
list(l.extend(row) for row in output)
some timeits for new extend and the improvement gotten by just removing list(...) for [...]:
import timeit
t = timeit.timeit
o = "output=list(zip(range(1000000000), range(10000000))); l=[]"
steps_ext = "for row in output: l.extend(row)"
steps_ext_old = "list(l.extend(row) for row in output)"
steps_ext_remove_list = "[l.extend(row) for row in output]"
steps_com = "[item for sublist in output for item in sublist]"
print(f"{steps_ext}\n>>>{t(steps_ext, setup=o, number=10)}")
print(f"{steps_ext_remove_list}\n>>>{t(steps_ext_remove_list, setup=o, number=10)}")
print(f"{steps_com}\n>>>{t(steps_com, setup=o, number=10)}")
print(f"{steps_ext_old}\n>>>{t(steps_ext_old, setup=o, number=10)}")
Time it results:
for row in output: l.extend(row)
>>> 7.022608777000187
[l.extend(row) for row in output]
>>> 9.155910597999991
[item for sublist in output for item in sublist]
>>> 9.920002304000036
list(l.extend(row) for row in output)
>>> 10.703829122000116
Solution 5 - Python
>>> flat_list = []
>>> nested_list = [(1, 2, 4), (0, 9)]
>>> for a_tuple in nested_list:
... flat_list.extend(list(a_tuple))
...
>>> flat_list
[1, 2, 4, 0, 9]
>>>
you could easily move from list of tuple to single list as shown above.
Solution 6 - Python
use itertools
chain:
>>> import itertools
>>> list(itertools.chain.from_iterable([(12.2817, 12.2817), (0, 0), (8.52, 8.52)]))
[12.2817, 12.2817, 0, 0, 8.52, 8.52]
Solution 7 - Python
Or you can flatten the list like this:
reduce(lambda x,y:x+y, map(list, output))
Solution 8 - Python
This is what numpy
was made for, both from a data structures, as well as speed perspective.
import numpy as np
output = [(12.2817, 12.2817), (0, 0), (8.52, 8.52)]
output_ary = np.array(output) # this is your matrix
output_vec = output_ary.ravel() # this is your 1d-array
Solution 9 - Python
In case of arbitrary nested lists(just in case):
def flatten(lst):
result = []
for element in lst:
if hasattr(element, '__iter__'):
result.extend(flatten(element))
else:
result.append(element)
return result
>>> flatten(output)
[12.2817, 12.2817, 0, 0, 8.52, 8.52]
Solution 10 - Python
def flatten_tuple_list(tuples):
return list(sum(tuples, ()))
tuples = [(5, 6), (6, 7, 8, 9), (3,)]
print(flatten_tuple_list(tuples))