How to set local variable in list comprehension?

PythonList Comprehension

Python Problem Overview


I have a method that takes a list and returns an object:

# input a list, returns an object
def map_to_obj(lst):
    a_list = f(lst)
    return a_list[0] if a_list else None

I want to get a list that contains all the mapped elements that aren't None.

Like this:

v_list = [v1, v2, v3, v4]

[map_to_obj(v) for v in v_list if map_to_obj(v)]

But it doesn't seem good to call the map_to_obj method twice in the list comprehension.

Is there a way to have local variables in list comprehensions so that it can have better performance?

Or does the compiler optimize it automatically?

Here is what I want:

(sml like)
[let mapped = map_to_obj(v) in for v in v_list if mapped end] 

Python Solutions


Solution 1 - Python

Use nested list comprehension:

[x for x in [map_to_obj(v) for v in v_list] if x]

or better still, a list comprehension around a generator expression:

[x for x in (map_to_obj(v) for v in v_list) if x]

Solution 2 - Python

Starting Python 3.8, and the introduction of assignment expressions (PEP 572) (:= operator), it's possible to use a local variable within a list comprehension in order to avoid calling twice the same function:

In our case, we can name the evaluation of map_to_obj(v) as a variable o while using the result of the expression to filter the list; and thus use o as the mapped value:

[o for v in [v1, v2, v3, v4] if (o := map_to_obj(v))]

Solution 3 - Python

A variable assignment is just a singular binding:

[x   for v in l   for x in [v]]

This is a more general answer and also closer to what you proposed. So for your problem you can write:

[x   for v in v_list   for x in [map_to_obj(v)]   if x]

Solution 4 - Python

You can avoid re-calculation by using python built-in filter:

list(filter(lambda t: t is not None, map(map_to_obj, v_list)))

Solution 5 - Python

A local variable can be set within a comprehension by cheating a bit and using an extra 'for' which "iterates" through a 1-element tuple containing the desired value for the local variable. Here's a solution to the OP's problem using this approach:

[o for v in v_list for o in (map_to_obj(v),) if o]

Here, o is the local variable being set equal to map_to_obj(v) for each v.

In my tests this is slightly faster than Lying Dog's nested generator expression (and also faster than the OP's double-call to map_to_obj(v), which, surprisingly, can be faster than the nested generator expression if the map_to_obj function isn't too slow).

Solution 6 - Python

List comprehensions are fine for the simple cases, but sometimes a plain old for loop is the simplest solution:

other_list = []
for v in v_list:
    obj = map_to_obj(v)
    if obj:
        other_list.append(obj)

Now if you really want a list comp and dont want to build an tmp list, you can use the iterator versions of filter and map:

import itertools as it
result = list(it.ifilter(None, it.imap(map_to_obj, v_list)))

or more simply :

import itertools as it
result = filter(None, it.imap(map_to_obj, v_list)))

The iterator versions don't build a temporary list, they use lazy evaluation.

Solution 7 - Python

I have figured out a way of using reduce:

def map_and_append(lst, v):
    mapped = map_to_obj(v)
    if mapped is not None:
        lst.append(mapped)
    return lst

reduce(map_and_append, v_list, [])

How about the performance of this?

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionHao TanView Question on Stackoverflow
Solution 1 - PythonLying DogView Answer on Stackoverflow
Solution 2 - PythonXavier GuihotView Answer on Stackoverflow
Solution 3 - PythonVincent GoossensView Answer on Stackoverflow
Solution 4 - Pythonbehzad.nouriView Answer on Stackoverflow
Solution 5 - PythonOvafloView Answer on Stackoverflow
Solution 6 - Pythonbruno desthuilliersView Answer on Stackoverflow
Solution 7 - PythonHao TanView Answer on Stackoverflow