Python AND operator on two boolean lists - how?

PythonListBooleanOperator Keyword

Python Problem Overview

I have two boolean lists, e.g.,


I want to AND these lists together, with the expected output:


I thought that expression x and y would work, but came to discover that it does not: in fact, (x and y) != (y and x)

Output of x and y: [True,False,True,False]

Output of y and x: [True,True,False,False]

Using list comprehension does have correct output. Whew!

xy = [x[i] and y[i] for i in range(len(x)]

Mind you I could not find any reference that told me the AND operator would work as I tried with x and y. But it's easy to try things in Python. Can someone explain to me what is happening with x and y?

And here is a simple test program:

import random
n = 10
x = [random.random() > 0.5 for i in range(n)]
y = [random.random() > 0.5 for i in range(n)]
# Next two methods look sensible, but do not work
a = x and y
z = y and x
# Next: apparently only the list comprehension method is correct
xy = [x[i] and y[i] for i in range(n)]
print 'x        : %s'%str(x)
print 'y        : %s'%str(y)
print 'x and y  : %s'%str(a)
print 'y and x  : %s'%str(z)
print '[x and y]: %s'%str(xy)

Python Solutions

Solution 1 - Python

and simply returns either the first or the second operand, based on their truth value. If the first operand is considered false, it is returned, otherwise the other operand is returned.

Lists are considered true when not empty, so both lists are considered true. Their contents don't play a role here.

Because both lists are not empty, x and y simply returns the second list object; only if x was empty would it be returned instead:

>>> [True, False] and ['foo', 'bar']
['foo', 'bar']
>>> [] and ['foo', 'bar']

See the Truth value testing section in the Python documentation:

> Any object can be tested for truth value, for use in an if or while condition or as operand of the Boolean operations below. The following values are considered false: > > [...] > > * any empty sequence, for example, '', (), []. > > [...] > > All other values are considered true — so objects of many types are always true.

(emphasis mine), and the Boolean operations section right below that:

> x and y
> if x is false, then x, else y > > This is a short-circuit operator, so it only evaluates the second argument if the first one is True.

You indeed need to test the values contained in the lists explicitly. You can do so with a list comprehension, as you discovered. You can rewrite it with the zip() function to pair up the values:

[a and b for a, b in zip(x, y)]

Solution 2 - Python

You could use numpy:

>>> import numpy as np
>>> x=np.array([True,True,False,False])
>>> y=np.array([True,False,True,False])
>>> x & y
array([ True, False, False, False], dtype=bool)

Numpy allows numerical and logical operations on arrays such as:

>>> z=np.array([1,2,3,4])
>>> z+1
array([2, 3, 4, 5])

You can perform bitwise and with the & operator.

Instead of a list comprehension, you can use numpy to generate the boolean array directly like so:

>>> np.random.random(10)>.5
array([ True,  True,  True, False, False,  True,  True, False, False, False], dtype=bool)

Solution 3 - Python

Here is a simple solution:


Solution 4 - Python

and is not necessarily a Boolean operator; it returns one of its two arguments, regardless of their type. If the first argument is false-ish (False, numeric zero, or an empty string/container), it returns that argument. Otherwise, it returns the second argument.

In your case, both x and y are non-empty lists, so the first argument is always true-ish, meaning x and y returns y and y and x returns x.

Solution 5 - Python

This should do what you want:

xy = [a and b for a, b in zip(x, y)]

The reason x and y returns y and y and x returns x is because boolean operators in python return the last value checked that determines the true-ness of the expression. Non-empty list's evaluate to True, and since and requires both operands to evaluate True, the last operand checked is the second operand. Contrast with x or y, which would return x because it doesn't need to check y to determine the true-ness of the expression.

Solution 6 - Python

Instead of using

[a and b for a, b in zip(x, y)]

one could just use the possibility of numpy to multiply bool-values:

>> array([ True, False, False, False], dtype=bool)

Or do I overlook a special case?

Solution 7 - Python

To generalize on the zip approach, use all and any for any number of lists.

all for AND:

[all(i) for i in zip(a, b, c)]  # zip all lists

and any for OR:

[any(i) for i in zip(a, b, c)]

Solution 8 - Python

You can use the zip function

z=[a and b for a,b in zip(x,y)]

Solution 9 - Python

In addition to what @Martijn Pieters has answered, I would just add the following code to explain and and or operations in action.

and returns the first falsy value encountered else the last evaluated argument.

Similarly or returns the first truthy value encountered else the last evaluated argument.

nl1 = [3,3,3,3,0,0,0,0]
nl2 = [2,2,0,0,2,2,0,0]
nl3 = [1,0,1,0,1,0,1,0]
and_list = [a and b and c for a,b,c in zip(nl1,nl2,nl3)]
or_list = [a or b or c for a,b,c in zip(nl1,nl2,nl3)]

Values are

and_list = [1, 0, 0, 0, 0, 0, 0, 0]

or_list = [3, 3, 3, 3, 2, 2, 1, 0]

Solution 10 - Python

Thanks for the answer @Martijn Pieters and @Tony. I dig into the timing of the various options we have to make the AND of two lists and I would like to share my results, because I found them interesting.

Despite liking a lot the pythonic way [a and b for a,b in zip(x,y) ], turns out really slow. I compare with a integer product of arrays (1*(array of bool)) * (1*(array of bool)) and it turns out to be more than 10x faster

import time
import numpy as np
array_to_filter = np.linspace(1,1000000,1000000)                # 1 million of integers :-)
value_limit = 100
cycles = 100

# METHOD #1:  [a and b for a,b in zip(x,y) ]
for jj in range(cycles):
    x = array_to_filter<np.max(array_to_filter)-value_limit   # filter the values > MAX-value_limit
    y = array_to_filter>value_limit                          # filter the values < value_limit
    z= [a and b for a,b in zip(x,y) ]                       # AND
    filtered = array_to_filter[z]
print('METHOD #1 = %.2f s' % ( (time.clock()-t0)))

# METHOD 1*(array of bool) AND  1*(array of bool)
for jj in range(cycles):
    x = 1*(array_to_filter<np.max(array_to_filter)-value_limit)   # filter the values > MAX-value_limit
    y = 1*(array_to_filter>value_limit)                          # filter the values < value_limit
    z = x*y                                                     # AND
    z = z.astype(bool)                                          # convert back to array of bool
    filtered = array_to_filter[z]
print('METHOD #2 = %.2f s' % ( (time.clock()-t0)))

The results are

METHOD #1 = 15.36 s
METHOD #2 = 1.85 s

The speed is almost affected equally by the size of the array or by the number of cycles.
I hope I helped someone code to be faster. :-)


All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionGlenn NView Question on Stackoverflow
Solution 1 - PythonMartijn PietersView Answer on Stackoverflow
Solution 2 - PythondawgView Answer on Stackoverflow
Solution 3 - PythonAlwaysLearningView Answer on Stackoverflow
Solution 4 - PythonchepnerView Answer on Stackoverflow
Solution 5 - PythonCyphaseView Answer on Stackoverflow
Solution 6 - PythonPhilippView Answer on Stackoverflow
Solution 7 - Pythonjak123View Answer on Stackoverflow
Solution 8 - PythonUri GorenView Answer on Stackoverflow
Solution 9 - PythonMSSView Answer on Stackoverflow
Solution 10 - PythonMarco TrichesView Answer on Stackoverflow