Combining two lists and removing duplicates, without removing duplicates in original list
PythonListPython Problem Overview
I have two lists that i need to combine where the second list has any duplicates of the first list ignored. .. A bit hard to explain, so let me show an example of what the code looks like, and what i want as a result.
first_list = [1, 2, 2, 5]
second_list = [2, 5, 7, 9]
# The result of combining the two lists should result in this list:
resulting_list = [1, 2, 2, 5, 7, 9]
You'll notice that the result has the first list, including its two "2" values, but the fact that second_list also has an additional 2 and 5 value is not added to the first list.
Normally for something like this i would use sets, but a set on first_list would purge the duplicate values it already has. So i'm simply wondering what the best/fastest way to achieve this desired combination.
Thanks.
Python Solutions
Solution 1 - Python
You need to append to the first list those elements of the second list that aren't in the first - sets are the easiest way of determining which elements they are, like this:
first_list = [1, 2, 2, 5]
second_list = [2, 5, 7, 9]
in_first = set(first_list)
in_second = set(second_list)
in_second_but_not_in_first = in_second - in_first
result = first_list + list(in_second_but_not_in_first)
print(result) # Prints [1, 2, 2, 5, 9, 7]
Or if you prefer one-liners 8-)
print(first_list + list(set(second_list) - set(first_list)))
Solution 2 - Python
resulting_list = list(first_list)
resulting_list.extend(x for x in second_list if x not in resulting_list)
Solution 3 - Python
You can use sets:
first_list = [1, 2, 2, 5]
second_list = [2, 5, 7, 9]
resultList= list(set(first_list) | set(second_list))
print(resultList)
# Results in : resultList = [1,2,5,7,9]
Solution 4 - Python
first_list = [1, 2, 2, 5]
second_list = [2, 5, 7, 9]
print( set( first_list + second_list ) )
Solution 5 - Python
You can bring this down to one single line of code if you use numpy:
a = [1,2,3,4,5,6,7]
b = [2,4,7,8,9,10,11,12]
sorted(np.unique(a+b))
>>> [1,2,3,4,5,6,7,8,9,10,11,12]
Solution 6 - Python
resulting_list = first_list + [i for i in second_list if i not in first_list]
Solution 7 - Python
Simplest to me is:
first_list = [1, 2, 2, 5]
second_list = [2, 5, 7, 9]
merged_list = list(set(first_list+second_list))
print(merged_list)
#prints [1, 2, 5, 7, 9]
Solution 8 - Python
You can also combine RichieHindle's and Ned Batchelder's responses for an average-case O(m+n) algorithm that preserves order:
first_list = [1, 2, 2, 5]
second_list = [2, 5, 7, 9]
fs = set(first_list)
resulting_list = first_list + [x for x in second_list if x not in fs]
assert(resulting_list == [1, 2, 2, 5, 7, 9])
Note that x in s
has a worst-case complexity of O(m), so the worst-case complexity of this code is still O(m*n).
Solution 9 - Python
Based on the recipe :
> resulting_list = list(set().union(first_list, second_list))
Solution 10 - Python
This might help
def union(a,b):
for e in b:
if e not in a:
a.append(e)
The union function merges the second list into first, with out duplicating an element of a, if it's already in a. Similar to set union operator. This function does not change b. If a=[1,2,3] b=[2,3,4]. After union(a,b) makes a=[1,2,3,4] and b=[2,3,4]
Solution 11 - Python
you can use dict.fromkeys
to return a list with no duplicates:
def mergeTwoListNoDuplicates(list1, list2):
"""
Merges two lists together without duplicates
:param list1:
:param list2:
:return:
"""
merged_list = list1 + list2
merged_list = list(dict.fromkeys(merged_list))
return merged_list
Solution 12 - Python
first_list = [1, 2, 2, 5]
second_list = [2, 5, 7, 9]
newList=[]
for i in first_list:
newList.append(i)
for z in second_list:
if z not in newList:
newList.append(z)
newList.sort()
print newList
[1, 2, 2, 5, 7, 9]