When should iteritems() be used instead of items()?
PythonPython 3.xPython Problem Overview
Is it legitimate to use items()
instead of iteritems()
in all places? Why was iteritems()
removed from Python 3? Seems like a terrific and useful method. What's the reasoning behind it?
Edit: To clarify, I want to know what is the correct idiom for iterating over a dictionary in a generator-like way (one item at a time, not all into memory) in a way that is compatible with both Python 2 and Python 3?
Python Solutions
Solution 1 - Python
In Python 2.x - .items()
returned a list of (key, value) pairs. In Python 3.x, .items()
is now an itemview
object, which behaves differently - so it has to be iterated over, or materialised... So, list(dict.items())
is required for what was dict.items()
in Python 2.x.
Python 2.7 also has a bit of a back-port for key handling, in that you have viewkeys
, viewitems
and viewvalues
methods, the most useful being viewkeys
which behaves more like a set
(which you'd expect from a dict
).
Simple example:
common_keys = list(dict_a.viewkeys() & dict_b.viewkeys())
Will give you a list of the common keys, but again, in Python 3.x - just use .keys()
instead.
Python 3.x has generally been made to be more "lazy" - i.e. map
is now effectively itertools.imap
, zip
is itertools.izip
, etc.
Solution 2 - Python
dict.iteritems
was removed because dict.items
now does the thing dict.iteritems
did in python 2.x and even improved it a bit by making it an itemview
.
Solution 3 - Python
The six library helps with writing code that is compatible with both python 2.5+ and python 3. It has an iteritems method that will work in both python 2 and 3. Example:
import six
d = dict( foo=1, bar=2 )
for k, v in six.iteritems(d):
print(k, v)
Solution 4 - Python
As the dictionary documentation for python 2 and python 3 would tell you, in python 2 items
returns a list, while iteritems
returns a iterator.
In python 3, items
returns a view, which is pretty much the same as an iterator.
If you are using python 2, you may want to user iteritems
if you are dealing with large dictionaries and all you want to do is iterate over the items (not necessarily copy them to a list)
Solution 5 - Python
Just as @Wessie noted, dict.iteritems
, dict.iterkeys
and dict.itervalues
(which return an iterator in Python2.x) as well as dict.viewitems
, dict.viewkeys
and dict.viewvalues
(which return view objects in Python2.x) were all removed in Python3.x
And dict.items
, dict.keys
and dict.values
used to return a copy of the dictionary's list in Python2.x now return view objects in Python3.x, but they are still not the same as iterator.
If you want to return an iterator in Python3.x, use iter(dictview)
:
$ python3.3
>>> d = {'one':'1', 'two':'2'}
>>> type(d.items())
<class 'dict_items'>
>>>
>>> type(d.keys())
<class 'dict_keys'>
>>>
>>>
>>> ii = iter(d.items())
>>> type(ii)
<class 'dict_itemiterator'>
>>>
>>> ik = iter(d.keys())
>>> type(ik)
<class 'dict_keyiterator'>
Solution 6 - Python
You cannot use items
instead iteritems
in all places in Python. For example, the following code:
class C:
def __init__(self, a):
self.a = a
def __iter__(self):
return self.a.iteritems()
>>> c = C(dict(a=1, b=2, c=3))
>>> [v for v in c]
[('a', 1), ('c', 3), ('b', 2)]
will break if you use items
:
class D:
def __init__(self, a):
self.a = a
def __iter__(self):
return self.a.items()
>>> d = D(dict(a=1, b=2, c=3))
>>> [v for v in d]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: __iter__ returned non-iterator of type 'list'
The same is true for viewitems, which is available in Python 3.
Also, since items returns a copy of the dictionary’s list of (key, value)
pairs, it is less efficient, unless you want to create a copy anyway.
In Python 2, it is best to use iteritems
for iteration. The 2to3
tool can replace it with items
if you ever decide to upgrade to Python 3.
Solution 7 - Python
future.utils
allows for python 2 and 3 compatibility.
# Python 2 and 3: option 3
from future.utils import iteritems
heights = {'man': 185,'lady': 165}
for (key, value) in iteritems(heights):
print(key,value)
>>> ('lady', 165)
>>> ('man', 185)
See this link: https://python-future.org/compatible_idioms.html
Solution 8 - Python
If for some reason someone still needs to do this and doesn't want to use a compatibility library, a simple approach is to make a top-level callable that calls the correct one by version:
from operator import methodcaller
try:
{}.viewitems()
except AttributeError:
# Py3, use items
viewitems = methodcaller('items')
else:
viewitems = methodcaller('viewitems')
mydict = {...}
for k, v in viewitems(mydict):
...
This will behave identically on 2.7 and 3+ (no copying, always gets a live view). You could change all references to view
from view
to iter
to support pre-2.7 Python, though the behavior would differ subtly in some cases (views are iterbles, but not iterators; you can call next
on an iterator, but not a non-iterator iterable).
In practice though:
- At this point Py2 is EOL, so just write for Py3, and
- If you must support both, write as portably as you can for Py2 (with all the
__future__
imports, explicitu
andb
prefixes for text vs. bytes literals, usingio.open
to replaceopen
, etc.) and use2to3
for the little things that you can't directly write portably.