Django template can't loop defaultdict

import collections

data = [
  {'firstname': 'John', 'lastname': 'Smith'}, 
  {'firstname': 'Samantha', 'lastname': 'Smith'}, 
  {'firstname': 'shawn', 'lastname': 'Spencer'}, 
]

new_data = collections.defaultdict(list)

for d in data:
    new_data[d['lastname']].append(d['firstname'])

print new_data

Here's the output:

defaultdict(<type 'list'>, {'Smith': ['John', 'Samantha'], 'Spencer': ['shawn']})

and here's the template:

{% for lastname, firstname in data.items %}
  <h1> {{ lastname }} </h1>
  <p> {{ firstname|join:", " }} </p>
{% endfor %}

But the loop in my template doesn't work. Nothing shows up. It doesn't even give me an error. How can i fix this? It's supposed to show the lastname along with the firstname, something like this:

<h1> Smith </h1>
<p> John, Samantha </p>

<h1> Spencer </h1>
<p> shawn </p>

You can avoid the copy to a new dict by disabling the defaulting feature of defaultdict once you are done inserting new values:

new_data.default_factory = None

Explanation

The template variable resolution algorithm in Django will attempt to resolve new_data.items as new_data['items'] first, which resolves to an empty list when using defaultdict(list).

To disable the defaulting to an empty list and have Django fail on new_data['items'] then continue the resolution attempts until calling new_data.items(), the default_factory attribute of defaultdict can be set to None.

try:

dict(new_data)

and in Python 2 it is better to use iteritems instead of items :)

Since the "problem" still exist years later and is inherint to the way Django templates work, I prefer writing a new answer giving the full details of why this behaviour is kept as-is.

How-to fix the bug

First, the solution is to cast the defaultdict into a dict before passing it to the template context:

context = {
    'data': dict(new_data)
}

You should not use defaultdict objects in template context in Django.

But why?

The reason behind this "bug" is detailed in the following Django issue #16335:

> Indeed, it boils down to the fact that the template language uses the same syntax for dictionary and attribute lookups.

... and from the docs:

> Dictionary lookup, attribute lookup and list-index lookups are implemented with a dot notation. [...] If a variable resolves to a callable, the template system will call it with no arguments and use its result instead of the callable.

When Django resolve your template expression it will try first data['items']. BUT, this is a valid expression, which will automatically creates a new entry items in your defaultdict data, initialized with an empty list (in the original author case) and returns the list created (empty).

The intented action would be to call the method items with no arguments of the instance data (in short: data.items()), but since data['items'] was a valid expression, Django stop there and gets the empty list just created.

If you try the same code but with data = defaultdict(int), you would get a TypeError: 'int' object is not iterable, because Django won't be able to iterate over the "0" value returned by the creation of the new entry of the defaultdict.