Named tuple and default values for optional keyword arguments
PythonDefault ValueNamedtupleOptional ArgumentsPython Problem Overview
I'm trying to convert a longish hollow "data" class into a named tuple. My class currently looks like this:
class Node(object):
def __init__(self, val, left=None, right=None):
self.val = val
self.left = left
self.right = right
After conversion to namedtuple
it looks like:
from collections import namedtuple
Node = namedtuple('Node', 'val left right')
But there is a problem here. My original class allowed me to pass in just a value and took care of the default by using default values for the named/keyword arguments. Something like:
class BinaryTree(object):
def __init__(self, val):
self.root = Node(val)
But this doesn't work in the case of my refactored named tuple since it expects me to pass all the fields. I can of course replace the occurrences of Node(val)
to Node(val, None, None)
but it isn't to my liking.
So does there exist a good trick which can make my re-write successful without adding a lot of code complexity (metaprogramming) or should I just swallow the pill and go ahead with the "search and replace"? :)
Python Solutions
Solution 1 - Python
Python 3.7
Use the defaults parameter.
>>> from collections import namedtuple
>>> fields = ('val', 'left', 'right')
>>> Node = namedtuple('Node', fields, defaults=(None,) * len(fields))
>>> Node()
Node(val=None, left=None, right=None)
Or better yet, use the new dataclasses library, which is much nicer than namedtuple.
>>> from dataclasses import dataclass
>>> from typing import Any
>>> @dataclass
... class Node:
... val: Any = None
... left: 'Node' = None
... right: 'Node' = None
>>> Node()
Node(val=None, left=None, right=None)
Before Python 3.7
Set Node.__new__.__defaults__
to the default values.
>>> from collections import namedtuple
>>> Node = namedtuple('Node', 'val left right')
>>> Node.__new__.__defaults__ = (None,) * len(Node._fields)
>>> Node()
Node(val=None, left=None, right=None)
Before Python 2.6
Set Node.__new__.func_defaults
to the default values.
>>> from collections import namedtuple
>>> Node = namedtuple('Node', 'val left right')
>>> Node.__new__.func_defaults = (None,) * len(Node._fields)
>>> Node()
Node(val=None, left=None, right=None)
Order
In all versions of Python, if you set fewer default values than exist in the namedtuple, the defaults are applied to the rightmost parameters. This allows you to keep some arguments as required arguments.
>>> Node.__new__.__defaults__ = (1,2)
>>> Node()
Traceback (most recent call last):
...
TypeError: __new__() missing 1 required positional argument: 'val'
>>> Node(3)
Node(val=3, left=1, right=2)
Wrapper for Python 2.6 to 3.6
Here's a wrapper for you, which even lets you (optionally) set the default values to something other than None
. This does not support required arguments.
import collections
def namedtuple_with_defaults(typename, field_names, default_values=()):
T = collections.namedtuple(typename, field_names)
T.__new__.__defaults__ = (None,) * len(T._fields)
if isinstance(default_values, collections.Mapping):
prototype = T(**default_values)
else:
prototype = T(*default_values)
T.__new__.__defaults__ = tuple(prototype)
return T
Example:
>>> Node = namedtuple_with_defaults('Node', 'val left right')
>>> Node()
Node(val=None, left=None, right=None)
>>> Node = namedtuple_with_defaults('Node', 'val left right', [1, 2, 3])
>>> Node()
Node(val=1, left=2, right=3)
>>> Node = namedtuple_with_defaults('Node', 'val left right', {'right':7})
>>> Node()
Node(val=None, left=None, right=7)
>>> Node(4)
Node(val=4, left=None, right=7)
Solution 2 - Python
I subclassed namedtuple and overrode the __new__
method:
from collections import namedtuple
class Node(namedtuple('Node', ['value', 'left', 'right'])):
__slots__ = ()
def __new__(cls, value, left=None, right=None):
return super(Node, cls).__new__(cls, value, left, right)
This preserves an intuitive type hierarchy, which the creation of a factory function disguised as a class does not.
Solution 3 - Python
Wrap it in a function.
NodeT = namedtuple('Node', 'val left right')
def Node(val, left=None, right=None):
return NodeT(val, left, right)
Solution 4 - Python
With typing.NamedTuple
in Python 3.6.1+ you can provide both a default value and a type annotation to a NamedTuple field. Use typing.Any
if you only need the former:
from typing import Any, NamedTuple
class Node(NamedTuple):
val: Any
left: 'Node' = None
right: 'Node' = None
Usage:
>>> Node(1)
Node(val=1, left=None, right=None)
>>> n = Node(1)
>>> Node(2, left=n)
Node(val=2, left=Node(val=1, left=None, right=None), right=None)
Also, in case you need both default values and optional mutability, Python 3.7 is going to have data classes (PEP 557) that can in some (many?) cases replace namedtuples.
Sidenote: one quirk of the current specification of annotations (expressions after
:
for parameters and variables and after ->
for functions) in Python is that they are evaluated at definition time*. So, since "class names become defined once the entire body of the class has been executed", the annotations for 'Node'
in the class fields above must be strings to avoid NameError.
This kind of type hints is called "forward reference" ([1], [2]), and with PEP 563 Python 3.7+ is going to have a __future__
import (to be enabled by default in 4.0) that will allow to use forward references without quotes, postponing their evaluation.
* AFAICT only local variable annotations are not evaluated at runtime. (source: PEP 526)
Solution 5 - Python
This is an example straight from the docs:
> Default values can be implemented by using _replace() to customize a
> prototype instance:
>
> >>> Account = namedtuple('Account', 'owner balance transaction_count')
> >>> default_account = Account('
So, the OP's example would be:
from collections import namedtuple
Node = namedtuple('Node', 'val left right')
default_node = Node(None, None, None)
example = default_node._replace(val="whut")
However, I like some of the other answers given here better. I just wanted to add this for completeness.
Solution 6 - Python
I'm not sure if there's an easy way with just the built-in namedtuple. There's a nice module called recordtype that has this functionality:
>>> from recordtype import recordtype
>>> Node = recordtype('Node', [('val', None), ('left', None), ('right', None)])
>>> Node(3)
Node(val=3, left=None, right=None)
>>> Node(3, 'L')
Node(val=3, left=L, right=None)
Solution 7 - Python
Here is a more compact version inspired by justinfay's answer:
from collections import namedtuple
from functools import partial
Node = namedtuple('Node', ('val left right'))
Node.__new__ = partial(Node.__new__, left=None, right=None)
Solution 8 - Python
In python3.7+ there's a brand new defaults= keyword argument.
> defaults can be None
or an iterable of default values. Since fields with a default value must come after any fields without a default, the defaults are applied to the rightmost parameters. For example, if the fieldnames are ['x', 'y', 'z']
and the defaults are (1, 2)
, then x
will be a required argument, y
will default to 1
, and z
will default to 2
.
Example usage:
$ ./python
Python 3.7.0b1+ (heads/3.7:4d65430, Feb 1 2018, 09:28:35)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from collections import namedtuple
>>> nt = namedtuple('nt', ('a', 'b', 'c'), defaults=(1, 2))
>>> nt(0)
nt(a=0, b=1, c=2)
>>> nt(0, 3)
nt(a=0, b=3, c=2)
>>> nt(0, c=3)
nt(a=0, b=1, c=3)
Solution 9 - Python
Short, simple, and doesn't lead people to use isinstance
improperly:
class Node(namedtuple('Node', ('val', 'left', 'right'))):
@classmethod
def make(cls, val, left=None, right=None):
return cls(val, left, right)
# Example
x = Node.make(3)
x._replace(right=Node.make(4))
Solution 10 - Python
Python 3.7: introduction of defaults
param in namedtuple definition.
Example as shown in the documentation:
>>> Account = namedtuple('Account', ['type', 'balance'], defaults=[0])
>>> Account._fields_defaults
{'balance': 0}
>>> Account('premium')
Account(type='premium', balance=0)
Read more here.
Solution 11 - Python
A slightly extended example to initialize all missing arguments with None
:
from collections import namedtuple
class Node(namedtuple('Node', ['value', 'left', 'right'])):
__slots__ = ()
def __new__(cls, *args, **kwargs):
# initialize missing kwargs with None
all_kwargs = {key: kwargs.get(key) for key in cls._fields}
return super(Node, cls).__new__(cls, *args, **all_kwargs)
Solution 12 - Python
You can also use this:
import inspect
def namedtuple_with_defaults(type, default_value=None, **kwargs):
args_list = inspect.getargspec(type.__new__).args[1:]
params = dict([(x, default_value) for x in args_list])
params.update(kwargs)
return type(**params)
This basically gives you the possibility to construct any named tuple with a default value and override just the parameters you need, for example:
import collections
Point = collections.namedtuple("Point", ["x", "y"])
namedtuple_with_defaults(Point)
>>> Point(x=None, y=None)
namedtuple_with_defaults(Point, x=1)
>>> Point(x=1, y=None)
Solution 13 - Python
Combining approaches of @Denis and @Mark:
from collections import namedtuple
import inspect
class Node(namedtuple('Node', 'left right val')):
__slots__ = ()
def __new__(cls, *args, **kwargs):
args_list = inspect.getargspec(super(Node, cls).__new__).args[len(args)+1:]
params = {key: kwargs.get(key) for key in args_list + kwargs.keys()}
return super(Node, cls).__new__(cls, *args, **params)
That should support creating the tuple with positional arguments and also with mixed cases. Test cases:
>>> print Node()
Node(left=None, right=None, val=None)
>>> print Node(1,2,3)
Node(left=1, right=2, val=3)
>>> print Node(1, right=2)
Node(left=1, right=2, val=None)
>>> print Node(1, right=2, val=100)
Node(left=1, right=2, val=100)
>>> print Node(left=1, right=2, val=100)
Node(left=1, right=2, val=100)
>>> print Node(left=1, right=2)
Node(left=1, right=2, val=None)
but also support TypeError:
>>> Node(1, left=2)
TypeError: __new__() got multiple values for keyword argument 'left'
Solution 14 - Python
I find this version easier to read:
from collections import namedtuple
def my_tuple(**kwargs):
defaults = {
'a': 2.0,
'b': True,
'c': "hello",
}
default_tuple = namedtuple('MY_TUPLE', ' '.join(defaults.keys()))(*defaults.values())
return default_tuple._replace(**kwargs)
This is not as efficient as it requires creation of the object twice but you could change that by defining the default duple inside the module and just having the function do the replace line.
Solution 15 - Python
Since you are using namedtuple
as a data class, you should be aware that python 3.7 will introduce a @dataclass
decorator for this very purpose -- and of course it has default values.
@dataclass
class C:
a: int # 'a' has no default value
b: int = 0 # assign a default value for 'b'
Much cleaner, readable and usable than hacking namedtuple
. It is not hard to predict that usage of namedtuple
s will drop with the adoption of 3.7.
Solution 16 - Python
Inspired by this answer to a different question, here is my proposed solution based on a metaclass and using super
(to handle future subcalssing correctly). It is quite similar to justinfay's answer.
from collections import namedtuple
NodeTuple = namedtuple("NodeTuple", ("val", "left", "right"))
class NodeMeta(type):
def __call__(cls, val, left=None, right=None):
return super(NodeMeta, cls).__call__(val, left, right)
class Node(NodeTuple, metaclass=NodeMeta):
__slots__ = ()
Then:
>>> Node(1, Node(2, Node(4)),(Node(3, None, Node(5))))
Node(val=1, left=Node(val=2, left=Node(val=4, left=None, right=None), right=None), right=Node(val=3, left=None, right=Node(val=5, left=None, right=None)))
Solution 17 - Python
The answer by jterrace to use recordtype is great, but the author of the library recommends to use his namedlist project, which provides both mutable (namedlist
) and immutable (namedtuple
) implementations.
from namedlist import namedtuple
>>> Node = namedtuple('Node', ['val', ('left', None), ('right', None)])
>>> Node(3)
Node(val=3, left=None, right=None)
>>> Node(3, 'L')
Node(val=3, left=L, right=None)
Solution 18 - Python
Here's a short, simple generic answer with a nice syntax for a named tuple with default arguments:
import collections
def dnamedtuple(typename, field_names, **defaults):
fields = sorted(field_names.split(), key=lambda x: x in defaults)
T = collections.namedtuple(typename, ' '.join(fields))
T.__new__.__defaults__ = tuple(defaults[field] for field in fields[-len(defaults):])
return T
Usage:
Test = dnamedtuple('Test', 'one two three', two=2)
Test(1, 3) # Test(one=1, three=3, two=2)
Minified:
def dnamedtuple(tp, fs, **df):
fs = sorted(fs.split(), key=df.__contains__)
T = collections.namedtuple(tp, ' '.join(fs))
T.__new__.__defaults__ = tuple(df[i] for i in fs[-len(df):])
return T
Solution 19 - Python
Using the NamedTuple
class from my Advanced Enum (aenum)
library, and using the class
syntax, this is quite simple:
from aenum import NamedTuple
class Node(NamedTuple):
val = 0
left = 1, 'previous Node', None
right = 2, 'next Node', None
The one potential drawback is the requirement for a __doc__
string for any attribute with a default value (it's optional for simple attributes). In use it looks like:
>>> Node()
Traceback (most recent call last):
...
TypeError: values not provided for field(s): val
>>> Node(3)
Node(val=3, left=None, right=None)
The advantages this has over justinfay's answer
:
from collections import namedtuple
class Node(namedtuple('Node', ['value', 'left', 'right'])):
__slots__ = ()
def __new__(cls, value, left=None, right=None):
return super(Node, cls).__new__(cls, value, left, right)
is simplicity, as well as being metaclass
based instead of exec
based.
Solution 20 - Python
Another solution:
import collections
def defaultargs(func, defaults):
def wrapper(*args, **kwargs):
for key, value in (x for x in defaults[len(args):] if len(x) == 2):
kwargs.setdefault(key, value)
return func(*args, **kwargs)
return wrapper
def namedtuple(name, fields):
NamedTuple = collections.namedtuple(name, [x[0] for x in fields])
NamedTuple.__new__ = defaultargs(NamedTuple.__new__, [(NamedTuple,)] + fields)
return NamedTuple
Usage:
>>> Node = namedtuple('Node', [
... ('val',),
... ('left', None),
... ('right', None),
... ])
__main__.Node
>>> Node(1)
Node(val=1, left=None, right=None)
>>> Node(1, 2, right=3)
Node(val=1, left=2, right=3)
Solution 21 - Python
If you want to retain the possibility of using type annotation, unfortunately the very nice solution by @mark-lodato isn't usable (it fails for me on setting __defaults__
).
An alternative is using attrs:
import attr
@attr.s
class Node(object):
val: str = attr.ib()
left: 'Node' = attr.ib(None)
right: 'Node' = attr.ib(None)
This has:
- type annotations
- nice
__str__
and__repr__
- customizable, since it's a real class
- same implementation wit all Python versions
Solution 22 - Python
Here's a less flexible, but more concise version of Mark Lodato's wrapper: It takes the fields and defaults as a dictionary.
import collections
def namedtuple_with_defaults(typename, fields_dict):
T = collections.namedtuple(typename, ' '.join(fields_dict.keys()))
T.__new__.__defaults__ = tuple(fields_dict.values())
return T
Example:
In[1]: fields = {'val': 1, 'left': 2, 'right':3}
In[2]: Node = namedtuple_with_defaults('Node', fields)
In[3]: Node()
Out[3]: Node(val=1, left=2, right=3)
In[4]: Node(4,5,6)
Out[4]: Node(val=4, left=5, right=6)
In[5]: Node(val=10)
Out[5]: Node(val=10, left=2, right=3)