Is it always safe to modify the `**kwargs` dictionary?

PythonFunctionDictionaryScope

Python Problem Overview


Using the Python function syntax def f(**kwargs), in the function a keyword argument dictionary kwargs is created, and dictionaries are mutable, so the question is, if I modify the kwargs dictionary, is it possible that I might have some effect outside the scope of my function?

From my understanding of how dictionary unpacking and keyword argument packing works, I don't see any reason to believe it might be unsafe, and it seems to me that there is no danger of this in Python 3.6:

def f(**kwargs):
    kwargs['demo'] = 9

if __name__ == '__main__':
    demo = 4
    f(demo=demo)
    print(demo)     # 4
    
    kwargs = {}
    f(**kwargs)
    print(kwargs)   # {}
    
    kwargs['demo'] = 4
    f(**kwargs)
    print(kwargs)    # {'demo': 4}

However, is this implementation-specific, or is it part of the Python spec? Am I overlooking any situation or implementation where (barring modifications to arguments which are themselves mutable, like kwargs['somelist'].append(3)) this sort of modification might be a problem?

Python Solutions


Solution 1 - Python

It is always safe. As the spec says

> If the form “**identifier” is present, it is initialized to a new > ordered mapping receiving any excess keyword arguments, defaulting to > a new empty mapping of the same type.

Emphasis added.

You are always guaranteed to get a new mapping-object inside the callable. See this example

def f(**kwargs):
    print((id(kwargs), kwargs))

kwargs = {'foo': 'bar'}
print(id(kwargs))
# 140185018984344
f(**kwargs)
# (140185036822856, {'foo': 'bar'})

So, although f may modify an object that is passed via **, it can't modify the caller's **-object itself.


Update: Since you asked about corner cases, here is a special hell for you that does in fact modify the caller's kwargs:

def f(**kwargs):
    kwargs['recursive!']['recursive!'] = 'Look ma, recursive!'

kwargs = {}
kwargs['recursive!'] = kwargs
f(**kwargs)
assert kwargs['recursive!'] == 'Look ma, recursive!'

This you probably won't see in the wild, though.

Solution 2 - Python

For Python-level code, the kwargs dict inside a function will always be a new dict.

For C extensions, though, watch out. The C API version of kwargs will sometimes pass a dict through directly. In previous versions, it would even pass dict subclasses through directly, leading to the bug (now fixed) where

'{a}'.format(**collections.defaultdict(int))

would produce '0' instead of raising a KeyError.

If you ever have to write C extensions, possibly including Cython, don't try to modify the kwargs equivalent, and watch out for dict subclasses on old Python versions.

Solution 3 - Python

Both of above answers are correct in stating that technically, mutating kwargs will never have an effect on the parent scopes.

But... that's not the end of the story. It is possible for a reference to kwargs to be shared outside of the function scope, and then you run into all the usual shared mutated state problems that you'd expect.

def create_classes(**kwargs):

    class Class1:
        def __init__(self):
            self.options = kwargs

    class Class2:
        def __init__(self):
            self.options = kwargs

    return (Class1, Class2)

Class1, Class2 = create_classes(a=1, b=2)

a = Class1()
b = Class2()

a.options['c'] = 3

print(b.options)
# {'a': 1, 'b': 2, 'c': 3}
# other class's options are mutated because we forgot to copy kwargs

Technically this answers your question, since sharing a reference to mutable kwargs does lead to effects outside of the function scope's.

I've been bitten multiple times by this in production code, and it's something that I explicitly watch out for now, both in my own code and when reviewing others. The mistake is obvious in my contrived example above, but it's much sneakier in real code when creating factory funcs that share some common options.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionPaulView Question on Stackoverflow
Solution 1 - Pythonuser2722968View Answer on Stackoverflow
Solution 2 - Pythonuser2357112View Answer on Stackoverflow
Solution 3 - PythonNick SweetingView Answer on Stackoverflow