Why does C++11's lambda require "mutable" keyword for capture-by-value, by default?

Short example:

#include <iostream>

int main()
{
    int n;
    [&](){n = 10;}();             // OK
    [=]() mutable {n = 20;}();    // OK
    // [=](){n = 10;}();          // Error: a by-value capture cannot be modified in a non-mutable lambda
    std::cout << n << "\n";       // "10"
}

The question: Why do we need the mutable keyword? It's quite different from traditional parameter passing to named functions. What's the rationale behind?

I was under the impression that the whole point of capture-by-value is to allow the user to change the temporary -- otherwise I'm almost always better off using capture-by-reference, aren't I?

Any enlightenments?

(I'm using MSVC2010 by the way. AFAIK this should be standard)

It requires mutable because by default, a function object should produce the same result every time it's called. This is the difference between an object orientated function and a function using a global variable, effectively.

Your code is almost equivalent to this:

#include <iostream>

class unnamed1
{
	int& n;
public:
	unnamed1(int& N) : n(N) {}
	
	/* OK. Your this is const but you don't modify the "n" reference,
	but the value pointed by it. You wouldn't be able to modify a reference
	anyway even if your operator() was mutable. When you assign a reference
	it will always point to the same var.
	*/
	void operator()() const {n = 10;}
};

class unnamed2
{
	int n;
public:
	unnamed2(int N) : n(N) {}
	
	/* OK. Your this pointer is not const (since your operator() is "mutable" instead of const).
	So you can modify the "n" member. */
	void operator()() {n = 20;}
};

class unnamed3
{
	int n;
public:
	unnamed3(int N) : n(N) {}
	
	/* BAD. Your this is const so you can't modify the "n" member. */
	void operator()() const {n = 10;}
};

int main()
{
    int n;
    unnamed1 u1(n); u1();    // OK
    unnamed2 u2(n); u2();    // OK
    //unnamed3 u3(n); u3();  // Error
    std::cout << n << "\n";  // "10"
}

So you could think of lambdas as generating a class with operator() that defaults to const unless you say that it is mutable.

You can also think of all the variables captured inside [] (explicitly or implicitly) as members of that class: copies of the objects for [=] or references to the objects for [&]. They are initialized when you declare your lambda as if there was a hidden constructor.

> I was under the impression that the whole point of capture-by-value is to allow the user to change the temporary -- otherwise I'm almost always better off using capture-by-reference, aren't I?

The question is, is it "almost"? A frequent use-case appears to be to return or pass lambdas:

void registerCallback(std::function<void()> f) { /* ... */ }

void doSomething() {
  std::string name = receiveName();
  registerCallback([name]{ /* do something with name */ });
}

I think that mutable isn't a case of "almost". I consider "capture-by-value" like "allow me to use its value after the captured entity dies" rather than "allow me to change a copy of it". But perhaps this can be argued.

FWIW, Herb Sutter, a well-known member of the C++ standardization committee, provides a different answer to that question in [Lambda Correctness and Usability Issues][1]:

> Consider this straw man example, where the programmer captures a local variable by > value and tries to modify the > captured value (which is a member variable of the lambda object): > > int val = 0; > auto x = [=](item e) // look ma, [=] means explicit copy > { use(e,++val); }; // error: count is const, need ‘mutable’ > auto y = [val](item e) // darnit, I really can’t get more explicit > { use(e,++val); }; // same error: count is const, need ‘mutable’ > > This feature appears to have been added out of a concern that the user > might not realize he got a copy, and in particular that since lambdas > are copyable he might be changing a different lambda’s copy.

His paper is about why this should be changed in C++14. It is short, well written, worth reading if you want to know "what's on [committee member] minds" with regards to this particular feature.

[1]: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3424.pdf "Lambda Correctness and Usability Issues"

You have to understand what capture means! it's capturing not argument passing! let's look at some code samples:

int main()
{
	using namespace std;
	int x = 5;
	int y;
	auto lamb = [x]() {return x + 5; };
	
	y= lamb();
	cout << y<<","<< x << endl; //outputs 10,5
	x = 20;
	y = lamb();
	cout << y << "," << x << endl; //output 10,20

}

As you can see even though x has been changed to 20 the lambda is still returning 10 ( x is still 5 inside the lambda) Changing x inside the lambda means changing the lambda itself at each call (the lambda is mutating at each call). To enforce correctness the standard introduced the mutable keyword. By specifying a lambda as mutable you are saying that each call to the lambda could cause a change in the lambda itself. Let see another example:

int main()
{
	using namespace std;
	int x = 5;
	int y;
	auto lamb = [x]() mutable {return x++ + 5; };
	
	y= lamb();
	cout << y<<","<< x << endl; //outputs 10,5
	x = 20;
	y = lamb();
	cout << y << "," << x << endl; //outputs 11,20
	
}

The above example shows that by making the lambda mutable, changing x inside the lambda "mutates" the lambda at each call with a new value of x that has no thing to do with the actual value of x in the main function

You need to think what is the closure type of your Lambda function. Every time you declare a Lambda expression, the compiler creates a closure type, which is nothing less than an unnamed class declaration with attributes (environment where the Lambda expression where declared) and the function call ::operator() implemented. When you capture a variable using copy-by-value, the compiler will create a new const attribute in the closure type, so you can't change it inside the Lambda expression because it is a "read-only" attribute, that's the reason they call it a "closure", because in some way, you are closing your Lambda expression by copying the variables from upper scope into the Lambda scope. When you use the keyword mutable, the captured entity will became a non-const attribute of your closure type. This is what causes the changes done in the mutable variable captured by value, to not be propagated to upper scope, but keep inside the stateful Lambda. Always try to imagine the resulting closure type of your Lambda expression, that helped me a lot, and I hope it can help you too.

See this draft, under 5.1.2 [expr.prim.lambda], subclause 5: >The closure type for a lambda-expression has a public inline function call operator (13.5.4) whose parameters and return type are described by the lambda-expression’s parameter-declaration-clause and trailingreturn- type respectively. This function call operator is declared const (9.3.1) if and only if the lambdaexpression’s parameter-declaration-clause is not followed by mutable.

Edit on litb's comment: Maybe they thought of capture-by-value so that outside changes to the variables aren't reflected inside the lambda? References work both ways, so that's my explanation. Don't know if it's any good though.

Edit on kizzx2's comment: The most times when a lambda is to be used is as a functor for algorithms. The default constness lets it be used in a constant environment, just like normal const-qualified functions can be used there, but non-const-qualified ones can't. Maybe they just thought to make it more intuitive for those cases, who know what goes on in their mind. :)

> I was under the impression that the > whole point of capture-by-value is to > allow the user to change the temporary > -- otherwise I'm almost always better off using capture-by-reference, aren't > I?

n is not a temporary. n is a member of the lambda-function-object that you create with the lambda expression. The default expectation is that calling your lambda does not modify its state, therefore it is const to prevent you from accidentally modifying n.

There is now a proposal to alleviate the need for mutable in lambda declarations: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n3424.pdf">n3424</a>

To extend Puppy's answer, lambda functions are intended to be pure functions. That means every call given a unique input set always returns the same output. Let's define input as the set of all arguments plus all captured variables when the lambda is called.

In pure functions output solely depends on input and not on some internal state. Therefore any lambda function, if pure, does not need to change its state and is therefore immutable.

When a lambda captures by reference, writing on captured variables is a strain on the concept of pure function, because all a pure function should do is return an output, though the lambda does not certainly mutate because the writing happens to external variables. Even in this case a correct usage implies that if the lambda is called with the same input again, the output will be the same everytime, despite these side effects on by-ref variables. Such side effects are just ways to return some additional input (e.g. update a counter) and could be reformulated into a pure function, for example returning a tuple instead of a single value.

I also was wondering about it and the simplest explanation why [=] requires explicit mutable is in this example:

int main()
{
 	int x {1};
  	auto lbd = [=]() mutable { return x += 5; };
    printf("call1:%d\n", lbd());
    printf("call2:%d\n", lbd());
    return 0;
}

Output:

call1:6
call2:11

By words:

You can see that the x value is different at the second call (1 for the call1 and 6 for the call2).

1. A lambda object keeps a captured variable by value (has its own copy) in case of [=].
2. The lambda can be called several times.

And in general case we have to have the same value of the captured variable to have the same predictable behavior of the lambda based on the known captured value, not updated during the lambda work. That's why the default behavior assumed const (to predict changes of the lambda object members) and when a user is aware of consequences he takes this responsibility on himself with mutable.

Same with capturing by value. For my example:

auto lbd = [x]() mutable { return x += 5; };