Demystifying C++ lambdas

A new (but not so welcome?) addition

Lambdas are one of the new features added to C++ and seem to cause considerable consternation amongst many programmers. In this article we’ll have a look at the syntax and underlying implementation of lambdas to try and put them into some sort of context.


Functors and the Standard Library

With STL algorithms the processing on each element is performed by a user-supplied unary or binary functor object. For common operations, the STL-supplied functors can be used (for example std::divides), but for bespoke manipulations a bespoke function or functor must be created.

image

A functor is a class that provides an implementation of operator().

In the case of functors used with the STL algorithms the operator() function must take either one parameter (for a unary procedure) or two parameters (for binary procedures) of appropriate types.

Creating bespoke functors can be a lot of effort; especially if the functor is only used in one specific place. These bespoke functors also unnecessarily ‘clutter up’ the code.

Introducing lambdas

A lambda is an ad-hoc, locally-scoped function (well, more strictly, a functor). Basically, lambdas are syntactic sugar, designed to reduce a lot of the work required in creating ad-hoc functor classes.

image

The brackets ([]) mark the declaration of the lambda; it can have parameters, and it should be followed by its body (the same as any other function).

When the lambda is executed the parameters are passed using the standard ABI mechanisms. One difference between lambdas and functions: lambda parameters can’t have defaults.

The body of the lambda is just a normal function body, and can be arbitrarily complex (although, as we’ll see, it’s generally good practice to keep lambda bodies relatively simple).

Note the lambda uses a trailing return type declaration. This is (no doubt) to simplify parsing (since types are not valid function parameters).

The return type may be omitted if:

  • The return type is void
  • The compiler can deduce the return type (lambda body is return <type>)

A lambda is an object (hence why we’re referring to it as a functor, rather than a function) so has a type and can be stored. However, the type of the lambda is only known by the compiler (since it is compiler-generated), so you must use auto for declaration instances of the lambda.

image

Lambdas allow ad-hoc functions to be declared at block scope (something that was illegal before in C++). The lambda function (functor) is only available within the scope of func() in this example; unlike a function, which would have global scope (or file scope if declared as static).

This means I can replace my manually-created functor in the STL algorithm with a lambda:

image

In the code above the for_each algorithm calls the lambda with each element in the container it turn.

So, that’s neat. But what has it really gained me?

Inline lambdas

Since a lambda is a scoped functor we can define it just in the scope where we actually need it – in our case within the for_each algorithm.

image

Now, the lambda is defined within the body of the algorithm and effectively only exists for the lifetime of the algorithm. We have reduced the scope of the code to just where it is needed (one of the principles of good modular design).

From a readability perspective we have put the functionality right where it is being used; unlike a functor, which may well be defined in another module (that the programmer has to go and find, and/or understand out of context to where it’s being used).

Under the hood

When you define a lambda the compiler uses that to create an ad-hoc functor class. The functor name is compiler-generated (and probably won’t be anything human readable)

image

The lambda body is used to generate the operator() method on the functor class. The client code is modified to use the new lambda-functor.

image

Capturing the context

Sometimes it’s useful and convenient to be able to access objects from the lambda’s containing scope – that is, the scope within which the lambda was defined. We could pass them in to the lambda as parameters (just like a normal function); however, this doesn’t work with algorithms since the algorithm has no mechanism for passing extra parameters from your code (how could it?)

If you were writing your own functor you could do this by passing in the appropriate parameters to the constructor of the functor. C++ provides a convenient mechanism for achieving this with lambdas called ‘capturing the context’.

The context of a lambda is the set of objects that are in scope when the lambda is called. The context objects may be captured then used as part of the lambda’s processing.

Capturing an object by name makes a lambda-local copy of the object.

image

Capturing an object by reference allows the lambda to manipulate its context. That is, the lambda can change the values of the objects it has captured by reference.

image

A word of warning here:  A lambda is, as we’ve seen, just an object and, like other objects it may be copied, passed as a parameter, stored in a container, etc. The lambda object has its own scope and lifetime which may, in some circumstances, be different to those objects it has ‘captured’.  Be very careful when capturing local objects by reference because a lambda’s lifetime may exceed the lifetime of its capture list. In other words, the lambda may have a reference to an object no longer in scope!

All variables in scope can be captured using a default-capture.  This makes available all automatic variables currently in scope.

image

Note, the compiler is only required to make copies of captured objects if they are actually used within the body of the lambda.

Under the hood (again)

When you add a capture list to a lambda the compiler adds appropriate member variables to the lambda-functor class and a constructor for initialising these variables.

image

It is relatively easy to see now why capturing the context has potential overheads: for every object captured by value a copy of the original is made; for every object captured by reference a reference is stored.

image

 

Lambdas within member functions

It is perfectly possible (and quite likely) that we may want to use lambdas inside class member functions. Remember, a lambda is a unique (and separate) class of its own so when it executes it has its own context. Therefore, it does not have direct access to any of the class’ member variables.

To capture the class’ member variables we must capture the this pointer of the class. We now have full access to all the class’ data (including private data, as we are inside a member function).

image

Callable objects

Callable object is a generic name for any object that can be called like a function:

  • A member function (pointer)
  • A free function (pointer)
  • A functor
  • A lambda

In C we have the concept of a pointer-to-function, which allows the address of any function to be stored (providing its signature matches that of the pointer). However, a pointer-function has a different signature to a pointer-to-member-function; which as a different signature to a lambda. What would be nice is a generalised ‘pointer-to-callable-object’ that could store the address of any callable object (providing its signature matched, of course).

std::function is a template class that can hold any callable object that matches its signature. std::function provides a consistent mechanism for storing, passing and accessing these objects.

image

std::function can be thought of as a generic pointer-to-function that can point at any callable object, provided the callable object matches the signature of the std::function. And, unlike C’s pointer-to-function, the C++ compiler provides strong type-checking on the parameters of the callable object (including the return type).

image

std::function provides an overload for operator!= to allow it to be compared to nullptr (so it can act like a function-pointer).

Our SimpleCallback class can be used with any callable type – functors, free functions or lambdas, without any change, since they all match the signature required by callback.

image

In summary

Despite the awkward syntax lambdas are not a mechanism to be feared or despised. They merely provides a useful way of simplify code and reducing programmer effort. In essence they are no more than syntactic sugar. In this respect they are no more detrimental than operator overloading.

 

For more information on other aspects of C++11, have a look at our  training course AC++11-401 – Transitioning to C++11.

For more on C++ programming – particularly for embedded and real-time applications – visit the Feabhas website. You may find the following of interest:

C++-501 – C++ for Embedded Developers

C++-502 – C++ for Real-Time Developers

AC++-501 – Advanced C++

Posted on March 7th, 2014
» Feed to this thread
» Trackback

22 Comments a “Demystifying C++ lambdas”

  1. Dan says:

    This is one of the best write-ups on lambdas I’ve seen, thank you.

    That said, and I know this is extremely subjective, I’m not the inline lambda syntax improves readability (e.g. for_each). It’s completely possible that it’s just so new that my brain is still fighting it, I’ll concede that. But I could quickly see a lambda becoming more complex, and soon the for_each looks like someone sneezed out a bunch of code (I realize lambdas should only be used for small snippets, but we all know how well code ages as new & different developers come on board).

    Anyway, thanks again, this one is definitely bookmarked. I haven’t found too many of the C++11 features to be compelling for deeply embedded systems, but this post has encouraged me to at least give lambdas another look.

  2. Antoine says:

    Very nice. Stright to the point, informative and pedagogical.
    May I ask what software you used to produce the nice code snapshots with the comments and all?
    Thanks.

  3. glennan says:

    Thanks Antoine!
    The code snippets are done with PowerPoint :-) They’re taking directly from our training course materials.

  4. Ioannis says:

    Great Work!

    I think you are missing a -> on the last picture

    int main(){
    SimpleCallback callback([]() -> {cout << "Lambda" << endl;});
    callback.execute();
    }

    Other than that, excellent work!

  5. FipS says:

    Very well written and nicely presented! Thanks for sharing.

  6. Paul Jurczak says:

    Nice article, thank you. You may want to revisit your Filter example though. std::remove and std::remove_if are the most confusingly named STL algorithms – they don’t remove anything, they just rearrange order of elements in the container and create a bunch of undefined values. You must follow them with erase, in order to achieve desired result.

  7. Henri Tuhola says:

    C++ – lots of carsinogenic abstractions

  8. Steve Little says:

    “It is relatively easy to see now why capturing the context has potential overheads: for every object captured by value a copy of the original is made; for every object captured by reference a reference is stored. (Therefore you might want to think twice about capturing everything in the context by value (using [=]))”

    I’m not sure that’s correct. The C++ language standard section [expr.prim.lambda] says:

    “If a lambda-expression has an associated capture-default and its compound-statement odr-uses (3.2) this or a variable with automatic storage duration and the odr-used entity is not explicitly captured, then the odr-used entity is said to be implicitly captured; such entities shall be declared within the reaching scope of
    the lambda expression.”

    Based on my reading of that, a compiler is *not required* to capture all objects (when using a capture-default) just all objects that are ODR-used in the body of the lambda-expression.

    Some implementations *may* choose to copy everything, but that would be the fault of an inefficient implementation, and not necessarily a good reason to avoid using a capture-default.

  9. Michael Hamilton says:

    @Ioannis: No, the author has it correct. Lambda declarations only require -> when the return type cannot be automatically deduced. In the case of no return statement it is trivially “void”.

  10. KrzaQ says:

    I liked the article, but I have one nit-pick: unless your lambda is mutable, its `operator()` should take `const this`.

  11. Andy Prowl says:

    Nice article :) One minor thing: the lambda’s call operator generated by the compiler is const-qualified, unless “mutable” is used.

  12. glennan says:

    Well spotted, Andy.

    I’ve updated the article to reflect this.

  13. glennan says:

    Very good point Steve.
    I’ve updated the article to reflect this.

    Thanks.

  14. Marcel Wid says:

    Your code examples in section Under the hood (again) are wrong:

    1. “It is unspecified whether additional unnamed non-static data members are declared in the closure type for entities captured by reference.” (§5.1.2p15)

    2. This is a serious mistake! The closure type (your class _SomeCompilerGeneratedName_) is local to main, i.e. it is defined inside main! And you don’t pass a pointer to for_each. The object _inst001_ is never created as lvalue. The correct compiler generated code is “for_each(v.begin(), v.end(), _SomeCompilerGeneratedName_(offset));”

    3. To be even more precise, the closure type has a deleted default constructor.

  15. glennan says:

    Marcel,

    Good points, there.

    I don’t disagree with you. The idea of the ‘conceptual equivalence’ example was to echo the earlier functor example, to help illustrate to the reader approximately what was happening. It was never intended to be a literal ‘what the compiler generates’.

    What you’ve written is, of course, quite correct. Apologies for the confusion.

  16. Marcel Wid says:

    Well, I see.

    But it is even conceptually wrong to pass a pointer to a functor (&_inst001_) to for_each. If you want the analogy to functors then you should pass (the lvalue) _inst001_ itself as you did in the functor example where you pass f and not &f.

  17. glennan says:

    Quite right; and updated. :-)

  18. panovr says:

    “This means I can replace my manually-created functor in the STL algorithm with a lambda:”

    auto lambda = [](X& elem) -> void { elem.op(); }

    should be:

    auto lambda = [](X& elem) -> void { elem.op(); };

  19. glennan says:

    It is now! :-)

    Thank you!

  20. panovr says:

    You wrote: “Callable object is a generic name for any object that can be called like a function:
    1. A member function (pointer)
    2. A free function (pointer)
    3. functor
    4. lambda”

    And there are three examples using std::function with functor, free function and lambda.

    How about a class member function? Can it be used with std::function?

  21. Andrei Zissu says:

    panovr,

    You can bind a member function indirectly, as you also have to bind to a specific object instance (unless it’s a static member function, which is just like any other non-member function). You can accomplish that by either providing a lambda which itself makes the actual call, or by using std::bind.

  22. Using C++11 Lambdas with ELLCC | The ELLCC Embedded Compiler Collection says:

    […] recent post on C++11 lambdas and a forum question led me to this topic. Lambdas are a new C++ feature, described very well in […]

Leave a Reply