Demystifying C++ lambdas

A new (but not so welcome?) addition

Lambdas are one of the new features added to C++ and seem to cause considerable consternation amongst many programmers. In this article we’ll have a look at the syntax and underlying implementation of lambdas to try and put them into some sort of context.

Functors and the Standard Library

With STL algorithms the processing on each element is performed by a user-supplied unary or binary functor object. For common operations, the STL-supplied functors can be used (for example std::divides), but for bespoke manipulations a bespoke function or functor must be created.

image

A functor is a class that provides an implementation of operator().

In the case of functors used with the STL algorithms the operator() function must take either one parameter (for a unary procedure) or two parameters (for binary procedures) of appropriate types.

Creating bespoke functors can be a lot of effort; especially if the functor is only used in one specific place. These bespoke functors also unnecessarily ‘clutter up’ the code.

Introducing lambdas

A lambda is an ad-hoc, locally-scoped function (well, more strictly, a functor). Basically, lambdas are syntactic sugar, designed to reduce a lot of the work required in creating ad-hoc functor classes.

image

The brackets ([]) mark the declaration of the lambda; it can have parameters, and it should be followed by its body (the same as any other function).

When the lambda is executed the parameters are passed using the standard ABI mechanisms. One difference between lambdas and functions: lambda parameters can’t have defaults.

The body of the lambda is just a normal function body, and can be arbitrarily complex (although, as we’ll see, it’s generally good practice to keep lambda bodies relatively simple).

Note the lambda uses a trailing return type declaration. This is (no doubt) to simplify parsing (since types are not valid function parameters).

The return type may be omitted if:

  • The return type is void
  • The compiler can deduce the return type (lambda body is return <type>)

A lambda is an object (hence why we’re referring to it as a functor, rather than a function) so has a type and can be stored. However, the type of the lambda is only known by the compiler (since it is compiler-generated), so you must use auto for declaration instances of the lambda.

image

Lambdas allow ad-hoc functions to be declared at block scope (something that was illegal before in C++). The lambda function (functor) is only available within the scope of func() in this example; unlike a function, which would have global scope (or file scope if declared as static).

This means I can replace my manually-created functor in the STL algorithm with a lambda:

image

In the code above the for_each algorithm calls the lambda with each element in the container it turn.

So, that’s neat. But what has it really gained me?

Inline lambdas

Since a lambda is a scoped functor we can define it just in the scope where we actually need it – in our case within the for_each algorithm.

image

Now, the lambda is defined within the body of the algorithm and effectively only exists for the lifetime of the algorithm. We have reduced the scope of the code to just where it is needed (one of the principles of good modular design).

From a readability perspective we have put the functionality right where it is being used; unlike a functor, which may well be defined in another module (that the programmer has to go and find, and/or understand out of context to where it’s being used).

Under the hood

When you define a lambda the compiler uses that to create an ad-hoc functor class. The functor name is compiler-generated (and probably won’t be anything human readable)

image

The lambda body is used to generate the operator() method on the functor class. The client code is modified to use the new lambda-functor.

image

Capturing the context

Sometimes it’s useful and convenient to be able to access objects from the lambda’s containing scope – that is, the scope within which the lambda was defined. We could pass them in to the lambda as parameters (just like a normal function); however, this doesn’t work with algorithms since the algorithm has no mechanism for passing extra parameters from your code (how could it?)

If you were writing your own functor you could do this by passing in the appropriate parameters to the constructor of the functor. C++ provides a convenient mechanism for achieving this with lambdas called ‘capturing the context’.

The context of a lambda is the set of objects that are in scope when the lambda is called. The context objects may be captured then used as part of the lambda’s processing.

Capturing an object by name makes a lambda-local copy of the object.

image

Capturing an object by reference allows the lambda to manipulate its context. That is, the lambda can change the values of the objects it has captured by reference.

image

A word of warning here:  A lambda is, as we’ve seen, just an object and, like other objects it may be copied, passed as a parameter, stored in a container, etc. The lambda object has its own scope and lifetime which may, in some circumstances, be different to those objects it has ‘captured’.  Be very careful when capturing local objects by reference because a lambda’s lifetime may exceed the lifetime of its capture list. In other words, the lambda may have a reference to an object no longer in scope!

All variables in scope can be captured using a default-capture.  This makes available all automatic variables currently in scope.

image

Note, the compiler is only required to make copies of captured objects if they are actually used within the body of the lambda.

Under the hood (again)

When you add a capture list to a lambda the compiler adds appropriate member variables to the lambda-functor class and a constructor for initialising these variables.

image

It is relatively easy to see now why capturing the context has potential overheads: for every object captured by value a copy of the original is made; for every object captured by reference a reference is stored.

image

 

Lambdas within member functions

It is perfectly possible (and quite likely) that we may want to use lambdas inside class member functions. Remember, a lambda is a unique (and separate) class of its own so when it executes it has its own context. Therefore, it does not have direct access to any of the class’ member variables.

To capture the class’ member variables we must capture the this pointer of the class. We now have full access to all the class’ data (including private data, as we are inside a member function).

image

Callable objects

Callable object is a generic name for any object that can be called like a function:

  • A member function (pointer)
  • A free function (pointer)
  • A functor
  • A lambda

In C we have the concept of a pointer-to-function, which allows the address of any function to be stored (providing its signature matches that of the pointer). However, a pointer-function has a different signature to a pointer-to-member-function; which as a different signature to a lambda. What would be nice is a generalised ‘pointer-to-callable-object’ that could store the address of any callable object (providing its signature matched, of course).

std::function is a template class that can hold any callable object that matches its signature. std::function provides a consistent mechanism for storing, passing and accessing these objects.

image

std::function can be thought of as a generic pointer-to-function that can point at any callable object, provided the callable object matches the signature of the std::function. And, unlike C’s pointer-to-function, the C++ compiler provides strong type-checking on the parameters of the callable object (including the return type).

image

std::function provides an overload for operator!= to allow it to be compared to nullptr (so it can act like a function-pointer).

Our SimpleCallback class can be used with any callable type – functors, free functions or lambdas, without any change, since they all match the signature required by callback.

image

In summary

Despite the awkward syntax lambdas are not a mechanism to be feared or despised. They merely provides a useful way of simplify code and reducing programmer effort. In essence they are no more than syntactic sugar. In this respect they are no more detrimental than operator overloading.

 

For more information on other aspects of C++11, have a look at our  training course AC++11-401 – Transitioning to C++11.

For more on C++ programming – particularly for embedded and real-time applications – visit the Feabhas website. You may find the following of interest:

C++-501 – C++ for Embedded Developers

C++-502 – C++ for Real-Time Developers

AC++-501 – Advanced C++

Glennan Carnie
Dislike (0)
Website | + posts

Glennan is an embedded systems and software engineer with over 20 years experience, mostly in high-integrity systems for the defence and aerospace industry.

He specialises in C++, UML, software modelling, Systems Engineering and process development.

About Glennan Carnie

Glennan is an embedded systems and software engineer with over 20 years experience, mostly in high-integrity systems for the defence and aerospace industry. He specialises in C++, UML, software modelling, Systems Engineering and process development.
This entry was posted in C/C++ Programming and tagged , , , , , , . Bookmark the permalink.

39 Responses to Demystifying C++ lambdas

  1. Dan says:

    This is one of the best write-ups on lambdas I've seen, thank you.

    That said, and I know this is extremely subjective, I'm not the inline lambda syntax improves readability (e.g. for_each). It's completely possible that it's just so new that my brain is still fighting it, I'll concede that. But I could quickly see a lambda becoming more complex, and soon the for_each looks like someone sneezed out a bunch of code (I realize lambdas should only be used for small snippets, but we all know how well code ages as new & different developers come on board).

    Anyway, thanks again, this one is definitely bookmarked. I haven't found too many of the C++11 features to be compelling for deeply embedded systems, but this post has encouraged me to at least give lambdas another look.

    Like (15)
    Dislike (0)
  2. Antoine says:

    Very nice. Stright to the point, informative and pedagogical.
    May I ask what software you used to produce the nice code snapshots with the comments and all?
    Thanks.

    Like (2)
    Dislike (0)
  3. glennan says:

    Thanks Antoine!
    The code snippets are done with PowerPoint 🙂 They're taking directly from our training course materials.

    Like (0)
    Dislike (0)
  4. Ioannis says:

    Great Work!

    I think you are missing a -> on the last picture

    int main(){
    SimpleCallback callback([]() -> {cout << "Lambda" << endl;});
    callback.execute();
    }

    Other than that, excellent work!

    Like (1)
    Dislike (1)
  5. FipS says:

    Very well written and nicely presented! Thanks for sharing.

    Like (2)
    Dislike (0)
  6. Paul Jurczak says:

    Nice article, thank you. You may want to revisit your Filter example though. std::remove and std::remove_if are the most confusingly named STL algorithms - they don't remove anything, they just rearrange order of elements in the container and create a bunch of undefined values. You must follow them with erase, in order to achieve desired result.

    Like (0)
    Dislike (0)
  7. Henri Tuhola says:

    C++ - lots of carsinogenic abstractions

    Like (0)
    Dislike (2)
  8. Steve Little says:

    "It is relatively easy to see now why capturing the context has potential overheads: for every object captured by value a copy of the original is made; for every object captured by reference a reference is stored. (Therefore you might want to think twice about capturing everything in the context by value (using [=]))"

    I'm not sure that's correct. The C++ language standard section [expr.prim.lambda] says:

    "If a lambda-expression has an associated capture-default and its compound-statement odr-uses (3.2) this or a variable with automatic storage duration and the odr-used entity is not explicitly captured, then the odr-used entity is said to be implicitly captured; such entities shall be declared within the reaching scope of
    the lambda expression."

    Based on my reading of that, a compiler is *not required* to capture all objects (when using a capture-default) just all objects that are ODR-used in the body of the lambda-expression.

    Some implementations *may* choose to copy everything, but that would be the fault of an inefficient implementation, and not necessarily a good reason to avoid using a capture-default.

    Like (1)
    Dislike (0)
  9. @Ioannis: No, the author has it correct. Lambda declarations only require -> when the return type cannot be automatically deduced. In the case of no return statement it is trivially "void".

    Like (2)
    Dislike (0)
  10. KrzaQ says:

    I liked the article, but I have one nit-pick: unless your lambda is mutable, its `operator()` should take `const this`.

    Like (1)
    Dislike (0)
  11. Andy Prowl says:

    Nice article 🙂 One minor thing: the lambda's call operator generated by the compiler is const-qualified, unless "mutable" is used.

    Like (1)
    Dislike (0)
  12. glennan says:

    Well spotted, Andy.

    I've updated the article to reflect this.

    Like (1)
    Dislike (0)
  13. glennan says:

    Very good point Steve.
    I've updated the article to reflect this.

    Thanks.

    Like (1)
    Dislike (0)
  14. Marcel Wid says:

    Your code examples in section Under the hood (again) are wrong:

    1. "It is unspecified whether additional unnamed non-static data members are declared in the closure type for entities captured by reference." (§5.1.2p15)

    2. This is a serious mistake! The closure type (your class _SomeCompilerGeneratedName_) is local to main, i.e. it is defined inside main! And you don't pass a pointer to for_each. The object _inst001_ is never created as lvalue. The correct compiler generated code is "for_each(v.begin(), v.end(), _SomeCompilerGeneratedName_(offset));"

    3. To be even more precise, the closure type has a deleted default constructor.

    Like (0)
    Dislike (0)
  15. glennan says:

    Marcel,

    Good points, there.

    I don't disagree with you. The idea of the 'conceptual equivalence' example was to echo the earlier functor example, to help illustrate to the reader approximately what was happening. It was never intended to be a literal 'what the compiler generates'.

    What you've written is, of course, quite correct. Apologies for the confusion.

    Like (1)
    Dislike (0)
  16. Marcel Wid says:

    Well, I see.

    But it is even conceptually wrong to pass a pointer to a functor (&_inst001_) to for_each. If you want the analogy to functors then you should pass (the lvalue) _inst001_ itself as you did in the functor example where you pass f and not &f.

    Like (1)
    Dislike (0)
  17. glennan says:

    Quite right; and updated. 🙂

    Like (1)
    Dislike (0)
  18. panovr says:

    "This means I can replace my manually-created functor in the STL algorithm with a lambda:"

    auto lambda = [](X& elem) -> void { elem.op(); }

    should be:

    auto lambda = [](X& elem) -> void { elem.op(); };

    Like (1)
    Dislike (0)
  19. glennan says:

    It is now! 🙂

    Thank you!

    Like (0)
    Dislike (0)
  20. panovr says:

    You wrote: "Callable object is a generic name for any object that can be called like a function:
    1. A member function (pointer)
    2. A free function (pointer)
    3. functor
    4. lambda"

    And there are three examples using std::function with functor, free function and lambda.

    How about a class member function? Can it be used with std::function?

    Like (0)
    Dislike (0)
  21. Andrei Zissu says:

    panovr,

    You can bind a member function indirectly, as you also have to bind to a specific object instance (unless it's a static member function, which is just like any other non-member function). You can accomplish that by either providing a lambda which itself makes the actual call, or by using std::bind.

    Like (0)
    Dislike (0)
  22. Pingback: Using C++11 Lambdas with ELLCC | The ELLCC Embedded Compiler Collection

  23. dorodnic says:

    Hi,

    Great article, thank you!

    I was curious how the following code works under the hood:
    function createFunc(int x)
    {
    return [=]() { return x; };
    }
    In particular, where the generated object is stored?

    Like (0)
    Dislike (0)
  24. JayArby says:

    Thoroughly demystifying. Thank you!

    Like (0)
    Dislike (0)
  25. Andreas says:

    would you mind telling how you created your code images? I really like the the pseudo hand-writting.

    Like (0)
    Dislike (0)
  26. All the code snippets are created in PowerPoint (yup, really!).
    The font used for the code is Consolas
    The hand-written font is Segoe Print.

    Like (1)
    Dislike (0)
  27. shashank says:

    one of the best writeups on lambdas I ve found! really helpful , thanks!

    Like (1)
    Dislike (0)
  28. Sharath says:

    This is exactly the detailed documentation I was looking for . Kudos for writing up this !

    Like (0)
    Dislike (0)
  29. Really glad it was useful.

    I'll be delivering this material as a webinar next month, so please sign up! 🙂

    Like (0)
    Dislike (0)
  30. Enrico says:

    A question about "Lambdas within member functions": what if I need to capture "this" in a static member function?
    Use case is I have a library which exposes a setCallback() and so the function I define need to be static. However, what if I want to register a member function instead of a "classic" function? Examples around the web are not enlightening... 🙂

    Like (0)
    Dislike (0)
  31. Pingback: [Перевод] Лямбды: от C++11 до C++20. Часть 1 – CHEPA website

  32. Chris says:

    Thank you for this clearly-written and helpful article. Whilst reading it, I almost started to like lambdas 🙂

    Unfortunately, the fact that C++ lambda syntax conflicts with C syntax for designated initializers (well-established since 1999) is reason enough for me to hate lambdas. Designated initializers are incredibly useful for declaring look-up tables as static const arrays.

    Making such an incompatible change to C++ seems either careless or malicious, given that there are so many other ways that lambdas could have been specified. Either way, it is a pain for anyone maintaining mixed C/C++ codebases, and it can never be undone.

    Like (0)
    Dislike (0)
  33. Andrei Zissu says:

    I don't understand how the two might conflict, since:
    1.lambdas require at the very least an empty capture list [] which should disambiguate them from designated initializers.
    2.either feature won't compile in the other language, so #ifdef CPP would be required anyway.
    What am I missing?

    Like (1)
    Dislike (0)
  34. Dan says:

    Regarding lambdas and designated initializers, I haven't looked at it closely, but maybe what Chris was talking about was an array designator inside an initializer list, because that would require the square brackets. I found this PDF (go to last page for this specific issue)

    https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0329r4.pdf

    Note that now starting with C++20, designated initializers are supported. But they aren't quite supported with the same rules as C.

    Changing topics a little: I was OK with many of the changes in C++11, I felt that they increased the language's size and the "cognitive load" required to work on a large code base (in spite of many pleading to the contrary), but I felt on the whole, it was a step forward. But with C++14, then 17, now 20 and 23 coming very soon, I'm just wondering for whom all these changes are being made.

    Something like concepts, OK fine, that is probably long overdue, and maybe modules... but many of us have been building large systems without these for years and getting by. And now when you introduce these new features and 95% of the team doesn't understand them, it almost seems like some of the people on the standards committee are in an echo chamber.

    I am a sample size of 1, I have no illusions otherwise. but as a consultant I work with a lot of companies, and I see more groups becoming skeptical of C++ and its expansion, and taking a serious look at Rust.

    Sorry for getting a little off-topic. It's just that most of my colleagues scratch their head when they see each new version of C++ and its changes, but if you watch any C++ conference videos, with all due respect, they seem to largely of the nature, "Hey look at this cool esoteric not-very-useful thing I can do with this new C++ feature that I pushed through in the last standards committee meeting." (Examples from C++20: "default constructible and assignable stateless lambdas" -- huh? "allow pack expansions in lambda init-capture" -- what was that again?)

    Peace out.

    Like (3)
    Dislike (0)
  35. Andrei Zissu says:

    Dan, I'm just wondering, for every person you hear complaining that C++ is too bloated, how many others (or perhaps even the same ones...) are crying out for some missing feature?

    Like (0)
    Dislike (0)
  36. Dan says:

    Hi Andrei -- you make a good point, and sometimes I can imagine they are the same person.

    I don't claim my own experience or colleagues are representative of the larger C++ community, but virtually all of the experienced developers I work with are asking / seeking / hoping / crying out for ** zero new features ** .

    I work entirely on deeply embedded systems, maybe if I did more embedded Linux or desktop/server programming I would have a different perspective. For example, C++ now has std::thread, but I'm normally running an RTOS and use the RTOS threading model, so "hooray" for the committee adding std::thread, but it doesn't help me. But now when I'm helping a company and I see an engineer use the "shiny new hammer" (e.g., std::thread in an RTOS system), I'm just at a loss for words. Same thing for adding graphics, networking, etc. into the language (seems to be trying to keep up with Python?) -- none of my colleagues are asking for that.

    Like (0)
    Dislike (0)
  37. Andrei Zissu says:

    Dan, none of your colleagues may be asking for that, but have you considered some of the over 5 million C++ devs actually might?

    Like (0)
    Dislike (0)
  38. Dan says:

    Andrei, did you read my comment? (Not trying to be snarky). "I work entirely on deeply embedded systems, maybe if I did more embedded Linux or desktop/server programming I would have a different perspective". I suspect most people don't fall into the same category as me. Probably 95%. Most of the safety-critical projects I'm working on are on C++11 or even C++03.

    Out of curiosity, what features do you think are missing in C++? What types of systems are you working on?

    Like (0)
    Dislike (0)
  39. Andrei Zissu says:

    Dan, obviously you have a vested interest stemming from your specific needs, don't we all? My point is it makes no sense IMHO to complain against features that are of no use to you but may definitely be useful to others - unless they are utterly detrimental to your needs. It makes much more sense to suggest missing features or modifications/additions required in existing ones. My apologies in case I've misunderstood your intention on the first part.

    I've worked on lots of different systems in various industries, though I've been staying clear of hardcore embedded ones for over a decade. Currently I work at a medical startup.

    Probably the thing I most want to see finally make its appearance is reflection, including code generation. Coroutine library support will also be welcome, hopefully as soon as C++ 23. And then there's pattern matching, also most welcome. There are lots of other prospects too, which I'll be slowly digging into and making my mind about now that I've finally reached WG21 membership status (as of one month ago).

    Like (0)
    Dislike (0)

Leave a Reply