Coder Perfect

What do lambda function closures capture?


Recently I started playing around with Python and I came around something peculiar in the way closures work. Consider the following code:

adders=[None, None, None, None]

for i in [0,1,2,3]:
   adders[i]=lambda a: i+a

print adders[1](3)

It creates a basic array of functions that take a single input and adds a number to it. The methods are built in a for loop with an iterator I that ranges from 0 to 3. A lambda function is generated for each of these integers, which captures I and adds it to the function’s input. The final line uses 3 as an argument to call the second lambda function. Surprisingly, the output was 6.

I was hoping for a 4. My logic was that everything in Python is an object, thus every variable is just a pointer to it. I expected the lambda closures for I to store a pointer to the integer object currently pointed to by I when I created them. That is, when I attach a new integer object, the previously constructed closures should not be affected. Unfortunately, examining the adders array in a debugger reveals that it does. Adders[1](3) returns 6 because all lambda functions relate to the latest value of I 3.

Which leads me to consider the following:

Asked by Boaz

Solution #1

You can use an argument with a default value to force the capture of a variable:

>>> for i in [0,1,2,3]:
...    adders[i]=lambda a,i=i: i+a  # note the dummy parameter with a default value
>>> print( adders[1](3) )

the idea is to declare a parameter (cleverly named i) and give it a default value of the variable you want to capture (the value of i)

Answered by Adrien Plisson

Solution #2

Your second question has already been answered, but here’s what you need to know about your first:

Python’s scoping is both dynamic and lexical. The name and scope of the variable will always be remembered by a closure, not the object it’s pointing to. Because all of the functions in your example were written in the same scope and used the same variable name, they all refer to the same thing.

In response to your second question, there are two approaches that come to mind:

The scope is constructed by sending the value you want to bind as an argument to a new function (a lambda, for brevity), which binds its argument. In practical code, though, you’ll almost always use an ordinary function to establish the new scope instead of the lambda:

def createAdder(x):
    return lambda y: y + x
adders = [createAdder(i) for i in range(4)]

Answered by Max Shawabkeh

Solution #3

To solve your second point completely, you might use partial in the functools module.

With Chris Lutz’s suggestion of importing add from operator, the example becomes:

from functools import partial
from operator import add   # add(a, b) -- Same as a + b.

adders = [0,1,2,3]
for i in [0,1,2,3]:
    # store callable object with first argument given as (current) i
    adders[i] = partial(add, i) 

print adders[1](3)

Answered by Joma

Solution #4

Consider the following:

x = "foo"

def print_x():
    print x

x = "bar"

print_x() # Outputs "bar"

Most people, I believe, will not be perplexed by this. It’s just what you’d anticipate.

So, why do people think it would be different when it is done in a loop? I know I did that mistake myself, but I don’t know why. It is the loop? Or perhaps the lambda?

After all, the loop is nothing more than a condensed form of:

adders= [0,1,2,3]
i = 0
adders[i] = lambda a: i+a
i = 1
adders[i] = lambda a: i+a
i = 2
adders[i] = lambda a: i+a
i = 3
adders[i] = lambda a: i+a

Answered by mthurlin

Solution #5

Here’s a new example that emphasizes the data structure and contents of a closure to help you understand when the enclosing context is “saved.”

def make_funcs():
    i = 42
    my_str = "hi"

    f_one = lambda: i

    i += 1
    f_two = lambda: i+1

    f_three = lambda: my_str
    return f_one, f_two, f_three

f_1, f_2, f_3 = make_funcs()

What exactly does a closure entail?

>>> print f_1.func_closure, f_1.func_closure[0].cell_contents
(<cell at 0x106a99a28: int object at 0x7fbb20c11170>,) 43 

My str, for example, is not included in f1’s closure.

What’s in the final chapter of f2?

>>> print f_2.func_closure, f_2.func_closure[0].cell_contents
(<cell at 0x106a99a28: int object at 0x7fbb20c11170>,) 43

It’s worth noting that both closures contain the same objects (based on memory addresses). As a result, the lambda function can be thought of as having a reference to the scope. However, my str is absent from the closures for f 1 and f 2, and I is absent from the closure for f 3 (not shown), implying that the closure objects are separate.

Is it possible that the closure objects are the same thing?

>>> print f_1.func_closure is f_2.func_closure

Answered by Jeff

Post is based on