Problem
What is the purpose of __slots__ in Python, particularly in terms of when I should use it and when I shouldn’t?
Asked by Jeb
Solution #1
The special attribute __slots__ lets you specify which instance properties your object instances should have, with the following intended results:
The space savings are due to
One caveat: in an inheritance tree, you should only declare a certain position once. Consider the following scenario:
class Base:
__slots__ = 'foo', 'bar'
class Right(Base):
__slots__ = 'baz',
class Wrong(Base):
__slots__ = 'foo', 'bar', 'baz' # redundant foo and bar
When you get this wrong, Python doesn’t complain (though it definitely should); problems may not appear, but your objects will take up more space than they should. Python 3.8 is a new version of Python.
>>> from sys import getsizeof
>>> getsizeof(Right()), getsizeof(Wrong())
(56, 72)
This is because the Base’s slot descriptor has a slot separate from the Wrong’s. This shouldn’t happen very often, but it could:
>>> w = Wrong()
>>> w.foo = 'foo'
>>> Base.foo.__get__(w)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: foo
>>> Wrong.foo.__get__(w)
'foo'
The most important caveat is that several “parent classes with nonempty slots” cannot be mixed.
Follow these best practices to work around this restriction: Remove all but one or all of your parents’ abstractions, which their concrete classes and your new concrete class will inherit from collectively, leaving the abstraction(s) empty (just like abstract base classes in the standard library).
For an example, see the section on multiple inheritance below.
If you want to keep reading, there are a lot more details.
Guido van Rossum, the author of Python, claims that he invented __slots__ to speed up attribute access.
It is simple to show that faster access is measurably significant:
import timeit
class Foo(object): __slots__ = 'foo',
class Bar(object): pass
slotted = Foo()
not_slotted = Bar()
def get_set_delete_fn(obj):
def get_set_delete():
obj.foo = 'foo'
obj.foo
del obj.foo
return get_set_delete
and
>>> min(timeit.repeat(get_set_delete_fn(slotted)))
0.2846834529991611
>>> min(timeit.repeat(get_set_delete_fn(not_slotted)))
0.3664822799983085
In Python 3.5 on Ubuntu, slotted access is about 30 percent faster.
>>> 0.3664822799983085 / 0.2846834529991611
1.2873325658284342
I measured it to be roughly 15% faster in Python 2 on Windows.
Another goal of __slots__ is to reduce the amount of memory used by each object instance.
The reasons for this are clearly stated in my contribution to the documentation:
__slots__ is credited with a lot of memory savings in SQLAlchemy.
To test this, I used the Anaconda Python 2.7 distribution on Ubuntu Linux with guppy.hpy (aka heapy) and sys.getsizeof to find that the size of a class instance without __slots__ specified is 64 bytes. The __dict__ is not included in this. Python, thank you for allowing lazy evaluation. Again, it appears that the __dict__ is not created until it is accessed, but classes without data are usually meaningless. The __dict__ attribute is a minimum of 280 bytes when it is called into existence.
A class instance with __slots__ stated as () (no data) is just 16 bytes long, and 56 bytes long with one item in slots, and 64 bytes long with two.
I show the memory consumption in bytes for __slots__ and __dict__ (no slots defined) in Python 2.7 and 3.6 for each point where the dict increases in 3.6 (except for 0, 1, and 2 attributes):
Python 2.7 Python 3.6
attrs __slots__ __dict__* __slots__ __dict__* | *(no slots defined)
none 16 56 + 272† 16 56 + 112† | †if __dict__ referenced
one 48 56 + 272 48 56 + 112
two 56 56 + 272 56 56 + 112
six 88 56 + 1040 88 56 + 152
11 128 56 + 1040 128 56 + 240
22 216 56 + 3344 216 56 + 408
43 384 56 + 3344 384 56 + 752
Despite the smaller dicts in Python 3, we can see how well __slots__ scale for instances to conserve memory, which is one of the main reasons why should use __slots__.
Just for completeness of my notes, note that there is a one-time cost per slot in the class’s namespace of 64 bytes in Python 2, and 72 bytes in Python 3, because slots use data descriptors like properties, called “members”.
>>> Foo.foo
<member 'foo' of 'Foo' objects>
>>> type(Foo.foo)
<class 'member_descriptor'>
>>> getsizeof(Foo.foo)
72
You must subclass object to prevent the formation of a __dict__. In Python 3, everything is an object subclass, whereas in Python 2, you had to be explicit:
class Base(object):
__slots__ = ()
now:
>>> b = Base()
>>> b.a = 'a'
Traceback (most recent call last):
File "<pyshell#38>", line 1, in <module>
b.a = 'a'
AttributeError: 'Base' object has no attribute 'a'
Alternatively, subclass a class that defines __slots__.
class Child(Base):
__slots__ = ('a',)
and now:
c = Child()
c.a = 'a'
but:
>>> c.b = 'b'
Traceback (most recent call last):
File "<pyshell#42>", line 1, in <module>
c.b = 'b'
AttributeError: 'Child' object has no attribute 'b'
Simply add ‘__dict__’ to the __slots__ (notice that slots are ordered, and you shouldn’t repeat slots that are already in parent classes) to allow __dict__ generation when subclassing slotted objects:
class SlottedWithDict(Child):
__slots__ = ('__dict__', 'b')
swd = SlottedWithDict()
swd.a = 'a'
swd.b = 'b'
swd.c = 'c'
and
>>> swd.__dict__
{'c': 'c'}
Alternatively, you don’t need to specify __slots__ in your subclass; you can still use slots from the parents, but the creation of a __dict__ is unrestricted:
class NoSlots(Child): pass
ns = NoSlots()
ns.a = 'a'
ns.b = 'b'
And:
>>> ns.__dict__
{'b': 'b'}
However, __slots__ may cause problems for multiple inheritance:
class BaseA(object):
__slots__ = ('a',)
class BaseB(object):
__slots__ = ('b',)
Because it is impossible to create a kid class from parents with both non-empty slots:
>>> class Child(BaseA, BaseB): __slots__ = ()
Traceback (most recent call last):
File "<pyshell#68>", line 1, in <module>
class Child(BaseA, BaseB): __slots__ = ()
TypeError: Error when calling the metaclass bases
multiple bases have instance lay-out conflict
If you run into this issue, you can either delete __slots__ from the parents, or give them empty slots if you have authority over the parents, or restructure to abstractions:
from abc import ABC
class AbstractA(ABC):
__slots__ = ()
class BaseA(AbstractA):
__slots__ = ('a',)
class AbstractB(ABC):
__slots__ = ()
class BaseB(AbstractB):
__slots__ = ('b',)
class Child(AbstractA, AbstractB):
__slots__ = ('a', 'b')
c = Child() # no problem!
class Foo(object):
__slots__ = 'bar', 'baz', '__dict__'
and now:
>>> foo = Foo()
>>> foo.boink = 'boink'
So, with ‘__dict__’ in slots, we sacrifice some of the size advantages in exchange for dynamic assignment and slots for the names we do expect.
When you inherit from a non-slotted object, you get the same semantics as when you use __slots__: names in __slots__ point to slotted values, while all other values are stored in the instance’s __dict__.
Avoiding __slots__ because you want to be able to add attributes on the fly isn’t an acceptable justification; if you need this, simply add “__dict__” to your __slots__.
If you need the feature, you can explicitly add __weakref__ to __slots__.
The namedtuple builtin creates immutable instances that are very lightweight (basically, the size of tuples), but if you subclass them, you must do it yourself to reap the benefits:
from collections import namedtuple
class MyNT(namedtuple('MyNT', 'bar baz')):
"""MyNT is an immutable and lightweight object"""
__slots__ = ()
usage:
>>> nt = MyNT('bar', 'baz')
>>> nt.bar
'bar'
>>> nt.baz
'baz'
We have stopped the formation of __dict__: Assigning an unexpected attribute causes an AttributeError.
>>> nt.quux = 'quux'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'MyNT' object has no attribute 'quux'
By omitting __slots__ = (, you can allow __dict__ construction, but you can’t use non-empty __slots__ with tuple subtypes.
Even if many parents have non-empty slots that are the same, they cannot be utilized together:
class Foo(object):
__slots__ = 'foo', 'bar'
class Bar(object):
__slots__ = 'foo', 'bar' # alas, would work if empty, i.e. ()
>>> class Baz(Foo, Bar): pass
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Error when calling the metaclass bases
multiple bases have instance lay-out conflict
Using an empty __slots__ in the parent appears to provide the most flexibility, letting the child to choose whether to avoid or enable the construction of a __dict__: by adding ‘__dict__’ to get dynamic assignment, see section above.
class Foo(object): __slots__ = ()
class Bar(object): __slots__ = ()
class Baz(Foo, Bar): __slots__ = ('foo', 'bar')
b = Baz()
b.foo, b.bar = 'foo', 'bar'
You don’t have to have slots, so adding them and then removing them shouldn’t cause any issues.
In terms of flexibility for subclassers, an empty __slots__ in those parents seems to be the best way to go if you’re creating mixins or utilizing abstract base classes that aren’t supposed to be instantiated.
To demonstrate, let’s develop a class with code that we’d like to use in a multiple inheritance situation.
class AbstractBase:
__slots__ = ()
def __init__(self, a, b):
self.a = a
self.b = b
def __repr__(self):
return f'{type(self).__name__}({repr(self.a)}, {repr(self.b)})'
By inheriting and stating the expected slots, we might utilize the above directly:
class Foo(AbstractBase):
__slots__ = 'a', 'b'
But we don’t care about it; single inheritance is straightforward; we need another class from which we may inherit, perhaps with a noisy attribute:
class AbstractBaseC:
__slots__ = ()
@property
def c(self):
print('getting c!')
return self._c
@c.setter
def c(self, arg):
print('setting c!')
self._c = arg
We couldn’t do the following if both bases had non-empty slots. (In reality, we could have provided AbstractBase nonempty slots a and b and left them out of the declaration below if we chose; leaving them in would be incorrect):
class Concretion(AbstractBase, AbstractBaseC):
__slots__ = 'a b _c'.split()
We may still reject __dict__ and __weakref__ instantiation because we have functionality from both via multiple inheritance:
>>> c = Concretion('a', 'b')
>>> c.c = c
setting c!
>>> c.c
getting c!
Concretion('a', 'b')
>>> c.d = 'd'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'Concretion' object has no attribute 'd'
You might be able to eke out more caveats from the remainder of the __slots__ documentation (the 3.7 dev docs are the most recent), to which I’ve contributed significantly recently.
The current top answers cite outdated information and are quite hand-wavy and miss the mark in some important ways.
I quote:
The collections module’s Abstract Base Classes, for example, are not instantiated, but __slots__ are designated for them.
Why?
If a user wishes to deny __dict__ or __weakref__ creation, those things must not be available in the parent classes.
When building interfaces or mixins, __slots__ helps with reusability.
Although many Python programmers do not write for reusability, having the option to prevent wasteful space usage is useful.
When pickingling a slotted object, you may notice that it complains with a TypeError:
>>> pickle.loads(pickle.dumps(f))
TypeError: a class that defines __slots__ without defining __getstate__ cannot be pickled
This is actually not the case. This message is generated by the default protocol, which is the oldest. The -1 option can be used to select the most recent protocol. This is 2 in Python 2.7 (which was introduced in 2.3), and 4 in Python 3.6.
>>> pickle.loads(pickle.dumps(f, -1))
<__main__.Foo object at 0x1129C770>
in Python 2.7:
>>> pickle.loads(pickle.dumps(f, 2))
<__main__.Foo object at 0x1129C770>
in Python 3.6
>>> pickle.loads(pickle.dumps(f, 4))
<__main__.Foo object at 0x1129C770>
As it is a solved problem, I would keep this in mind.
The first paragraph is split into two parts: a brief explanation and a prediction. This is the only part of the document that actually answers the question.
The second half is wishful thinking, and off the mark:
Python achieves something similar by just creating the __dict__ when it is needed, however creating a large number of objects with no data is absurd.
The second paragraph exaggerates and overlooks the real reasons to avoid __slots__. The following is not a valid reason to avoid slots (read the rest of my answer above for valid reasons):
It then goes on to talk about various ways to achieve that absurd aim with Python, avoiding any mention of __slots__.
More wishful thinking in the third paragraph. It’s generally off-the-mark information that the answerer didn’t even write, and it adds to the site’s detractors’ ammunition.
Make some regular and slotted objects as follows:
>>> class Foo(object): pass
>>> class Bar(object): __slots__ = ()
Make a million of them at once:
>>> foos = [Foo() for f in xrange(1000000)]
>>> bars = [Bar() for b in xrange(1000000)]
Inspect with guppy.hpy().heap():
>>> guppy.hpy().heap()
Partition of a set of 2028259 objects. Total size = 99763360 bytes.
Index Count % Size % Cumulative % Kind (class / dict of class)
0 1000000 49 64000000 64 64000000 64 __main__.Foo
1 169 0 16281480 16 80281480 80 list
2 1000000 49 16000000 16 96281480 97 __main__.Bar
3 12284 1 987472 1 97268952 97 str
...
Examine the regular objects and their __dict__ once more:
>>> for f in foos:
... f.__dict__
>>> guppy.hpy().heap()
Partition of a set of 3028258 objects. Total size = 379763480 bytes.
Index Count % Size % Cumulative % Kind (class / dict of class)
0 1000000 33 280000000 74 280000000 74 dict of __main__.Foo
1 1000000 33 64000000 17 344000000 91 __main__.Foo
2 169 0 16281480 4 360281480 95 list
3 1000000 33 16000000 4 376281480 99 __main__.Bar
4 12284 0 987472 0 377268952 99 str
...
This is in line with Python’s heritage, which dates back to Python 2.2’s Unifying types and classes.
Answered by Aaron Hall
Solution #2
Quoting Jacob Hallen:
Answered by Jeff Bauer
Solution #3
If you’re planning to create a large number of objects (hundreds or thousands), you’ll want to use __slots__. Only the memory optimization tool __slots__ exists.
Using __slots__ to constrain attribute creation is strongly discouraged.
Pickling objects with __slots__ will not operate with the default (oldest) pickle protocol; a later version must be specified.
Python’s introspection features may be harmed in other ways as well.
Answered by Ryan
Solution #4
Each Python object has a __dict__ attribute, which is a dictionary that contains all of the other attributes. When you type self, for example. Python’s self. dict [‘attr’] is actually self. dict [‘attr’]. As you might expect, utilizing a dictionary to hold attributes takes up more space and time to access.
When you use __slots__, however, any object produced for that class will be missing the __dict__ property. Instead, pointers are used to access all attributes.
If you desire a C-style structure rather than a full-fledged class, you can utilize __slots__ to reduce attribute access time and condense the size of the objects. A Point class with the attributes x and y is an excellent example. If you’re going to have a lot of points, you might want to use __slots__ to save memory.
Answered by Suraj
Solution #5
Here’s an example of how to use __slots__ in addition to the other answers:
>>> class Test(object): #Must be new-style class!
... __slots__ = ['x', 'y']
...
>>> pt = Test()
>>> dir(pt)
['__class__', '__delattr__', '__doc__', '__getattribute__', '__hash__',
'__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__',
'__repr__', '__setattr__', '__slots__', '__str__', 'x', 'y']
>>> pt.x
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: x
>>> pt.x = 1
>>> pt.x
1
>>> pt.z = 2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'Test' object has no attribute 'z'
>>> pt.__dict__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'Test' object has no attribute '__dict__'
>>> pt.__slots__
['x', 'y']
So, all it takes to add __slots__ is an extra line (and, if your class isn’t already, making it a new-style class). This allows you to minimize the memory footprint of those classes by a factor of five, at the cost of needing to implement bespoke pickle logic if and when it is required.
Answered by Evgeni Sergeev
Post is based on https://stackoverflow.com/questions/472000/usage-of-slots