Lesson 13 - Magic Methods in Python - Collections and Descriptors

In the previous lesson, Magic Methods in Python - Math methods, we were dealing with mathematical functions and operators. In this Python tutorial, we're going to look at the magic collection methods and their abstract base classes.

Magic methods

__len__(self)

Returns the number of items in the collection.

__getitem__(self, key)

Returns an item from the collection by a specified key, this syntax applies to classes similar to dictionaries. If it fails, it should raise KeyError.

__missing__(self, key)

The method is called by the __getitem__() method if the given key is missing in the dictionary subclass.

__getitem__(self, index)

Returns an item from the collection at a given position. In case of failure, it should raise IndexError (in the for loop, IndexError indicates the end of the sequence).

__setitem__(self, key, value)

Sets the value of an item specified by a given key in the collection, this syntax applies to classes similar to dictionaries. If it fails, he should raise KeyError.

__delitem__(self, key)

Deletes an entry with a given key from the collection, this syntax applies to classes similar to dictionaries. If it fails, he should raise KeyError.

__iter__(self)

Returns an iterator for the collection. We can create an iterator using the iter() function from an object that has a magical __iter__() or __getitem__() method.

__reversed__(self)

Returns an inverted iterator for the collection, we should re-implement this method if there's a faster implementation than:

for i in reversed(range(len(self))):
    yield self[i]

Abstract base classes

Abstract base classes provide us with a list of interfaces that we should implement. Interfaces are all public methods, including magic methods. If we implement the required minimum, other methods are derived from them.

Class Inherits from Abstract methods Derived methods
Container   __contains__  
Hashable   __hash__  
Iterable   __iter__  
Iterator Iterable __next__ __iter__
Sized   __len__  
Callable   __call__  
Sequence Sized, Iterable, Container __getitem__, __len__ __contains__, __iter__, __reversed__, index, and count
MutableSequence Sequence __getitem__, __setitem__, __delitem__, __len__, insert Sequence methods + append, reverse, extend, pop, remove and __iadd__
Set Sized, Iterable, Container __contains__, __iter__, __len__ __le__, __lt__, __eq__, __ne__, __gt__, __ge__, __and__, __or__, __sub__, __xor__, and isdisjoint
MutableSet Set __contains__, __iter__, __len__, add, discard The Set method + clear, pop, remove, __ior__, __iand__, __ixor__, and __isub__
Mapping Sized, Iterable, Container __getitem__, __iter__, __len__ __contains__, keys, items, values, get, __eq__ a __ne__
MutableMapping Mapping __getitem__, __setitem__, __delitem__, __iter__, __len__ The methods Mapping + pop, popitem, clear, update, and setdefault
MappingView Sized   __len__
ItemsView MappingView, Set   __contains__, __iter__
KeysView MappingView, Set   __contains__, __iter__
ValueView MappingView   __contains__, __iter__

Attribute methods

__getattr__(self, name)

This method is called if the attribute isn't found using the usual approach. The method should return the attribute's value or invoke AttributeError.

__setattr__(self, name, value)

It's called when setting attributes instead of assigning variables to __dict__. If the class wants to save the attribute, it should call the __setattr__() superclass method.

__delattr__(self, name)

We should only implement this method if:

del obj.name

makes some meaningful sense for the class.

An example:

class Foo:

    def __setattr__(self, name, value):
        print("Setting")
        super().__setattr__(name, value)

    def __getattr__(self, name):
        print("Getting")
        return 1

foo = Foo()
print(foo.x)
print(foo.__dict__)
foo.x = 2
print(foo.x)
print(foo.__dict__)

Descriptors

Descriptors allow us to override attribute access using another class. We can avoid redundant properties like this. The class must define at least one of the __get__(), __set__(), or __delete__() methods. With the descriptor, we can improve the validation of attribute values. We'll create a sample Typed descriptor that will ensure that the attribute is of a certain type:

from weakref import WeakKeyDictionary

class Typed:

    dictionary = WeakKeyDictionary()

    def __init__(self, ty=int):
        self.ty = ty

    def __get__(self, instance, owner=None):
        print("Getting instance")
        return self.dictionary[instance]

    def __set__(self, instance, value):
        print("Setting instance", instance, "to", value)
        if not isinstance(value, self.ty):
            raise TypeError("Value must be type {}".format(self.ty))
        self.dictionary[instance] = value

__get__(self, instance, owner)

The method accepts an instance and a class as parameters. The instance's class can be obtained as follows:

owner = type(instance)

In the code, we return the property value stored in the descriptor.

__set__(self, instance, value)

The method accepts an instance as a parameter and the value to which the attribute should be set. When setting, we check if the attribute is of the given type. It's normally set to int (integer). If not, we raise the TypeError exception.

Creating a class

Now we'll create a class that will represent a human. We set him three attributes: name, age, and weight. The name will have to be str (string), age of the int (integer) type, and weight will be float (decimal number):

class Person:

    name = Typed(ty = str)
    age = Typed()
    weight = Typed(ty = float)

    def __init__(self, name, age, weight):
        self.name = name
        self.age = age
        self.weight = weight

We create the descriptors in the way that each property name will be a descriptor. We create descriptors at the class level. Each descriptor then contains a dictionary with instances and property values.

When calling attributes, Python recognizes that they are descriptors and calls the __get__(), __set__(), or __delete__() methods.

Of course, we can also create subclasses without redefining the properties.

For example, a subclass with an additional iq property:

class CleverPerson(Person):

    iq = Typed(ty = int)

    def __init__(self, name, age, weight, iq):
        super().__init__(name, age, weight)
        self.iq = iq

Of course, we can create subclasses of descriptors as well:

class Ranged(Typed):

    def __init__(self, ty=int, min=0, max=100):
        super().__init__(ty)
        self.min = min
        self.max = max

    def __set__(self, instance, value):
        if value < self.min:
            raise TypeError("Value must be greater than {}".format(self.min)
        elif value > self.max:
            raise TypeError("Value must be lower than {}".format(self.max)
        super().__set__(instance, value)

Weak-reference dictionaries

We use a weak reference dictionary (WeakKeyDictionary) from the weakref module to store the attributes. It works like a normal dictionary but differs in one detail. If there are no more references to an object in the dictionary than the one in the dictionary, the object is deleted from the dictionary. This will prevent memory leaks. An ordinary dictionary would prevent instances from being deleted even if they were no longer used, because it would be the dictionary using them. If we use a __slots__ class, we need to add weakref to __slots__ to support weak links.

Getting the descriptor

If we want to modify the descriptor, we need to access the object class and slightly modify the __get__() method. For example:

from weakref import WeakKeyDictionary

class Typed:

    dictionary = WeakKeyDictionary()

    def __init__(self, ty=int):
        self.ty = ty

    def __get__(self, instance, owner=None):
        if instance is None: # we access through the class
            return self
        print("Getting instance")
        return self.dictionary[instance]

    ...


CleverPerson.name # gets the descriptor

This way we can create and call our own methods on the descriptor.


 

Download

Downloaded 0x (2.46 kB)
Application includes source codes in language Python

 

 

Article has been written for you by gcx11
Avatar
Do you like this article?
No one has rated this quite yet, be the first one!
Previous article
Magic Methods in Python - Math methods
All articles in this section
Object-Oriented Programming in Python
Activities (2)

 

 

Comments

To maintain the quality of discussion, we only allow registered members to comment. Sign in. If you're new, Sign up, it's free.

No one has commented yet - be the first!