Python iterable protocol

Question

I've made an iterable Squares object in 4 way. The classical class, generator style and just for fun a closure style and an improved(?) closure style.

class

class Squares:
    def __init__(self, n):
        self.n = n

    def __iter__(self):
        return self.SquaresIterator(self.n)

    class SquaresIterator:
        def __init__(self, n):
            self.n = n
            self.i = 0

        def __iter__(self):
            return self

        def __next__(self):
            if self.i >= self.n:
                raise StopIteration

            result = self.i ** 2
            self.i += 1
            return result

It works as expected

sq_class = Squares(5)
list(sq_class)
[0, 1, 4, 9, 16]
# calling again will create a new iterator
list(sq_class)
[0, 1, 4, 9, 16]

generator

def squares(n):
    for i in range(n):
        yield i ** 2

sq_gen = squares(5)
list(sq_gen)
[0, 1, 4, 9, 16]
# calling again will return empty [] is OK, generator exhausted.
list(sq_gen)
[]

closure

def Squares(n):
    def inner():
        return squares_gen(n)

    def squares_gen(n):
        for i in range(n):
            yield i ** 2

    return inner

sq_closure = Squares(5)
list(sq_closure())
[0, 1, 4, 9, 16]
# calling again the inner will create a new generator so it is not exhausted
list(sq_closure())
[0, 1, 4, 9, 16]

improved(?) closure

def Squares(n):
    def inner():
        return squares_gen(n)

    def squares_gen(n):
        for i in range(n):
            yield i ** 2

    inner.__iter__ = lambda : squares_gen(n)
    return inner

sq_closure = Squares(5)
list(sq_closure)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-13-beb02e61ccfb> in <module>
----> 1 for i in sq:
      2     print(i)

TypeError: 'function' object is not iterable

What I wanted to try is attach the __iter__ function to the inner function object and return a generator and I can leave the ()-s and I can use the returned inner as an iterable object. What is the exact definition of the iterable protocol? It seems, only the presence of the __iter__ function is not enough.

Magicmethods are only invoked when they are class attributes - you cannot set them as instance attributes (well you can but they wont be invoked ). — bruno desthuilliers, Dec 18 '18 at 16:11
Possible duplicate: https://stackoverflow.com/questions/38133096/assigning-instead-of-defining-a-getitem-magic-method-breaks-indexing — Patrick Haugh, Dec 18 '18 at 16:13
You missed the simplest way, which is to just use the built-in tools for generating infinite sequences: `squares = map(lambda x: x*x, itertools.count())` or `squares = (x*x for x in itertools.count())`. Use `islice` to take a finite prefix of the sequence: `list(itertools.islice(squares, 5)) == [0,1,4,9,16]` — chepner, Dec 18 '18 at 16:29

score 2 · Accepted Answer · answered Dec 18 '18 at 16:20

The "exact definition of the iterable protocol" is in the documentation:

iterable: An object capable of returning its members one at a time. Examples of iterables include (...) objects of any classes you define with an __iter__() method or with a __getitem__() method that implements Sequence semantics.

BUT: __magicmethods__ are only invoked when defined on the class - you cannot override them on a per-instance basis (well you can but they won't be invoked).

Python iterable protocol

1 Answers1