Exploring Iterators in python : iter(object)

After reading this you should be able to take your code from this:

blocks = []
while True:
  block = f.read(32)
  if block == '':
    break
   blocks.append(block)

to this

from functools import partial

blocks = []
for block in iter( partial(f.read, 32), '')):
  blocks.append(block)

enter python iterators

iter(object, sentinel)

The iter() function creates an iterator from an object. Without a second argument, it requires an iterable (supporting __iter__() or __getitem__()).

If a sentinel value is provided, iter() repeatedly calls a given callable until the sentinel value is returned, raising StopIteration when encountered.

By default list, tuples and dictionary are iterable i.e they all has __iter__() method implicitly.

a = [10, 20, 30, 40]

# Convert the list into an iterator
iterator = iter(a)  # same as  a.__iter__()

# Access elements using next() : Both are equaly same
print(next(iterator))   # 10
print(a.__next__())     # 20

More examples

# With a string
s = "Python"
iterator = iter(s)

print(next(iterator))  
print(next(iterator))  


# Even Dictionary
d = {'a': 1, 'b': 2, 'c': 3}
iterator = iter(d)
for key in iterator:
    print(key)  

With ‘sentinel’ value provided you can call any object as long as they meet the condition

# Generate numbers until sentinel value is encountered
import random

# Keep printing random numbers until it encounter '7'
# if it meet '7' it'll stop
iterator = iter(lambda: random.randint(1, 10), 7) 
for num in iterator:
    print(num)

great example showing this:

from functools import partial


# Keep printing file in chunk of 64 byte and if it encounter an empty read, then stop.
with open('mydata.db', 'rb') as f:
    for block in iter(partial(f.read, 64), b''):
        process_block(block)

Building Custom Iterators

Building an iterator from scratch is easy in Python. We just have to implement the __iter__() and the __next__() methods,

• __iter__() returns the iterator object itself. If required, some initialization can be performed.

• __next__() must return the next item in the sequence. On reaching the end, and in subsequent calls, it must raise StopIteration.

class Fibonacci:
    """Class to implement an iterator of Fibonacci numbers"""
    def __init__(self, max):
        self.max = max

    def __iter__(self):
     "" Use this to initialize variables ""

        self.a = 0
        self.b = 1
        self.count = 0  # Keep track of how many numbers we've generated
        return self

    def __next__(self):
        if self.count < self.max:
            result = self.a
            
            # Efficiently calculate the next Fibonacci number
            self.a, self.b = self.b, self.a + self.b  
            self.count += 1
            
            return result
        else:
            # This is where you stop iteration
            raise StopIteration

Demonstrate various ways to use the Fibonacci iterable

1. Explicitly using iter() and next()

   fib_sequence = Fibonacci(10)  # Generate up to 10 Fibonacci numbers
   my_iterator = iter(fib_sequence)
   print("Using next() explicitly:")
   for _ in range(5): # Print first 5
       print(next(my_iterator))

2. Using in a for loop (implicit iteration)

   fib_sequence = Fibonacci(7) # Generate up to 7 Fibonacci numbers
   print("\nUsing in a for loop:")
   for num in fib_sequence:
       print(num)

3. Converting to a list

   fib_sequence = Fibonacci(15)  # Generate up to 15 Fibonacci numbers
   fib_list = list(fib_sequence) # Consumes the iterator and creates a list
   print("\nConverting to a list:")
   print(fib_list)

4. Slicing an iterator (requires itertools)

   import itertools
   
   fib_sequence = Fibonacci(20)
   sliced_fib = itertools.islice(fib_sequence, 5, 10) # Get elements from index 5 up to (but not including) 10
   print("\nSlicing the iterator:")
   for num in sliced_fib:
       print(num)

5. Using with the * operator for unpacking (after converting to a list)

   fib_sequence = Fibonacci(8)
   fib_list = list(fib_sequence)
   print("\nUnpacking with *:")
   print(*fib_list)

Lastly, demonstrating the power of iterables: Memory efficiency

large_fib_sequence = Fibonacci(10**6) # Imagine a very large sequence
# We don't have to store all the numbers in memory at once!

for i, num in enumerate(large_fib_sequence):
    if i < 10:  # Print the first 10
        print(num)
    if i > 1000: # Stop after a while to avoid printing too much
        break
# The large_fib_sequence iterable only generates numbers as needed.