TITLE OF PAPER: Iterators and Generators URL OF PRESENTATION: _URL_of_powerpoint_presentation_ PRESENTED BY: Thomas Wouters REPRESENTING: _name_of_the_company_they_represent_ CONFERENCE: PyCON 2004 DATE: 20040324 LOCATION: _venue_and_room_in_venue_ -------------------------------------------------------------------------- REAL-TIME NOTES / ANNOTATIONS OF THE PAPER: {If you've contributed, add your name, e-mail & URL at the bottom} Overview iteration Iteration is the act of going over a collection Explicit in the form of 'for' Implicit in many forms: list(), tuple(), dict() map(), reduce(), zip() 'in' in absence of __contains__I 'extended call' syntax: func(*...) but not apply() Uses __getitem__ in Python before 2.2.2 iterators added in python 2.2 protocol of 2 methods (no special class): __iter__(): get iterator because __iter__() returns self you can exchange them next(): get next value raises StopIteration when done Explicit itertor creation with iter() called explicitly by all places where iterators are used Turn iteration(-state) into objects Interchangeable with iterable for iteration iter() Creates a new iterator for an object Calls __iter__() for creation Falls back to __getiem__() Called mplicitly for iteration Wraps a function in an iterator iter(f, o) calls f until o is returned: if you want to mix iterators and readline Examples l = [1,2,3,4,5,6] it = iter(l) for num in it: if num > 1: break for num in it: print num; break writing iterators: class IRange: def __init__(self, end): self.end = end self.cur = 0 def next(self): cur = self.cur if cur >= self.end: raise StopIteration self.cur += 1 return cur def __iter__(self) Generators optional in Python 2.2 Use new keyword 'yield' any function with 'yield' is special Turn function-state into objects Use the iterator protocol Not unavoidable just very very efficient IRange generator def irange(end): cur = 0 while cur < end: yield cur cur += 1 print irange(10) many useful generators in the itertools module in Python 2.3 written in C. Example def map(func, iterable): result = [] for item in iterable: result.append(func(item)) return result def imap(func, iterable): for item in iterable: yield func(item) 'yield' and 'try'/'finally' Python does not allow'yield' inside a 'try' block with a 'finally' clause: try: yield x yield x finally: print x 'yield' inside 'finally' or in 'try'/'except' is allowed yield inside 'finally' or in 'try'/'except' is allowed Generators as "tasklets" (Tasklets are functions which are scheduled cooperatively; they can yield control to the scheduler, which can call other tasklets, and be resumed later, when the scheduler has time to do so)(dp) Write function as generator At all pause points before the final result, yield a sentinel value (such as None) Avoid blocking functions, or write them as generators and call in a loop Convince callers to use iteration Remember: you may never be called again TryAgainLater = object() class Bucket: def __init__(self): self.data = [] self.done = False def __iter__(self): return self def next(self): if self.data: return self.data.pop(0) if self.done: raise StopIteration return TryAgainLater def add(self, item): self.data.append(item) def end(self): self.done = True class Tasklet: def__init__(self, *args, **kwargs): self._work= self._workgen(*args, **kwargs) def run(self): try: return self._work.next() except StopIteration: return None def _workgen(self, bucket): for item in bucket: if item is TryAgainLater: yield TryAgainLater continue for returnval in self._process(item): yield returnval def _process(self, item): while not item.done: yield TryAgainLater yield item.value class Tasklet: def __init__(self, *args, **kwargs): self.run = self._workgen(*args).next def _workgen(self, bucket, process): for item in bucket: if item is TryAgainLater: yield TryAgainLater continue for endmarker in process(item): if endmarker is TryAgainLater: yield TryAgainLater break else: raise SuitableError for skipitem in bucket: # eat bucket items until #endmarker if skipitem is not endmarker: yield TryAgaiLater break else: warnings.warn(SuitableWarning, 'marker not found') yield DoneProcessing raise TaskletError, 'task has finished' questions Q: Iterators for dictionaries A: There are multiple types - there is a default for key based -------------------------------------------------------------------------- REFERENCES: {as documents / sites are referenced add them below} -------------------------------------------------------------------------- QUOTES: -------------------------------------------------------------------------- CONTRIBUTORS: {add your name, e-mail address and URL below} Ted Leung, twl@osafoundation.org, http://www.sauria.com/blog Donovan Preston, dp@divmod.org, http://www.ulaluma.com/pyx -------------------------------------------------------------------------- E-MAIL BOUNCEBACK: {add your e-mail address separated by commas } -------------------------------------------------------------------------- NOTES ON / KEY TO THIS TEMPLATE: A headline (like a field in a database) will be CAPITALISED This differentiates from the text that follows A variable that you can change will be surrounded by _underscores_ Spaces in variables are also replaced with under_scores This allows people to select the whole variable with a simple double-click A tool-tip is lower case and surrounded by {curly brackets / parentheses} These supply helpful contextual information. -------------------------------------------------------------------------- Copyright shared between all the participants unless otherwise stated...