From: Raymond Hettinger <python@rcn.com>
Date: Mon, 2 Jul 2012 03:00:09 +0000 (-0700)
Subject: The StopIteration API applies to both tokenize() and generate_tokens()
X-Git-Tag: v2.7.4rc1~726
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=2aa8519f07bff9e7a04e62e2e4d11c87d18bb846;p=thirdparty%2FPython%2Fcpython.git

The StopIteration API applies to both tokenize() and generate_tokens()
---

diff --git a/Doc/library/tokenize.rst b/Doc/library/tokenize.rst
index 707503528157..d0374521cd1a 100644
--- a/Doc/library/tokenize.rst
+++ b/Doc/library/tokenize.rst
@@ -29,7 +29,8 @@ The primary entry point is a :term:`generator`:
    which must be a callable object which provides the same interface as the
    :meth:`readline` method of built-in file objects (see section
    :ref:`bltin-file-objects`).  Each call to the function should return one line
-   of input as a string.
+   of input as a string. Alternately, *readline* may be a callable object that
+   signals completion by raising :exc:`StopIteration`.
 
    The generator produces 5-tuples with these members: the token type; the token
    string; a 2-tuple ``(srow, scol)`` of ints specifying the row and column