From: Ezio Melotti <ezio.melotti@gmail.com>
Date: Sun, 20 Dec 2009 12:24:57 +0000 (+0000)
Subject: Fixed capitalization and markup; added imports in the example
X-Git-Tag: v3.2a1~2005
X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=a8f6f1e2d63b90549278b1162d264bb279459007;p=thirdparty%2FPython%2Fcpython.git

Fixed capitalization and markup; added imports in the example
---

diff --git a/Doc/library/tokenize.rst b/Doc/library/tokenize.rst
index deb7826393d7..d66b07c62d43 100644
--- a/Doc/library/tokenize.rst
+++ b/Doc/library/tokenize.rst
@@ -94,16 +94,19 @@ function it uses to do this is available:
     (as a string) and a list of any lines (not decoded from bytes) it has read
     in.
 
-    It detects the encoding from the presence of a utf-8 bom or an encoding
-    cookie as specified in pep-0263. If both a bom and a cookie are present,
+    It detects the encoding from the presence of a UTF-8 BOM or an encoding
+    cookie as specified in :pep:`263`. If both a BOM and a cookie are present,
     but disagree, a SyntaxError will be raised.
 
-    If no encoding is specified, then the default of 'utf-8' will be returned.
+    If no encoding is specified, then the default of ``'utf-8'`` will be returned.
 
 
 Example of a script re-writer that transforms float literals into Decimal
 objects::
 
+    from tokenize import tokenize, untokenize, NUMBER, STRING, NAME, OP
+    from io import BytesIO
+
     def decistmt(s):
         """Substitute Decimals for floats in a string of statements.