From: Raymond Hettinger <python@rcn.com>
Date: Mon, 14 Jul 2014 08:52:00 +0000 (-0700)
Subject: Issue 21977:  Minor improvements to the regexes in the tokenizer example.
X-Git-Tag: v3.4.2rc1~227
X-Git-Url: http://git.ipfire.org/gitweb.cgi?a=commitdiff_plain;h=8323f68f3e50f73ce549d885c40bb17c06894376;p=thirdparty%2FPython%2Fcpython.git

Issue 21977:  Minor improvements to the regexes in the tokenizer example.
---

diff --git a/Doc/library/re.rst b/Doc/library/re.rst
index 451ee2d2a34e..a835c14f0ca3 100644
--- a/Doc/library/re.rst
+++ b/Doc/library/re.rst
@@ -1340,9 +1340,9 @@ successive matches::
             ('ASSIGN',  r':='),          # Assignment operator
             ('END',     r';'),           # Statement terminator
             ('ID',      r'[A-Za-z]+'),   # Identifiers
-            ('OP',      r'[+*\/\-]'),    # Arithmetic operators
+            ('OP',      r'[+\-*/]'),     # Arithmetic operators
             ('NEWLINE', r'\n'),          # Line endings
-            ('SKIP',    r'[ \t]'),       # Skip over spaces and tabs
+            ('SKIP',    r'[ \t]+'),      # Skip over spaces and tabs
         ]
         tok_regex = '|'.join('(?P<%s>%s)' % pair for pair in token_specification)
         get_token = re.compile(tok_regex).match