]> git.ipfire.org Git - thirdparty/Python/cpython.git/commit
Add untokenize() function to allow full round-trip tokenization.
authorRaymond Hettinger <python@rcn.com>
Fri, 10 Jun 2005 11:05:19 +0000 (11:05 +0000)
committerRaymond Hettinger <python@rcn.com>
Fri, 10 Jun 2005 11:05:19 +0000 (11:05 +0000)
commit68c04534182f2c09783b6506701a8bc25c98b4a9
tree4e5f2b764eff65a3201dd2e666355c487e88a9b7
parentbf7255fffb5dda1b9541892cc40412bb6bbd4409
Add untokenize() function to allow full round-trip tokenization.

Should significantly enhance the utility of the module by supporting
the creation of tools that modify the token stream and writeback the
modified result.
Doc/lib/libtokenize.tex
Lib/test/regrtest.py
Lib/test/test_tokenize.py
Lib/tokenize.py
Misc/NEWS