git.ipfire.org Git - thirdparty/Python/cpython.git/commit

]> git.ipfire.org Git - thirdparty/Python/cpython.git/commit

projects / thirdparty / Python / cpython.git / commit

author	Lysandros Nikolaou <lisandrosnik@gmail.com>
	Thu, 6 Oct 2022 23:07:17 +0000 (16:07 -0700)
committer	GitHub <noreply@github.com>
	Thu, 6 Oct 2022 23:07:17 +0000 (16:07 -0700)
commit	cbf0afd8a1474d68310331af9218606959d4cc22
tree	cd421653d73d25c89fa8c02e5819517efa76f7b3	tree \| snapshot
parent	b9d2e8171696514e9226164005f7bf24bf69e66d	commit \| diff

gh-97973: Return all necessary information from the tokenizer (GH-97984)

Right now, the tokenizer only returns type and two pointers to the start and end of the token.
This PR modifies the tokenizer to return the type and set all of the necessary information,
so that the parser does not have to this.

Misc/NEWS.d/next/Core and Builtins/2022-10-06-20-41-29.gh-issue-97973.gB-xWi.rst	[new file with mode: 0644]	blob
Parser/pegen.c		diff \| blob \| blame \| history
Parser/pegen_errors.c		diff \| blob \| blame \| history
Parser/tokenizer.c		diff \| blob \| blame \| history
Parser/tokenizer.h		diff \| blob \| blame \| history
Python/Python-tokenize.c		diff \| blob \| blame \| history

Mirror of https://github.com/python/cpython.git

RSS Atom