The lexer cannot distinguish the difference between a float literal and a
tuple index in some cases. This means we should fix this while parsing
depending on the context.
gcc/rust/ChangeLog:
* expand/rust-macro-invoc-lexer.cc (MacroInvocLexer::split_current_token):
Add implementation for multiple token split.
* expand/rust-macro-invoc-lexer.h: Add function prototype.
* expand/rust-proc-macro-invoc-lexer.cc (ProcMacroInvocLexer::split_current_token):
Add implementation for 2+ token split for procedural macros.
* expand/rust-proc-macro-invoc-lexer.h: Add function prototype.
* lex/rust-lex.cc (Lexer::split_current_token): Add function to split a
token in multiple other tokens.
* lex/rust-lex.h: Add function prototype for split_current_token.
* parse/rust-parse-impl.h (Parser::left_denotation): Handle float tuple
index identified as a float literal.