>Parsers typically use regexes for the tokenization stage - indeed, what else would you use?
This is completely wrong. One can also just write their own tokenizer reading one character at a time with a state machine. It's trivial compared to the complexity of the rest of the parser.
This is completely wrong. One can also just write their own tokenizer reading one character at a time with a state machine. It's trivial compared to the complexity of the rest of the parser.