> The downside is that, if you perform this as a separate pass, your parser now has to iterate over all of the source code twice. This may not be the end of the world: tokenizing isn't the most expensive operation. But it isn't ideal, so some parsers combine the two passes into a single one, saving cycles at the expense of readability.
This paragraph is a bit muddled:
* If you keep the two passes separate, then the second pass is over the token stream. So it's irrelevant that "tokenizing isn't the most expensive operation" because you're never going to do the actual tokenising a second time.
* If you combine the two passes then you're not "saving cycles" because you're still doing both of the loops - it's just that they're now happening in parrallel rather than in sequence. This confusion is repeated later when he says the combined one is "without the runtime overhead of a second loop".
* What you are saving is memory, because you don't have to keep the whole token stream in memory at once.
But really this is a nitpick on a small note. The overall article is great.
This paragraph is a bit muddled:
* If you keep the two passes separate, then the second pass is over the token stream. So it's irrelevant that "tokenizing isn't the most expensive operation" because you're never going to do the actual tokenising a second time.
* If you combine the two passes then you're not "saving cycles" because you're still doing both of the loops - it's just that they're now happening in parrallel rather than in sequence. This confusion is repeated later when he says the combined one is "without the runtime overhead of a second loop".
* What you are saving is memory, because you don't have to keep the whole token stream in memory at once.
But really this is a nitpick on a small note. The overall article is great.