> Table 4 | Relevant formatting control tokens used for Gemma models
> User turn: user
> Model turn: model
> Start of conversation turn: <start_of_turn>
> End of conversation turn: <end_of_turn>
> Beginning of sequence: <bos>
> End of sequence: <eos>
You know I keep wondering why <bos> and <eos> tokens are even a thing in general. No model is tuned to keep generating multiple turns after its <end_of_turn> equivalent is sent, and what's the point of <bos> when you're parsing the entire context anyway. If it's an attempt to ignore text before it... then why is that text there? Just remove it from context, you're throwing away compute.
> User turn: user
> Model turn: model
> Start of conversation turn: <start_of_turn>
> End of conversation turn: <end_of_turn>
> Beginning of sequence: <bos>
> End of sequence: <eos>
You know I keep wondering why <bos> and <eos> tokens are even a thing in general. No model is tuned to keep generating multiple turns after its <end_of_turn> equivalent is sent, and what's the point of <bos> when you're parsing the entire context anyway. If it's an attempt to ignore text before it... then why is that text there? Just remove it from context, you're throwing away compute.