Comment by nphard85

Could the (not so perfect but technically simple) solution be to transform the style of content under each tag to the correct expected style for the tag, via a smaller or purpose-built LLM, before the data stream is fed into the main LLM? Perhaps the two LLMs can be co-trained to keep the overall quality of the output stable while role confusion is minimized.