If the AI knew what it was about to generate, sure. The problem is the text you see appearing word-after-word appears to be live output. The AI doesn't know the complete output as it's writing it to you. Then it checks what it said and oops! It was hateful.
It probably could work like you how you mention, but then you're left with a 5-10 second wait while the AI 'thinks' after you send a message. I suspect someone made a decision to be more responsive than safe.
ChatGPT is the same way, though I've had ChatGPT cut itself off mid-response before. Maybe they might be calling the moderation API after every token is generated instead of once at the end?
It probably could work like you how you mention, but then you're left with a 5-10 second wait while the AI 'thinks' after you send a message. I suspect someone made a decision to be more responsive than safe.
ChatGPT is the same way, though I've had ChatGPT cut itself off mid-response before. Maybe they might be calling the moderation API after every token is generated instead of once at the end?