The term speech is very broadly defined because there are a lot of ways to convey meaning. Some of them then become ambiguous and you have to resolve those ambiguities and that gets messy. But only the messy cases are messy. Riots characteristically aren't a messy case, they're violence in the same way that publishing a newspaper article is speech.
Moreover, if you mess up the messy cases then you should try to do better but society will probably survive, whereas if you censor in the cases that are pure speech or don't punish the actions that are pure violence, you're the baddies.
Moreover, if you mess up the messy cases then you should try to do better but society will probably survive, whereas if you censor in the cases that are pure speech or don't punish the actions that are pure violence, you're the baddies.