I love Unicode, but I'm more and more coming to the conclusion that strings are evil and should be treated as opaque byte arrays, whose only available operation is rendering into a bounded area. I now see any other string operation as code smell.
It's scary how much of our infrastructure relies on strings, given how few guarantees string operations actually give. Take files names, for example. Two visually identical file names may map to different files (because confusables[1]), or two different names map to the same file (because normalization[2]), or the ".jpg" at the end may not actually be the extension (because right-to-left override[3]), not to mention names with newlines or backspaces in them, and inconsistencies between operating systems.
I would go as far as blaming our overreliance on strings for all the injection attacks we see (XSS, SQL, command, etc).
It's scary how much of our infrastructure relies on strings, given how few guarantees string operations actually give. Take files names, for example. Two visually identical file names may map to different files (because confusables[1]), or two different names map to the same file (because normalization[2]), or the ".jpg" at the end may not actually be the extension (because right-to-left override[3]), not to mention names with newlines or backspaces in them, and inconsistencies between operating systems.
I would go as far as blaming our overreliance on strings for all the injection attacks we see (XSS, SQL, command, etc).
[1] https://unicode.org/cldr/utility/confusables.jsp
[2] https://developer.apple.com/library/archive/qa/qa1173/_index...
[3] https://krebsonsecurity.com/2011/09/right-to-left-override-a...