In my opinion, one argument for internally representing `String`s as UTF8 is it prevents accidentally saving a file as Latin1 or other encodings. I would like to read a file my coworker sent me in my favorite language without having to figure out what the encoding of the file is.
For example, my most recent Julia project has the following line:
Figuring out that I had to use Windows-1252 (and not Latin1) took a lot more time than I would have liked it to.
I get that there's some ergonomic challenges around this in languages like Julia that are optimized for data analysis workflows, but imho all data analysis languages/scripts should be forced to explicitly list encodings/decodings whenever reading/writing a file or default to UTF-8.
For example, my most recent Julia project has the following line:
Figuring out that I had to use Windows-1252 (and not Latin1) took a lot more time than I would have liked it to.I get that there's some ergonomic challenges around this in languages like Julia that are optimized for data analysis workflows, but imho all data analysis languages/scripts should be forced to explicitly list encodings/decodings whenever reading/writing a file or default to UTF-8.