Nice, until the CSV part really. I mean you have a nice DataFrame there but then it gets serialized using probably the worst format out there for totally variable data like what you scrape from a site.
Unless the file uses the actual ASCII record separator you'll end up with a CSV file which can only be read by a handful of software, after telling it explicitly what the separator and quoting rules are. And even then it's hit or miss.
And likely it does not use the RS because even though the chances it's unambiguous greatly increase and the RS was actually meant for that, software doesn't typically use it because when CSV was invented it's existence was unknown or ignored beacue it's not human readable (I guess, don't really know actually) and so the sad story began.
As you can see: I'm not a fan of CSV :) Just today I again had to waste time because at one point in the development of this otherwise fine piece of software I'm working on - even though I knew I'd regret it - I allowed it to export CSV files. Customer moved software to another machine, forgot that they once told the CSV exporter part to use the system settings, and now has CSV files with a comma separator (you know, the C in CSV). Oh the irony, that's not what they wanted.
Unless the file uses the actual ASCII record separator you'll end up with a CSV file which can only be read by a handful of software, after telling it explicitly what the separator and quoting rules are. And even then it's hit or miss. And likely it does not use the RS because even though the chances it's unambiguous greatly increase and the RS was actually meant for that, software doesn't typically use it because when CSV was invented it's existence was unknown or ignored beacue it's not human readable (I guess, don't really know actually) and so the sad story began.
As you can see: I'm not a fan of CSV :) Just today I again had to waste time because at one point in the development of this otherwise fine piece of software I'm working on - even though I knew I'd regret it - I allowed it to export CSV files. Customer moved software to another machine, forgot that they once told the CSV exporter part to use the system settings, and now has CSV files with a comma separator (you know, the C in CSV). Oh the irony, that's not what they wanted.