STOP.
People insist on making data exchange formats used to transfer data between software 99.99% of the time "easy for humans" that 0.001% of the time, at the expense of the common case. This inevitably results in squishy, inefficient formats like HTML and JSON that become performance quagmires and security land mines: http://seriot.ch/projects/parsing_json.html
CSV is not even easy for humans to process, with or without tools like grep. Can you successfully handle quote escaping and line-wrapping?
This ,"is ", valid
"Hello",123,"''from the
not,what,you,
""think"",next line"""
That's one record, by the way.
Your level one challenge is: Extract the unescaped value of the "valid" column. Write your robust, tested, "grep" command-line below.
Level two: Make that parse consistently across a dozen tools from a dozen vendors. Try ETL tools from Oracle, Sybase, and SAP. Try Java, PowerShell, and Excel. Then import it via an ODBC text driver, and also into PowerBI. I wish you luck.
People insist on making data exchange formats used to transfer data between software 99.99% of the time "easy for humans" that 0.001% of the time, at the expense of the common case. This inevitably results in squishy, inefficient formats like HTML and JSON that become performance quagmires and security land mines: http://seriot.ch/projects/parsing_json.html
CSV is not even easy for humans to process, with or without tools like grep. Can you successfully handle quote escaping and line-wrapping?
That's one record, by the way.Your level one challenge is: Extract the unescaped value of the "valid" column. Write your robust, tested, "grep" command-line below.
Level two: Make that parse consistently across a dozen tools from a dozen vendors. Try ETL tools from Oracle, Sybase, and SAP. Try Java, PowerShell, and Excel. Then import it via an ODBC text driver, and also into PowerBI. I wish you luck.