So I'm on ProPublica's web team -- the organization mentioned in the first video -- and we deal with the types of messy data Refine is made for on a day to day basis.
We've been using it pretty much daily for about 5 months now and cleaning messy government data used to be time consuming and destructive, with google Refine it's so easy and fast to join, cleanup and do rudimentary analysis on said data.
It especially shines when you have to merge many disparate data sets into one. My colleague, Dan Nguyen, did just that for our Dollars for Doctors app:
http://projects.propublica.org/docdollars/
and he scraped the data from reports like this:
http://www.pfizer.com/responsibility/working_with_hcp/paymen...
(one company even put the disclosures up as a flash movie).
Of course we could write scripts, use grep/awk/sed or import it into a database, but Refine is really it. I encourage you to give it a try if you have questionable data you'll need to clean.
We've been using it pretty much daily for about 5 months now and cleaning messy government data used to be time consuming and destructive, with google Refine it's so easy and fast to join, cleanup and do rudimentary analysis on said data.
It especially shines when you have to merge many disparate data sets into one. My colleague, Dan Nguyen, did just that for our Dollars for Doctors app:
http://projects.propublica.org/docdollars/
and he scraped the data from reports like this:
http://www.pfizer.com/responsibility/working_with_hcp/paymen...
(one company even put the disclosures up as a flash movie).
Of course we could write scripts, use grep/awk/sed or import it into a database, but Refine is really it. I encourage you to give it a try if you have questionable data you'll need to clean.