Discussion

Ask HN: Interested in a Wikipedia parsing tool for data mining?

I recently developed a C library that parses Wikipedia, with the goal of being _fast_. For example, I was able to parse and extract text from the entire wikipedia dump (~35 GB uncompressed) in under an hour (on a 5yr old iMac).

It is a work in progress. If there is sufficient interest, then I will clean up the code and put up some documentation.

5 points ajeet 14 years ago 6 comments

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton