creekorful

Just a tor proxy (socks) instance, and a HTTP client configured to use the proxy. Other than that nothing more than what a web crawler should have.

I'll certainly publish an index of interesting (legal) hostnames on my GitHub account.

I have an instance running for about 5 days+ with 1,300,000 docs. It's mostly huge amount of porns, some weird forums & lot of marketplaces.

There's still some good finds like a 'sysadmin' forum with huge amount of good posts & tips.

Release 1.0.0 will be out very soon. Check it out guys!