Comment by kdowns

I made it to a third round interview at anthropic in 2024 and they had me build a web crawler as their programming test. Part way through I started on making it respect robots.txt and I could immediately tell they were no longer interested in me.