I'm ready to believe the most charitable explanation for this: the new code search (which I have found to be exceptionally good) has a lot more going on under the hood than a normal search engine, which makes it a lot more resource-intensive - limiting it to signed-in accounts saves a huge amount of server resources that would otherwise be used to serve crawlers.
My guess is that the trade-off here is genuinely a question of if they spend 3-4x (I'm guessing, but I think this is likely a low-ball estimate) on their search infrastructure, v.s. having people angry at them for requiring login to use the feature.
My guess is that the trade-off here is genuinely a question of if they spend 3-4x (I'm guessing, but I think this is likely a low-ball estimate) on their search infrastructure, v.s. having people angry at them for requiring login to use the feature.
I actually built my own code search tool for use with GitHub repos, but I've mostly stopped using that because the new GitHub code search is so useful by default: https://simonwillison.net/2020/Nov/28/datasette-ripgrep/