We definitely need to decouple discovery from hosting.
I'm not even sure you would need to do any crawling. Maybe you could just let people submit links to videos hosted elsewhere. Kind of like you can submit any URL to the Internet Archive. But in this case you would use the frequency of duplicate submissions (along with in-house upvotes) to build out a recommendation platform.
I'm not even sure you would need to do any crawling. Maybe you could just let people submit links to videos hosted elsewhere. Kind of like you can submit any URL to the Internet Archive. But in this case you would use the frequency of duplicate submissions (along with in-house upvotes) to build out a recommendation platform.