ospider
Joined 13 years ago
Joined 13 years ago
The web spec is vast and endless, if you try to build something actually works for modern sites, it will be huge. If you want to keep it lightweight, it won't work for most sites.
Just use chrome-headless-shell or webkit. Don't reinvent the wheels.
As for performance, Python 3.x was not even as fast as Python 2.x in the beginning, but with the correct mindset, it's eventually faster.
Do you mind elaborating on what features are missing?
Basically, there are 2 legislation in the world, legistlation and the China legislation. In China, there are laws on the surface and there are rules underneath. For example, the government never admitted that the GFW exists, yet it keeps blocking more and more sites. The government never bans online forums, yet it never grants license to open a online bbs, since like ten years ago.
During some political sensitive times, the government would send secret requirement to local companies like ByteDance and Tencent on how to censor the social media. Back when I worked at ByteDance, when the 19th Communist Party congress was open, the auditors would be in a war room, just for making sure that no negative news or comments would be released. American companies also work with the government on censorship, more or less, but that's another story.
It's very common for Chinese people who have been fooled by the government to say that, these western companys left by themselves. But it's not the laws that on the surface drives them away, it's the rules underneath.