ccgreg

We need to abandon the Gregorian calendar -- it's clearly a software maintenance nightmare -- and return to the Julian calendar.

Can you point out an example where someone made that mistake?

It's as if Common Crawl is attempting to be a sample of the web.

> The CC CDX endpoint, index.commoncrawl.org, historically has been easily overwhelmed and unreliable

Use our Parquet index.

It's also worth noting that archive.org downloads all of our crawl data and adds it to the IA Wayback Machine.

How does that work for non-profit web crawls like the Common Crawl Foundation?

Common Crawl is text-only.

Are you talking about the 8087 transcendental instructions? Which are well known and well-written software doesn't have a problem with it.

Huh. A quick google says that's a feature bit and there's also an OS call to turn it on.

There are? We aren't talking about feature bits, and the only known software that checks the vendor are Intel's math libraries.

Another aspect of the x86_64 architectural swamp is that Intel deliberately makes their optimized math libraries fail if run on a chip that is not 'INTEL INSIDE'. That block totally ignores cpu feature bits.

I've always wondered if virtualization software vendors made everything claim to be INTEL INSIDE, just to avoid this problem.

How is this different from x86_64 and aarch64? Or even the various Alpha and Mips64 chips.

David recently announced he was leaving MLCommons, so maybe we'll get him back as an industry analyst.

Common Crawl has very little content from Reddit and 4chan -- both blocked us years ago.