arbot360

Not beating the allegations on "Claude doesn't speak human because Anthropic employees don't speak human."

Many SaaS vendors forbid benchmarking, I find it crazy that such anti-competitive terms are standard across the industry but they are. Generally the goal of such terms is to "control the narrative" around the product, regardless of the truth of performance being better or worse than competitors.

Starting with the Bell Riots in SF, September 2024 which happened as a consequence of the unemployment and homelessness crisis there.

Isn't that already on the tax return? Your dependent count would increment from the prior year. The IRS can also distinguish births vs adoptions and step children by the checking for novel SSNs.

Clearly this means new SaaS companies need to start buying forks, issues and PRs for their repos now too.

Another great example of this working is the genomic variant calling models from Deepmind "DeepVariant". They use the "alignment pile-up" images which are also used by humans to debug genomic alignments, with some additional channels to further feature engineer the CNN.

This field of study is known as "Curriculum Learning" for your Googling pleasure (or I guess ChatGPT Deep Research now).

> What was true last year may be false today. For instance, ...

Good example of a medical QA dataset shifting but not a good example of a medical "fact" since it is an opinion. Another way to think about shifting medical targets over time would be things like environmental or behavioral risk factors changing.

Anyways, thank you for putting this dataset together, certainly we need more third-party benchmarks with careful annotations done. I think it would be wise if you segregate tasks between factual observations of data, population-scale opinions (guidelines/recommendations), and individual-scale opinions (prognosis/diagnosis). Ideally there would be some formal taxonomy for this eventually like OMOP CDM, maybe there is already in some dusty corner of pubmed.

Star Trek outlined a sequence of steps but you might not like them...

I use the Viture which supports negative prescriptions by an adjustment dial on the optics.

Looks like I've been ninja'ed.

That paragraph is referring to mitochondria, which the scientists discovered are not present in the subject of the article.

I played $60k and ended with +$64k net. Going to open a Robinhood account next.