Corence

Steam Next Fest is unique in that the first two days of it have the default view for the page be random. This is intended to give more games exposure, and then two days into the Fest they have enough data to start personalizing recommendations. So the event format's design makes slop particularly bad compared to most other places in the storefront.

I feel like this is a quirk of LMSYS-style optimization where humans in SxS preference evals will pick based on formatting more than actual content.

Google search is worse because of recent AI tech flooding the internet with misinformation and low quality articles.

I think LLMs are better at finding the most helpful sources now, but that's more a testament to how much the front page of web search has lost to low value LLM content.

I've used it for live service video games, it's pretty good at summarizing major changes to a game since you've played it last. With regular web search you'd have to go to every major patch and a lot of games don't even have good patch notes / it's all stuck in content creator videos.

Though I still prefer Claude for this since it's better at citing sources.

Unironically you can put "what are good demos for agentic workflows at Google I/O that would be received well by the general public" into Gemini's AI Mode and get better suggestions for use cases than what they're showing.

Note the scoring function is significantly different for ARC-AGI-3. It isn't the percentage of tests passed like previous versions, it's the square of the efficiency ratio -- how many steps the model needed vs the second best human.

So if a model can solve every question but takes 10x as many steps as the second best human it will get a score of 1%.

It is a fair question. I'd expect the numbers are all real. Competitors are going to rerun the benchmark with these models to see how the model is responding and succeeding on the tasks and use that information to figure out how to improve their own models. If the benchmark numbers aren't real their competitors will call out that it's not reproducible.

However it's possible that consumers without a sufficiently tiered plan aren't getting optimal performance, or that the benchmark is overfit and the results won't generalize well to the real tasks you're trying to do.

Focusing on the AI content generation aspect of this is disappointing to me, since those tools are fuzzy at best and even if the post was AI generated, it isn't necessarily a red flag since the user could be trying to disguise their writing style.

There are plenty of other signs this story is likely fake. The author claiming to be posting from a library on New Years' Day (most government buildings are closed) and was responding over 10 hours on the account. He's using a throwaway and a "burner laptop" at the library, but he also says he put his two weeks' notice in yesterday (also odd that this is on New Years' Eve) which would make identifying him trivial.

Fake stories get to the front page of Reddit every day, I wish journalists were pointing out the actual signs not to trust something to act as a better example.

FYI: Card 8's transcription is different than the image. In the image 5, 8, 12 is a Set but the transcription says Card 8 only has 2 symbols which removes that Set.

See: "How do we know it’s training data?" from the posted link.

Tesla's example is most alarming to me since people who seemingly have no business purpose to access highly sensitive data have access to it. Culturally people feel safe sharing this data on internal chats which means they don't think coworkers will report the data access violations and since the content is spreading "like wildfire" there's a significant number of people at the company who are abusing data access as opposed to an individual abusing their elevated access.