matt4711

I made the same choice. Also did not even consider the Blazer knowing they are identical.

You need ways to sift through large amounts of unstructured data and do data analysis after. We created these functions to be able to use LLMs to parse unstructured data from the into structured data which we then process with a SQL like engine to map/group/filter etc.

Thanks for the feedback on the cancel/abort/edit requests. we are working on this already.

Hey! I’m on the team working on this. The semantic web comparison is pretty close to how we think about this.

One key difference is that we’re using LLMs to create structured data on the fly based on the query. That means we don't need to relying on webmasters to explicitly annotate pages with metadata which in my view was one of the main reasons why the semantic web did not work out.

> Besides the clear AI smell, this nonsensical claim also plainly contradicts the methodology's key evaluation claim that the quality of an engine's results should be measured against how much it overlaps with the reranked aggregate of the other engines. The benchmark thus seemingly values an engine's ability to "answer unanswerable questions" at zero.

The engine itself is part of the reranked aggregate so if it finds something useful and everybody else does not it gets full credit.

> Yeah? Care to cite anything for that?

This automatically optimizing for clicks using ML is the main way google and other "human" focused search engines have been improving for 20 years. Not sure what citation is needed here.

The judgements are available on huggingface so training is possible. But given that we evaluate on new queries daily there would need to be some generalization happening for this to show up in the benchmark.

One of the authors here. We have been seeing lots of benchmaxxing and leakage in standard web search benchmarks such as BrowseComp.

We developed this live benchmark with daily/hourly sampled fresh queries matching real agentic search traffic to estimate actual search performance of different AI search providers.

I remember a couple of years ago Amazon added a new leadership principle:

Success and Scale Bring Broad Responsibility

We started in a garage, but we’re not there anymore. We are big, we impact the world, and we are far from perfect. We must be humble and thoughtful about even the secondary effects of our actions. Our local communities, planet, and future generations need us to be better every day. We must begin each day with a determination to make better, do better, and be better for our customers, our employees, our partners, and the world at large. And we must end every day knowing we can do even more tomorrow. Leaders create more than they consume and always leave things better than how they found them.

these actions are exactly the opposite of what they claim to try to strive towards.