Otterly99

I just commented the same thing about a magic cake bush my dog found. I am deligthed to see it is a universal dog (and human?) thing.

My dog had a similar thing with a bush.

One day, she found a whole cake under a bush and brought it to us (I'm talking a cake big enough to feed 8 persons).

From that day forward, she would intensely forage under the "cake bush" every time we passed it.

It has always seemed to me crazy (and maybe a bit lazy) that people will just blindly use self-generated confidence scores from LLMs without having some sort of validation mechanism.

Also, as another poster said: decomposing your rating into as many dimension as possible is better way to generate some confidence score grounded in fact.

I tried to be in good faith and read the paper, but I can't deal with these "wall of text" figures that have been popping up all the time in the last few years.

Also it is not obvious from the introduction/conclusion what the goal of the study is, other than showcasing technical abilities. I'm hoping they can now use it on real-world problem because it seems like a nice tool.

From all the comments in the thread it seems to be personnal assistants: I've seen halding groceries, ordering things, booking lessons, writing emails etc.

In a sense, I get it because they are not always the most fun part of life, but it seems odd to me to want to automate away every single little inconvenience of your life away.

Not a joke at all. It is well documented. Attention peaks a few hour after you wake up then it's all downhill from there. It seems counterintuitive, but somebody working at full capacity for "only" 4h a day can be more efficient that somebody working 12h every day (of course, these all ballpark numbers and depend on a lot of factors, sleep being one of the most crucial one).

I don't see how this would have been different is the author of the paper had used Google to get their fake stats published instead.

Yes, it is deceptively easy to post fake information on the internet, and with social media and AI, it is even easier to spread them.

I wonder however, would prompting a LLM to use only reliable sources return the fake stats website?

Disclaimer: I did not work in protein design but in biophysics. However some of my collaborators did.

It definitely is a small help. Although the lab I was working with had supercalculators to generate microtubule-binding proteins, They were kind of limited in how many they could simulate and try, so being able to go through 318K molecules is useful. Our lab was then tasked with doing the testing (I designed the automated assay) and this part is going on right now, I expect it to last around a year (small lab, many tests to run).

Once potential medical targets have been found, the actual work takes several years as you have mentionned as you would switch to cell cultures, then animal models, then human trials. That matches the 6-7 years timeline you mentionned.

So overall, you can crunch a ~6-months period into 1 month, which is definitely a time saver.

My favorite is mustard rabbit (Lapin à la moutarde). It's basically rabbit stewed in white wine, cream or butter and mustard. Mustards helps make a really thick sauce and you can use a copious amount since its flavor mellows down when you cook it.

The "not bothering to check" part is the most annoying.

There are already several of these "proofs" that have been retracted because they did not make sense.

I would have no problem accepting this whole ordeal if the humans behind each proof took the time to re-read, understand and synthesize the result in a readable paper.

There is ample evidence of how damaging fake information can be, and paper retraction never fully undoes the damage (see the Andrew Wakefield and his antivax "paper" for instance).

To me, the weakest link of this whole chain (and a lot of automation chains) is the classifier. Something (probably a Claude model) decides which part is sent to a human reviewer.

How is it calibrated? Do you rank every file in the project by their criticality? Do you train a classifier? Do you base it on some heuristic like number of downstream files affected?

Same sentiment here.

I always wondered why so much of discussed politics centers around race and gender, but I think this is a good hypothesis.

"More generally, you can have a fruitful discussion about a topic only if it doesn't engage the identities of any of the participants."

I think the more interesting result you provided is kind of buried in the article: after you re-word the question and put the negative consequences of pulling the lever, the models switch decision.

This has always been obvious to me, but I wish more people would understand this: uncalibrated decision models are basically worthless. The model clearly doesn't "understand" the semantics of the question since the result heavily depends on how you word it.

It is nothing new thought. Before that, a lot of people would ask LLM for confidence scores while classifying text (I even saw a paper from a team at Google).

So with that in mind, I really don't get the hype behind decision models. If you need to curate a dataset to build a classifier anyway, why not train it yourself and have an even faster model?

It really depends on your teeth layout.

If you are lucky enough to have very straight and packed teeth, there is very little room between your teeth for any food to be lodged in. I used to never floss then one day I tried it and it took me 3 weeks of flossing before the wire came out clean and odorless.