rosspackard

Half the tasks they were not allowed to use AI.

Great for the authors. But everyone else seems to be extrapolating. Authors have a responsibility and should recognize how their work will be used.

Metr may overall have an ok mission, but their motivation is questionable. They published something like this to get attention. Mission accomplished on that but they had to have known how this would be twisted.

It was just published. Too new for someone to conduct a direct study to critique and journals don't just publish critiques anyway. It would have to be a study that disputes the results.

They used 16 developers. The confidence intervals are wide and a few atypical issues per dev could swing the headline figure.

Veteran maintainers on projects they know inside-out. This is a bias.

Devs supplied the issue list (then randomized) which still leads to subtle self-selection bias. Maintainers may pick tasks they enjoy or that showcase deep repo knowledge—exactly where AI probably has least marginal value.

Time was not independently logged and was self-reported.

No possible direct quality metric is possible. Could the AI code be better?

The Hawthorne effect. Knowing they are observed paid may make devs over-document, over-prompt, or simply take their time.

Many of the devs were new to Cursor

Bias in forecasting.

One mediocre paper/study (it should not even be called that with all the bias and sample size issues) and now we have to put up with stories re-hashing and dissecting it. I really hope these don't get upvoted more in the future.

16 devs. And they weren't allowed to pick which tasks they used the AI on. Ridiculous. Also using it on "old and >1 million line" codebases and then extrapolating that to software engineering in general.

Writers like this then theorize why AI isn't helpful, then those "theories" get repeated until it feels less like a theory and more like a fact and it all proliferates into an echo chamber of AI isn't a useful tool. There have been too many anecdotes and my own personal experience to ignore that it isn't useful.

It is a tool and you have to learn it to be successful with it.

I have a suspicion it does use budget forcing. The word "alternatively" also frequently show up and it happens when it seems logically that a </think> tag could have been place.

What books or material on learning marketing would you suggest?

Could this be a case like investment alpha? If you have a real life use case and share it then you could lose the opportunity.

So some "experts" could be staying quiet because they don't have one. But some may stay quiet because they are working on or benefiting from it?

What do you mean when you use the word taught for open-interpreter?

Looking at the OI docs wasn't too helpful.

"I did spend a non-trivial amount of time fiddling with the prompts" was it writing prompts?

I am really interested and this seems like a cool use case that I want to explore. Could you share the prompts on a github gist?

Which James Clear articles did you like the best? There are a lot to choose from.

Fully anonymized medical history/demographics data across countries and open to the public.

This would allow crowd sourcing and statistics to be applied by the best minds in the world.

I know regulations can make this difficult but many difficult boundaries are being pressed now.

Would you share anything about your lifestyle business?

What industry? How long you worked on it, etc?

IANAL but you should be careful about sharing private information about a large publicly traded company