astrophysician

I work for a data science consulting company and previously in advertising, so I can say that there are (1) a number of very real problems that cannot realistically be solved _without_ "AI" (though what does that term even mean these days?), and (2) endless piles of "AI"-branded nonsense.

First of all, what do you consider AI? Deep learning? Hierarchical bayesian models? Linear regression? I would call all of these "AI" simply because I have seen all of these labeled as such at one point or another.

Does your business revolve around computer vision tasks ("how can I track people/cars/things in this image/video?")? You almost definitely need "AI" even if you have almost no historical data.

Does your business revolve around optimizing decisions based on large amounts of historical and relevant data (e.g. user interactions on your website, etc.)? You _probably_ need "AI" (but it depends).

There are an infinite number of ways to incompetently apply AI and upsell what you're doing to people who don't know the difference. It happens all the time and is a natural consequence of any hype cycle. But underneath all of the hype there is obvious substance (just maybe not at your company).

Concrete and very public examples of competent AI:

* Tesla's "auto"-pilot (computer vision/SLAM)

* Recommender systems that you see everywhere (Facebook Watch/News Feed, YouTube, Netflix, etc.) -- can be reinforcement learning based, factorization based, etc.

* Real-time bidding markets in advertising (DSP's like Google's DV360, Trade Desk, Xandr run optimized bidding "AI")

In my experience I’ve never found an instance where you would use Brier scores over cross entropy/Bernoulli/Binomial log likelihoods. Does anybody know a concrete example when you would prefer Brier??

Yea, some people are disappointed; some of the more interesting and exciting moments in physics are when we find out we're wrong, but not always. E.g. I will never forget the time and place I heard about the preliminary detection of primordial B-modes by BICEP (which turned out to be dust contamination) -- that was a predicted detection from canonical inflation models, as the Higgs was a standard prediction from the standard model (also a pretty exciting moment).

Not seeing something when we "expect" to not see anything (from the perspective of certain models) might be more boring, but it's definitely not a "waste" (again speaking purely from a physicist's standpoint).

We know the standard model is incomplete, but where and how are not well known. Not seeing evidence for new physics rules out certain models, and places upper/lower limits on others. It's progress either way.

From a physicists standpoint, not seeing something unexpected is not a waste at all.

I agree -- as ML becomes increasingly easy to be applied by non-experts or people without a heavy math/stats background, I've seen an increasing volume of arguments against the data science profession (someone the other day called DS the "gate-keepers") but: there be dragons.

Anyone can use SOTA deep learning models today, but in my experience, it's more important to understand the answer to "what are the shortcomings/consequences of using a particular method to solve this problem?" "what is (or could be) biases in this dataset?", etc. It requires a non-trivial understanding of the underlying methodology and statistics to reliably answer these questions (or at least worry about them).

Can you apply deep reinforcement learning to your problem? Maybe. Should you? Well, it depends, and you should understand the pros and cons, which requires more than just the knowledge of how to make API calls. There are consequences to misusing ML/AI, and they may not even be obvious from offline testing and cross validation.

I agree: in retrospect, in todays context, those statements are not good public health policy. And saying "they are not effective in preventing general public from catching coronavirus" may indeed be a false statement, even at the time. But, when masks were in short supply and when it wasn't really known how much protection they provide (so saying "hey masks might help" may give people a false sense of comfort thinking that they're protected and able to live as normal), those comments, while not the most objective, make way more sense.

But I think this "they lied to us, we shouldn't trust them" rhetoric is silly and dangerous. Again, I agree, it's definitely a lesson to learn. Public health professionals are human beings too and I think it's fair to say they are doing their jobs to the best of their ability.

I mean I see the point, I think their comments made a lot more sense in the context of where things were back then, and they didn't consider how they would be viewed after the initial panic phase was over.

> the result is eroded trust that health advice being given is in your immediate best interest.

I would argue that they are making health advice that minimizes the population-wide risk; I don't expect them to tell me the optimal thing for me to do to protect myself if that advice means that the population-wide payoff is worse (i.e. "hey everybody the best thing you can do is to grab an N95 mask" might be optimal for everyone who is able to do that but that is not the optimal thing to do for the population). I kind of understand people's frustration but I just don't personally feel that this was that controversial.

You should wear masks in public when you are vaccinated because while you're protected from severe symptoms, you can still carry the virus and transmit it to others.

> The ultra risk aversion is what many people are souring on, especially considering the risks are low to begin with for the vast majority.

Yea but...they are not low for a significant portion of the population, which should matter? And those people are affected when people don't wear masks because many of them are not able to completely isolate. I don't really understand this line of reasoning -- wearing a mask helps people, so why not do it? Is it really that debilitating? I don't like wearing a mask either but if wearing one helps, why not wear it?

Yea but....he's right? What is the controversy here? There was a huge concern about supply shortages and people hoarding PPE. Masks are the best thing the public can do at this point but they aren't perfect protection and the medical community are the ones with the greatest need for PPE...Not to mention, early on in the pandemic it wasn't clear how much protection masks would really give, and people thinking "oh i just need to wear a mask, then I can just do what I normally do" was something they were trying to fight against.

I agree and have been doing some similar analysis on the backblaze dataset. I suppose you can use this for prediction but I'm personally just interested in post-hoc analysis and (1) getting better AFR estimates when failure counts are low + (2) exploring time-dependence of hazard functions with different priors (GP priors, etc.). This post and your comment have motivated me to make a post this weekend! Thanks!

Unfortunately not my experience -- went to nearest ER for sudden and agonizing abdominal pain, got some basic pain meds, got a CAT scan to diagnose pneumonia, was sent home within a couple of hours (lots of waiting).

Turned out the hospital was not in network, they charged me $15,000 (including $10,000 for a "level 5" ER room stay, which is twice the most expensive price listed on the OP's website and is 10x the local average price for that service, despite the fact that my experience could not possibly have been Level 5 service). Eventually they just sold the debt to a collections agency. I tried to fight but they didn't seem to care, and why would they? They already sold the debt!

I'm guessing just venting personal frustrations due to their own experiences, plus maybe poor hiring and guidance of data scientists in their own teams? I have definitely seen DS people in my experience that fit some of the descriptions here, but I think it's a mistake to trivialize the DS position itself. A good DS is a valuable asset, but depending on your company/data, maybe not worth the cost. Plus there is no "single" DS candidate or role, a lot of these roles (data engineer, DS, analyst, swe) blend together at times, and it's about finding the right balance of skills.

Sometimes I think a company (not having the DS experience themselves) mistakenly over-hire DS roles in today's hype of "AI" when their data is mostly run-of-the-mill and only requires simple linear models that can be architected an understood by a stats/math-savvy engineer. Even then, a good DS is still useful (even linear models can be complex: e.g. what priors do you want to use? Do you want a multi-task solution? etc.), but maybe not worth the cost.