frogamel

The research in this space is very conflicting about what methods actually work. In the graph on the page, the ETS model (basically just a weighted moving average) outperforms multiple, recent deep learning models. But the papers for those models claim they outperform ETS and other basic methods by quite a bit.

You can find recent papers from researchers about how their new transformers model is the best and SOTA, papers which claim transformers is garbage for time series and claim their own MLP variant is SOTA, other papers which claim deep learning in general underperforms compared to xgboost/lightgbm, etc.

Realistically I think time series is incredibly diverse, and results are going to be highly dependent on which dataset was cherry-picked for benchmarking. IMO this is why the idea of a time series foundation model is fundamentally flawed - transfer learning is the reason why foundation models work in language models, but most time series are overwhelmingly noise and don't provide enough context to figure out what information is actually transferrable between different time series.

I've worked in many research scientist/MLE roles over the years and I haven't met any IC who has this much of a fixation on AI being a moral evil/good. The ones who do are inevitably nontechnical hucksters, usually just trying to get money or self-promote.

I might be miunderstanding something here, but what complexity here is resolved by making this a framework? Isnt this just:

1. Train model like normal

2. Evaluate model using self

3. Use eval results for DPO finetune

IMO if you're going to profit off open research, you should at least make your own work available for other researchers. The white paper has 10 pages of performance benchmarks but 5 sentences on methodology.

Dplyr actually supports some really cool join functionalities that I wish were in SQL implementations, including:

- Ability to specify whether your join should be one-to-one, many-to-one, etc. So that R will throw an error instead of quietly returning 100x as many rows as expected (which I've seen a lot in SQL pipelines).

- A direct anti_join function. Much cleaner than using LEFT JOIN... WHERE b IS NULL to replicate an anti join.

- Support for rolling joins. E.g. for each user, get the price of their last transaction. Super common but can be a pain in SQL since it requires nested subqueries or CTEs.

I know many people who've gotten big raises over the last 2-3 years, including individuals who have more than doubled their pay. And they always attribute their raises to hard work, job hopping, or climbing the promotion ladder. At the same time, they'll also act like the 20% or whatever inflation we've had over the past few years is equivalent to the government stealing their hard earned money.

From a macro perspective, the same factors that drove inflation is what drove their big pay hikes. But most people only see that "I earned more money and now I'm being screwed by inflation". These are often the people who spend like crazy while simultaneously complaining about how bad the economy is!

Every tech company minus the few doing core research have been doing this for at least half a year. Generate training data with GPT4 or sometimes even 3.5 -> use it to do a QLoRA finetune on a llama or mistral base -> roll it out as a "proprietary" AI model -> management claims a big win and talks about how they're leaders in "[industry name] AI".

It is remarkably easy - it takes practically zero knowledge of ML and can usually be done with less than <$1k of cloud compute costs. The issue is that for most realistic tasks you can expect to end up with something roughly on the level of GPT-3.5, and its actually really hard to compute with GPT-3.5 on a cost level, at least if you use cloud GPUs.

I've noticed a huge surge in negativity and pessimism on English-language social media within the last year or so, roughly corresponding with to the spread of LLM tech. I do wonder whether these people are mostly just bots.

I dont think the first part of this article is quite right.

1. I doubt many people would think of 9/11 as a "1 in 50 years" event if it didn't actually happen. If you had a year-by-country level dataset of every developed country post ww2, you'd have thousands of observations but none would have as many fatalies from terrorism as the US in 2001.

2. If a genuinely super rare event occurs (like one in thousands), its often more reasonable to think that theres been some fundamental shift in the world that you failed to recognize, rather than that you just got super lucky or unlucky to have lived through it.

In my experience, you can get extremely good performance in this style of sports prediction with extremely basic methods. E.g. doing some sort of basic Elo rating system + choosing the higher rated competitor to win every game. This is simply because almost all human competitors will frequently try to predict upsets (as its more fun), but it will harm their performance overall.

I would be cool to see if something like this can do can outperform a benchmark Elo system but even that is an extremely high bar that most sophisticated models can't beat.