ajb117

Holy marketing

I get what you're saying, but I think confidence is actually more important than enjoyment.

A lot of people confuse enjoying [x] with enjoying being good at [x]. This is why so many students switch subjects later on in life; when a field suddenly doesn't come naturally to them, they seek to play to their strengths elsewhere. Problems occur when they quit too early, and building confidence early on is important for stopping this.

In my experience, when you think you're bad at something, it's almost impossible to enjoy doing it, which makes preliminary mastery actually the first step to enjoyment and therefore downstream success.

I'm curious about the AI research job market in fields that aren't LLMs and RL. They're all the rage now, but what about researchers in fields like vision or graphs or more fundamental topics like optimization?

I don't know much physics, but is there a reason we should be hoping for superconductivity specifically rather than a cheap material that's just a very very good conductor? Also, even if LK-99 were superconductive, that wouldn't imply it would be useful for computing, right? Surely there are materials that are better conductors than silicon but have mechanical and other physical quirks preventing us from using then for chips.

Makes me wish python had a pipe operator like Julia's |> and R's %>%

Besides Nvidia cards usually being faster and having higher memory bandwidth, Ada cards also have FP8 cores. I'm not sure how well Apples' M-series chips handle low/mixed precision tensors, but I wouldn't be surprised if Nvidia cards perform better with them.

I think that's more in line with transfer learning, a variant of fine-tuning. If I'm reading this article correctly, they're fine-tuning the LMs end-to-end.

My guess is that it's because they've already done RLHF on top of the standard next token prediction. In other words, they can't cheaply fine tune ChatGPT without undoing the RLHF objective by training on next token prediction with post-2021 data, and then retraining with RLHF to make sure it still gives good human-like output.

I mention the "undoing RLHF" since it's not uncommon for fine-tuned models to increase in error in the original training objective after being fine-tuned with a different one. I think people saw this happen in BERT.

Also ChatGPT is almost certainly huge.

I honestly don't know how people can stand Gnome/Unity. I've been running Xubuntu LTS with the Plank dock and have had nothing but smooth sailing save Nvidia drivers.