eugenhotaj

This is because everyone is training with synchronous sgd. all gpus need to synchronize on each gradient step so tail latency will kill you.

In my experience processes like these rarely work out as intended and usually add layers of bureaucracy for marginal benefit. It’s usually senior engineers or middle managers looking for “org wide impact” so they can get promoted.

This post would sound so dumb if it didn’t come from the almighty pg.

The issue is not that ChatGPT will kill things off, the issue is that ChatGPT 4.0 will kill things off. If you don’t think that’s a real possibility, you’re sleeping.

Some of these are so odd. When Kylie Jenner launched her company she could have sold celery and still made $1B+. It wasn’t because of the effectiveness of her small team.

My experience is exactly the opposite in almost all cases. Most software is much more complicated than it needs to be. Reads like the author is just butthurt at feedback they received about their work.

I’ve likely had it all my life, but really started noticing about two years ago during the pandemic. Now I can’t unhear it. Went to a doctor a couple of times but nothing they tried really helped. Thankfully mine is not too bad and I’m mostly able to ignore it.

The network communication overhead would be way too high to make this useful. At least for current methods of training large models.

Source: trust me bro

Now someone just needs to pipe the output into stable diffusion.

Just take a look at big tech salaries on levels.fyi. Very senior engineers (L8/L9) easily make 10x the salary of entry to mid-level engineers (L3/L4/L5). For a lot of tech companies, L5 is a terminal level meaning most engineers will spend the rest of their careers there. Clearly the market thinks 10x engineers exist.