necroforest

yeah, and as far as I'm aware there isn't even a definition of "civilization" or "intelligence" that doesn't boil down to "sufficiently like me".

I don't entirely disagree with you, but "what products do people want" is overly conservative. Pre-ChatGPT, very few people wanted a (more or less) general purpose chatbot.

It's branding (see: TensorFlow); also, pretty much anything (linear) you would do with an arbitrarily ranked tensor can be expressed in terms of vector ops and matmuls

cosine similarity is (isomorphic to) "distances to nearby objects". and not all embeddings are word embeddings.

If you need to relearn your iPhone/Mac interfaces every year, you might want to get screened for Alzheimers.

> Who doesn’t think China has a ten year AI algorithm to takeover Taiwan?

anybody who works in either AI or natsec

> We arent going to see more progress until we have a way to generalize the compute graph as a learnable parameter

That's a bold statement since a ton of progress has been made without learning the compute graph.

which laws are broken exactly? it's not remotely settled law that "training an NN = copyright infringement"

What's an order of magnitude between friends?

Maybe "best word processor for mac" is a better search for you?

LLMs are trained in parallel. The model weights and optimizer state are split over a number (possibly thousands) of accelerators.

The main bottleneck to doing distributed training like this is the communication between nodes.

which jargon here is "just for show"?

>it's actually impossible to have a non-Hermitian matrix C such that x†Cx > 0

Nice catch, it's been a few years since I had to think about these details.

no, at least not for anything non-contrived.

I used Bayesian analysis for an industry problem where we had a lot of data (~1M samples), but the relationship between the observed data and the latent variables of interest was complicated and non-identifiable. In the non-identifiable setting, point estimates don't converge in the large data limit and quantifying uncetainty becomes critical.