Comment by BoredomIsFun

ehhh....to be pedantic, LLMs, like any NNs derive power from _not_ being pure matrix multiplication - they are "halved" matrix algebra, due to ReLU