Comment by aithrowaway1987

Because they can learn a bunch of symbolic formal arithmetic without learning anything about quantity. They can learn

  5 x 3 = 15

without learning

  *****    ****     *******
  ***** =  *****  = *******
  *****    ******   *

And this generalizes to almost every sentence an LLM can regurgitate.