Comment by goethes_kind

We got so far with transformers and huge amounts of compute and huge amounts of data. If we can get an architecture that is able to extract a slightly higher order of reasoning it will have a cascading effect when we apply the same level of compute and data. I see lots of potential for progressive improvement in this direction. Problem is, it's quite expensive to develop and test new architectures, but that's were the financial capital comes in.