I have a tiny suspicion that a new technique is going to emerge soon that will cause some frustration for Nvidia's investors. For instance, observing how little quantization error seems to matter makes me believe there are probably some other massive wins lurking throughout. Before I'd invest, I'd consider the "doomsday" scenario wherein someone develops a technique for running an OpenAI, GPT-4-scale model on a MacBook or iPhone.
I know it probably seems impossible to many on HN that there could be another bucket of 3-4 orders of magnitude sitting on the table, but progress over the last ~6 months seems to provide a compelling argument against that point of view.
There are also radically-different architectures that have seen zero serious effort put towards them. Mostly things that are CPU-bound. In my view, GPU is starting to cause more harm than good with regard to development of new concepts for problem solving. There are neural network architectures that simply don't work well across the PCIe bus. Eventually, someone is going to start playing around with these ideas as GPU scarcity rages onwards.
I know it probably seems impossible to many on HN that there could be another bucket of 3-4 orders of magnitude sitting on the table, but progress over the last ~6 months seems to provide a compelling argument against that point of view.
There are also radically-different architectures that have seen zero serious effort put towards them. Mostly things that are CPU-bound. In my view, GPU is starting to cause more harm than good with regard to development of new concepts for problem solving. There are neural network architectures that simply don't work well across the PCIe bus. Eventually, someone is going to start playing around with these ideas as GPU scarcity rages onwards.