> Every year, training and running AI will demand more resources over and over and we will continue to be charged increasing amounts for everything because everything costs more either directly or upstream of us.
Unless we are really going to hit the Singularity and create Godlike superintelligence that improves beyond all limits, sooner or later the models will stop improving, no matter how much compute we throw at them. After that happens, the best models will be etched into model-specific ASICs with weights fixed in mask ROM (like Taalas is already doing), which will result in massive, order of magnitude inference efficiency increase. Couple this with the training datacenters suddenly becoming available for inference, and the cost of compute will plummet.
Yes, it might take a few years, but the hardware will be cheap again. And if the result of the whole process is human-level AI, then it will be worth it in the end.
Unless we are really going to hit the Singularity and create Godlike superintelligence that improves beyond all limits, sooner or later the models will stop improving, no matter how much compute we throw at them. After that happens, the best models will be etched into model-specific ASICs with weights fixed in mask ROM (like Taalas is already doing), which will result in massive, order of magnitude inference efficiency increase. Couple this with the training datacenters suddenly becoming available for inference, and the cost of compute will plummet.
Yes, it might take a few years, but the hardware will be cheap again. And if the result of the whole process is human-level AI, then it will be worth it in the end.