To solve mnist without mathematical tricks like convolutions or attention heads you would nees 2.5e42 weights. Assuming that you're using 16 bit weights that 5e42 bytes. A yotta byte is 10e24.
That is you'd need 5 exa yotta bytes to solve it.
Currently the whole world has around 200 zetabytes of storage.
I short for the next 120 years mnist will need mathematical tricks to be solved.
That is you'd need 5 exa yotta bytes to solve it.
Currently the whole world has around 200 zetabytes of storage.
I short for the next 120 years mnist will need mathematical tricks to be solved.