The key to the claim that other computational estimates are way off is essentially that there's a lot of data crunching happening within a single neuron, rather than it being something we can model as collecting a bunch of inputs and either firing or not. He's arguing that each neuron does a ton of internal computation that can itself be modeled as a (sometimes very large) convolutional network.
I think in a sense this is well-established when looking at real neurons, but several times in the article he uses phrases like "shown to be important for information processing", and that's where I get off the boat a bit. When you're saying that it's so important for information processing that it warrants a 1000x or more increase in the computational power necessary to implement an algorithm, I think it's necessary to dig into what the actual work being done there is, not just that there's some non-trivial transformation. A lot of interesting and extremely tough to model fluid and chemical dynamics are in play when I drink too much water and have to pee, but that doesn't mean that we need to understand them to build a waste disposal system using pipes.
In particular, does the within-neuron processing actively tune itself based on the data it processes to an extent on-par with inter-neuron connections (in which case the argument that it's fundamental to the learning process would hold a lot more weight), or is it mostly static? I think a lot of us consider "important for information processing" to mean "is a meaningfully dynamic parameter involved in a learning algorithm", rather than an accidental shmearing of data.
I'd really love more info on what the actual processing that's happening is.
I think in a sense this is well-established when looking at real neurons, but several times in the article he uses phrases like "shown to be important for information processing", and that's where I get off the boat a bit. When you're saying that it's so important for information processing that it warrants a 1000x or more increase in the computational power necessary to implement an algorithm, I think it's necessary to dig into what the actual work being done there is, not just that there's some non-trivial transformation. A lot of interesting and extremely tough to model fluid and chemical dynamics are in play when I drink too much water and have to pee, but that doesn't mean that we need to understand them to build a waste disposal system using pipes.
In particular, does the within-neuron processing actively tune itself based on the data it processes to an extent on-par with inter-neuron connections (in which case the argument that it's fundamental to the learning process would hold a lot more weight), or is it mostly static? I think a lot of us consider "important for information processing" to mean "is a meaningfully dynamic parameter involved in a learning algorithm", rather than an accidental shmearing of data.
I'd really love more info on what the actual processing that's happening is.