Comment by optimalsolver

Has anyone tried searching for new basic operations like convolution or pooling?

We've been using these methods for years, and I doubt the first major development in NN vision (convolution) is the most optimal method possible.

Consider the extreme case of searching over all (differentiable) mathematical operations to see if something really novel can be discovered.

How feasible would this be?