In this regard, it is similar to how natural sciences are done. The hyperparameter space of possible experiments is immense, they are expensive, so one has to go with intuition and luck. Reporting this is difficult.
[edit:] In this analogy, deep learning currently misses any sort of a general theory (in the sense of theories explaining experiments).
[edit:] In this analogy, deep learning currently misses any sort of a general theory (in the sense of theories explaining experiments).