Comment by jvmancuso

Thanks for the suggestion! The reason average pooling is faster than max pooling is that the former uses only addition, which is quite a bit faster than comparison in MPC. The smooth max requires both exponentiation and division, both of which are significantly more expensive than doing the comparison in ReLU or max pooling.