Comment by BenoitEssiambre

I agree, to me, the maximum entropy argument for the Gaussian most convincingly makes it one of the most fundamental distribution.

Click on chapter 7 here for a good discussion: http://omega.albany.edu:8008/JaynesBook.html