Comment by twiecki

No, back-propagation would not give full Bayesian inference (although there are some tricks [0]). They instead use variational inference[1], which allows for fast inference of continuous PGMs.

[0] http://mlg.eng.cam.ac.uk/yarin/blog_2248.html

[1] https://arxiv.org/abs/1603.00788