Comment by hexwab

I can highly recommend the late David Mackay's _Information Theory, Inference and Learning Algorithms_. To my mind it's a much easier read than the above book.

http://www.inference.org.uk/mackay/itila/book.html

Associated lecture series: https://www.youtube.com/playlist?list=PLruBu5BI5n4aFpG32iMbd...