Comment by svalorzen

Shameless self-promotion post: if you're interested in AI based on Markov, I maintain an open-source library here https://github.com/Svalorzen/AI-Toolbox. It can be used for general autonomous agents which need to act in response to environmental stimuli. I've used it to learn to play StarCraft and to control cameras to predict people's movements around buildings. Any comments and suggestions are welcome =)