alessiodm

I’ve released a major expansion of my open-source deep reinforcement learning course. Last year's initial release got positive feedback, so I've added a new module with advanced topics and practical productionization techniques by curating and refining materials I collected over the years. This final update includes hands-on implementations of RND, AlphaZero, RLHF, MBPO, and more. I hope it's a valuable resource for the community.

Really happy to hear you enjoyed the notebooks! And thank you very much for the patch in the simulate_mdp for the cliff world!

Great feedback, I didn't even think about that the TODOs could be indeed confusing! I updated the instructions in the README.md calling them out explicitly as the coding sections to be completed. Thanks again!

Thank you so much for this feedback! Indeed, this is definitely confusing in the notebook. I pushed a small commit to make it a little bit more clear that the non-determinism comes from the probabilistic nature of the environment dynamics (and not b/c the agent chooses a different action by mistake).

As a side note, initially I meant to go through it in a video to fill the gaps in the text with my voice. But given that I didn't have time for those, I am fixing those gaps first :) Thanks again!

Thank you! I'll be be curious if / how these notebooks help and how your experience is! Any feedback welcome!

Thank you. It is true, indeed the material does assume some prior knowledge (which I mention in the introduction). In particular: being proficient in Python, or at least in one high-level programming language, be familiar with deep learning and neural networks, and - to get into the theory and mathematics (optional) - basic calculus, algebra, statistics, and probability theory.

Nonetheless, especially for RL foundations, I found that a practical understanding of the algorithms at a basic level, writing them yourself, and "playing" with them and their results (especially in small toy settings like the grid world) provided the best way to start getting a basic intuition in the field. Hence, this resource :)

Yes, the material relies heavily on Python. I intentionally used popular open-source libraries (such as Gymnasium for RL environments, and PyTorch for deep learning) and Python itself given their popularity in the field, so that the content and learnings could be readily applicable to real-world projects.

The theory and algorithms per-se are general: they can be re-implemented in any language, as long as there are comparable libraries to use. But the notebooks are primarily in Python, and the (attempted) "frictionless" learning experience would lose a bit if the setup is in a different language, and it'll likely take a little bit more effort to follow along.

Thank you so much! Unfortunately, that is a mistake in the README that I just noticed (thank you for pointing it out!) :( As I mentioned in the first post, I didn't get to make the YouTube videos yet. But it seems the community would be indeed interested.

I will try to get to them (and in the meantime fix the README, sorry about that!)

Thank you for the amazing links as well! You are right that the article [1] is 6 years old now, and indeed the field has evolved. But the algorithms and techniques I share in the GitHub repo are the "classic" ones (dating back then too), for which that post is still relevant - at least from an historical perspective.

You bring up a very good point though: more recent advancements and assessments should be linked and/or mentioned in the repo (e.g., in the resources and/or an appendix). I will try to do that sometime.

Didn't know that, but now I have an excuse to go watch a movie :D

Thank you, I appreciate it.

Thank you!

Thanks a lot. It makes me feel better to hear that the post is not completely confusing and appropriating - I really didn't mean that, or to use it as a trick for attention.