> At the core RL is just updating a table of values, and then using function approximation (aka, machine learning) for more complex cases.
this seems to be a common assertion about ml. other refrains include "ml is just matrix multiplication" and "ml is just affine transformations followed by nonlinearity".
while technically true, it is an unhelpful comment as it doesnt shed any light on the salient questions, such as:
* how do you reduce bias/variance, since youre sampling a minuscule slice of the state/action space?
* how do you construct a value function (or something like it) in a sparse reward environment, with possibly thousands of time steps?
* how do you know your policy network is exploring new states?
it is the equivalent of learning how to draw an owl: draw a few circles, then draw the fucking owl.
this seems to be a common assertion about ml. other refrains include "ml is just matrix multiplication" and "ml is just affine transformations followed by nonlinearity".
while technically true, it is an unhelpful comment as it doesnt shed any light on the salient questions, such as:
* how do you reduce bias/variance, since youre sampling a minuscule slice of the state/action space? * how do you construct a value function (or something like it) in a sparse reward environment, with possibly thousands of time steps? * how do you know your policy network is exploring new states?
it is the equivalent of learning how to draw an owl: draw a few circles, then draw the fucking owl.