Comment by kikimora

You can say it this way. Other way to put is optimal policy makes optimal decisions on every step as if it knows the future. If you looking for shortest distance in a graph then at every vertex it picks the right step, even if the edge is longer than other available in the vertex, as if it knows the future. This is why you cannot just build an algo implementing the policy, you have to find it with dynamic programming.