I often wondered that entropy is often not the right measure which you want and something like what he describes as complexity is actually what you want. It's the first time that I have heard of this concept and that it's formally defined as sophistication or logical depth. The only downside is that this is not computable.
This lead me to the question, whether there is a simpler variant of complexity (with similar properties) which is computable: https://math.stackexchange.com/questions/2387690/simple-comp...