>> not working towards an optimization problem set-up by humans
I am suggesting neither of the two.
Not a great analogy but a parent may be working for an infant's benefit without the infant yet being able to understand. Taking an arbitrarily broad example, the problem to optimize for could be "help humanity advance", and other aspects could be subgoals of the same including what mathematics is useful.
>> not working towards an optimization problem set-up by humans
I am suggesting neither of the two.
Not a great analogy but a parent may be working for an infant's benefit without the infant yet being able to understand. Taking an arbitrarily broad example, the problem to optimize for could be "help humanity advance", and other aspects could be subgoals of the same including what mathematics is useful.