Comment by spider-mario

Those are still numerical scores and, as far as I know, essentially based on “does it solve the task correctly or not”. By “qualitative”, I meant for example some analysis of how it succeeds (e.g. quality of prose or code) or its failure modes (lack of persistence, etc.).