I don't know that "hard-working" has generally been used to mean "working at hard physical labor". I have not been hard-working in that sense since the age of about twenty, the occasional home or volunteer project excepted.
Having said that, would you settle for "diligent"?
Having said that, would you settle for "diligent"?