It's not ambiguous because you have a directional reference built into the phrase and speaking directly to a singular person: "ask you about your left hand".
It is certainly clear if one would say "left facing stern" or "left facing aft", but that's a mouthful when you can just shorten it (and the reference facing direction is not relevant). Bonus points if the shortened version can't be mistaken for another direction...
BTW, I'm 100% down for introducing dedicated words for "my left", "your left" etc vs just "on the left". It would certainly save me a bit of time when my family asks me to look for something and they flip between the two meanings in the same sentence.
It is certainly clear if one would say "left facing stern" or "left facing aft", but that's a mouthful when you can just shorten it (and the reference facing direction is not relevant). Bonus points if the shortened version can't be mistaken for another direction...
BTW, I'm 100% down for introducing dedicated words for "my left", "your left" etc vs just "on the left". It would certainly save me a bit of time when my family asks me to look for something and they flip between the two meanings in the same sentence.