Comment by simply_sam

There’s an app on sidequest (for the oculus quest) that can track your piano and finger moves. You can input a midi file and then it sees you playing it and shows your virtual fingers. It might be possible to do hand tracking like that with an iphone. Have you tried concerting sound from a phone microphone to midi?