Indeed! This is an age-old method, this is what a dandy horse is for! In France, and surely many other places, you see the kids of young age on dandy horses ("draisiennes") coming to and from the school supervised by their parents. As a rite of passage towards the bicycle :)
https://en.wikipedia.org/wiki/Dandy_horse