No that is not a solution.
If they swapped the public key they can read the message being sent back (it is encrypted with their public key), then encrypt it again with the real public key.
The only solution is to use another channel to authenticate the other's key, be it GPG's web of trust, or any other imperfect way (phone call, physically meeting, ...)
The only solution is to use another channel to authenticate the other's key, be it GPG's web of trust, or any other imperfect way (phone call, physically meeting, ...)