Comment by bingo-bongo

Try “UsePAM no” and ensure “UseDNS no” in sshd_config on Ubuntu and you’ll probably experience the same fast login time.