Comment by Philippe_H

well just whitelist your Public IPs or use a combo of IPset & port knockd. Works fine for me for variable IPs.