Comment Comment by li-ch Seems that CNTK already implemented this.https://github.com/Microsoft/CNTK/wiki/Enabling-1bit-SGD parent li-ch 10 years ago
https://github.com/Microsoft/CNTK/wiki/Enabling-1bit-SGD