Comment Comment by paul_mk1 Goes back before then. This got popularized by BinaryConnect in 2015, and groups were training binary networks as early as 2011.You are probably referring to XNOR net, and the novel piece there was also using binary activations (which bitnet is not).So as far as I can tell, bitnet is basically BinaryConnect applied to LLMs.https://arxiv.org/abs/1511.00363 parent paul_mk1 2 years ago
You are probably referring to XNOR net, and the novel piece there was also using binary activations (which bitnet is not).
So as far as I can tell, bitnet is basically BinaryConnect applied to LLMs.
https://arxiv.org/abs/1511.00363