It can't work with gradient descent, which requires about 16 bits of precision. The problem is the gradients are small, and so are rounded down to zero. Then when added together they get zero, even though the number should be higher.
However it's been shown this can be solved by using stochastic rounding. But I believe that would require specialized hardware to implement. I'm not sure.
However it's been shown this can be solved by using stochastic rounding. But I believe that would require specialized hardware to implement. I'm not sure.