For neural networks, it makes sense to skip pure digital design.
When I learned how to design an ALU to say, add, and wait for the propagation of carry bits that’s like O(n) where n is the number of bits in the number, it made me want to just use superposition for addition, which is physical and instantaneous. Of course, that has all sorts of other problems that make it worse (so much worse).
Once you learn how to slow down the earlier bits, you end up with all the bits arriving at the same time, and you can have up to n adds pipelined and timed to the clock when output matters.
But with NN I think now would be a fun time (and likely the last decade as well) to rethink the basics right down to the basics. Everything need not be a NAND gate. :)
When I learned how to design an ALU to say, add, and wait for the propagation of carry bits that’s like O(n) where n is the number of bits in the number, it made me want to just use superposition for addition, which is physical and instantaneous. Of course, that has all sorts of other problems that make it worse (so much worse).
Once you learn how to slow down the earlier bits, you end up with all the bits arriving at the same time, and you can have up to n adds pipelined and timed to the clock when output matters.
But with NN I think now would be a fun time (and likely the last decade as well) to rethink the basics right down to the basics. Everything need not be a NAND gate. :)