agreed, I find it pretty useful to check if I remember where LSTM stuff connects
I remember reading the original paper a while ago but always forget (pun intended) where to connect stuff
then I realized that memorizing it visually is not the best approach, it's better to think about it in this sorta loose fashion -- I remember there is forget gate, well it forgets previous stuff so there is probably some hadamard product somewhere, it probably needs some inputs and previous hidden states...there was some -1,1 forcing in candidate memory so probably needs tanh instead of sigmoid...and then piece by piece i can reconstruct it pretty closely
I remember reading the original paper a while ago but always forget (pun intended) where to connect stuff
then I realized that memorizing it visually is not the best approach, it's better to think about it in this sorta loose fashion -- I remember there is forget gate, well it forgets previous stuff so there is probably some hadamard product somewhere, it probably needs some inputs and previous hidden states...there was some -1,1 forcing in candidate memory so probably needs tanh instead of sigmoid...and then piece by piece i can reconstruct it pretty closely