Bayes7

>"The first species of Paulinella was discovered on Dec. 24, 1894, in the sediments of a backwater of the Rhine River by the German biologist Robert Lauterborn. He named the genus after his stepmother, Pauline."

How can they say this without providing more background?

| „[…] was a friend telling me his LaTeX thesis took 90 seconds to compile towards the end“

Sure, but in order to iterate you won’t have to compile the whole document but can just keep the chapter you are working on by structuring it with \includes

Would you mind sharing how it turned out useful?

thanks!

Is there any good article/paper that describes how it actually works or is implemented not just in high-level and hand-waving terms?

great project!

This was a great read, thanks a lot! One a side note, any one has a good guess what tool/software they used to create the visualisations for matrix multiplications or memory outline?

Okay, I see that for inference. But for training it shouldn't matter because I need to hold on to all my activations for my backwards pass anyways? But yeah, fair point!

"[...] modern neural network (NN) architectures have complex designs with many components [...]"

I find the Transformer architecture actually very simple compared to previous models like LSTMs or other recurrent models. You could argue that their vision counterparts like ViT are conceptually maybe even simpler than ConvNets?

Also, can someone explain why they are so keen to remove the skip connections? At least when it comes to coding, nothing is simpler than adding a skip connection and computationally the effect should be marginal?