I've been thinking a lot about the ability of neural networks to develop understanding and wanted to share my perspective on this. For me it seems absolutely necessary for a NN to develop an understanding of its training data.
Take Convolutional Neural Networks (CNNs) used in computer vision, for example. One can observe how the level of abstraction increases in each layer. It starts with detecting brightness transitions, followed by edges, then general shapes, and eventually specific objects like cars or houses. Through training, the network learns the concept of a car and understands what a car is.
The same principle applies to Transformer networks in text processing. Instead of pixels, they process textual elements. Neurons in different layers learn to recognize complex relationships and understand abstract concepts.
Take Convolutional Neural Networks (CNNs) used in computer vision, for example. One can observe how the level of abstraction increases in each layer. It starts with detecting brightness transitions, followed by edges, then general shapes, and eventually specific objects like cars or houses. Through training, the network learns the concept of a car and understands what a car is.
The same principle applies to Transformer networks in text processing. Instead of pixels, they process textual elements. Neurons in different layers learn to recognize complex relationships and understand abstract concepts.