Comment by rxm

What used to be feature engineering a decade or more ago now seems to have shifted to developing distributed representations. LLMs use word tokens (for words or the entities in images). But there are many more. The 3D Fields (or whatever they have evolved to) developed by Fei-Fei Li's group represent visual information in a way better suited for geometrical tasks. Wav2Vec, the convolutional features for YOLO and friends, and these sentence representations are other examples. I would love to read a review of this circle of ideas.