Comment

Comment by godelski

Yes but it doesn't generalize very well. Even on simple features like gender. If you go look at embeddings you'll find that man and woman are neighbors, just as king and queen are[0]. This is a better explanation for the result as you're just taking very small steps in the latent space.

Here, play around[1]

  mother - parent + man = woman
  father - parent + woman = man
  father - parent + man = woman
  mother - parent + woman = man
  woman - human + man = girl
Or some that should be trivial

  woman - man + man = girl
  man - man + man = woman
  woman - woman + woman = man
  
Working in very high dimensions is funky stuff. Embedding high dimensions into low dimensions results in even funkier stuff

[0] https://projector.tensorflow.org/

[1] https://www.cs.cmu.edu/~dst/WordEmbeddingDemo/

parent godelski 1 year ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton