Comment by danielmorozoff

Similar work has been pursued for a number of years now in a Darpa program called Machine Common Sense: https://www.darpa.mil/news-events/2018-10-11

I recall Tenenbaum's lab had a similar paper a few years back.