EDIT: As @rubatuga said,they have been avoided largely as embeddings are continuous random vars and MI is for distributions. Nonetheless I think there can be merit in exploring these
PS: I am the author and just wanted to talk about the familiar basic stuff on this one
I found something for MI if you wanna check it out : https://www.aclweb.org/anthology/2020.acl-main.741.pdf
EDIT: As @rubatuga said,they have been avoided largely as embeddings are continuous random vars and MI is for distributions. Nonetheless I think there can be merit in exploring these
PS: I am the author and just wanted to talk about the familiar basic stuff on this one