Ask HN: Managers of remote employees, what's hard for you?

I've been a remote person for a little while now and a big advocate for it. I've thought a lot about what's hard for me, but, I'd love to hear the other side of it. What's hard about managing a remote person?