Comment by hundt

One possibility is that he "works hard" at that, too, e.g. by consciously thinking about what the best things to do with them are and endeavoring to spend as much as possible doing them. Perhaps if you internalize the idea that spending time with your family is in fact work, of a different but also valuable nature, then you will not find it as difficult.