My impression, which others may correct me on, is that he started down that road while in Los Alamos: I vaguely recall that his "You just ask them?" story was in that setting, anyway. I've always had the sense that he was deeply (and monogamously) devoted to his wife until her death. After her death, well, he says in this letter that he's had multiple girlfriends but that those relationships never went anywhere; my guess would be that he's telling his wife about precisely those early flings, and what they meant to him.
But yeah, for all his brilliance and magnetism and vision, Feynman was also (eventually) pretty terrible toward women. I gaze in grateful awe and reverence at his personal contributions to human knowledge, even as I grimace in frustration and disappointment at his contributions to a culture that continues to drive too many potentially brilliant contributors away. So this letter is bittersweet to me, since it calls to mind one of the few significant areas where Feynman's life did harm to humanity rather than good. I wonder if that might have been different if his wife had lived?
But yeah, for all his brilliance and magnetism and vision, Feynman was also (eventually) pretty terrible toward women. I gaze in grateful awe and reverence at his personal contributions to human knowledge, even as I grimace in frustration and disappointment at his contributions to a culture that continues to drive too many potentially brilliant contributors away. So this letter is bittersweet to me, since it calls to mind one of the few significant areas where Feynman's life did harm to humanity rather than good. I wonder if that might have been different if his wife had lived?