Long time vegan here. While I think that saying has some truth to it, there are also those that sympathize with the ethics, try it out for a short while, but find the pressure of social backlash too costly. The biggest problem isn't nutrition; It's ingroup/outgroup policing (predominantly by the vast majority of non-vegans).
https://www.youtube.com/watch?v=qQUo53o6QXg#t=0m30s (strawman opening skipiped) is clearly an exaggeration, but highlights the uncomfortable reality of being "the different one" at the dinner table.