Ask HN: Self Hosted vs. Gmail / Outlook?

I'm currently using Gmail and getting more and more worried about my privacy. I'd like to know my options if I decide to jump ship. Should I setup my own email server ? Or use another less invasive service ? What are your thoughts ? How do you concile email and privacy ?