One compromise that I can recommend is hosting the Email server yourself for incoming mail, but using an external SMTP relay such as SendGrid or SMTP2GO to send outgoing email. That way you never have to worry about your emails being blocked due to some misconfigured setting. Switching SMTP relay provider is easy so vendor lock-in is no issue.
Also, those services are mostly intended for broadcasting mass spam mails for better or worse, so for personal use their free tiers are almost too good to be true.
Also, those services are mostly intended for broadcasting mass spam mails for better or worse, so for personal use their free tiers are almost too good to be true.