Are libraries always the best choice in secure development?

I attended a secure development course where the instructor insisted on always using third-party libraries. While I agree that relying on well-audited libraries—especially for cryptography—is often the safest route, there have been plenty of cases where libraries were exploited. Has anyone experienced scenarios where building custom solutions outperformed using existing libraries? Also, how do you typically audit third-party libraries to assess their security, and what factors do you consider when weighing library trustworthiness against in-house development?