What are your experiences with customizing Twitter Bootstrap?

Whenever I run into a problem with Bootstrap (usually a layout issue), I try to customize it. Every time I look over the docs, I become extremely anxious, jettison bootstrap from the project, and write the whole site from a clean slate.

Am I missing out on incredible time-savings by not taking the time to learn Twitter Bootstrap customization?