I wouldn't call some cases of UB like overflowing a signed int or running into a dangling reference "actively pushing the compiler". I'm having a hard time believing you didn't ever run into these or similar issues having "programmed for a long time".
For other cases (e.g. aliasing, alignment), I'd agree that one is rather safe as long as the dangerous tools (e.g. reinterpret_cast) are not used.
For other cases (e.g. aliasing, alignment), I'd agree that one is rather safe as long as the dangerous tools (e.g. reinterpret_cast) are not used.