Not a "hard" bug but a useful lesson in any case. I worked on a set of stress tests for a major middleware product and came into the office on a Monday morning to check the 72-hour over-weekend runs. We were getting close to release date and things were settling down so I wasn't expecting anything major. Except they'd ALL failed. It took us far longer than I'd care to admit to figure out what had gone wrong - I wasn't working on it non-stop but I definitely remember it taking quite some time. I think it was a colleague who figured it out later that week.
Anyway, what had happened was that our Perl test harness was tracking time elapsed in the 72-hour run as seconds since the Unix epoch, but was comparing them using the lexicographical order operator (lt versus <). Everything worked until the time ticked over from 999,999,999 seconds to 1,000,000,000.
I just looked up those timestamps to check my memory, and I can now see why fixing it wasn't our top priority that week... the 999999999/1000000000 transition happened the weekend before 9/11.
Anyway, what had happened was that our Perl test harness was tracking time elapsed in the 72-hour run as seconds since the Unix epoch, but was comparing them using the lexicographical order operator (lt versus <). Everything worked until the time ticked over from 999,999,999 seconds to 1,000,000,000.
I just looked up those timestamps to check my memory, and I can now see why fixing it wasn't our top priority that week... the 999999999/1000000000 transition happened the weekend before 9/11.