>Unless it's one of those cursed things installed at the customer thousands of miles away that never happens back in the lab.
I had a bug like that in my previous (telcom embedded dev) career. Ended up driving to the customer premises (luckily mere 200km), working two weeks on collecting traces for the repro, and a day on the patch. Once I figured out how to trace the repro, the rest was trivial - the bug was glaringly obvious in the trace. Which in hindsight means I didn't really need to drive there at all, I merely needed to properly implement tracing the repro, and send the built artifact to the customer.
The problem would have been trivially solved if I had sufficient experience with tracing, or found a colleague with sufficient experience. However this one time the experience was bought & paid for with the trip.
I had a bug like that in my previous (telcom embedded dev) career. Ended up driving to the customer premises (luckily mere 200km), working two weeks on collecting traces for the repro, and a day on the patch. Once I figured out how to trace the repro, the rest was trivial - the bug was glaringly obvious in the trace. Which in hindsight means I didn't really need to drive there at all, I merely needed to properly implement tracing the repro, and send the built artifact to the customer.
The problem would have been trivially solved if I had sufficient experience with tracing, or found a colleague with sufficient experience. However this one time the experience was bought & paid for with the trip.