pigs

I rarely visit HN anymore because of shit like this, but I decided to check back over here to see what everyone had to see. not surprised

We use Clojure extensively at Room Key. Our main web API, various scheduled cron-like jobs, and AWS Lambda functions are all written in Clojure.

Are you willing to say how much money was at risk?

Yes, significantly upping the ulimit would have probably been sufficient for all practical purposes, although we might run out of JVM heap space if we are up long enough. Plus, it really bothered me that I didn't know what was going on.

There were only two threads stuck, so monitoring the threads wouldn't have set off any alarms. One was a Clojure agent thread, which basically allowed all the clients marked for retirement to queue up.

I used both Yourkit and Java Mission Control for monitoring the JVM. They both have some functionality of deadlock detection, but it did not flag these two threads. Yourkit identified these threads as waiting, not blocked, but I'm not sure how it makes that distinction.

Damn, this is sad. I'll echo the other comments here that he was a nice guy. I was an undergrad at Duke circa 1999-2000 and knew him through the Duke LUG. I was a noob, and he always showed a fair amount of patience with my questions on the mailing list. Later on the LUG started meeting for beers on Friday afternoons and he was just as friendly in real life. He was one of those guys that seemed like a computing god for me so early in my career.