Comment by draaglom

Yes, serious forecasters predict on hundreds of questions & the relevant websites prominently feature "binary calibration" plots.

This is mine on Metaculus (mentioned in the article):

https://imgur.com/a/d3s67xk

There's still a bit of noise at 205 predictions but you can see a pattern emerging!